目录
一、ELK介绍
二、ELK安装准备工作
三、安装es
四、 配置es
五、curl查看es
六、安装kibanan
七、安装logstash
八、配置logstash
九、kibanan上查看日志
十、收集nginx日志
十一、使用beats采集日志
十二、扩展部分
一、ELK介绍
需求背景
业务发展越来越庞大，服务器越来越多
各种访问日志、应用日志、错误日志量越来越多
开发人员排查问题，需要到服务器上查日志，不方便
运营人员需要一些数据，需要我们运维到服务器上分析日志ELK介绍
官网https://www.elastic.co/cn/
中文指南https://www.gitbook.com/book/chenryn/elk-stack-guide-cn/details
ELK Stack （5.0版本之后→ Elastic Stack ==（ELK Stack + Beats）
ELK Stack包含：ElasticSearch、Logstash、Kibana
ElasticSearch是一个搜索引擎，用来搜索、分析、存储日志。它是分布式的，也就是说可以横向扩容，可以自动发现，索引自动分片，总之很强大。文档https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html
Logstash用来采集日志，把日志解析为json格式交给ElasticSearch。
Kibana是一个数据可视化组件，把处理后的结果通过web界面展示
Beats在这里是一个轻量级日志采集器，其实Beats家族有5个成员
早期的ELK架构中使用Logstash收集、解析日志，但是Logstash对内存、cpu、io等资源消耗比较高。相比 Logstash，Beats所占系统的CPU和内存几乎可以忽略不计
x-pack对Elastic Stack提供了安全、警报、监控、报表、图表于一身的扩展包，是收费的ELK架构
二、ELK安装准备工作
准备3台机器130,132,128
角色划分：
3台全部安装elasticsearch(后续简称es) ，1主节点130，2数据节点132,128
es主130上安装kibana
1台es数据节点132上安装logstash 3台机器全部安装jdk8（openjdk即可）
yum install -y java-1.8.0-openjdk
#三个机器的hosts都改一下，方便管理
[root@minglinux-01 ~] vim /etc/hosts
···
8 192.168.162.130 minglinux-01
9 192.168.162.132 minglinux-02
10 192.168.162.128 minglinux-03
#安装jdk
[root@minglinux-01 ~] java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
[root@minglinux-01 ~] which java
/usr/local/jdk1.8/bin/java
[root@minglinux-02 ~] java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
[root@minglinux-02 ~] which java
/usr/local/jdk1.8/bin/java
[root@minglinux-03 ~] java -version
-bash: java: 未找到命令
[root@minglinux-03 ~] yum install -y java-1.8.0-openjdk
[root@minglinux-03 ~] java -version
openjdk version "1.8.0_201"
OpenJDK Runtime Environment (build 1.8.0_201-b09)
OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)
[root@minglinux-03 ~] which java
/usr/bin/java
三、安装es
官方文档 https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html
以下操作3台机器上都要执行
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
vim /etc/yum.repos.d/elastic.repo //加入如下内容
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
yum install -y elasticsearch //也可以直接下载rpm文件，然后安装
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0.rpm
rpm -ivh elasticsearch-6.0.0.rpm
#3台机器配置yum源，然后yum安装elasticsearch
[root@minglinux-01 ~] rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@minglinux-01 ~] vim /etc/yum.repos.d/elastic.repo
#加入以下内容
1 [elasticsearch-6.x]
2 name=Elasticsearch repository for 6.x packages
3 baseurl=https://artifacts.elastic.co/packages/6.x/yum
4 gpgcheck=1
5 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
6 enabled=1
7 autorefresh=1
8 type=rpm-md
[root@minglinux-01 ~] yum list |grep elastic
apm-server.i686 6.6.2-1 elasticsearch-6.x
apm-server.x86_64 6.6.2-1 elasticsearch-6.x
auditbeat.i686 6.6.2-1 elasticsearch-6.x
auditbeat.x86_64 6.6.2-1 elasticsearch-6.x
elastic-curator.noarch 3.2.3-1.el7 epel
elasticdump.noarch 2.2.0-2.el7 epel
elasticsearch.noarch 6.6.2-1 elasticsearch-6.x
filebeat.i686 6.6.2-1 elasticsearch-6.x
filebeat.x86_64 6.6.2-1 elasticsearch-6.x
heartbeat-elastic.i686 6.6.2-1 elasticsearch-6.x
heartbeat-elastic.x86_64 6.6.2-1 elasticsearch-6.x
journalbeat.i686 6.6.2-1 elasticsearch-6.x
journalbeat.x86_64 6.6.2-1 elasticsearch-6.x
kibana.x86_64 6.6.2-1 elasticsearch-6.x
kibana-oss.x86_64 6.3.0-1 elasticsearch-6.x
logstash.noarch 1:6.6.2-1 elasticsearch-6.x
metricbeat.i686 6.6.2-1 elasticsearch-6.x
metricbeat.x86_64 6.6.2-1 elasticsearch-6.x
packetbeat.i686 6.6.2-1 elasticsearch-6.x
packetbeat.x86_64 6.6.2-1 elasticsearch-6.x
pcp-pmda-elasticsearch.x86_64 4.1.0-5.el7_6 updates
python-elasticsearch.noarch 1.9.0-1.el7 epel
rsyslog-elasticsearch.x86_64 8.24.0-34.el7 base
[root@minglinux-01 ~] yum install -y elasticsearch
[root@minglinux-01 ~] ln -s /usr/local/jdk1.8/bin/java /usr/bin/java
#报错了，做一个软连接后安装成功
could not find java; set JAVA_HOME or ensure java is in PATH
error: %pre(elasticsearch-0:6.6.2-1.noarch) scriptlet failed, exit status 1
Error in PREIN scriptlet in rpm package elasticsearch-6.6.2-1.noarch
[root@minglinux-01 ~] yum install -y elasticsearch
[root@minglinux-02 ~] rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@minglinux-02 ~] vim /etc/yum.repos.d/elastic.repo
[root@minglinux-02 ~] yum install -y elasticsearch
[root@minglinux-02 ~] ln -s /usr/local/jdk1.8/bin/java /usr/bin/java
[root@minglinux-02 ~] yum install -y elasticsearch
[root@minglinux-03 ~] rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@minglinux-03 ~] vim /etc/yum.repos.d/elastic.repo
[root@minglinux-03 ~] yum install -y elasticsearch
四、 配置es
elasticsearch配置文件/etc/elasticsearch和/etc/sysconfig/elasticsearch
参考https://www.elastic.co/guide/en/elasticsearch/reference/6.0/rpm.html
在130上编辑配置文件vim /etc/elasticsearch/elasticsearch.yml//增加或更改
cluster.name: minglinux
node,name: minglinux-01
node.master: true//意思是该节点为主节点
node.data: false //是不是数据节点
network.host: 192.168.162.130 //0.0.0.0是监听全部ip，不安全
discovery.zen.ping.unicast.hosts: ["192.168.162.130", "192.168.162.132", "192.168.162.128"]
在132和128上同样编辑配置文件vim /etc/elasticsearch/elasticsearch.yml//增加或更改
cluster.name: minglinux
node.name: minglinux-02 或minglinux-03
node.master: false
node.data: true //是数据节点
network.host: 192.168.162.132 或192.168.162.128
discovery.zen.ping.unicast.hosts: ["192.168.162.130", "192.168.162.132", "192.168.162.128"]
#130编辑配置文件
[root@minglinux-01 ~] vim /etc/elasticsearch/elasticsearch.yml
#再对应区域添加如下内容
18 cluster.name: minglinux
26 node.name: minglinux-01
30 node.master: true
31 node.data: false
60 network.host: 192.168.162.130
74 discovery.zen.ping.unicast.hosts: ["192.168.162.130", "192.168.162.132", "192.168.162.128"]
#将01机器修改好的配置文件传到02和03机器，然后修改
[root@minglinux-01 ~] scp /etc/elasticsearch/elasticsearch.yml minglinux-02:/tmp/
elasticsearch.yml 100% 3076 1.6MB/s 00:00
[root@minglinux-01 ~] scp /etc/elasticsearch/elasticsearch.yml minglinux-03:/tmp/
elasticsearch.yml 100% 3076 2.6MB/s 00:00
[root@minglinux-02 ~] cp /tmp/elasticsearch
elasticsearch-6356716019382047520/ elasticsearch.yml
[root@minglinux-02 ~] cp /tmp/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml
cp：是否覆盖"/etc/elasticsearch/elasticsearch.yml"？ y
[root@minglinux-02 ~] vim /etc/elasticsearch/elasticsearch.yml
···
cluster.name: minglinux
node.name: minglinux-02
node.master: false
node.data: true
network.host: 192.168.162.132
discovery.zen.ping.unicast.hosts: ["192.168.162.130", "192.168.162.132", "192.168.162.128"]
[root@minglinux-03 ~] cp /tmp/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml
cp：是否覆盖"/etc/elasticsearch/elasticsearch.yml"？ y
[root@minglinux-03 ~] vim !$
vim /etc/elasticsearch/elasticsearch.yml
···
cluster.name: minglinux
node.name: minglinux-03
node.master: false
node.data: true
network.host: 192.168.162.128
discovery.zen.ping.unicast.hosts: ["192.168.162.130", "192.168.162.132", "192.168.162.128"]
- ELK安装 – 安装x-pack（可省略）
3台机器上都要执行
cd /usr/share/elasticsearch/bin/ （可省略）
./elasticsearch-plugin install x-pack //如果速度慢，就下载x-pack压缩包（可省略）
cd /tmp/; wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.0.0.zip （可省略）
./elasticsearch-plugin install file:///tmp/x-pack-6.0.0.zip （可省略）
启动elasticsearch服务
启动失败查看日志/var/log/elasticsearch/minglinux.log
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
以下操作只需要在130上执行
安装x-pack后就可以为内置用户设置密码了，如下
/usr/share/elasticsearch/bin/x-pack/setup-passwords interactive （可省略）
curl localhost:9200 -u elastic //输入密码，可以查看到输出信息（可省略）
#由于x-pack是收费的就不安装了，接下来启动服务
[root@minglinux-01 ~] systemctl start elasticsearch
[root@minglinux-01 ~] ps aux |grep elasticsearch
elastic+ 35666 34.5 52.4 3444748 977676 ? Ssl 20:25 0:22 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-4079948052346171381 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
root 35840 0.0 0.0 112720 984 pts/0 R+ 20:26 0:00 grep --color=auto elasticsearch
#01机器监听的端口一直看不到，而且启动起来后会自动退出，检查到有iptables规则，所以清空规则，然后在杀掉一些进程，服务器负载太高了。启动都要等好一会才看到9200和9300端口。
[root@minglinux-01 ~] systemctl stop iptables.service
[root@minglinux-01 ~] iptables -nvL
[root@minglinux-01 ~] pkill mongod
[root@minglinux-01 ~] ps aux |grep mongo
root 41950 0.0 0.0 112720 984 pts/0 S+ 21:15 0:00 grep --color=auto mongo
[root@minglinux-02 ~] systemctl start elasticsearch
[root@minglinux-02 ~] ps aux |grep elasticsearch
elastic+ 5101 37.8 60.5 1591216 696520 ? Ssl 20:48 0:02 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-8969078006330607850 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
root 5155 23.0 0.0 112720 984 pts/1 S+ 20:48 0:00 grep --color=auto elasticsearch
[root@minglinux-03 ~] systemctl start elasticsearch
[root@minglinux-03 ~] ps aux |grep elasticsearch
elastic+ 21046 69.0 74.7 1550420 746264 ? Ssl 20:49 0:02 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-7191065286840389837 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
root 21093 36.0 0.0 112720 984 pts/0 S+ 20:49 0:00 grep --color=auto elasticsearch
五、curl查看es
130上执行
curl '192.168.162.130:9200/_cluster/health?pretty' 健康检查
curl '192.168.162.130/_cluster/state?pretty' 集群详细信息
参考 http://zhaoyanblog.com/archives/732.html
[root@minglinux-01 ~] curl '192.168.162.130:9200/_cluster/health?pretty'
{
"cluster_name" : "minglinux",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 2,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
[root@minglinux-01 ~] curl '192.168.162.130:9200/_cluster/state?pretty'
#太长了不贴出来了
六、安装kibanan
以下在130上执行
前面已经配置过yum源，这里就不用再配置了
yum install -y kibana
若速度太慢，可以直接下载rpm包
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-x86_64.rpm
rpm -ivh kibana-6.0.0-x86_64.rpm
kibana同样也需要安装x-pack（可省略）
安装方法同elasticsearch的x-pack
cd /usr/share/kibana/bin （可省略）
./kibana-plugin install x-pack //如果这样安装比较慢，也可以下载zip文件（可省略）
wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.0.0.zip//这个文件和前面下载的那个其实是一个（可省略）
./kibana-plugin install file:///tmp/x-pack-6.0.0.zip （可省略）以下也是在130上执行
vim /etc/kibana/kibana.yml //增加
server.host: 192.168.162.130 //未安装x-pack无需密码登录，安全起见只监听内网
elasticsearch.url: "http://192.168.162.130:9200"
logging.dest: /var/log/kibana.log
touch /var/log/kibana.log; chmod 777 /var/log/kibana.log
systemctl restart kibana
浏览器里访问http://192.168.162.130:5601/
用户名elastic，密码为之前你设置过的密码（如果未安装x-pack，不需要用户名密码）
若无法输入用户名密码，查日志/var/log/kibana.log
出现错误 Status changed from uninitialized to red - Elasticsearch is still initializing the kibana index.
解决办法：curl -XDELETE http://192.168.162.130:9200/.kibana -uelastic
#安装
[root@minglinux-01 ~] yum install -y kibana
#修改配置文件
[root@minglinux-01 ~] vim /etc/kibana/kibana.yml
#新增如下内容
2 server.port: 5601
8 server.host: 192.168.162.130
28 elasticsearch.hosts: ["http://192.168.162.130:9200"]
97 logging.dest: /var/log/kibana.log
#创建日志目录并修改权限
[root@minglinux-01 ~] touch /var/log/kibana.log; chmod 777 /var/log/kibana.log
#启动kibana
[root@minglinux-01 ~] systemctl restart kibana
[root@minglinux-01 ~] ps aux |grep kibana
kibana 49103 109 8.0 1236344 149788 ? Rsl 22:13 0:14 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml
root 49142 0.0 0.0 112720 984 pts/0 S+ 22:13 0:00 grep --color=auto kibana
[root@minglinux-01 ~] netstat -lntp |grep 5601
tcp 0 0 192.168.162.130:5601 0.0.0.0:* LISTEN 49103/node
-
浏览器访问
七、安装logstash
以下在132上执行
logstash目前不支持java9
直接yum安装（配置源同前面es的源）
yum install -y logstash //如果慢，就下载rpm包
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.0.0.rpm
cd /usr/share/logstash/bin/（可省略）
./logstash-plugin install file:///tmp/x-pack-6.0.0.zip （可省略）
#02机器上安装logstash
[root@minglinux-02 ~] yum install -y logstash
- 先进行logstash收集syslog日志的一个测试
以下在132上操作
编辑配置文件 vim /etc/logstash/conf.d/syslog.conf//加入如下内容
input {
syslog {
type => "system-syslog" #这里将syslog输出到logstash监听端口10514
port => 10514 #也可以直接指定文件
}
}
output {
stdout {
codec => rubydebug
}
}
检测配置文件是否有错
cd /usr/share/logstash/bin
./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf --config.test_and_exit以下在132上操作
前台形式启动logstash
./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf//这样可以在屏幕上查看到日志输出，不能敲命令
再开一个终端
检测是否开启10514端口：netstat -lnp |grep 10514
vi /etc/rsyslog.conf//在#### RULES下面增加一行
. @@127.0.0.1:10514
systemctl restart rsyslog
从130ssh到132上，可以在logstash前台的终端上看到ssh登录的相关日志
结束logstash，在前台的那个终端上按ctrl c
[root@minglinux-02 ~] vim /etc/logstash/conf.d/syslog.conf
#新增如下内容
input {
syslog {
type => "system-syslog"
port => 10514
}
}
output {
stdout {
codec => rubydebug
}
}
#检测配置文件是否有错
[root@minglinux-02 ~] cd /usr/share/logstash/bin
[root@minglinux-02 /usr/share/logstash/bin] ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf --config.test_and_exit
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2019-03-20T23:28:03,361][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/var/lib/logstash/queue"}
[2019-03-20T23:28:03,398][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/var/lib/logstash/dead_letter_queue"}
[2019-03-20T23:28:05,583][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2019-03-20T23:28:20,681][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
#前台形式启动logstash
[root@minglinux-02 /usr/share/logstash/bin] ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf
#等待在屏幕上查看日志输出，不能敲命令
#另开一个终端执行一些操作，然后查看日志是否输出到10514端口，即显示在启动logstash的终端的前台
[root@minglinux-02 /usr/share/logstash/bin] vim /etc/rsyslog.conf
#RULES下面增加一行
#### RULES ####
*.* @@192.168.162.132:10514
[root@minglinux-02 ~] systemctl restart rsyslog
[root@minglinux-02 ~] netstat -lnp |grep 10514
tcp6 0 0 :::10514 :::* LISTEN 5681/java
udp 0 0 0.0.0.0:10514 0.0.0.0:* 5681/java
#进行ssh登录
[root@minglinux-01 ~] ssh minglinux-02
Last login: Wed Mar 20 23:33:54 2019 from 192.168.162.1
[root@minglinux-02 ~] 登出
Connection to minglinux-02 closed.
[root@minglinux-01 ~] ssh minglinux-02
Last login: Wed Mar 20 23:40:30 2019 from gitlab.example.com
[root@minglinux-02 ~] 登出
Connection to minglinux-02 closed.
#可以查看到ssh登录的相关日志
[root@minglinux-02 /usr/share/logstash/bin] ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2019-03-20T23:49:07,728][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-03-20T23:49:07,772][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.6.2"}
[2019-03-20T23:49:20,782][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-03-20T23:49:21,498][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x9b299a7 run>"}
[2019-03-20T23:49:21,719][INFO ][logstash.inputs.syslog ] Starting syslog tcp listener {:address=>"0.0.0.0:10514"}
[2019-03-20T23:49:21,728][INFO ][logstash.inputs.syslog ] Starting syslog udp listener {:address=>"0.0.0.0:10514"}
[2019-03-20T23:49:21,749][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-03-20T23:49:22,257][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-03-20T23:50:01,797][INFO ][logstash.inputs.syslog ] new connection {:client=>"192.168.162.132:53582"}
{
"message" => "Started Session 38 of user root.\n",
"@version" => "1",
"priority" => 30,
"timestamp" => "Mar 20 23:50:01",
"severity" => 6,
"host" => "192.168.162.132",
"type" => "system-syslog",
"facility_label" => "system",
"severity_label" => "Informational",
"@timestamp" => 2019-03-20T15:50:01.000Z,
"program" => "systemd",
"logsource" => "minglinux-02",
"facility" => 3
}
{
"message" => "action 'action 0' resumed (module 'builtin:omfwd') [v8.24.0 try http://www.rsyslog.com/e/2359 ]\n",
"@version" => "1",
"priority" => 46,
"timestamp" => "Mar 20 23:50:01",
"severity" => 6,
"host" => "192.168.162.132",
"type" => "system-syslog",
"facility_label" => "syslogd",
"severity_label" => "Informational",
"@timestamp" => 2019-03-20T15:50:01.000Z,
"program" => "rsyslogd",
"logsource" => "minglinux-02",
"facility" => 5
}
八、配置logstash
以下在132上操作
后台形式启动logstash
编辑配置文件 vim /etc/logstash/conf.d/syslog.conf//配置文件内容改为如下
input {
syslog {
type => "system-syslog"
port => 10514
}
}
output {
elasticsearch {
hosts => ["192.168.162.132:9200"]
index => "system-syslog-%{+YYYY.MM}"
}
}
systemctl start logstash //启动需要一些时间，启动完成后，可以看到9600端口和10514端口已被监听
130上执行curl 'localhost:9200/_cat/indices?v' 可以获取索引信息
curl -XGET 'localhost:9200/indexname?pretty' 可以获指定索引详细信息
curl -XDELETE 'localhost:9200/logstash-xxx-*' 可以删除指定索引
浏览器访问192.168.162.130:5601，到kibana配置索引
左侧点击“Managerment”-> “Index Patterns”-> “Create Index Pattern”
Index pattern这里需要根据前面curl查询到的索引名字来写，否则下面的按钮是无法点击的
[root@minglinux-02 /usr/share/logstash/bin] vim /etc/logstash/conf.d/syslog.conf
#配置文件改为如下
input {
syslog {
type => "system-syslog"
port => 10514
}
}
output {
elasticsearch {
hosts => ["192.168.162.132:9200"]
index => "system-syslog-%{+YYYY.MM}"
}
}
#检测一下配置文件有没有错
[root@minglinux-02 /usr/share/logstash/bin] ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf --config.test_and_exit
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2019-03-21T00:07:23,462][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2019-03-21T00:07:33,481][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
#后台以服务方式启动logstash
[root@minglinux-02 /usr/share/logstash/bin] systemctl start logstash
Failed to start logstash.service: Unit not found.
[root@minglinux-02 /usr/share/logstash/bin] /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd #生成logstash.service
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
Successfully created system startup script for Logstash
[root@minglinux-02 /usr/share/logstash/bin] systemctl start logstash
[root@minglinux-02 /usr/share/logstash/bin] ps aux |grep logstash
logstash 6481 190 33.7 3477180 387680 ? SNsl 00:14 0:47 /bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit.threshold=0 -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -cp /usr/share/logstash/logstash-core/lib/jars/animal-sniffer-annotations-1.14.jar:/usr/share/logstash/logstash-core/lib/jars/commons-codec-1.11.jar:/usr/share/logstash/logstash-core/lib/jars/commons-compiler-3.0.8.jar:/usr/share/logstash/logstash-core/lib/jars/error_prone_annotations-2.0.18.jar:/usr/share/logstash/logstash-core/lib/jars/google-java-format-1.1.jar:/usr/share/logstash/logstash-core/lib/jars/gradle-license-report-0.7.1.jar:/usr/share/logstash/logstash-core/lib/jars/guava-22.0.jar:/usr/share/logstash/logstash-core/lib/jars/j2objc-annotations-1.1.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-annotations-2.9.5.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-core-2.9.5.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-databind-2.9.5.jar:/usr/share/logstash/logstash-core/lib/jars/jackson-dataformat-cbor-2.9.5.jar:/usr/share/logstash/logstash-core/lib/jars/janino-3.0.8.jar:/usr/share/logstash/logstash-core/lib/jars/javassist-3.22.0-GA.jar:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.1.13.0.jar:/usr/share/logstash/logstash-core/lib/jars/jsr305-1.3.9.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-api-2.9.1.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-core-2.9.1.jar:/usr/share/logstash/logstash-core/lib/jars/log4j-slf4j-impl-2.9.1.jar:/usr/share/logstash/logstash-core/lib/jars/logstash-core.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.commands-3.6.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.contenttype-3.4.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.expressions-3.4.300.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.filesystem-1.3.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.jobs-3.5.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.resources-3.7.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.core.runtime-3.7.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.app-1.3.100.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.common-3.6.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.preferences-3.4.1.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.equinox.registry-3.5.101.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.jdt.core-3.10.0.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.osgi-3.7.1.jar:/usr/share/logstash/logstash-core/lib/jars/org.eclipse.text-3.5.101.jar:/usr/share/logstash/logstash-core/lib/jars/slf4j-api-1.7.25.jar org.logstash.Logstash --path.settings /etc/logstash
root 6509 0.0 0.0 112720 980 pts/1 R+ 00:14 0:00 grep --color=auto logstash
#启动过程较久，可以看看日志是否有错
[root@minglinux-02 /usr/share/logstash/bin] less /var/log/logstash/logstash-plain.log
#日志没有更新，判断是权限问题，更改所属用户
[root@minglinux-02 /usr/share/logstash/bin] chown logstash /var/log/logstash/logstash-plain.log
[root@minglinux-02 /usr/share/logstash/bin] chown -R logstash /var/lib/logstash
[root@minglinux-02 /usr/share/logstash/bin] systemctl restart logstash
#终于成功了
#等到10514和9600端口被监听才是启动成功
[root@minglinux-02 /usr/share/logstash/bin] netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 4232/master
tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 3818/zabbix_agentd
tcp 0 0 192.168.162.132:27017 0.0.0.0:* LISTEN 3981/mongod
tcp 0 0 127.0.0.1:27017 0.0.0.0:* LISTEN 3981/mongod
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3804/sshd
tcp6 0 0 ::1:25 :::* LISTEN 4232/master
tcp6 0 0 127.0.0.1:9600 :::* LISTEN 7323/java
tcp6 0 0 :::10050 :::* LISTEN 3818/zabbix_agentd
tcp6 0 0 :::3306 :::* LISTEN 4276/mysqld
tcp6 0 0 :::111 :::* LISTEN 1/systemd
tcp6 0 0 192.168.162.132:9200 :::* LISTEN 5101/java
tcp6 0 0 :::10514 :::* LISTEN 7323/java
tcp6 0 0 192.168.162.132:9300 :::* LISTEN 5101/java
tcp6 0 0 :::22 :::* LISTEN 3804/sshd
九、kibanan上查看日志
130上执行curl '192.168.162.130:9200/_cat/indices?v' 可以获取索引信息
curl -XGET '192.168.162.130:9200/indexname?pretty' 可以获指定索引详细信息
curl -XDELETE '192.168.162.130:9200/logstash-xxx-*' 可以删除指定索引
浏览器访问192.168.162.130:5601，到kibana配置索引
左侧点击“Managerment”-> “Index Patterns”-> “Create Index Pattern”
Index pattern这里需要根据前面curl查询到的索引名字来写，否则下面的按钮是无法点击的
#获取索引信息
[root@minglinux-01 ~] curl '192.168.162.130:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana_1 IpJkISvGSBecrHJR5dNMtA 1 1 3 0 24kb 12kb
green open system-syslog-2019.03 cWR1XimDQE6PkexAIHCYkA 5 1 5 0 81.5kb 40.7kb
[root@minglinux-01 ~] curl -XGET '192.168.162.130:9200/system-syslog-2019.03?pretty'
-
到kibana配置索引查看日志
[root@minglinux-02 /usr/share/logstash/bin] tail -f /var/log/messages
Mar 21 00:38:33 minglinux-02 logstash: [2019-03-21T00:38:33,852][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x1354ed0d run>"}
Mar 21 00:38:34 minglinux-02 logstash: [2019-03-21T00:38:34,787][INFO ][logstash.inputs.syslog ] Starting syslog udp listener {:address=>"0.0.0.0:10514"}
Mar 21 00:38:34 minglinux-02 logstash: [2019-03-21T00:38:34,792][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Mar 21 00:38:34 minglinux-02 logstash: [2019-03-21T00:38:34,789][INFO ][logstash.inputs.syslog ] Starting syslog tcp listener {:address=>"0.0.0.0:10514"}
Mar 21 00:38:35 minglinux-02 logstash: [2019-03-21T00:38:35,908][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
Mar 21 00:40:01 minglinux-02 systemd: Started Session 45 of user root.
Mar 21 00:40:01 minglinux-02 rsyslogd: action 'action 0' resumed (module 'builtin:omfwd') [v8.24.0 try http://www.rsyslog.com/e/2359 ]
Mar 21 00:40:01 minglinux-02 rsyslogd: action 'action 0' resumed (module 'builtin:omfwd') [v8.24.0 try http://www.rsyslog.com/e/2359 ]
Mar 21 00:40:01 minglinux-02 logstash: [2019-03-21T00:40:01,528][INFO ][logstash.inputs.syslog ] new connection {:client=>"192.168.162.132:34460"}
Mar 21 00:50:01 minglinux-02 systemd: Started Session 46 of user root.
十、收集nginx日志
132上 编辑配置文件 vi /etc/logstash/conf.d/nginx.conf//加入如下内容
input {
file {
path => "/tmp/elk_access.log"
start_position => "beginning"
type => "nginx"
}
}
filter {
grok {
match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} [%{HTTPDATE:timestamp}] "(?:%{WORD:http_verb} %{NOTSPACE:http_request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_http_request})" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"}
}
geoip {
source => "clientip"
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["192.168.162.132:9200"]
index => "nginx-test-%{+YYYY.MM.dd}"
}
}
检测配置文件是否有错
cd /usr/share/logstash/bin
./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/nginx.conf --config.test_and_exit
yum install -y nginx
vim /usr/local/nginx/conf/vhost/elk.conf//创建虚拟主机配置文件写入如下内容
server {
listen 80;
server_name elk.ming.com;
location / {
proxy_pass http://192.168.162.130:5601;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
access_log /tmp/elk_access.log main2;
}vim /etc/nginx/nginx.conf//增加如下内容
log_format main2 '$http_host $remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$upstream_addr" $request_time';
/usr/local/nginx/sbin/nginx -t
systemctl start nginx
windows绑定hosts 192.168.162.132 elk.ming.com
浏览器访问，检查是否有日志产生
systemctl restart logstash
[root@minglinux-02 /usr/share/logstash/bin] vi /etc/logstash/conf.d/nginx.conf
#加入以下内容
input {
file {
path => "/tmp/elk_access.log"
start_position => "beginning"
type => "nginx"
}
}
filter {
grok {
}
geoip {
input {
file {
path => "/tmp/elk_access.log"
start_position => "beginning"
type => "nginx"
}
}
filter {
grok {
match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:http_verb} %{NOTSPACE:http_request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_http_request})\" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"}
}
geoip {
source => "clientip"
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["192.168.162.132:9200"]
index => "nginx-test-%{+YYYY.MM.dd}"
}
}
#检测
[root@minglinux-02 /usr/share/logstash/bin] ./logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/nginx.conf --config.test_and_exit
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2019-03-21T02:03:17,049][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2019-03-21T02:03:33,526][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
#重启logstash
[root@minglinux-02 /usr/share/logstash/bin] systemctl restart logstash
#新建虚拟主机配置文件
[root@minglinux-02 /usr/local/nginx/conf/vhost] vim elk.conf
#内容如下
server {
listen 80;
server_name elk.ming.com;
location / {
proxy_pass http://192.168.162.130:5601;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
access_log /tmp/elk_access.log main2;
}
[root@minglinux-02 /usr/local/nginx/conf/vhost] vim /usr/local/nginx/conf/nginx.conf
#加入以下内容
···
log_format main2 '$http_host $remote_addr - $remote_user [$time_local] "$request"'
'$status $body_bytes_sent "$http_referer"'
'"$http_user_agent" "$upstream_addr" $request_time';
···
#检测nginx配置文件是否有错
[root@minglinux-02 /usr/local/nginx/conf/vhost] /usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[root@minglinux-02 /usr/local/nginx/conf/vhost] /usr/local/nginx/sbin/nginx -s reload
最后
130上curl '192.168.162.130:9200/_cat/indices?v'
检查是否有nginx-test开头的索引生成
如果有，才能到kibana里去配置该索引
左侧点击“Managerment”-> “Index Patterns”-> “Create Index Pattern”
Index pattern这里写nginx-test-*
之后点击左侧的Discover
#130上查看索引
[root@minglinux-01 ~] curl '192.168.162.130:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana_1 IpJkISvGSBecrHJR5dNMtA 1 1 4 0 39.4kb 19.7kb
green open system-syslog-2019.03 cWR1XimDQE6PkexAIHCYkA 5 1 2887 0 2.2mb 1mb
green open nginx-test-2019.03.20 hVWCcygQRvSfmE54H_vyIg 5 1 2821 0 1.4mb 703.4kb
-
到kibana里去配置新索引
-
日志页面
[root@minglinux-02 /usr/share/logstash/bin] tail -f /tmp/elk_access.log
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:11:54 +0800] "GET /bundles/kibana.bundle.js HTTP/1.1"304 0 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.038
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:11:59 +0800] "GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_6_0 HTTP/1.1"200 19139 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.104
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /api/spaces/space HTTP/1.1"200 114 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.095
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /api/saved_objects/_find?type=index-pattern&fields=title&search=*&search_fields=title&per_page=1 HTTP/1.1"200 179 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.091
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /bundles/ebdca7741674eca4e1fadeca157f3ae6.svg HTTP/1.1"304 0 "http://elk.ming.com/bundles/commons.style.css""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.193
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /api/security/v1/me HTTP/1.1"200 2 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.175
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /ui/fonts/open_sans/open_sans_v15_latin_regular.woff2 HTTP/1.1"304 0 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.224
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:01 +0800] "GET /api/xpack/v1/info HTTP/1.1"200 679 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.079
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:02 +0800] "GET /ui/fonts/open_sans/open_sans_v15_latin_700.woff2 HTTP/1.1"200 14720 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.157
elk.ming.com 192.168.162.1 - - [21/Mar/2019:02:12:02 +0800] "GET /ui/fonts/open_sans/open_sans_v15_latin_600.woff2 HTTP/1.1"200 14544 "http://elk.ming.com/app/kibana""Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0" "192.168.162.130:5601" 0.161
十一、使用beats采集日志
https://www.elastic.co/cn/products/beats
filebeat metricbeat packetbeat winlogbeat auditbeat heartbeat
可扩展，支持自定义构建
在128上执行
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-x86_64.rpm
rpm -ivh filebeat-6.0.0-x86_64.rpm
首先编辑配置文件
vim /etc/filebeat/filebeat.yml //增加或者更改
filebeat.prospectors:
- type: log
paths:
- /var/log/messages
output.console:
enable: true
/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml //可以在屏幕上看到对应的日志信息
再编辑配置文件
vim /etc/filebeat/filebeat.yml //增加或者更改
filebeat.prospectors:
- input_type: log
paths:
- /var/log/elasticsearch/minglinux.log
output.elasticsearch:
hosts: ["192.168.128.130:9200"]
systemctl start filebeat
[root@minglinux-03 ~] wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-x86_64.rpm
[root@minglinux-03 ~] rpm -ivh filebeat-6.0.0-x86_64.rpm
准备中... ################################# [100%]
正在升级/安装...
1:filebeat-6.0.0-1 ################################# [100%]
#修改配置文件
[root@minglinux-03 ~] vim /etc/filebeat/filebeat.yml
#修改或新增如下内容
15 filebeat.prospectors:
21 - type: log
24 # enabled: false
27 paths:
28 - /var/log/messages
141 output.console:
142 enable: true
143 #-------------------------- Elasticsearch output ------------------------------
144 #output.elasticsearch:
#远程登录
[root@minglinux-02 ~] ssh minglinux-03
root@minglinux-03's password:
Last login: Thu Mar 21 02:30:37 2019 from minglinux-02
[root@minglinux-03 ~] 登出
Connection to minglinux-03 closed.
#屏幕可以显示对应的日志信息
^C[root@minglinux-03 ~] /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml
···
···
{"@timestamp":"2019-03-20T18:35:59.219Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.0.0"},"source":"/var/log/messages","offset":13019,"message":"Mar 21 02:35:56 minglinux-03 systemd: Starting Session 73 of user root.","prospector":{"type":"log"},"beat":{"version":"6.0.0","name":"minglinux-03","hostname":"minglinux-03"}}
{"@timestamp":"2019-03-20T18:35:59.219Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.0.0"},"prospector":{"type":"log"},"beat":{"name":"minglinux-03","hostname":"minglinux-03","version":"6.0.0"},"source":"/var/log/messages","offset":13084,"message":"Mar 21 02:35:57 minglinux-03 systemd-logind: Removed session 73."}
#再编辑配置文件
#增加或者更改如下内容
[root@minglinux-03 ~] vim /etc/filebeat/filebeat.yml
···
27 paths:
28 - /var/log/elasticsearch/minglinux.log #es的日志
141 #output.console:
142 # enable: true
144 output.elasticsearch:
145 # Array of hosts to connect to.
146 hosts: ["192.168.162.130:9200"]
···
#启动filebeat
[root@minglinux-03 ~] systemctl start filebeat
[root@minglinux-03 ~] ps aux |grep filebeat
root 21859 12.5 1.2 287348 12112 ? Ssl 02:50 0:00 /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
root 21867 0.0 0.0 112720 984 pts/0 S+ 02:50 0:00 grep --color=auto filebeat
#查看索引
[root@minglinux-01 ~] curl '192.168.162.130:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana_1 IpJkISvGSBecrHJR5dNMtA 1 1 5 0 52.8kb 26.4kb
green open system-syslog-2019.03 cWR1XimDQE6PkexAIHCYkA 5 1 16469 0 7mb 3.5mb
green open filebeat-6.0.0-2019.03.21 WKCdeKzARK2npJwjkgHFtA 3 1 9 0 48.6kb 21.6kb
green open nginx-test-2019.03.20 hVWCcygQRvSfmE54H_vyIg 5 1 16403 0 6.5mb 3.5mb
-
到kibana里去配置新索引
-
日志页面
十二、扩展部分
x-pack 收费，免费 http://www.jianshu.com/p/a49d93212eca
https://www.elastic.co/subscriptions
Elastic stack演进 http://70data.net/1505.html
基于kafka和elasticsearch，linkedin构建实时日志分析系统 http://t.cn/RYffDoE
使用redis http://blog.lishiming.net/?p=463
ELK+Filebeat+Kafka+ZooKeeper 构建海量日志分析平台 https://www.cnblogs.com/delgyd/p/elk.html
http://www.jianshu.com/p/d65aed756587
