用ELK去分析與儲存log紀錄吧! Use ELK Stack to log

Preface

Before, we installed and configured the rsyslog , MySQL and LogAnalyzer.
Now we use anothor tools to help us collect logs , and get better statistics.

Objective

Deploy the ELK and get statistics to analyze logs.

System environvent

Transport iptables log file to log server
Same environment in this note,and install the ELK in log server.

Let's Deploy

Install ELK

Install Java-OpenSDK

yum install java

Download ElasticSearch and extract it

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.0-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.3.0-linux-x86_64.tar.gz
cd elasticsearch
bin/elasticsearch  #run to install

modify config file from config/elasticsearch.yml

cluster.name: log-elasticsearch
network.host: $SERVER_IP
http.port: 9200
discovery.seed_hosts:["127.0.0.1","[::1]","[$SERVER_IP]"]

Download Kibana and extract it

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.3.0-linux-x86_64.tar.gz
tar -zxvf kibana-7.3.0-linux-x86_64.tar.gz
cd kibana/config
vim kibana.yml

modify config file from config/kibana.yml

server.port: 5601
server.host: $SERVER_NAME
elasticsearch.hosts: ["http://$elasticsearch_SERVER_IP:9200"]

Download Logstash and extract it

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.3.0.tar.gz
tar -zxvf logstash-7.3.0.tar.gz

Transport method

Transport rsyslog log files through LogStash

Logstash config file

imput{
    syslog{
        port => "514" 
    }
}
output{
    elasticsearch{hosts => ["$Elasticsearch_SERVER:9200"]}
    stdout{}
}

start up logstash

bin/logstash -f config/syslog.conf

Transport log files through Filebeat

Download filebeat plugin

https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.3.0-linux-x86_64.tar.gz
tar -zxvf filebeat-7.3.0-linux-x86_64.tar.gz

Modify filebeat config from ./filebeat.yml

#=====Filebeat inputs=====
filebeat.inputs:
    -type: log
    enabled: true
    paths:
        -/var/log/*.log
        
#=====Kibana=====
setup.kibana:
    host: "192.168.0.250:5601"

#=====Outputs=====

#-----ElasticSearch output-----
output.elasticsearch:
    hosts: ["$elasticsearch_SERVER_IP:9200"]

Browse kibana/discover then can show log in screen.

Some Errors

Insufficient space for shared memory file

clean the disk.

df -h
du -h -x --max-depth=1
ps aux
kill

Create Kibana index pattern forbidden

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

Screenshot

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

log from rsyslog client through logstash