EFK 指令 - HackMD

EFK 指令 = Fluentd 在 Linux 中的套件名稱為 td-agent 查看 td-agent 設定 ``` cd /etc/td-agent/ vi td-agent.conf <source> @type tail path /var/log/test1.log pos_file /var/log/td-agent/test1.log.pos tag td.test <parse> @type json </parse> </source> <match **> @type elasticsearch host localhost port 9200 index_name fluentd type_name log </match> ``` 修改完 td-agent.conf 後須重啟 td-agent ``` systemctl restart td-agent ``` 查看log ``` cd /var/log ``` 啟動 Elasticsearch ``` systemctl start elasticsearch ``` 啟動 td-agent ``` systemctl start td-agent ``` 在 Elasticsearch 上建新的 index ``` curl -XPUT http://localhost:9200/[index名稱]/ ``` 在 Elasticsearch 上刪除 index ``` curl -XDELETE 'localhost:9200/[index名稱]' ``` 查看特定 index ``` curl -XGET 'http://localhost:9200/[index名稱]/_search' ``` 查看目前所有的 indices: ``` curl http://localhost:9200/_cat/indices ``` 查看單一 index 狀態 (size預設顯示10個)： ``` curl http://localhost:9200/[index名稱]?[size=100]&pretty ``` 若要餵資料進 elasticsearch，有以下幾種： ``` 1. curl -XPOST -H "Content-Type: application/json" 'http://localhost:9200/customer/external/2?pretty' -d '[JSON語法]' 2. curl -XPOST -H "Content-Type: application/json" 'http://localhost:9200/customer/external/1?pretty' --data-binary @[內含JSON語法的檔案路徑] ``` 網卡設定檔路徑(可從該路徑去把gateway的設定刪除讓vm連不到外網) ``` /etc/sysconfig/network-scripts/ifcfg-eth0 #重啟network systemctl restart network #測試連線至Google ping 8.8.8.8 ``` 修改IP指令(下方將網卡名稱為eth0的IP修改為192.168.8.101/24) ``` $ nmcli connection modify eth0 ipv4.addresses 192.168.8.101/24 ipv4.dns 192.168.8.1 ipv4.method manual $ nmcli connection up eth0 $ nmcli connection modify eth0 ipv4.addresses 10.240.131.101/24 ipv4.method manual $ nmcli connection up eth0 ``` 清除terminal指令 ``` control + u ``` 查看td-agent版本 ``` https://docs.treasuredata.com/articles/td-and-td-agent-deprecation-201510 ``` 把檔案從本機丟到遠端主機指令(要在本機端下指令) ``` scp [FileName] [User]@[遠端IP]:[Path] ``` 相依關係失敗 ![](https://i.imgur.com/D4mUxT2.png) 解決方式： 1.到/etc/yum.repos.d/創一個repo檔 (e.g., rheldvd.repo) 2.裡面寫入 (主要是調整baseurl，寫到mnt那層即可) ``` [local] name=Red Hat Enterprise Linux $releasever - $basearch - Source baseurl=file:///mnt/ enabled=1 gpgcheck=0 gpgkey=file:///opt/data/RPM-GPG-KEY-redhat-release ``` 刪除td-agent ``` $ sudo yum remove td-agent ``` cluster td-agent.conf ``` <source> @type tail format json read_lines_limit 1000 path /tmp1/*/*.log pos_file /tmp/pos_file/all_project.log.pos read_from_head true tag UP0125_ai_all_project </source> <match UP0125_ai*> @type copy <store> @type elasticsearch logstash_format true logstash_prefix UP0125_ai logstash_prefix separator logstash_dateformat %Y-%m-%d hosts 192.168.8.141:9200,192.168.8.142:9200,192.168.8.143:9200 scheme https ssl_verify yes flush_interval 5s id_key tail_hash write_operation create buffer_type memory retry_limit 5 ssl_version TLSv1_2 user UP0125_FUser password UP0125123 time_key timestamp time_key_format %Y-%m-%dT%H:%M:%S.%N type_name _doc </store> <store> @type elasticsearch logstash_format true logstash_prefix UP0125_ai logstash_prefix separator logstash_dateformat %Y-%m-%d hosts 192.168.8.131:9200,192.168.8.132:9200,192.168.8.133:9200 scheme https ssl_verify yes flush_interval 5s id_key tail_hash write_operation create buffer_type memory retry_limit 5 ssl_version TLSv1_2 user UP0125_FUser password UP0125123 time_key timestamp time_key_format %Y-%m-%dT%H:%M:%S.%N type_name _doc </store> </match> ``` 放ssh-key的位置滿了的解決方法： ``` $rm ~/.ssh/known_hosts ``` 在command中找尋套件： ``` find / -iname elasticsearch-certutil ```