Fluentd === ###### tags: `QCT` `Data Center` ## Install td-agent ``` sudo curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent2.sh | sh td-agent -c CONFIG_PATH -v ``` ## Elasticsearch Output Plugin * Install Elasticsearch ``` docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.3.0 ``` * Install td-agent Elasticsearch plugin ``` sudo td-agent-gem install elasticsearch ``` ``` # /etc/td-agent/td-agent.conf <match {bmc,system.**}> @type elasticsearch host 10.103.3.89 port 9200 index_name nfvd include_timestamp true </match> ``` * REST APIs * List all indices ``` curl http://10.103.3.89:9200/_cat/indices?v ``` * Delete index ``` curl -X DELETE http://10.103.3.89:9200/fluentd | jq ``` * Check logs ``` curl http://10.103.3.89:9200/fluentd/_search | jq ``` ``` curl http://10.103.3.89:9200/fluentd/_search?q=FIELD_VALUE | jq ``` ``` curl http://10.103.11.100:9200/nfvd/_search?q=SerialNo:QTFCR2725007B | jq ``` * Data format ``` 2019-08-21T11:34:25+08:00 system.loadavg {"key1":"0.00","key2":"0.00","key3":"0.00"} ``` :::info 每筆紀錄要有時間欄位，且格式```2019-08-21T11:34:25+08:00```要一樣。 ![](https://i.imgur.com/GyDlxic.png) ::: ## Syslog Input Plugin * **in_syslog** is included in Fluentd's core. ``` # /etc/td-agent/td-agent.conf <source> @type syslog port 5140 bind 0.0.0.0 tag log.system protocol_type tcp </source> ``` * Rsyslog client ``` # /etc/rsyslog.conf should include $IncludeConfig /etc/rsyslog.d/*.conf ``` ``` # /etc/rsyslog.d/00-td-agent.conf should include *.* @@10.103.3.84:5140 ``` ``` sudo service rsyslog restart ``` :::info To forward messages to another host via UDP, prepend the hostname with the at sign (```@```). To forward it via plain tcp, prepend two at signs (```@@```). To forward via RELP, prepend the string ```:omrelp:``` in front of the hostname. ::: * Check log ``` logger test tail -f /var/log/syslog ```