Monitor system
===
# Zabbix&Grafana-install
## PreInstall

apt install mysql-server mysql

1. Install Zabbix repository

# wget https://repo.zabbix.com/zabbix/5.2/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.2-1+ubuntu18.04_all.deb
# dpkg -i zabbix-release_5.2-1+ubuntu18.04_all.deb
# apt update

2. Install Zabbix server, frontend, agent

# apt install zabbix-server-mysql zabbix-frontend-php zabbix-apache-conf zabbix-agent

3. Create initial database

# mysql -uroot -p
password
mysql> create database zabbix character set utf8 collate utf8_bin;
mysql> create user zabbix@localhost identified by '1qaz@WSX';
mysql> grant all privileges on zabbix.* to zabbix@localhost;
mysql> quit;

* On Zabbix server host import initial schema and data. You will be prompted to enter your newly created password.

# zcat /usr/share/doc/zabbix-server-mysql*/create.sql.gz | mysql -uzabbix -p zabbix

4. Configure the database for Zabbix server

DBPassword=password

/etc/zabbix/zabbix_server.conf

5. Start Zabbix server and agent processes

# systemctl restart zabbix-server zabbix-agent apache2
# systemctl enable zabbix-server zabbix-agent apache2

6. Configure Zabbix frontend
* Connect to your newly installed Zabbix frontend: http://server_ip_or_name/zabbix

7. create monitor host
* Crete host
* Config
* Data templates

# Grafana
1. download Grafana

wget https://dl.grafana.com/oss/release/grafana_7.2.0_amd64.deb
sudo dpkg -i grafana_7.2.0_amd64.deb
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server

2. Install Grafana Plugin

wget https://grafana.com/api/plugins/alexanderzobnin-zabbix-app/versions/4.0.2/download

grafana-cli plugins install alexanderzobnin-zabbix-app

3. 連線 http://server_ip_server:3000/
* enable plugin
* 設定連線
* 帳密並update
* 加入新dashboard
* Grafana設定Varible
* 設定varible
* 呈現

# 維運
* zabbix unreachable poller processes more than 75 busy

tail -f /var/log/zabbix/zabbix-server.log

* out of memory問題，memory不足修改以下內容

### Option: VMwareCacheSize
# VMwareCacheSize=8M
### Option: CacheSize
CacheSize=1024M


/etc/zabbix/zabbix_server.conf

* polling 太多Out time主機 unreachable poller processes處於busy狀態
* 增加StartPoller量

### Option: StartPollers
# Number of pre-forked instances of pollers.
#
# Mandatory: no
# Range: 0-1000
# Default:
# StartPollers=50

* connection to database 'zabbix' failed: [1040] Too many connections
* 代表超過mysql最大連線數

[(mysql)]> show variables like 'max_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 151   |
+-----------------+-------+

* vim /etc/my.cnf

max_connections=1000

* systemctl restart mariadb.service

[(mysql)]> show variables like 'max_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 214   |
+-----------------+-------+

* vim /usr/lib/systemd/system/mariadb.service

LimitNOFILE=10000
LimitNPROC=10000

* 重啟sql服務

# systemctl --system daemon-reload
# systemctl restart mariadb.service

* 驗證是否1000

[(mysql)]> show variables like 'max_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 1000  |
+-----------------+-------+

* zabbix 其他情況：

| 參數 | 描述 |
| ----------------------- | ------------------------------------------------------------------ |
| CacheSize | 快取監控memory大小默認8M |
| CacheUpdateFrequency | zabbix 快取更新頻率 |
| DebugLevel | 基本資訊1 - 重要資訊2 - 錯誤資訊3 - 警告資訊4 - 調適5 |
| HistoryCacheSize | 歷史cache大小 HistoryCache=64M |
| HistoryIndexCacheSize | 快取一個item需要100bits history=監控數量(10^5)*100/1024/1024 =9.6M |
| HousekeepingFrequency | Zabbix 執行housekeeping 頻率 |
| StartPollersUnreachable | 沒連到主機polling數量預設20 |
| StartPollers | Polling真實數量 建議大小StartPollers=30 |
| StartDiscoverers | 自動發現subprocess數量 StartDiscovers =5 |
| StartTrappers | Trappers process instance數量 Trappers來自Zabbix發送者 |
| Timeout | agent、SNMP 設備或外部檢查的時長，Timeout=6 |
| TrendCacheSize | 快取大小共存memory大小 |