# Apach Nifi

### 基本的系統調整設定

Create user
$ useradd nifi -d /home/nifi -u 1009
Change security parameter ==> 和 too many open files error 有關
$ vi /etc/security/limits.conf  <-- append below  
nifi hard nofile 65536
nifi soft nproc 65536
nifi hard nproc 65536 
nifi soft nproc 65536
 
用 ulimit -a 檢查一下數字對不對

### 系統參數配置
#### 增加TCP socket可存取的port 範圍

# sysctl -w net.ipv4.ip_local_port_range="10000 65000"
# Set how long sockets stay in a TIMED_WAIT state when closed
# sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1"
# Linux 不讓 NiFi  swap,需要去改 '/etc/sysctl.conf' t
vm.swappiness = 0
# '/etc/fstab' file and for the partition(s) of interest add the 'noatime' option.
# Get package from below link.
# curl -O https://archive.apache.org/dist/nifi/1.12.1/nifi-1.12.1-bin.tar.gz

### NiFi 安裝

Nifi 有分成兩種 mode (1)standalone (2)cluster
差異在於需要安裝 zookeeper ，cluster node 之間事需要透過 zookeeper 溝通。

### 安裝的步驟簡述:
1. 安裝胞姊壓縮
2. Conf/ 內幾個重要的檔案設定
3. 產稱憑證
4. Zookeeper(standalone 可略過) 或 jps 安裝
5. Lib 或user 提供 .nar 補充
6. Web UI 介面功能性確認以及權限調整

### Nifi 的重要核心設定檔

* **bootstrap.conf** => jvm setting
* **authorizers.xml** => nifi main configrution
* **nifi.properties** => nifi main configrution
* **login_identity-providers.xml** => AD login setting & superuser
* **state-management.xml** => login about setting
* **authorizations.xml** => processor or component authorizations
* **users.xml** => user info
* **logback.xml** => log output & format

# cd /nifi
# tar xvfz /nifi/nifi-1.12.1-bin.tar.gz
# chown -R nifi:nifi nifi*


/nifi/nifif-1.12.1/bootstrap.conf
java.arg.7=-XX:ReservedCodeCacheSize=256m
java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m
java.arg.9=-XX:+UseCodeCacheFlushing
# 放置 JDBC library
# mkdir /nifi/nifi-1.12.1/jdbc
# cp /nifi/nifi_JDBC/* /nifi/nifi/jdbc

1. 先去編輯 /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

10.37.39.114 AVANIFIAPD04.wistron.com AVANIFIAPD04
10.37.32.115 AVANIFIAPD05.wistron.com AVANIFIAPD05
10.37.32.116 AVANIFIAPD06.wistron.com AVANIFIAPD06
1. 安裝 zookeeper 3.6.0
 改 conf/zoo.cfg
 在 data/ 下面產生 myid
o https://www.itread01.com/content/1549499436.html => 發現空格會導致啟動不起來
2. 下載 nifi -1.12.1
3. 在/nifi/nifi-1.12.1/ 創一個 key 目錄，在裡面產生憑證

[nifi@AVANIFIAPQ02 key]$ 
keytool -genkeypair -alias avanifiapq02.wistron.com -keyalg RSA -keystore /nifi/nifi-1.12.1/key/keystore.jks -keysize 2048 -dname "CN=avanifiapq02.wistron.com,OU=NIFI" -ext san=dns:avanifiapq02.wistron.com -storepass zaqwedcxs
Enter key password for <dxnifiapq02.wistron.com>
        (RETURN if same as keystore password): zaqwedcxs
Re-enter new password: zaqwedcxs

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /nifi/nifi-1.12.1/key/keystore.jks -destkeystore /nifi/nifi-1.12.1/key/keystore.jks -deststoretype pkcs12".


[nifi@AVANIFIAPQ02 key]$  
keytool -certreq -alias avanifiapq02.wistron.com -keystore /nifi/nifi-1.12.1/key/keystore.jks -file /nifi/nifi-1.12.1/key/avanifiapq02.wistron.com.csr -ext san=dns:avanifiapq02.wistron.com -storepass zaqwedcxs

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /nifi/nifi-1.12.1/key/keystore.jks -destkeystore /nifi/nifi-1.12.1/key/keystore.jks -deststoretype pkcs12".

[nifi@AVANIFIAPQ02 key]$ ls -al
total 8
drwxrwxr-x 2 nifi nifi 62 Jun 15 14:20 .
drwxrwxr-x 8 nifi nifi 124 Jun 15 13:51 ..
-rw-rw-r-- 1 nifi nifi 1067 Jun 15 14:20 avanifiapq02.wistron.com.csr
-rw-rw-r-- 1 nifi nifi 2182 Jun 15 14:18 keystore.jks


[nifi@AVANIFIAPQ02 key]$ cat avanifiapq02.wistron.com.csr

-----BEGIN NEW CERTIFICATE REQUEST----- (範例)
MIICzDCCAbQCAQAwMjENMAsGA1UECxMETklGSTEhMB8GA1UEAxMYYXZhbmlmaWFw
*****-------------------------------------------------**********
*****-------------------------------------------------**********
*****-------------------------------------------------**********
e/hPn3ezaZR8KEX8Qyz0n9fs5NLPNLFa5iV1RmVP+uFM99anmPM5CXZSydpfG2V1
*****-------------------------------------------------**********
*****-------------------------------------------------**********
*****-------------------------------------------------**********
*****------------------------------------------------***********
*****-------------------------------------------------**********
XUgmKEy6rvjA97sNH6ofu3NYDh2dYn3X1dy7TvN6kwcMqUxDmx/E0AUa6Twou56c
-----END NEW CERTIFICATE REQUEST-----

cp /nifi/nifi-1.12.1/key/*jks /nifi/nifi-1.12.1/conf

1. /nifi/nifi-1.12.1/conf/ 下的幾個檔案要編輯
o nifi.properties
o authorizers.xml
o login-identity-providers.xml
2. 就可以啟動測試

________________________________________
## 產生 key
$ keytool -genkeypair -alias avanifiapq04.wistron.com -keyalg RSA -keystore /nifi/nifi-1.12.1/newkey/keystore.jks -keysize 2048 -dname "CN=avanifiapq04.wistron.com,OU=NIFI" -ext san=dns:avanifiapq04.wistron.com -storepass zaqwedcxs

## 產生憑證
$ keytool -certreq -alias avanifiapq04.wistron.com -keystore /nifi/nifi-1.12.1/newkey/keystore.jks -file /nifi/nifi-1.12.1/newkey/avanifiapq04.wistron.com.csr -ext san=dns:avanifiapq04.wistron.com -storepass zaqwedcxs

$ cat avanifiapq04.wistron.com.csr
# 去申請憑證 https://adca.wistron.com/certsrv/
Select “Wistron Client and Server Authentication
san=dns:avanifiapq04.wistron.com

##用 ftp 上傳後
root 切換
$ chown nifi:nifi /nifi/nifi-1.12.1/newkey/*jks

su - nifi
cd /nifi/nifi-1.12.1/newkey/

# 憑證改名
$ mv avanifiapq04.cer avanifiapq04.wistron.com.pem

$ mv rootc.cer rootca.pem

$ openssl x509 -in ./avanifiapq04.wistron.com.pem -noout -text
檢查憑證時效

# 做 keystore
$ keytool -keystore /nifi/nifi-1.12.1/newkey/truststore.jks -alias rootca -import -file /nifi/nifi-1.12.1/newkey/rootca.pem -storepass zaqwedcxs

$ keytool -keystore /nifi/nifi-1.12.1/newkey/keystore.jks -alias rootca -import -file /nifi/nifi-1.12.1/newkey/rootca.pem -storepass zaqwedcxs

$ keytool -import -alias avanifiapq02.wistron.com -file /nifi/nifi-1.12.1/newkey/avanifiapq04.wistron.com.pem -keystore /nifi/nifi-1.12.1/newkey/keystore.jks -storepass zaqwedcxs

#### copy 一份 jks 去 /nifi/nifi-1.12.1/conf
$ cp /nifi/nifi-1.12.1/newkey/*jks /nifi/nifi-1.12.1/conf

#### 啟動 zookeeper

#### 啟動 nifi
$ sh /nifi/nifi-1.12.1/bin/nifi.sh start
$ tail -f /nifi/nifi-1.12.1/logs/nifi-app.log

其他維運管理常見需求
## 權限調整

## update lib
把 .nar 放到 lib => AP team 會有一些改良的應用 lib

## smtp & domain-name 申請 => ITSR 設定

## mount nifi-file空間 => /etc/fstab 設定配置

## 設 housekeeping disk 空間清理

# site-to-site 設定