# Service Log installation environment
###### tags: `servicelog`

## Installation Environment
1. install open-jdk(1.8 above)
https://openjdk.java.net/install/
2. install nifi
https://nifi.apache.org/docs/nifi-docs/html/getting-started.html#downloading-and-installing-nifi
3. install docker(19.03 above)
https://docs.docker.com/install/linux/docker-ce/ubuntu/
4. install nvidia driver
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#package-manager-installation
5. install nvidia-docker
https://github.com/NVIDIA/nvidia-docker
- restart docker
$ sudo service docker restart
6. docker execute without sudo
https://docs.docker.com/install/linux/linux-postinstall/

## Docker 指令
- docker inference 指令
docker run --gpus '"device=1"' -i --rm --name servicelog_predict -v "$PWD":/usr/src/myapp -w /usr/src/myapp model-gpu python servicelog_predict.py

- docker training bert model 指令
docker run --gpus '"device=1"' -i --rm --name servicelog_training -v "$PWD":/usr/src/myapp -w /usr/src/myapp model-gpu python servicelog_bertv2.py

## NiFi位置
http://10.109.6.11:8080/nifi/

## Service Log Ftp
- sftp://10.109.6.11
- user:serviceloguser
- password:@WSX3edc
- folder: /mnt/hdd/Service_Log

## Develop Info
- inference code: /home/servicelog/AA_dev/ServiceLog/servicelog_predict_summary_tf2.1_toDB.py

## Deployment Info

- inference code: /mnt/hdd/Service_Log/project/servicelog_predict_summary_tf2.1_toDB.py

- inference input: project/input/
- inference output: project/output/

- training input: project/tr_input/
- training output: project/tr_output/

- backup folder:
user: servicelog
password: Pass1105
1. log: /mnt/hdd/Service_Log_Installation/ftp_log
2. output:/mnt/hdd/Service_Log/project/backup_ftp_output
3. origin and error:/mnt/hdd/Service_Log_Installation/backup_ftp_data

## Database
connection: mysql://root:123456@10.109.6.148:3306
database: servicelog
table: file_info

## Install mysql package
apt-get install libmysqlclient-dev （ubuntu）
yum install mysql-devel (centos)

## Folder structure
├── fail_file
│   └── taipei_20190302_153432.txt
├── project
│   ├── input
│   │   └── taipei_20190302_153444.xlsx
│   ├── output
│   │   └── Calllog_Output_taipei_20190302_153444.xlsx
└── success_file
└── taipei_20190302_153434.xlsx

## NiFi set service
/etc/systemd/system$ sudo vi nifi.service

[Unit]
Description=Apache NiFi
After=network.target

[Service]
Type=forking
User=root
Group=root
ExecStart=/mnt/hdd/Service_Log_Installation/nifi-1.10.0/bin/nifi.sh start
ExecStop=/mnt/hdd/Service_Log_Installation/nifi-1.10.0/bin/nifi.sh stop
ExecRestart=/mnt/hdd/Service_Log_Installation/nifi-1.10.0/bin/nifi.sh restart

[Install]
WantedBy=multi-user.target

$ sudo systemctl daemon-reload
$ sudo systemctl start nifi.service
$ sudo systemctl status nifi.service
$ sudo systemctl enable nifi.service

## 外網

- 防火牆開通資訊

- Source IP: 10.106.9.10, 10.109.6.11, 10.109.6.12, 10.109.6.13, 10.109.6.14
- Destination IP: Any
- Destination Port: 80, 443
- URL Domain:
archive.canonical.com
beacons.gcp.gvt2.com
cdn.fwupd.org
clients4.google.com
clientservices.googleapis.com
connectivity-check.ubuntu.com
dashboard.snapcraft.io
dell.archive.canonical.com
dl.google.com
download.docker.com
extensions.gnome.org
files.pythonhosted.org
motd.ubuntu.com
nvidia.github.io
oem.archive.canonical.com
pypi.org
security.ubuntu.com
ssl.gstatic.com
tw.archive.ubuntu.com
update.googleapis.com
apt.kubernetes.io
raw.githubusercontent.com