# Jetson Nano 筆記
:::spoiler 目錄
[TOC]
:::
## Intro
* 安裝步驟參考：[Getting Started with Jetson Nano Developer Kit](https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit#write)、[Nvidia Jetson Nano 使用心得](https://hackmd.io/@0p3Xnj8xQ66lEl0EHA_2RQ/Skoa_phvB)、[環境安裝、最詳細燒錄教學](https://d246810g2000.medium.com/nvidia-jetson-nano-for-jetpack-4-4-01-%E7%92%B0%E5%A2%83%E5%AE%89%E8%A3%9D-fd48d5658a13)
## Version
## 安裝
* [SD卡燒錄軟體BalenaEtcher](https://www.balena.io/etcher)
* [Nvidia Jetson Nano SD Image](https://developer.nvidia.com/jetson-nano-sd-card-image)
* [PuTTY SSH連線軟體](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html)
1. 需透過micro USB port + putty Serial Software 連結 bard rate=9600
2. 連線成功後進入初始化設定：語言、地點、編碼、UTC
3. user nmae 可填user 密碼userpassword
4. 繼續設定APP Partition Size(預設)、系統分區(預設)、網路(eth0有線)、Hostname(Jetson-Nano)、電源模式：(MAXN)/5W
5. 完成-自動段開連線
6. 使用SSH連線到Jetson Nano(同一區網)進行基本設定

## 基本設定 _{(全部設定需要約3小時)}
### 更新系統
* 切換為superuser：`sudo -s`

    sudo apt-get update
    sudo apt-get full-upgrade -y

* 中間若遇到`nvidia-tegra.conf (Y/I/N/O/D/Z) [default=N] ?` 請選擇`N`
* 自動重啟 docker(YES)
* 重新安裝 Vim

    sudo apt-get remove vim-common -y
    sudo apt-get install vim -y
    

* 安裝Nano：`sudo apt-get install nano`

### 設定swap
#### 手動設定
設定4G的空間作為swap，避免記憶體不足，流程：
1. 停止swap, 第一次設定會有錯誤訊息, 因為不存在
2. 新增一個空白4G檔案
3. 該檔案設定為swap
4. 啟用並設定開機自動執行

sudo swapoff /swapfile
sudo dd if=/dev/zero of=/swapfile bs=1M count=4096
sudo mkswap /swapfile
sudo swapon /swapfile

sudo cp /etc/fstab /etc/fstab.bak
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

#### 腳本自動設定

git clone https://github.com/JetsonHacksNano/installSwapfile
./installSwapfile/installSwapfile.sh

sudo reboot

### 檢查CUDA
Jetson-nano已經內建CUDA, 修改環境變數確保可以使用CUDA

sudo cat >> ~/.bashrc <<EOF

# Add Cuda to path
export CUDA_HOME=/usr/local/cuda/
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:\$LD_LIBRARY_PATH
export PATH=\${CUDA_HOME}bin:$PATH
EOF

source ~/.bashrc

* Check: `nvcc -V`

### 設定自動風扇
`sudo sh -c 'echo 255 > /sys/devices/pwm-fan/target_pwm'`
`sudo sh -c 'echo 0 > /sys/devices/pwm-fan/target_pwm'`
* 風扇控制腳本
:::spoiler 展開程式碼

    sudo cat > /usr/local/bin/fan-control.sh <<EOF
    #!/bin/bash

    # Check parameters
    if [[ "\$1" != "start" && "\$1" != "stop" ]]; then
        echo "Usage: \$0 start|stop"
        exit 1
    fi

    # Constants
    BASE_SPEED=50
    NML_TEMP=30
    PIDFILE=/var/run/fan-control.pid

    # Function to set fan speed based on CPU temperature
    set_fan_speed() {
        CPU_TEMP=\$(cat /sys/class/thermal/thermal_zone0/temp)
        CPU_TEMP=\$((CPU_TEMP/1000))
        FAN_SPEED=\$BASE_SPEED

        if [ \$CPU_TEMP -gt \$NML_TEMP ]; then
            FAN_SPEED=\$((BASE_SPEED + (CPU_TEMP - NML_TEMP)*10))
        fi
        if [ \$FAN_SPEED -gt 255 ]; then
            FAN_SPEED=255
        fi
        echo \$FAN_SPEED > /sys/devices/pwm-fan/target_pwm
    }

    # Start fan-control process
    if [ "\$1" == "start" ]; then
        # Check if fan-control is already running
        if [[ -f "\$PIDFILE" ]]; then
            if ps -p \$(cat "\$PIDFILE") > /dev/null; then
                echo "Error: fan-control is already running."
                exit 1
            else
                # Remove PID file if it doesn't correspond to a running process
                rm "\$PIDFILE"
            fi
        fi

        # Create PID file
        echo "\$\$" > "\$PIDFILE"

        # Continuously set fan speed
        while true; do
            set_fan_speed
            sleep 10
        done
    fi

    # Stop fan-control process
    if [ "\$1" == "stop" ]; then
        echo 0 > /sys/devices/pwm-fan/target_pwm
        if [[ -f "\$PIDFILE" ]]; then
            PID=\$(cat "\$PIDFILE")
            if ps -p \$PID > /dev/null; then
                kill \$PID
            fi
            rm "\$PIDFILE"
        else
            echo "fan-control is not running."
        fi
    fi
    EOF

:::
* 檢查：`cat /usr/local/bin/fan-control.sh`
* 設定權限：`sudo chmod +x /usr/local/bin/fan-control.sh`
* 設定服務檔

    sudo cat > /etc/systemd/system/fan-control.service <<EOF
    [Unit]
    Description=Fan Control Automatically Service

    [Service]
    Type=simple
    ExecStart=/usr/local/bin/fan-control.sh start
    ExecStop=/usr/local/bin/fan-control.sh stop
    Restart=on-failure
    User=root

    [Install]
    WantedBy=multi-user.target
    EOF

* 檢查：`cat /etc/systemd/system/fan-control.service`
* 重新載入systemd：`sudo systemctl daemon-reload`
* 設定開機自動執行：`sudo systemctl enable fan-control`
* 啟動風扇控制：`sudo systemctl start fan-control`
* 關閉風扇控制：`sudo systemctl stop fan-control`
* 查看風扇服務狀態：`sudo systemctl status fan-control`
* 查看風扇服務Log：`sudo journalctl -u fan-control`
* 查看目前溫度(使用ctrl-c終止)

	while true; do
	cat /sys/class/thermal/thermal_zone0/temp
	sleep 1
	done

### CuDNN
* 測試CuDNN，執行結果最後有 "Test passed!"

sudo apt-get install -y libfreeimage3 libfreeimage-dev
cd /usr/src/cudnn_samples_v8/mnistCUDNN
sudo make
sudo chmod a+x mnistCUDNN
./mnistCUDNN 
cd

### JTOP
* 安裝 pip3 工具

    sudo apt-get install -y python3-pip python3-dev build-essential 
    sudo -H pip3 install --upgrade pip
    sudo -H pip3 install jetson-stats
    #pip3 install -U jetson-stats
    sudo systemctl restart jtop.service

* 執行Jtop監控工具，可查看CPU與GPU的相關資訊，也可將目前安裝的library顯示出來

    sudo jtop
    jetson_release

### OpenCV with GPU _{(需要約2小時)}
#### 腳本自動設定
* [參考資料](https://github.com/Qengineering/Install-OpenCV-Jetson-Nano)
* 指令

    wget https://github.com/Qengineering/Install-OpenCV-Jetson-Nano/raw/main/OpenCV-4-7-0.sh
    sudo chmod 755 ./OpenCV-4-7-0.sh
    ./OpenCV-4-7-0.sh
    # 使用了 nohup 命令來忽略掉 SIGHUP 訊號
    nohup ./OpenCV-4-7-0.sh >> ./OpenCV_install.log 2>&1 &
    # 查看進度
    sudo tail -f ./OpenCV_install.log
    jobs -l

#### 手動設定
* [安裝流程參考資料](https://chiachun0818.medium.com/jetson-nano-%E9%87%8D%E6%96%B0%E7%B7%A8%E8%AD%AF-opencv-cuda-a65daf5988ab)
1. 安裝相依套件
:::spoiler 展開程式碼

    # reveal the CUDA location
    sudo sh -c "echo '/usr/local/cuda/lib64' >> /etc/ld.so.conf.d/nvidia-tegra.conf" -y
    sudo ldconfig -y

    # third-party libraries
    sudo apt-get install build-essential cmake git unzip pkg-config -y
    sudo apt-get install libjpeg-dev libpng-dev libtiff-dev -y
    sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev -y
    sudo apt-get install libgtk2.0-dev libcanberra-gtk* -y
    sudo apt-get install python3-dev python3-numpy python3-pip -y
    sudo apt-get install libxvidcore-dev libx264-dev libgtk-3-dev -y
    sudo apt-get install libtbb2 libtbb-dev libdc1394–22-dev -y
    sudo apt-get install gstreamer1.0-tools libv4l-dev v4l-utils -y
    sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev -y
    sudo apt-get install libavresample-dev libvorbis-dev libxine2-dev -y
    sudo apt-get install libfaac-dev libmp3lame-dev libtheora-dev -y
    sudo apt-get install libopencore-amrnb-dev libopencore-amrwb-dev -y
    sudo apt-get install libopenblas-dev libatlas-base-dev libblas-dev -y
    sudo apt-get install liblapack-dev libeigen3-dev gfortran -y
    sudo apt-get install libhdf5-dev protobuf-compiler -y
    sudo apt-get install libprotobuf-dev libgoogle-glog-dev libgflags-dev -y

:::
2. 查看最新版本：https://opencv.org/releases/
3. 下載並解壓縮

    cd ~
    wget -O opencv.zip https://github.com/opencv/opencv/archive/4.7.0.zip
    wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.7.0.zip
    unzip opencv.zip && unzip opencv_contrib.zip

    mv opencv-4.7.0 opencv
    mv opencv_contrib-4.7.0 opencv_contrib
    # clean up the zip files
    rm opencv.zip && rm opencv_contrib.zip

    cd ~/opencv
    mkdir build && cd build

4. 設定CMake file
:::spoiler 展開程式碼

    cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
    -D EIGEN_INCLUDE_PATH=/usr/include/eigen3 \
    -D WITH_OPENCL=OFF \
    -D WITH_CUDA=ON \
    -D CUDA_ARCH_BIN=5.3 \
    -D CUDA_ARCH_PTX=”” \
    -D WITH_CUDNN=ON \
    -D WITH_CUBLAS=ON \
    -D ENABLE_FAST_MATH=ON \
    -D CUDA_FAST_MATH=ON \
    -D OPENCV_DNN_CUDA=ON \
    -D ENABLE_NEON=ON \
    -D WITH_QT=OFF \
    -D WITH_OPENMP=ON \
    -D WITH_OPENGL=ON \
    -D BUILD_TIFF=ON \
    -D WITH_FFMPEG=ON \
    -D WITH_GSTREAMER=ON \
    -D WITH_TBB=ON \
    -D BUILD_TBB=ON \
    -D BUILD_TESTS=OFF \
    -D WITH_EIGEN=ON \
    -D WITH_V4L=ON \
    -D WITH_LIBV4L=ON \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D INSTALL_C_EXAMPLES=OFF \
    -D INSTALL_PYTHON_EXAMPLES=OFF \
    -D BUILD_opencv_python3=TRUE \
    -D OPENCV_GENERATE_PKGCONFIG=ON \
    -D BUILD_EXAMPLES=OFF ..

:::
5. 編譯(可能需要90分鐘)：`make -j4`。註：`-j4`為使用4核心編譯
* 背景執行不輸出進度 `make -j4 > /dev/null 2>&1 &`、jobs
6. 刪除舊有的OpenCV

    sudo rm -r /usr/include/opencv4/opencv2
    sudo make install
    sudo ldconfig
    make clean
    sudo apt-get update

7. 清除OpenCV 來源包

    sudo rm -rf ~/opencv
    sudo rm -rf ~/opencv_contrib

8. 檢查是否成功：`jetson_release`

### 安裝FTP & Filezilla
* [參考連結](https://sites.google.com/site/raspberrypidiy/pc-to-rpi/vsftpd)
* 安裝：`sudo apt-get install -y vsftpd`
* 更新讀寫規則：`sudo vim /etc/vsftpd.conf`
* setup `listen=YES`
* setup `listen_ipv6=NO`
* remove "#" at line `local enable =yes`
* remove "#" at line `write enable =yes`
* 設定`root`帳號也能登入：`sudo vim /etc/ftpusers`
* add "#" at line `root`
* 啟動

    sudo service vsftpd restart

* PC端：[Filezilla下載](https://filezilla-project.org/download.php)

## 官方壓力測試(跑分) _{(跑分需要約1小時)}
[Benchmarks Targeted for Jetson (Using GPU+2DLA)](https://github.com/NVIDIA-AI-IOT/jetson_benchmarks)
* **務必啟用散熱風扇**(手動/自動)
* 下載Benchmark指令 Set up instructions

    cd ~
    git clone https://github.com/NVIDIA-AI-IOT/jetson_benchmarks.git
    cd jetson_benchmarks
    mkdir models

* 安裝相依套件：` sudo sh install_requirements.sh`
* Note: All libraries will be installed for `python3`
* 設定路徑

    export BENCHMARK_PATH=$(pwd)
    echo $BENCHMARK_PATH

* 下載 Models

    python3 utils/download_models.py --all --csv_file_path ${BENCHMARK_PATH}/benchmark_csv/tx2-nano-benchmarks.csv --save_dir ${BENCHMARK_PATH}/medels/

* Running All Benchmark Models at Once on Jetson Nano

    sudo python3 benchmark.py --all --csv_file_path ${BENCHMARK_PATH}/benchmark_csv/tx2-nano-benchmarks.csv \
                                --jetson_clocks \
                                --model_dir ${BENCHMARK_PATH}/medels/ \
                                --jetson_devkit nano \
                                --gpu_freq 921600000 --power_mode 0 --precision fp16

* 出現錯誤`No such file or directory: '/sys/devices/gpu.0/devfreq/57000000`[解決方式](https://forums.developer.nvidia.com/t/benchmarks-targeted-for-jetson-tx2/232954)：新增`--jetson_clocks`

## VNC遠端桌面
* 參考[官方教學](https://raw.githubusercontent.com/Aravindseenu/Nvidia-jetson-VNC-remote-access/master/VNC%20on%20NVIDIA)、[Set VNC on Jetson-Nano](https://raspberry-valley.azurewebsites.net/NVIDIA-Jetson-Nano/)、[錯誤解決](http://justamaker.blogspot.com/2019/05/jetson-nano-vnc-vinono-such-key-enabled.html)
* `sudo -s`切換到root
1. 安裝 Vino

    sudo apt-get update
    sudo apt-get install vino -y

2. 編輯設定檔：`sudo vim /usr/share/glib-2.0/schemas/org.gnome.Vino.gschema.xml`
* 新增

        <key name='enabled' type='b'>
          <summary>Enable remote access to the desktop</summary>
          <description>
            If true, allows remote access to the desktop via the RFB
            protocol. Users on remote machines may then connect to the
            desktop using a VNC viewer.
          </description>
          <default>false</default>
        </key>

* 編輯

    dbus-launch gsettings set org.gnome.Vino enabled true
    dbus-launch gsettings set org.gnome.Vino prompt-enabled false
    dbus-launch gsettings set org.gnome.Vino require-encryption false
    dbus-launch gsettings set org.gnome.Vino authentication-methods "['vnc']"
    dbus-launch gsettings set org.gnome.Vino vnc-password $(echo -n 'password'|base64)
    cGFzc3dvcmQ=

* 編譯設定：`sudo glib-compile-schemas /usr/share/glib-2.0/schemas`
* 檢查設定

    gsettings get org.gnome.Vino enabled
    gsettings get org.gnome.Vino prompt-enabled
    gsettings get org.gnome.Vino require-encryption
    gsettings get org.gnome.Vino authentication-methods
    gsettings get org.gnome.Vino vnc-password

3. 設定桌面自動登入
* [參考資料](https://askubuntu.com/questions/4474/enable-remote-vnc-from-the-commandline)
* 要將`AutomaticLogin`設定為user
* `sudo vim /etc/gdm3/custom.conf`

    [daemon]
    AutomaticLoginEnable=true
    AutomaticLogin=user

* 檢查 `cat /etc/gdm3/custom.conf`
4. 設定無頭預設桌面
* 參考資料：[xconfigoptions](https://download.nvidia.com/XFree86/Linux-x86_64/435.17/README/xconfigoptions.html)、[Headless jetson-nano](https://forums.developer.nvidia.com/t/headless-jetson-nano-capabilities/110304/2)
* `sudo vim /etc/X11/xorg.conf`

    # Allow X server to be started even if no display devices are connected.
        Option      "AllowEmptyInitialConfiguration" "true"

    # The desktop resolution depends on the attached display's capabilities. 
    # If no display is attached, the resolution defaults to 1280x800.

    Section "Screen"
        Identifier    "Default Screen"
        Monitor       "Configured Monitor"
        Device        "Tegra0"
        SubSection    "Display"
            Depth     24
            Virtual   1280 800
        EndSubSection
    EndSection

* 檢查：cat /etc/X11/xorg.conf
5. 啟用VNC (**此步驟不要使用`root`帳號**)
a. 開機自動執行
* [參考資料](https://blog.cavedu.com/2019/12/19/jetson-nano-remote-desktop-windows-mac-osx/)

    mkdir -p ~/.config/autostart
    cat > ~/.config/autostart/vino-server.desktop <<EOF
    [Desktop Entry]
    Type=Application
    Exec=/usr/lib/vino/vino-server  --display=:0
    Hidden=false
    NoDisplay=false
    X-GNOME-Autostart-enable=true
    Name[en_US]=Vino server startup
    Name=Vino server startup
    Comment[en_US]=Remote desktop
    Comment=Remote desktop
    EOF

* 檢查：`cat ~/.config/autostart/vino-server.desktop`
* 設定預設顯示器(之後就不用每次都`export DISPLAY=:0`)

    sudo cat >> ~/.bashrc <<EOF

    # Set default monitor
    export DISPLAY=:0
    EOF

    source ~/.bashrc

b. 手動啟動
* Check Monitor ([參考資料](https://unix.stackexchange.com/questions/17255/is-there-a-command-to-list-all-open-displays-on-a-machine))
`ps e | grep -Po " DISPLAY=[\.0-9A-Za-z:]* " | sort -u`
* 啟動
`export DISPLAY=:0`
`/usr/lib/vino/vino-server`
或
`/usr/lib/vino/vino-server --display=:0`
8. 其他
* 無頭模式下，不要使用GUI的設定去設定Display
* [Setting HDMI or DP Screen Resolution (Nvidia official)](https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3251/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/display_configuration.html)
* [Start a GUI software on a remote Linux PC via SSH](https://askubuntu.com/questions/47642/how-to-start-a-gui-software-on-a-remote-linux-pc-via-ssh)

        export DISPLAY=:0
        libreoffice --writer

## 其他
使用和不使用風扇測試 NVIDIA Jetson Nano 開發者套件
https://www.cnx-software.com/2019/12/09/testing-nvidia-jetson-nano-developer-kit-with-and-without-fan/

功能安裝大全
https://pyimagesearch.com/2020/03/25/how-to-configure-your-nvidia-jetson-nano-for-computer-vision-and-deep-learning/

## 模型訓練相關知識
[加速ML模型訓練的兩大方法(如何設定batch/檢查loss頻率)、batch size, iteration, epoch的概念和比較](https://ithelp.ithome.com.tw/articles/10219945)
[Epoch, Batch size, Iteration, Learning Rate](https://medium.com/%E4%BA%BA%E5%B7%A5%E6%99%BA%E6%85%A7-%E5%80%92%E5%BA%95%E6%9C%89%E5%A4%9A%E6%99%BA%E6%85%A7/epoch-batch-size-iteration-learning-rate-b62bf6334c49)

## 其他設定
[Jetson-Nano Setup](https://hackmd.io/GuRijZOFRk-2-1lwRKnUsw?view)
### 待整理

watchdog-device         = /dev/watchdog

# Uncomment and edit this line for hardware timeout values that differ
# from the default of one minute.

watchdog-timeout        = 60

max-load-1              = 24
max-load-5              = 18
max-load-15             = 12

min-memory              = 1

gsettings set org.gnome.desktop.background picture-uri file:///usr/share/backgrounds/NVIDIA_Wallpaper.jpg