# User Guide - NTU BDSRC HPC
:::info
- If you encounter any problem, please contact [e-mail]
- Change your password as soon as you log into the system.(Check out the User Setup section)
:::
## Overview
### What is a terminal?
Terminal is a text-based interface allowing us to interact with the operating system. We can perform any task without using the graphical interface, that colorful desktop you usually see on the screen.
Different OS provides diffrerent name for terminal, you can find it with the following names:
- **macOS**: terminal.app
- **windows**: PowerShell
- **Linux**: Terminal (right click to open if you are accessing through the desktop)

### How to Access the computing resource?
There are 7 servers you can access in total. You can access all the HPC (High Performance Computing) server via the IP address `140.112.176.245`. With this address, you will connect to `master` server by default. To access other servers, you need to specify which port you want to connect to.
:::info
- **Local machine**: The computers you have on hand.
- **Remote machine**: The servers you access via the Internet.
:::

## SSH Connection
SSH is a remote access protocal that allows you to connect any remote decive with terminal. The problem with this method is that it does not provide graphical interface like Desktop in Windows.

# In your terminal: 
## SSH Syntax
ssh <username>@140.112.176.245 -p <port number>

## If the terminal keeps yielding some weird failing messages that doe not affect the connection
ssh -o LogLevel=ERROR <username>@140.112.176.245 -p <port number>

## access server master by default
ssh <username>@140.112.176.245

## access node01 (port 2022)
ssh <username>@140.112.176.245 -p 2022

### HPC SSH Port Number
| Machine | master | node01 | node02 | node03 | gpu01 | gpu02 | gpu03 |
|:-------:|:------:|:------:|:------:|:------:|:-----:|:-----:|:-----:|
| **Port** | - | 2022 | 2023 | 2024 | 2025 | 2026 | 2027 |

### Premium Server Port Number
**(Additional Application Required)**

| Machine | gpu04 |
|:--------:|:-----:|
| **Port** | 2030 |

To end the SSH connection, you can type `exit` or use `ctrl-D` or `ctrl-C`
:::info
You can connect to other server "from master" simply by typing down the server's name.

# Example
ssh gpu01

:::

## NoMachine Remote Desktop
### Installation
Please go to https://www.nomachine.com/download to download NoMachine for your local machine.
:::info
NoMachine app installed on your local machine is called *NoMachine Client*.
:::

### Using NoMachine
Open NoMachine app on your local machine, and click *Add*

Add connection:
- Name: call it whatever you like
- Host: 140.112.176.245
- Port: 4000 (for connecting master. Other servers require different ports)

### Ports available for NoMachine
| master | node03 | gpu01 | gpu02 |
| ------ | ------ | --- | ----- |
| 4000 | 4002 | 4001 | 4003 |
:::info
Servers not listed in the above table do not have remote desktop access.
:::

Log in with your account:

Create a virtual desktop:

Configure the resolution of the virtual desktop:

And then you are good to go:

:::info
To upload any file to the server via NoMachine, you can simply drag the desired files to NoMachine's window.
:::
### Configure the resolution or other settings
Click the upper-right corner:

Then you can configure the desktop:

:::warning
- Your desktop will be killed if you do not log in for 7 days.
- Each NoMachine server can accommodate a maximum of 10 sessions across all users simultaneously. If you cannot log in, Please use other servers or try again later.
:::
## User Setup
### Change Password
Use `passwd` command and enter the current password to change it.
:::danger
***Only change your password from "master"***
:::
### Some Utilities
There are several softwares you can launch on the terminal of the server.

# Launching MATLAB
matlab

# Stata17/IC 2GB memory limited
xstata

# Stata17/MP has more memory capacity
xstata-mp

# Stata in batch mode
nohup stata-mp -b do [do-filename] &

# Python 
python3 <filename>.py

# Conda
conda

# Julia (only available in gpu01 and gpu02)
julia

### Use Jupyter Notebook on Remote Machine
Step 1: Run Jupyter server

jupyter-notebook

Step 2: open this URL (often it's `http://locolhost:8888/tree`) using Google Chrome
### Use Remote Jupyter Notebook on Local Machine

# On "Remote" Server
jupyter notebook --no-browser --port=8888

# On "Local" machine
ssh -L 8888:localhost:8888 <username>@140.112.176.245 -p <port_number>

Then you can open the browser on your local machine, and type in the URL:`http://localhost:8888`, you will be able to edit the notebook from your locak machine.

### Virtual Environment
#### Conda
Conda is a package and environment manager that allows you to isolate the environments of different projects.

# initial conda for shell
conda init

# create environment
conda create --name <your project name>

# activate the environment
conda activate <your project name>

# Deactivate the current active conda environment
conda deactivate

#### Others
You can also use built-in `venv` or thrid-party package `virtualenv` to create virtual environments.
Please refer to https://python.land/virtual-environments/virtualenv

### File Access Control
:::danger
Your files might be visible to others. To prevent this, you need to set up access control. However, if your data isn't sensitive, you can choose to ignore this.
:::
3 types of Permission: `r`(read), `w`(write), `x`(execute)

#### Checking File Permissions

# Example
ls -l test.txt

# Output
-rw-rw-r-- 1 ryan ryan 0  九   2 14:58 test.txt

# or you can use `getfacl`
getfacl <filename>

For example a file's permission is `-rw-rw-r--`, you can interpret it as:
- Is it a directory: `-` (it will be `d` if it's a directory)
- Owner: `rw-` (you can read and write)
- Group: `rw-` (people in your group can read and write)
- Others: `r--` (people outside of your group can only read)

:::info
`rw-` can be translated into `110`, which is `6` in decimal value. So the permission of the example above can be represented as `660`.

If you want to set the permission as `rwxr-x---`, you can simply type:

chmod 750 <filename>

:::

#### Modify File Permission

# Syntax
chmod <setting> <target name>

# Example: add execute permission for yourself.
chmod u+x <filename>

# Example: Take away other's rwx permission  
chmod o-rwx <filename>

#### Modify Directory Permissoin

# Syntax
chmod -R <your setting> <dir name>

:::info
`u`: you
`o`: others
`g`: users in your group
`+`: add
`-`: remove
`=`: assign
:::

#### Group Control
You might want to share files and directories with your project members. For convenience, you can ask us to create a group for your project by submitting the application https://forms.gle/6Ro9YciS68czZi5E9

##### Group Basics

# show all the groups you are in
groups

# You might want to switch bewteen the groups 
# when you creare a new file
newgrp <group name>

# Type exit when finishing your operation for that group
exit

If you are the owner of a group, you can add/delete the group member at your own will:

# add user
gpasswd -a <username> <group name>

# delete user
gpasswd -d <username> <group name>

#### Further info on File Permission
- https://www.redhat.com/sysadmin/linux-file-permissions-explained
- https://www.geeksforgeeks.org/how-to-set-file-permissions-in-linux/

### Utilize GPU
For some package on python like Tensorflow, you can designate the GPU you want to run on.

# show all GPUs on this machine
nvidia-smi

# run on GPU 0
export CUDA_VISIBLE_DEVICES=0

# run on GPU 0, GPU 1 and GPU 2
export CUDA_VISIBLE_DEVICES=0,1,2

:::danger
- Do not add space in assigning the variable. It can cause trouble by typing:`=0, 1, 2`
- Export the variable everytime before you run the python script.
:::
## VS Code Remote Editing
You might not want to transfer your file to the remote server everytime you modify your codes in your local machine. Therefore, directly editing the files on the remote server through VS code can be super convenient.

1. Install extension *"Remote Development"*, *"Remote - SSH"* and "*Remote Explorer*"

2. Open a remote window

3. Connect to Host

4. Add new SSH host > Configure SSH Hosts

5. Choose which file you want to save the configuration(you can choose the first one that appears)

6. Edit the configuration

# Format
Host <name whatever you like>
    HostNmae 140.112.176.245
    User <your user name>
    Port <port number>


7. Establish Connection

## Rstudio
You can also connect to Rstudio directly using your local browser with the following URL:
• **master**: http://140.112.176.245:8787
• **node01**: http://140.112.176.245:8788
• **gpu03**: http://140.112.176.245:8789
• **gpu02**: http://140.112.176.245:8790

## Basic Linux Guide
https://www.geeksforgeeks.org/linux-tutorial/