1. Login to LB cluster

Either for regular users of LB or the one using temporary usernames and passwords, please try to login to LB cluster using the correct method.

Once login to LB, we are now at the path /leonardo/home/userexternal/<YOUR-USER-ACCOUNT>.

2. Create a working directory and clone files from Github

Create a working directory at your login node.

mkdir multiGPU_AI_TTT_course_DL
cd multiGPU_AI_TTT_course_DL/

You can directly get files from Github repository via running the following commands:

git clone https://github.com/ENCCS/deep-learning-mini.git
cp -fr deep-learning-mini/jupyter-notebook-for-castiel2-course/* .

or you can copy files from the directory for this course with the following commands:

cp -r /leonardo_work/tra25_castiel2/COURSE_MATERIAL/DAY1/afternoon/* .

After running the commands listed above, you will have the following files/folder in your working directory (here it is multiGPU_AI_TTT_course_DL)

• 0_DL_job.sh
• 1_DL_test.py
• 2-Classification-NN-Keras-PenguinsClassification.ipynb
• 3-Monitor-training-process-WeatherPrediction.ipynb
• 4-Advanced-layer-types-ImageClassification.ipynb
• penguins_dataset.csv
• weather_prediction_dataset.csv
• image_classification (this is a folder with four npy files)

Submit job script sbatch 0_DL_job.sh to ask for computational resources and also test the programming environment.

• check the status of the job via command squeue --me
• once working, you will see the name of the compute node, like lrdn1234

3. Generating ssh keys for LB cluster

Method 1:

• Follow instructions HERE to get access to LB compute node from local computer.

Method 2:

• Using 2FA: smallstep to generate ssh key for LB
  • step ssh certificate <useremail> --provisioner cineca-hpc <mykey>
  • you will be asked to create a password for mykey
  • after running this command, you will get three files
    • mykey
    • mykey-cert.pub
    • mykey.pub
• copy three .pub files above to the login node of LB.
  • scp mykey* <YOUR-USER-NAME>@login01-ext.leonardo.cineca.it:~/.ssh
• login to login node again and go to .ssh directory
  • set up privilege for .ssh folder and mykey* files
  • use mykey for authorizing login for compute nodes

cd .ssh
cat mykey*.pub > authorized_keys
chmod go-rwx authorized_keys

4. Set up 2FA access for both login and compute node

Following Step 2, if your job is allocated, you can similar results if you run squeue --me

      JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
   idxxxxxx boost_usr  jupyter username  R       0:44      1 lrdnxxxx

• At the login node, run the command ssh -i ../.ssh/mykey -L 9777:localhost:8888 -N -f lrdnxxxx
• Open a new terminal at your computer, and run the command to make a ssh port forwarding between your local computer and login node ssh -L 7777:localhost:9999 -N -f <YOUR-USER-NAME>@login01-ext.leonardo.cineca.it
• Access jupyter lab/notebook through browser on your local computer http://127.0.0.1:7777/lab?token=tokenxxxxxxxxxxxxxxxxxx, in which the token is available at the slurm-idxxxxxx.out file.

Always ask questions at the very bottom of this document, right above this.