# Planning HPC Python at HPC2N and UPPMAX

## Materials pages
- material: https://uppmax.github.io/HPC-python/index.html
- source: https://github.com/UPPMAX/HPC-python/tree/main/docs

## Links to Python-Julia-R

- planning: https://hackmd.io/iHDNS90TRY-HFT7WlxFqVg
- source: https://github.com/UPPMAX/R-python-julia-HPC/
- material: https://uppmax.github.io/R-python-julia-HPC/

## Autumn
- 2-days
### Dates
- On-boarding Tue Dec 3, 1-2pm
- Thur-Fri Dec 5-6, 9-16

## Content
- 2nd day: Pandas more parallel/big data

## Spring
* Wed 15 May
* onboarding 14 May at 1.15-2pm

## Next meeting May 29 13.00
- Evaluation
- big data
- point to other on-line material for specific science topics
- To discuss further
- jupyter
- extend parallelism and ML
- in application: ask knowledge of parallel, slurm, gpu, ML
- more course links: https://enccs.github.io/gpu-programming/6-language-support/
- rewrite course goal:
- Deploy hpc-resources for different problems
- not learning the solutions of the problems!

## Next meeting May 8 13:00
- Status
- number: 57?
- Rebecca is contacted for fall python/matlab training
- Meeting 20 May? Check.

- Discuss
- practicals
- people usually knows how to behave
- shorten code of conduct and similar?
- or just not mention it too much
- Hardware
- keep it in the same manner as 3-day course (update accordingly)
- on-boarding at 13.15-14.00

- TODOs
- Björn
- correct links to new Uppmax page
- Talk less!
- reservation
- pandas/matplotlib link lunarc course
-
-
## Next meeting May 2 10:00
- Status
- BC
- just started due to other duties (new web going live tomorrow)
- schedule update
- will work more from Monday next week
- BB
- pandas/plot in load/run
- lots of tests!

- Discuss
- perhaps ML module is working?
- tf check by Diana
- We keep 3.9.5 in Pedro's part
- Rebecca has a course about matplotlib etc..
- advertise
- ask to collab in fall

- ToDos
- our own parts
- BC
- clear measurable goals
-

## Next meeting April 26 11:00

* Postponed

## Next meeting April 10 13.00
- Short one
- Status
- what is needed?

- ToDos
- Exercise with pandas, matplotlib
- BB: Prepares .
- BC: implements in load/run session
- BB: Same exercise under batch, but now with the graphs saved to files
- BC: First scratch on goals/teaching outcome (with students' perspective)
- Change Python version to 3.11.8 in text and exercises.

#### Timings
|Session| Dec 2023 | Suggestions May |
|-------|-------|----------|
|9.00 "Syllabus"| 30 | 15 |
|9.15 Intro| 10 | 10 |
9.25 Load-run + system inst. packages| 35 | 30
9.55 **break**|15 |15
10.10 self-install. packages + isolated| 46 | 35
10:45 batch + arrays | 35 | 35
11.05 break during | 10 | 10
11.30 interactive + jupyter (together)| 15 (a little too short) | 25 |
12:00 Lunch

|Session| Dec23 | Suggestions 15-May-2024|
|-------|-------|----------|
13.00 parallel| 55| 60 |
14:00 break | | 10
14:10 GPU | 30 | 40 |
14.50 BREAK
15.05 ML (incl all exercises) | 23 | 30
15:35 Sum+eval| 18| 20
15:55 Q/A + extra exercise| 0 | 35
16:30 END-OF-DAY
**sum**| 150|

### Lecturer

|Session| Björn | Birgitte | Pedro|
|-------|-------|----------|------|
|"Syllabus"+intro| X | |
Load-run+packages| X
self-inst. isolated | | X |
batch| | X |
parallel| | | X
interactive| X
GPU| | | X
ML (incl exercises)| | X
Exercises (exercises)| several different rooms |x|x
Question session (breakout)| X | X | X
Summary| X | X | X

## Meeting 27 March
- HPC Python: May 15th: leave everything the same: Bi + Bj + Pe teach, Ri assists
- with updates of course from earlier learnings
- py/3.11 (GPU UPPMAX perhaps other solution)

### Meeting 15 feb
- ToDos
- [x] UPPMAX web page

## Autumn 2023

### Next Meeting 28 Nov
- Course:
- registered: 31
- On-boarding:
- common problems?
- Status:
- 3.9.5
- Björn
- Site-packages added to first lesson
- problem with mermaid in rst
- will start to check functioning on Rackham
- what is done already?
- install packages
- Björn checks Dask on Rackham
- Birgitte checks for the batch session and ML session

### Next meeting 21 Nov

- Status:
- spelling is active
- Björn
- updated index, intro, mermaid
- Birgitte
- updated index + Q/A
- one project for UPPMAX
- others use existing kebne-projects

- To discuss:
- recording
- during
- before
- after
- Do we fill these slots with 40 minutes?
- 13.00 parallel 55
- live coding
- exercise
- dask: chunks, notebook
- "read more": starPU
- break 10
- 14:05 GPU 40
- 14:45 15 min break
- 15:0 ML 35
- 15: summary & Q/A

- ToDos:
- Björn
- adds link check later (steals time)
- test packages and scripts
- dask at rackham
- project numbers
- Birgitte
- ML and packages
- extra exercise
- Pedro
- dask at kebnekaise
- Evaluation
- skip but inform more than usual in the end

### Next meeting 6 Nov 13:00
- Evaluation
- links to StarPU
- interesting with DASK chunks
- ToDo:s
- BC: Intro
- [ ] a little more hpc
- diagram
- mention purposes of the clusters
- [x] state why we involve two centres
- mention the editor as a tool for file creation and change
- more info
- exercise time
- objectives
- More to extra reading!
- boxes with extra tips

- For the workshop
- BO:s
- have voluntary exercise rooms
- silent room
- paste messages to BO:s in chat and/or document so one can re-read
- checking in with students more frequently about if they are done or not
- let students join the main room again when they are done
- No time to cover all material
- More to extra reading!
- More time on code-along and exercises

#### Spring courses
- 15-18 Jan UPPMAX intro
- 18 Jan HPC2N intro
- 22-23 Jan Awk
- 13 Feb Singularity
- 12-14 March Py-jl-R
- May (flexible day) HPC-Python
- May/June Programming formalisms

### Meeting 30 Oct 13:00

- Helpdesk 1 hour, 16-17
- Have in mind:
- do recordings before without students
- or afterwards!
- have more interaction during lecture
- no flipped-class room (this time)
- not show recordings during class time
- decide like 2 weeks before
- (zoom recording of oneself, for instance)
- ToDos: See last meeting

### Meeting 24 Oct 15.15
* Suggestions from Björn
* minimize packages sections
* [x] links and keypoints from October course
* although use venvs for course
* More focus on the HPC
* Discuss
* same timing: 9-16? **yes**
* need for on-boarding: **yes**
* recording: **Yes** but hoe
* more examples and exercises on how to use in HPC **Yes**
* DASK: **Yes**
* Are some packages specifically interesting for HPC? **Yes**
* Differences between ML packages, which are there, when are they used - with or without GPUs. CUDA. ML on CPU/GPU: **Yes**
* Inspiration from:
* https://enccs.github.io/hpda-python/
* https://aaltoscicomp.github.io/python-for-scicomp/parallel/

* ToDos
* embarassingly parallel?
* [x] Björn updates UPPMAX web

#### Timings
|Session| May23 | Suggestions Dec23 |
|-------|-------|----------|
|9.00 "Syllabus"| 10 | 10 |
|9.10 Intro| 10 | 10 |
9.20 Load-run + system inst. packages| 30 | 30
9.50 **break**|15 |15
10.05 self-install. packages + isolated| 35 | 25
10:30 batch + arrays | 25 | 35
|11.05 break| 10 | 10
11.15 interactive + jupyter (together)| 30 | 30 |
11.45 |Exercises and informal chat + help
Lunch

|Session| May23 | Suggestions Dec23 |
|-------|-------|----------|
13.00 parallel| 40| 40 |
13:40 GPU | 30 | 40 |
14:20 break | | 10
14:30 ML (incl all exercises) | 40 | 40
15:10 Extra exercise time | 15 | 20
Sum| |
Q/A | 20 |
**sum**| 150|

### Lecturer

|Session| Björn | Birgitte | Pedro|
|-------|-------|----------|------|
|"Syllabus"+intro| X | |
Load-run+packages| X
self-inst. isolated | | X |
batch| | X |
parallel| | | X
interactive| X
GPU| | | X
ML (incl exercises)| | X
Exercises (exercises)| several different rooms |x|x
Question session (breakout)| X | X | X
Summary| X | X | X

* Invitation

### Next meeting 11 Aug 11-12

- Content changes
- more HPC!!!
- more exercises!
- assume students attend prereqs
- how to do on other Naiss systems
- tabs
- or cheat sheets
- Newletter
- dates
- define prereqs
- modify content text

### Meeting 15 June 11:00
- Dates (Oct-Dec)
- py-R-jl
- Oct 17-19 (Tue-Thu)
- Git
- 13-17 Nov
- HPC-python
- Dec 5
- 1 day!!

- Courses to have in mind to avoid clashes
- Python for Scicomp
- Programming formalisms (Dec?)
- CR i Gothenburg (Nov/Dec?)
- Bianca 1 day only (x2)

- Changes
- more HPC!!!
- more exercises!
- assume students attend prereqs
- how to do on other Naiss systems
- tabs
- or cheat sheets

--------------------------------
## Spring 2023
### When?
- Tue 23 May 9-16

### Meeting 7 June 13:00
- Go through evaluation
- Word document (by Björn) in Slack
- Learnings
- Make most of morning session prereqs
- Py-R-jl course
- More time for type-along and exercises (no hurry)
- Keep silent breakout room all the time

### Meeting Mon 22 May 10:00
- To discuss
- status
- registered: 84 (3 no account yet)
- jupyter at UPPMAX rephrase
- björn moves preparation/practicals
- py-R-jl course in Oct
- HPC-python in Dec
- Birgitte tests ML extra exercise
- Birgitte tarballs of exercises

### Meeting Wed 17 May 9:30
- To discuss
- status

#### Changes of content (Björn's suggestions)

- Extra packages
- [x] setup.py

- isolated (only Birgitte, because conda is in packages)
- [x] Björn can change venv to virtualenv for UPPMAX?
- [x] rm setup.py
- extra isolated
- [x] link to course: make own packages

### Meeting Fri 12 May 13:15
- To discuss
- status
- topics
- schedule
- FAQS (or implement in material) from previous Q/A
- gpu on UPPMAX test
- GPU + ML exercises
- fix julia interaction on UPPMAX
- Björn Look
- reservations
- magnetic reservation GPU

#### Reservation
- tell in intro
- https://support.naiss.se/Ticket/Display.html?id=274512
- ``scontrol -M snowy show res naiss2023-22-500``

#### Topics
- "Syllabus"
- add: practicals here from Intro
- mv to extra:
- comments: wait with schedule until 17 May
- Intro
- add: paths to tarboll when avail
- mv to extra: sites and hardware
- comments:
- Load-run
- add: (updates of versions)
- mv to extra: nope
- comments:
- packages
- add: pip install exercise
- mv to extra:
- comments: keep because of goal of course
- isolated
- add:
- mv to extra: "self-installed..."
- comments: venv also at HPC2N, keep (goal of course )
- in extra: point to develop packages yourself links
- batch
- add:
- mv to extra:
- comments: In order! (minor fixes?)
- interactive
- add: jupyter speed up in thinlinc
- mv to extra: Jupyter
- (UPPMAX vs HPC2N tabs
- comments:

- parallel
- add:
- mv to extra: nope
- comments: (minor fixes?)
- GPU
- add: demo2 --> exercise
- mv to extra: nope
- comments: (checking)
- ML
- add:
- mv to extra:
- comments: merge ML

#### Timings
|Session| Feb23 | Suggestions May23 |
|-------|-------|----------|
|"Syllabus"| 15 | 10 |
|Intro| 10 | 10
Load-run| 15 | 15
packages| 20+15 conda | 35
isolated | 30 | 20
batch| 25 | 25
interactive| 15 | 20
**sum**| 145| 135 |
(add exercises)
Lunch??

|Session| Sep22 (in practice) | Suggestions May23 |
|-------|-------|----------|
parallel| 36| 40|
GPU|17 | 30 |
ML (incl all exercises)|21+30 | 40
Extra exercise time| |15
Sum| |5
Q/A ||20
**sum**| 104| 150

### Meeting Thu 4 May 11:00
- To discuss
- follow up
- issues

#### Follow up
- [x] Web pages
- [x] Invitation by Thu
- [x] Form by Thu
- [x] Make a branch for old course
- Björn
- [x] Remove site specific
- Björn
- [x] Course project
- Birgitte
- [x] Issues
- but otherwise work directly in main
- [x] Copy files from python feb23 course
- Björn
- [x] integrated shared document questions into *each* sessions
- [x] all teachers
- [x] in text
- [x] FAQ:s
- [x] Test the code on UPPMAX
- (there are updates from julia-python wrt gpu)
- ML
- [x] GPU
- integrate from Feb
- [ ] Extra reading
- [x] files (e.g. "packages_deeper")
- [ ] content
- [x] Timings (when first versions of updates are done)
- Björn

Q/A page from the R/Julia/Python course: https://umeauniversity.sharepoint.com/:w:/s/HPC2N630/EbHWglWYU_VNpTpdD2CtSfYBlpsAF6DyD_4RMwWCie_B0g?e=UddlVs

Q/A page from the Python in HPC course in Sept. 2022: https://hackmd.io/@bclaremar/HPC-python

### Meeting Mo 24 Apr 10:30
- NAISS
- course project UPPMAX
- if already account on HPC2n let's keep
- reservations
- snowy reservations for GPU (some hours)
- rackham for cpu runs for a period of time
- Learnings from last time(s)
- HPC python
- check evaluation
- Python-R-julia
- use updates
- check evaluation
- Teachers
- How?
- Still seperate sessions
- No!
- **tabs instead**
- record?
- Yes!
- Material
- still much packaging?
- a little bit less
- more time for hands-on
- packaging and isolated
- briefly go through
- put deeper material from python-julia-R as "optional" but included in RTD material
- Björn
- very briefly on Bianca but point to special bianca course
- ML on UPPMAX
- Diana/Björn test?
- intro
- NAISS and the different centres overview
- links to other NAISS python web pages
- beginning? or for each section?
- purpose
- how to load
- conda or not?

- Exercise rooms
- main
- silent
- hpc2n
- several help rooms

- Shared document
- sharepoint 365 (Umeå hosts)

### Lecturer

|Session| Björn | Birgitte | Pedro|
|-------|-------|----------|------|
|"Syllabus"+intro| | X| X
Load-run| X
packages| X
isolated | conda| pip
batch| | X
interactive| X
parallel| | | X
GPU| | | X
ML (incl exercises)| | X
Exercises (exercises)| several different rooms |x|x
Question session (breakout)| X | X | X
Summary| X | X | X

### Until next time
- [x] Web pages
- [x] Invitation by Thu
- [x] Form by Thu
- [ ] Timings
- first see what is left after moving things to extra material
- Björn
- [x] Make a branch for old course
- Björn
- [ ] Copy files from python feb23 course
- Björn
- [x] Issues
- but otherwise work directly in main
- [x] Course project
- Birgitte
- [ ] Test the code on UPPMAX
- (there are updates from julia-python wrt gpu)
- ML
- [ ] GPU
- integrate from Feb
-
### Invitation text
- [x] Thursday

**Online workshop: "Python in an HPC environment", *May 23*, 2023**

UPPMAX and HPC2n is organising a joint workshop on how to run Python and install additional Python packages on HPC resources and how to use the HPC capabilities. Participants will be able to bring their particular software request for discussion as well.

The instructors will use UPPMAX's systems for demos and there will be hands-on exercises for the participants.

**The following will be covered:**
* Loading the appropriate Python module
* Packages
* checking installed packages
* pip install
* Conda install
* Isolated environments
* HPC
* batch jobs with Python
* parallel computing
* GPU computing
* Machine Learning environment

Prerequisities:
- basic Linux
- basic Python
- basic Slurm

When: Tue, May 23, 2023.
Time: 9:15 - 12:00, 13.15-16.00
Where: online.

Course web pages

### Outline
#### Common
- Syllabus
- Intro (incl. very briefly about UPPMAX/HPC2N)
- terminal vs thinlinc (up to users)
- Load/Run
- Packages especially pip
- Isolated general
- Interactive general
- rm `-p`
- Batch general
- this one before interactive
- Parallel session 1
- UPPMAX: Conda, venv (exercises),
- Kebnekaise: virtual env (exercises)
##### LUNCH break
- Parallel computing general
- also exercises
- GPU general
- only demos
- snowy
- ML general
- some links to UPPMAX
- cpu exercise
- Parallel session 2
- UPPMAX: Bianca, jupyter
- HPC2N:
- gpu exercises at Kebne
- Machine Learning given by HPC2N

---
## Meeting 7 Sep
- status
- ideas
- todo:s
- the recording
- is editing afterwards necessary?
- primary speaker view
- settings in meeting before
- mute at entrance

### Status?
- Björn keeps filling in UPPMAX info and works with formatting
- in addition we need better merges in some sessions
- load-run
- isolated general
- interactive

### Ideas
- many small breakoutrooms
- many hosts (helpers)
- also ask for help in HackMD
- install numba, mpi4py etc in Isolated envs sessions?
- Maybe not, but virtual env name created in Isolated sessions and with name connected to course name
- UPPMAX users will be able to run CPU versions of ML in exercises unless they already have
- I have booked Snowy for a demo using the GPUs in the general GPOU session

### Todo:s
[x] Björn will make HackMD for course

- Introduction
- HPC2N fill in table: Mem/node+GPU

- evaluation form
- Birgitte

## More
- inform about needs from helpers
- HackMD
- chat
- breakoutrooms
- during exercise
- technical problems
- identify helpers

## Background
98 registered
### Experience with LINUX
Orientation 94%
Bash 86%
Batch 75%
Packages 87%
### Cluster (multiple)
HPC2N 47/84 55
Rackham 47/84 56
Bianca 18/84 21
Snowy 14/84 17

### Size of your research problem (multiple)
Single 25 %
**Node 63%**
Multinode 41%
Don’t know 25%

## Suggested outline
### Common
- Syllabus
- Intro (incl. very briefly about UPPMAX/HPC2N)
- terminal vs thinlinc (up to users)
- Load/Run
- Packages especially pip
- Isolated general
- Interactive general
- rm `-p`
- Batch general
- this one before interactive
- Parallel session 1
- UPPMAX: Conda, venv (exercises),
- Kebnekaise: virtual env (exercises)
#### LUNCH break
- Parallel computing general
- also exercises
- GPU general
- only demos
- snowy
- ML general
- some links to UPPMAX
- cpu exercise
- Parallel session 2
- UPPMAX: Bianca, jupyter
- HPC2N:
- gpu exercises at Kebne
- **Machine Learning given by HPC2N**

- Summary

## Time estimations (min)

| Time (UPPMAX) | Time (HPC2N) | Topic | Activity | Description | Goal |
| -----| ----- | -------- |------------ | ---- |--|
**Morning 3 hours**| 50 + *45m* +*40*
10 |9.00 |Syllabus
15 | |Intro |Lecture | Outline | Get overview Cluster+Python and LOs
15 | |Load and run| Lecture+code along | Show modules | Learn how to load a python version
15 | |packages intro |Lecture *+ code along + (exercise)* |Check current, Pip | List packages, do pip installation
|**15**| 9.55|**Coffee**||
15 | 10.10 |Isolated Intro |Lecture + code along | | Understand virtual environ
20 | |batch|
15 | |Interactive |
5| 11.00 |**short leg stretch**
45| 11.05|UPPMAX session1| Lecture +Ex? |Conda, isolated,
45| |HPC2N session1|Lecture + Ex?|Kebnekaise,isolated| Cluster specific practice and installed packages, User interaction |
|**Afternoon 3 hours** | *45+45m*| |
30 |13.00 |parallel| +exercises
20 | |gpu| +demos
5|13.50 | **Leg stretch**
20 | 13.55 |ML intro|+demos
25| 14.15 |UPPMAX session2| Lecture +Ex? | jupyter, Bianca
25| 14.15|HPC2N session2|Lecture + Ex?|ML exercise| User interaction |
|**15**|14.40 |**Coffee**||
15| 14.55 |Summary| Lecture|Describe when to do what| Keypoints
25| 15.10| Exercises not recorded
25|15.35|Questions not recored

## Lecturer

|Session| Björn | Birgitte | Pedro|
|-------|-------|----------|------|
|"Syllabus"| X | X| (X)
|Intro incl cluster| X | X | (X)
Load-run| X
packages| X
isolated intro| X
batch| | X
interactive| X
parallel| | | X
GPU| | | X
ML| | X
UPPMAX1 (Co,Is,)| X
HPC2N1 (Keb,Is )| | X|
UPPMAX2 (Bi, (Jupyter)| X
HPC2N2 (ML exercises)| | X| X
Exercises (exercises)| exercise rooms and help rooms
Question session (breakout)| X | X | X
Summary| X input | X | X input

## To discuss

- parallel
- copy paste exercises or given files?
- gpu
- only HPC2N?
- demo on Snowy

## Course material

### General
- shorter titles in drop-down
- [Link](http://www.thing.com) (http://www.thing.com)
- move split sessions to afternoon

- index.rst
- [name=Björn]
- Intro
- [name=Björn]
- only overview
- move "facts" to sections
- merge more and split if different (true for all section)
- load_run.rst
- [name=Björn]
- Load
- tabs
- Run
- tabs
- be clear what is meant by interactive
- packages.rst
- [name=Björn]
- tabs
- install with conda (move to UPPMAX)
- isolated.rst
- [name=Björn]
- split and insert to site-specific session
- pyenv only for uppmax
- bianca.rst
- [name=Björn]
- format
- demo pip wharf
- kebnekaise
- [name=Birgitte]
- [name=Pedro]
- interactive
- [name=Björn]
- [name=Birgitte]
- [name=Pedro]
- Introduce batch already here?
- batch
- [name=Björn]
- [name=Birgitte]
- [name=Pedro]
- parallel computing
- [name=Pedro]
- [name=Björn]
- ml
- [name=Birgitte]
- [name=Björn]
- split to sessions ()
- gpu
- [name=Birgitte]
- [name=Pedro]
- [name=Björn]
- summary.rst
- [name=Björn]
- [name=Birgitte]
- [name=Pedro]

## Latest version Letter

**Using Python in an HPC environment, UPPMAX/HPC2N September 9 2022. Online workshop**

UPPMAX and HPC2N are organising a joint workshop on how to run Python codes and install additional Python packages on the computer resources provided by these two HPC centres. Participants are encouraged to bring their particular software request for discussion as well.

The goal for the course is that you will be able to:

- Load Python modules and site-installed Python packages
- Create a virtual environment
- Install Python packages with pip (Kebnekaise, Rackham, Snowy)
- Install Python packages with conda (Bianca)
- Write batch scripts for running Python
- Use Python in parallel
- Use Python for ML
- Use GPUs with Python

Prerequisites: familiarity with the LINUX command line, basic Python, and batch jobs.

For more info and registration, please visit https://www.uppmax.uu.se/support/courses-and-workshops/HPC-python/ or https://www.hpc2n.umu.se/events/courses/2022/python-in-hpc

## Content

[TOC]

# 18 May
- Content
- line-up
- Letter info
- Workshop page

## How?
- one full day
- workshop
- Fri 2 YES
- Tue 6 sep? NO
- (7 Sep maintenance)

## Learning outcomes
- load and run python
- find installed packages
- install package with pip
- install package with conda
- create a venv
- use a venv
- jupyter on calculation node (more or less about)
- HPC2N perhaps working but different approach? Pedro looks up
- run batch jobs with python code
- run parallel
- calc pi with mpi
- run workflows
- how much??
- load and use ML packages
- tensor flow example (UPPMAX)
- gpu
## Learning experience
- webinar
- pair-teaching
- code along
- problem-based approach
- find out an example or two?
- group or pair work? TBD
- exercise
- more?
## Prerequisites
- basic Bash commandline + scirpting
- explicit
- basic Python
-
- basic Slurm

## Content
### Morning
- Python modules (LMOD)
- python packages
- installations
- Kebnekaise/Bianca separated session
- user specific session as well
- workflow
-
### Afternoon
- interactive
- jupyter on calculation nodes
- batch
- parallel computing
- tensorflow??
- Problem-based approach
- how to approach a problem at the site to solve it
- a number of example problems
- OOM-related (out of memory?)
- numerical- as well as data/ I/O-based
- check how the resources are used
- graphana
- jobstats

## Plan
| Time | Topic | Activity | Description | Goal |
| -----| ----- | -------- |------------ | ---- |
**Morning 2 hours**| 45 + 40m
10 |Intro |Lecture | Outline | Get overview and LOs
10 |Load and run| Lecture+code along | Show modules | Learn how to load a python version
25 |packages |Lecture + code along + (exercise) |Check current, Pip, Conda | List packages, do pip installation,do conda installation
10 |Isolated environ |Lecture + code along | Venv | Understand virtual environ
25 | Kebnekaise/Bianca separated session| Lecture | Cluster specific practice and installed packages, User interaction | Understand cluster limitations
5 |Summary| Lecture| Describe when to do what| Keypoints
**Afternoon 2 hours** | 45+45m
30| batch with python and conda |Lecture+code along + exercise | python with batch | write batch script with right envs
15| interactive jupyter| Lecture + code along +exercise | run jupyter on calculation nodes| run jupyter on calculation nodes
30| workflows?
15| Summary

## Letter info
- [ ] Sep 2
- [ ] Sep 6
- [X] Sep 9

**Online workshop: "Python at UPPMAX & HPC2N", *Sep 9*, 2022**

UPPMAX and HPC2n is organising a joint workshop on how to run Python and install additional Python packages on the computer resources provided by UPPMAX and HPC2N and how to use the HPC capabilities. Participants will be able to bring their particular software request for discussion as well.

**The following will be covered:**
* Loading the appropriate Python module
* Packages
* checking installed packages
* pip install
* Conda install
* Virtual environments
* Kebnekaise / Bianca session
* HPC
* interactive jupyter
* batch jobs with Python
* Machine Learning environment
* workflows

Prerequisities:
- basic Linux
- basic Python
- basic Slurm

When: Friday, Sep 9, 2022.
Time: 9:15 - 12:00, 13.15-16.00
Where: online.

Registration opens in June

### SNIC training letter
- 25 May (Wed)

## Web page later

# 21 April

## To be decided today
- [x] Content
- [x] Just UPPMAX/HPC2N?
- [x] Length
- [x] Style (one-way, workshop, course)
- [x] One or two instances
- [x] Month (at least)
- [x] Who wants to contribute

## Intro
- Pedro asked:
"I thought we could join efforts and collaborate in a Python course where we could talk about the workflows at UPPMAX and HPC2N?
If you think this is doable, maybe we could run it in the summer/autumn? "

## Content
- core from “Python at Uppmax” and made general
- https://python-at-uppmax.readthedocs.io/en/main/
- https://uppmax.uu.se/support/user-guides/python-user-guide
HPC

:::warning
same at HPC2N? (except Bianca)
- not Conda
:::

- More workflows
- snakemake?
- can submit slurm jobs
- Own workshop?

- Hpc possibilities
- Focus on site-specific systems
- modules
- sbatch
- interactive
- parallel computing
- GPU?
- TensorFlow
- More…?
- jupyter on calculation nodes??
- Problem-based approach
- how to approach a problem at the site to solve it
- a number of example problems
- OOM-related
- numerical- as well as data/ I/O-based
- check how the resources are used
## What other courses/workshops/training are there? Not to collide (*in content*) with:
- Python intro course at UPPMAX
- no mpi4py there
- ENCCS: **HPDA Python**, https://enccs.se/events/2022-05-hpda-python
- **May** (closed)
- tools for performant processing
- (netcdf, numpy, pandas, scipy) on single workstations
- **parallel**, **distributed** and **GPU** computing
- (**snakemake**, numba ( just-in-time compiler for Python), **dask**, **multiprocessing**, **mpi4py**).
- CodeRefinery: **Python for Scientific Computing** course together with Aalto University and CodeRefinery
- https://aaltoscicomp.github.io/python-for-scicomp/
- 4 half-days
- dates not yet set, most likely in **October**
- **mpi4py**
- **distrib**
- **packages**
- CodeRefinery workshop
- **Sep 20-22 and 27-29**, (6 half-days)
- online
- Git, collab version control
- reproducible research, social coding
- jupyter
- **workflows with snakemake** , and more
- software testing (**pytest**)
- modular code devel

- PDC Summer School 2022: Introduction to High Performance Computing, **15-26 August 2022**
- https://www.pdc.kth.se/summer-school/2022
- **No python??**
- parallel algorithms
- parallel programming
- modern HPC architectures
- performance analysis and engineering
- software engineering
- Earlier SNIC parallel courses (**Dec 2020**)
- SNIC Intro to MPI (C, Fortran, **Python**)
- https://github.com/SNIC-MPI-course/MPI-course
- PDC/ENCCS: https://pdc-support.github.io/introduction-to-mpi/index.html
- don't know the status
- more?
- NBIS snakemake (general)
- https://nbis-reproducible-research.readthedocs.io/en/course_1911/snakemake/
-
## Prerequisites
- basic Slurm
- basic Bash scripting
- basic Python

## How?
- [x] Divided into
- [x] packages and
- [x] HPC python computing
- [x] One instance
- [ ] Seminar(s)
- [x] Workshop
- [ ] Hackaton? Another time?
- [ ] Course?
- [x] Also updated documentation
- [X] new topics/examples gradually
## Length?
- [ ] 3 hours
- [x] Full day
- [ ] Several days?

## When?
- [ ] Python-at-UPPMAX+HPC2 May 18 10-12
- [ ] Aug, probably not
- [x] Sep 2, Sep 6-8 somewhere
- [ ] Oct
## Who?
- Any other SNIC centre?
- not this time
- they can learn from
## Other
-
## Next steps
- Further planning
- Content
- line-up
- Letter info
- Workshop page
- Who wants to be part?
- Björn
- (Lars)
- Pedro
- SNIC training letter
- 28 April
- 12 May
- 25 May (Wed)
## Next meeting
- 19 or 20 May, Doodle
-

import numpy
from mpi4py import MPI
comm = MPI.COMM_WORLD 
size = comm.Get_size()
rank = comm.Get_rank()

## Abc

X:1
T:Twinkle, twinkle little Star Wars
M:4/4
C:Trad.
K:C clef:treble
|C2 C2 G2 G2|A2 A2 G4|F2 F2 E2 E2|D2 D2 C4|
K:Cm
M:4/4
|C4 C4 |A,2> E2 C4| A,2> E2 C4|-C4 z4|