---
---
# 3-day Introduction to running R, Python, Julia, MATLAB in HPC

- Course ZOOM: https://umu.zoom.us/j/63961186558?pwd=aXJXUnhrdlZpNjMvOTlXaU9zS3BIUT09

## Materials page
- https://github.com/UPPMAX/R-python-julia-HPC/
- https://uppmax.github.io/R-python-julia-HPC/

## Links to HPC-Python
- planning: https://hackmd.io/_J-EjtsYQhWUgS0QTUh0Ww
- material: https://uppmax.github.io/HPC-python/index.html
- source: https://github.com/UPPMAX/HPC-python/tree/main/docs

# Autumn 2024

## Dates

week 43: Mon 21 Oct - Fri 25 Oct:

- Onboarding: Mon 21 Oct
- Python: Tuesday 22nd Oct: Re + ?
- Julia: Wednesday 23rd Oct:
- R: Thursday 24th Oct:
- Matlab: Friday 25th Oct: Bi + Bj + Pe + Re?

## MATLAB planning

### Next meeting 19 Aug, 13-14

Zoom: https://umu.zoom.us/j/61208880201

- Look-up before:
- VS code implementation
- gurobi/comsol/other?
- home desktop

### Meeting 20 May
- Typical sessions
- load, run, scripts, thinlinc
- packages/plug-ins
- check vs code implementation
- https://content.mathworks.com/viewer/6616d1a654a921f7493cf4f6
- gurobi/comsol etcetra
- isolated? check
- batch
- interactive
- parallelisms
- gpus
- ML

## Meeting in May ??
- Evaluations of R and Julia

# Spring 2024

## Dates

* 11 March 11:00-12:00: On-boarding
* 12 March: Python
* 13 March: Julia
* 14 March: R

## Evaluation meeting 2024-03-27 at 13.30

- Student evaluations:
- [Python](https://forms.office.com/Pages/AnalysisPage.aspx?AnalyzerToken=gYUW4FOr3fmmsYmBv20YfsRGEQUDXKUw&id=-aZLWjH1Mk-UZzmPGead5I9DA_EX6qtGpq8zJf2ohI1UMVpNM1dGUEhPVFpKUDVOUlBRWkgyUFlWMi4u)
- [Julia](https://forms.office.com/Pages/AnalysisPage.aspx?AnalyzerToken=gYUW4FOr3fmmsYmBv20YfsRGEQUDXKUw&id=-aZLWjH1Mk-UZzmPGead5I9DA_EX6qtGpq8zJf2ohI1UNlY3NFUwODFMWDlDVFhUUkRMS01CSkQzSS4u)
- [R](https://forms.office.com/Pages/AnalysisPage.aspx?AnalyzerToken=gYUW4FOr3fmmsYmBv20YfsRGEQUDXKUw&id=-aZLWjH1Mk-UZzmPGead5I9DA_EX6qtGpq8zJf2ohI1UQ0NUU1pWQ0taNlE2V0JRUllDQlMwMkdXSi4u)

- [Decided in meeting] Make the Python day last until 16:00
- [Decided in meeting] Seperate email for introduction to course and the prerequisites,
most important info at the top; it's OK to repeat info
- We conclude that the course is improving from the course ratings, all of them have gone up

- [Ri, all, suggest/vote, 5 mins] I think Python by far got the most attention by us. What about try to have seperate meetings going per day?
- BiBjRi: Yes
- [Ri, all] I am happy to see the prereqs page and the obvious love put in it. I am unsure of the page's purpose. Could we come up with a purpose of the page and vote? I volunteer to do the work. If we discuss: The goal of the prereqs page is ...
- BiBjRi: ... that our learners can test/verify that they have the minimum skillset and have installed the minimum requirements for the course
- [other votes]: [other suggestions]
- [Pedro, discuss] Make course material (even) more course agnostic by adding more links to other centers in course material. Bi: we should focus on one or two centers. Bi: in the near future, we may already be using Dardel, so we'll do this automatically. Bi: for UmU, course material must gathered at the course's site (i.e. cannot only link to UmU pages).
- [Bi, discuss] Will we use Rackham or Dardel in the fall? We will decide before summer.
- [Richel, Python & R, discuss] I think the summary of the evaluations of Python and R is 'talk less, demonstrate slower, more time for exercises'. To me, this seems like a recurring pattern. What about we help encourage outselves to commit to fixing this? My suggestion: to the learners' evaluation, add 'Which section(s) scheduled enough time for exercises?' as a means to encourage use to do better
- [VOTED] BiBjPeRi: Yes
- [Ri, Python & R, discuss] When reflecting on the evaluation results, Most questions were useless to me, as they did not help answer my teaching questions. I cannot predict which questions are mandatory, nor which are useful to others. Result: we inherited these questions and we do use these to some extent
- [Agreed] We should check on adding text comments, next to the 'give a rating' qestions
- [Agreed] Which section(s) scheduled enough time for exercises?
- [Is more-or-less already in, keep the original question] What should we keep doing?
- [Is more-or-less already in, keep the original question] What should we improve?
- [Is more-or-less already in, keep the original question] Other comments?
- [AGREED][Ri, Python] Suggest at least 30 mins of time for UPPMAX's interactive+Jupyter: 15 mins is too short

- Fall planning
- week 43: Tue 22 Oct -Fri 25 Oct
- onboarding: Mon 21 Oct
- Python: Tuesday 22nd Oct:
- Julia: Wednesday 23rd Oct:
- R: Thursday 24th Oct:
- Matlab: Friday 25th Oct: Bi + Bj + Pe
- HPC Python: May 15th: leave everything the same: Bi + Bj + Pe teach, Ri assists
- with updates of course from earlier learnings
- py/3.11 (GPU UPPMAX perhaps other solution)
## Planning meeting 2024-03-11 9:30-(max)10:30

- [Richel, 1 min, FYI] Richel has not gotten the link checker to work, probably does not have the time to do so
- skip for now
- .wordlist can be updated by all of us
- [Richel, 3 mins, FYI] Richel has not tested [Python packages](https://uppmax.github.io/R-python-julia-HPC/python/packages.html) and [Python isolated environments](https://uppmax.github.io/R-python-julia-HPC/python/isolated.html) on Kebnekaise yet, instead used the code that was present. AFAIK, I have no access to Kebnekaise
- Birgitte has already double-checked, so we assume it to work :-)
- Should `--devcore` (UPPMAX only) be included in the session on batch?
- [ ] Björn will add it :-)
- Onboarding at 11:00, we predict few people show up :-)
- [Richel, 3 mins, discuss] Time to put schedule of HackMD on Sphinx website?
- Schedule can be assumed to be up-to-date...
- Tip: Richel uses [this website](https://tableconvert.com/restructuredtext-generator) to create RST tables :-)
- [Richel, 3 mins, discuss] Evaluations before Q&A on all days?
- [x] If yes: who does that? Björn will do :+1:
- Evaluations: Birgitte has them, will be in the Q&A document
- Parallel functions
- [x] move to last Jl and R days
- demo on kebnekaise
- [x] Björn added demo box
- exercises on batch
- make clearer instructions how to run
- UPPMAX: Björn
- HPC2N: Pedro
- When parallel tracks, an Umeåner needs to start the recording in both Zoom rooms
- First Zoom room:
- https://umu.zoom.us/j/61922264837?pwd=aUZJNGc2R2ZqdjAwZHIvVTI0cDdLUT09
- Meeting ID: 619 2226 4837
- Passcode: 416707
- Second Zoom room:
- In the Q&A document: https://umu.zoom.us/j/67224822226?pwd=cW1zcjNveUp5OWpMYnl4dTdEQVF0dz09
- [x] Reschedule R to mimic Python day: Björn will do so
- [x] Reschedule Julia to mimic Python day: Björn will contact Pedro when ready
- Exercises Julia added by Pedro
- [x] all except last exercise
- [ ] batch exercise

## Planning meeting 2024-03-06 at 13:00-14:00

- Status
- [Richel, Python, technical] : `python_ML_packages/3.9.5-gpu` seems to works perfectly with the `pytorch_fitting_gpu.py` script:

module load python_ML_packages/3.9.5-gpu
git clone https://github.com/UPPMAX/R-python-julia-HPC
cd R-python-julia-HPC
pip install -r requirements.txt
cd Exercises
python pytorch_fitting_gpu.py

- Birgitte: `pytorch_fitting_gpu-snowy.sh` does not work. Richel: I indeed needed `pip install -r requirements.txt`. Hence, the metapackage does not work correctly, as you need to install pytorch using pip
- Birgitte: `seaborn-code.py` in [the course content](https://uppmax.github.io/R-python-julia-HPC/python/batchPython.html#serial-code-self-installed-package-in-virt-env) does not work
- comment(BC):
- Seems we have to stick with 3.9.5, since installation of 3.11.8 has not succeeded yet
- If 3.11.8 is installed, use that one. Else, use 3.9.5.
- Decision: Thursday March 7 12:00

- Discussion
- [Richel, Python, isolated environments] The schedule states `**10.45**|Isolated environments (ML, venv, **conda**)|Richel (45-->60)`, with a stress on conda. Sharing the new teaching goals for isolated environments (below), however, showed the need for venv again. What need the teaching goals be?

        - learners have created, activated, used and deactivated a conda virtual environment
        - learners have used an ML package
        - learners have heard about venv

or back to the earlier: **YES**

        - learners have created, activated, used and deactivated a venv
        - learners have used an ML package
        - learners have heard about conda

Conclusion: use the good old venv, Richel misunderstood :-)

- Björn moved some content from index page to prereqs. https://uppmax.github.io/R-python-julia-HPC/prereqs.html
- the links for improving programming skills
- most of the cluster information
- Comments?
- Add using 2FA on the ThinLinc local client /Björn
- To discuss:
- Is it OK to take a quick explanation of the most important things
- login & compute nodes and cores
- similarities and differences between Kebnekaise and UPPMAX clusters
- DONE
- Richel: teacher is boss :-) . The UPPMAX docs are tested and nice and used by other courses: prefer working on one source of content
- SSH X forwarding<http://docs.uppmax.uu.se/software/ssh_x_forwarding/>
- DONE
- Björn adds XMING in prereqs
- DONE
- Refer to UPPMAX doc [Remote desktop via webbrowser](http://docs.uppmax.uu.se/getting_started/login_rackham/#remote-desktop), including using 2FA
- DONE
- Text editors: http://docs.uppmax.uu.se/software/text_editors/
- DONE
- Linux: <http://docs.uppmax.uu.se/getting_started/linux/>
- DONE

- [Richel, 5 mins] Suggest to conformize folder names at <https://github.com/UPPMAX/R-python-julia-HPC>: `Exercises`, `R`, `docs`, `summaryPython.rst`, `packages_suggest.rst`, `sequential-python.sh`
- Richel's suggestion: `all_lower_snake_case_with_underscores`, Richel volunteers to do so
- Birgitte: this year folders only, next time I'll promise to rename the files :-) .
- `exercises`, `folder_with_more_names`
- [ ] Richel does it today before 23:59 :+1:
- [ ] Richel checks link checker

- Birgitte: some UPPMAX questions. These were already answered :-)
- Type-alongs
- Björn added ``.. type-along::`` are the tags for type-along blocks.
- Björn can continue for R
- Instructors will add this to their content if it is used in Python
- Next events:
- Onboarding: Monday March 11th 11:00-(max)12:00
- Planning meeting: Monday March 11th 9:30-(max)10:30

## Planning meeting 2024-02-29 11:00-12:00
- Richel will be absent
- Björn and Richel thought that they had a nearly-mandatory event in the afternoon at the R day on March 14th. However, it was in the morning. Vote:
- Björn and Richel teach in the afternoon of the R day, Birgitte and Pedro teach in the morning: :-1:
- Björn and Richel cancel the nearly-mandatory event and teach in the morning: **We do this**
- Move the R day to another date: :-1:
- Latest info: the nearly-mandatory event may be prostponed for other reasons

- Status:
- Problems with ML_packages installation

- ToDos
- Richel
- check 3.11 compatibility with exercises at UPPMAX
- will go through https://uppmax.github.io/R-python-julia-HPC/python/isolated.html and check if it work with python/3.11.8 combined with ML_packages/3.11.8-GPU Tuesday 5th March
- Björn:
- type-along boxes
- julia done?
- index page strip
- Prereqs more info
- Julia parts updates
- quiz

## Planning meeting 2024-02-22 11:00–12:00

- Status
- Diana is installing ML_packages/3.11.8-GPU today
- theano problem??
- Theano is not compatible with modern numpy 1.25.x (as we have with Python 3.1
- pytensor is the continuation
- but use first in HPC-python course
- pytensor will be installed at Rackham

- ToDos
- Björn:
- check 3.11 compatibility with exercises at UPPMAX
- type-along boxes
- Prereqs more info

## Schedule for the spring 2024 version

### Python (Tuesday 2024-03-12)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Birgitte
**9.10** |Python in general |Birgitte
**9.20** |Load modules and run |Birgitte
**9.45** |Break |.
**10:00**|Packages (45-->30) |Richel
**10.30**|Break |.
**10.45**|Isolated environments (ML, venv, **conda**)|Richel (45-->60)
**11:45**|Break or informal chat |Richel
**12.00**|Lunch |.
**13.00**|Batch |Birgitte
**13:20**|GPU |Birgitte
**13.30**|Kebnekaise: Jupyter |Birgitte
. |Rackham: Interactive session and Jupyter |Richel
**13.45**|Break |.
**14.00**|Parallel and multithreaded functions|Pedro
**14.25**|Conclusion & Q/A |Birgitte
**14.45**|Evaluation |.
**15.00**|END

### Julia (Wednesday 2024-03-13)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus 7 |Björn
**9.10** |Intro, Julia in general 23 |Pedro
**9.30** |Load modules and run 16 |Björn
**9:45** |Break |.
**10:00**|Packages and isolated 47 |Pedro (and Björn)
**11:00**|Break |.
**11:15**|Parallel and multithreaded functions 30|Pedro
**11.45**|Break or informal chat or exer|all?
**12.00**|Lunch |.
**13.00**|Batch and GPU 46 |Pedro
**13.50**|Break |.
**14.00**|UPPMAX: Jupyter and interactive 25 |Björn
. |HPC2N: Jupyter |Pedro
**14.25**|Conclusion and summary, Q/A|Björn
**14.35**|Evaluation |Anon [R volunteers]
**14.45**|Q/A |Anon [R volunteers]
**15.00**|END |.

### R (Thursday 2024-03-14)

Time |Topic |Teacher
---------|---------------------------------|-------
**9:00** |Syllabus |Richel
**9:10** |R in general |Richel
**9:20** |Load modules and run |Richel
**9:45** |Break |.
**10:00**|Packages |Birgitte
**10:30**|Break |.
**10:45**|Kebnekaise, ThinLinc & RStudio |Pedro
. |UPPMAX: ThinLinc, RStudio, inter-|Björn <--- NEW
**11:15**|Parallel |Pedro
**11:50**|Break or informal chat |All
**12:00**|Lunch |.
**13:00**|Isolated environments |Birgitte
**13:20**|Batch |Birgitte
**13:50**|Break |.
**14:00**|ML |Birgitte or Pedro <--- NEW
**14:35**|Conclusion & Q/A |All
**14:45**|Evaluation |.
**15:00**|END |.

## Planning meeting 2024-02-15 11:00–12:00
- How many registered
- 55
- Status
- BC: started prereq page to be filled

- Discuss
- Skip bianca parallel session for all days. Vote yes!
- No intro at 9.00 for slurm

- ToDos:
- Björn:
- type-along boxes
- Py-ml UPPMAX
- Prereqs more info
- Pedro: several docs for parallel

## Planning meeting 2024-02-09 11:00–12:00

- Discuss:
- depth of parallelism/ML for python
- same level as for julia and R?
- Yes, show an ML example
- We use R 4.1.x! R versions 4.3.1 + 4.3. (Birgitte checks the packages)
- HPC2N has R/4.3.2 but a lot fewer packages than for R/4.0.4. I have inquired about getting some packages installed as modules
- 4.3.2 is lacking many packages, including the Bioconductor bundle which is not yet available for installation.
- **4.1.x chosen instead. **
- UPPMAX: 4.1.1
- python_ML new version would be nice [name=Björn] checks
- wish: 3.11.(3,8)
- At HPC2N, Python **3.11.3** is installed. All relevant packages except Theano and ASE are available for that version of Python. We can probably ask for those packages if we need them
- At UPPMAX: **3.11.8**
- julia 1.8.5
- [R, suggest vote] follow pace of schedule or follow pace of learners?
- [VOTED FOR] Pace of schedule with flexibility of 15 mins in both directions
- pro:
- easy to schedule who teaches what
- comfortable to teach
- higher evaluation scores
- clear for learners when takes place what
- con:
- can mismatch with learners
- votes: Birgitte, Björn, Pedro
- Pace of learners
- pro:
- the right speed
- needs procedure, e.g. https://richelbilderbeek.github.io/awk_course/misc/exercise_procedure/
- con:
- harder to schedule
- not everyone comfortable with teaching like that
- setting learners to work will result in lower evaluation scores
- votes: Richel
- [R, suggest vote] evaluation yes/no?
- [VOTED] Yes (pro: we get feedback, con: can be unpleasant): Birgitte, Björn, Pedro, Richel
- No (pro: not unpleasant, con: no feedback to improve on): -
- [R, suggest vote] order: evaluation-then-Q&A or Q&A-then-evaluation?
- [VOTED] evaluation-then-Q&A (pro: learners can leave at Q&A): Birgitte, Björn, Pedro, Richel
- Q&A-then-evaluation: -
- [R, suggest vote] publish evaluation results online yes/no?
- [VOTED] Yes (pro: shows transparency (e.g. https://github.com/richelbilderbeek/teaching), con: not everyone can rightly value an evaluation): Birgitte, Björn, Pedro, Richel
- No (pro: no trouble, con: links and info get lost):
- Menti responsible (the person needs to nbe available all day)
- Python day: no polls, so no person needed
- Julia day: Björn
- R day: Birgitte
- [False positive] Although https://www.uppmax.uu.se/support/courses-and-workshops/r-python-julia links to https://uppmax.github.io/uppmax_intro_python/ it does not matter: it is not the course material :-)

## Planning meeting 2024-01-31

* [Vote] Use Zoom Poll or Menti or text in shared document: will use Menti. Birgitte will prepare
* Zoom Poll: only host
* Votes: Birgitte, Pedro
* Menti: everyone more-or-less needs separate account
* Votes: Richel, Björn, Birgitte
* Plain text (in a shared document)
* Votes: Richel, Pedro, (Birgitte)
* MENTI WON!
* [Richel, FYI] Björn and/or Richel can teach R at March 14th in the morning, but not in the afternoon.
* We rescheduled so that Uppsala people teach in the morning and the Umeå people in the afternoon
* [Richel, vote] some UPPMAX courses are restructured by **referring to the UPPMAX documentation pages**. In that way, content can be found in exactly one place. An example can be found [here](https://uppmax.github.io/uppmax_intro_python/morning_session/working_with_python_scripts/#exercise-1-create-a-python-script). As this course also teaches Kebnekaise, referring to [its documentation](https://www.hpc2n.umu.se/support) would work too. Richel volunteers to do so for the Python course (as he taught another similar Python course too). Good idea yes/no?
* Important with new tab. Try this time!
* VOTE:
* for: Richel, Björn, Birgitte, (Pedro)
* against: (Pedro)
* Evaluations
* Both Julia & R: Type-along clearer (boxes) and talk about it!
* good level
* more exercises
* more about ThinLinc (technical prereqs can be expanded with more info and background and pros and cons)

### Schedule
- slurm is integrated with interactive/batch each day (take a break!)

#### R (Thursday 2024-03-14)

Time |Topic |Teacher
---------|---------------------------------|-------
**9:00** |Syllabus |Richel
**9:10** |R in general |Richel
**9:20** |Load modules and run |Richel
**9:45** |Break |.
**10:00**|Packages |Birgitte
**10:30**|Break |.
**10:45**|Kebnekaise, ThinLinc & RStudio |Pedro
. |UPPMAX: ThinLinc, RStudio, inter-|Richel
**11:15**|Kebnekaise: Parallel |Pedro
. |skip? *UPPMAX: Bianca* |Richel
**11:50**|Break or informal chat |All
**12:00**|Lunch |.
**13:00**|Isolated environments |Birgitte
**13:20**|Batch |Birgitte
**13:50**|Break |.
**14:00**|ML |?
**14:35**|Conclusion & Q/A |All
**14:45**|?Evaluation |.
**15:00**|END |.

* Teachers
* Python (Tue)
* Richèl, same topics as previous time: Packages, Isolated environments, Rackham: Jupyter, Bianca: Conda, interactive
* Birgitte, same topics as previous time
* Julia (Wed)
* Björn (same as last time)
* Pedro
* R (Thu)
* Richel can teach morning only
* Björn teach first hour (afternoon if needed) or standby

## Planning meeting (Jan 24 at 11)
* Discuss
* menti
* Zoom can also do something similar called a Poll, which is ideal for quicker things. It does need to be shared :-)
* :+1: :+1:

* We will vote between using Zoom Poll and Menti next week Wednesday 31 Jan at 13:00, Birgitte will prepare
* We used Python version 3.9.5, because that worked well with ML. We try to go to a newer version of Python, if that has made to works well with ML too.

* Evaluations
* python: done, see results [here](https://github.com/UPPMAX/R-python-julia-HPC/tree/main/evaluations/20231017)
* Julia
* R
* General
* cheat sheets
* short intros
* slurm first 30 minutes everyday or so
* think about extending
* Teachers
* Python (Tue)
* Richèl, same topics as previous time: Packages, Isolated environments, Rackham: Jupyter, Bianca: Conda, interactive
* Birgitte, same topics as previous time
* Julia (Wed)
* Björn (same as last time)
* Pedro
* R (Thu)
* Richel can teach morning only
* Björn teach first hour (afternoon if needed) or standby

* Needed updates
- Py module
- UPPMAX update ML

- R module
- newer?
- Don't go too deep. Examples:
- determining number of cores
- ML and multiple GPUs?

# Autumn 2023
## Dates
* 17-19 October
* py, jl, R
* 9-15

## Debriefing Python day 8 Nov 11.00

Present: Bj and R

- [Evaluation](https://forms.office.com/Pages/AnalysisPage.aspx?AnalyzerToken=gYUW4FOr3fmmsYmBv20YfsRGEQUDXKUw&id=-aZLWjH1Mk-UZzmPGead5I9DA_EX6qtGpq8zJf2ohI1URVZPSUFDRlpCV1BUNzIyQzY1V0pPSFVKUi4u)
- [marked summary](https://docs.google.com/document/d/1K7aw4LxL-PCT6xU3HWs_WTFuvOIc1h35fzj3TzN9p-E/edit?usp=sharing)
- Learnings
- continue
- friendly
- easy asking questions: also thanks to breakout rooms!
- material
- structure
- change?
- make clear in advertisement that packages are a big part in this workshop
- more HPC in the HPC-python
- BO:s
- checking in more frequently: we did well enough
- more flexible in returning to main room: we felt the setup was fine if the times are better matched to the exercise length
- have voluntary exercise rooms: we felt the breakout rooms worked fine, if the times are better matched to the exercise length
- [AGREED] use a silent room
- No time to cover all material
- More to extra reading!
- More time on code-along and exercises: will work when timing of the course matches the observed exercise durations; this will free up 25 mins
- Use more videos; both Bj and R think videos are a useful alternative over a monologue
- Bj comment and opinion:
- videos perhaps not in lecture
- but available as recordings (like before) and session-sliced and available as links on each session page?
- [AGREED] Suggest different lengths:

Topic |Was scheduled|Actual time used|Suggested time
--------|-------------|----------------|--------------
packages|45 mins |15 mins | 30 mins
venv |45 mins |15 mins | 30 mins
Jupyter |15 mins |15 mins | 15 mins
conda |25 mins |25 mins | 30 mins

* [AGREED (implicitly)] Have a pre-course login session for those that need it
* [AGREED (implicitly)] No Zoom chat, only use shared document
* [AGREED (implicitly)] No anonymous evaluation needed if the Google Form is so good
* [DONT: this would only work well for UPPMAX users] Suggest to follow this students' suggestion:
* 'The first should be how to run a simple script. "batch mode" should come before "virtual environments". Also, avoid "pip" at the start, take that later since modules are enough for standard tasks'
* General comments from BB:
* Yes to the pre-course session for onboarding etc.
* Yes to use Q/A document instead of Zoom chat
* Google form seems to be enough for evaluation purposes, yes
* Strongly NOT agreeing on the more videos - if we do, they should be shown in the main room and be strictly the lecture we would otherwise have held (and then I would favour flipped classroom instead of having a course where people just come and watch a video and then we do exercises or whatever).
* There seemed to be unhappiness about the "go to zoom room" -> "go to YouTube and watch a video and then do the exercises there" -> "go back to main room and see lectures etc."

### Planning for next workhop
- Suggested month
- Feb/March
- Suggested teachers
- py:
- Richel: willing to teach whole day if needed, same for being helper
- jl:
- Björn in some parts
- R:
- Richel: willing to be a helper whole day, but open to teach if needed
- Björn interested

## Follow-up after first day

### Learnings
- Batch takes longer time! 30-->45
- Parallel need longer time! 25-->35?

### Changes for julia
- Package+isolated 30 min
- Packages and isolated
- 10 min evaluation

### Changes for R
- Package 30 min
- Isolated 30
- 10 min evaluation

### Julia (Wednesday 2023-10-18)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Björn
**9.10** |Intro, Julia in general |Pedro
**9.30** |Load modules and run |Björn
**10:00** |Break |.
**10:15**|Packages and isolated |Pedro (and Björn)
**11:00**|Break |.
**11:15**|Parallel and multithreaded functions|Pedro
. |Bianca packages |Björn
**11.50**|Break or informal chat or exer|all?
**12.00**|Lunch |.
**13.00**|Batch and GPU |Pedro
**13.45**|Break |.
**14.00**|UPPMAX: Jupyter and interactive |Björn
. |HPC2N: Jupyter |Pedro
**14.25**|Conclusion and summary, Q/A|Björn
**14.35**|Evaluation |Anon [R volunteers]
**14.45**|Q/A |Anon [R volunteers]
**15.00**|END |.

### R (Thursday 2023-10-19)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Björn
**9.10** |R in general |Björn
**9.20** |Load modules and run |Björn
**9.45** |Break |.
**10:00**|Packages |Birgitte
**10.30**|Break |.
**10.45**|Isolated environments |Birgitte
**11.15**|Kebnekaise: Parallel |Pedro
. |*UPPMAX: Bianca* | Björn
**11:50**|Break or informal chat |All
**12.00**|Lunch |.
**13.00**|Batch |Birgitte
**13:20**|Parallel, ML |?
**13.30**|Kebnekaise, ThinLinc & RStudio|Pedro
. |UPPMAX: ThinLinkn, RStudio, inter-|Björn
**13.45**|Break |.
**14.25**|Conclusion & Q/A |All
**14.45**|Evaluation |Anon [R picks someone]
**15.00**|END |.

## Next meeting Mon 16 Oct 11.00
- Status
- all OK with parallel session materials
- R: not yet, working on it, will be ready, hence probably not in meeting. Looking forward to tomorrow :-) !
- R: to overcommunicate, for tomorrow: as agreed upon earlier, in the session 'Isolated environments' 11:00-11:45 I will only do `virtualenv`, as that works on both clusters. Conda is in a parallel session at 14:00.
- [x] BC: I move the conda part in material then to afternoon! If you don't do it first!
- [x] BC: Conda is removed from packages but available in condaUPPMAX. I kept an introduction in the packages session and gave a link to the afternoon.
- [X] BC: Updating schedule
- R: To overcommunicate: evaluator at 14:45 has been arranged (Wednesay and Thursdays R will do it). Using a person and doing an evaluation under course time has given 24 out of approx 27 responses [in another course](https://github.com/NBISweden/workshop-python/blob/ht23/evaluation/Intro%20to%20Python%202023%20evaluation.csv)

- To discuss
- same presentation style?
- half screen with material + terminal + history
- shell-sharing: https://enccs.github.io/instructor-training/teaching-mechanics/#shell-sharing
- R: I think this is a good idea, so I will try to do this most of the time. On the other hand, I feel each teacher should be free to do what he/she prefers
- we do as we prefer!

User

╔════════════╗ ╔══════════════╗
║            ║ ║  TERMINAL    ║
║            ║ ║   WINDOW     ║
║   ZOOM     ║ ╚══════════════╝
║  WINDOW    ║ ╔══════════════╗
║            ║ ║   BROWSER    ║
║            ║ ║  W/MATERIAL  ║
╚════════════╝ ╚══════════════╝

Teacher (sharing left/right portion of screen)

╔══════════════╗
║   BROWSER    ║
║  W/MATERIAL  ║
╚══════════════╝
╔══════════════╗
║  TERMINAL    ║
║   WINDOW     ║
╚══════════════╝
╔══════════════╗
║  HISTORY     ║
║   WINDOW     ║
╚══════════════╝

## Meeting Tue 10 Oct 15.00
- To discuss
- DASK?
- Is installed in py3.11 on UPPMAX
- just mention this time?
- include in hpc-python Dec 1?

- ToDos
- [ ] Björn checks everything is working on UPPMAX
- [x] First page common for all days
- changes, shortenings
- code of conduct
- [x] Björn COMMON
- parallel
- add dask+links in the end
- [x] Björn: list of links to centres
- langauage specific + common
- [x] Björn: courses at all levels in a good format

- Status
- exercises?
- start!
- discussions?
- perhaps not
- [x] can we copy the parallel?
- or put under a menu-bar header "COMMON" or something else. YES
-

- Workshop days
- BO rooms
- silent
- many rooms for exercises
- interaction room
- force: works!

## Next meeting Thu 28 Sep 13.00
- To discuss:
- evaluation in lesson-time?
- Publish results in raw form online?
- Status

## Schedule for the fall 2023 version

### Python (Tuesday 2023-10-17)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Birgitte
**9.10** |Python in general |Birgitte
**9.20** |Load modules and run |Birgitte
**9.45** |Break |.
**10:00**|Packages |Richel
**10.45**|Break |.
**11.00**|Isolated environments |Richel
**11:45**|Break or informal chat |Richel
**12.00**|Lunch |.
**13.00**|Batch |Birgitte
**13:20**|GPU |Birgitte
**13.30**|Kebnekaise: Jupyter |Birgitte
. |Rackham: Jupyter |Richel
**13.45**|Break |.
**14.00**|Parallel and multithreaded functions|Pedro
. |Bianca: Conda, interactive|Richel
**14.25**|Conclusion & Q/A |Birgitte
**14.45**|Evaluation |Anon [R picks someone]
**15.00**|END |.

### Julia (Wednesday 2023-10-18)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Björn
**9.10** |Intro, Julia in general |Pedro
**9.30** |Load modules and run |Pedro
**10:00** |Break |.
**10:15**|Packages and isolated |Pedro and Björn
**11:00**|Break |.
**11.15**|Exercises and informal chat|All
**12.00**|Lunch |.
**13.00**|Batch and GPU |Pedro
**13.30**|UPPMAX: Jupyter and interactive |Björn
. |HPC2N: Jupyter |Pedro
**13.45**|Break |.
**14.00**|Parallel and multithreaded functions|Pedro
. |Bianca packages |Björn
**14.25**|Conclusion and summary, Q/A|Björn
**14.45**|Evaluation |Anon [R volunteers]
**15.00**|END |.

### R (Thursday 2023-10-19)

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |Björn
**9.10** |R in general |Björn
**9.20** |Load modules and run |Björn
**9.45** |Break |.
**10:00**|Packages |Birgitte
**10.45**|Break |.
**11.00**|Isolated environments |Birgitte
**11:45**|Break or informal chat |All
**12.00**|Lunch |.
**13.00**|Batch |Birgitte
**13:20**|Parallel, ML |?
**13.30**|Kebnekaise, ThinLinc & RStudio|Pedro
. |UPPMAX: ThinLinkn, RStudio, inter-|Björn
**13.45**|Break |.
**14.00**|Kebnekaise: Parallel |Pedro
. |UPPMAX: Bianca | Björn
**14.25**|Conclusion & Q/A |All
**14.45**|Evaluation |Anon [R picks someone]
**15.00**|END |.

### Next meeting:
10th October 2023, 15:00-16:00

## Python-specific meeting 19 Sep 13.00-13.30
- Differences between clusters
- Do all students work on the same cluster: no
- Is it possible to get access to HPC2N: yes, B will add R
- Course material shows differences in tabs versus links? Decision: three tabs, 1 for UPPMAX, 1 for HPC2N, links in third tab
- Interactive jobs work best at UPPMAX, at end for UPPMAX only
- Preparation
- B sets up Zoom room, will be recorded, sent to YouTube channel, B edits movie as well :-)
- Planned changes
- More exercises for the quicker students: R and B will add for own sessions
- Assume students learn Python themselves
- Decision: Add parallel intro material, by Pedro. B asks Pedro
- Decision: Add GPU intro material, by Birgitte
- Teaching schedule (see schedule below)
- R and B will discuss on SNIC Slack
- TODO R: Scheduling 11:30 'Catching up' makes no sense: talk less and/or have less exercises instead. Or phrased differently: what would the learning outcome be of 'Catching up'?
- TODO R: vote: 15 mins break, scheduled at `x:45-y:00` (also from 11:45-12:00!)
- TODO R: I need 5 mins for the syllabus. Why is `+thinlinc` in the schedule at 'Intro syllabus'?
- TODO R: It feels needlessly clumsy that there are two teacher switches in the first 45 mins. What about one teacher does the full first 45 mins?
- TODO R: same needless switching for 13:00-13:50 (and why `50`, instead of `45`?)
- TODO Teach both `conda` (Binca only) and `venv`/`virtualenv`. `conda` is only for Bianca, so fewer students expected
- Evaluations
- R: Earlier evaluations:
- Where can we find them? B sends R the CSV file
- At the end of the third day
- R: When? For example, at the end of every day?
- At the end of 3rd day or every day?
- Vote: Split up: Richel,
- Actual vote at bigger meeting, B will (re)schedule and bring it up
- R: Publish results in raw form online
- Actual vote at bigger meeting and bring it up

### Present schedule

Suggested Python schedule by R&B (old is below this one):

Time |Topic |Teacher
---------|---------------------------|-------
**9.00** |Syllabus |B
**9.10** |Python in general |B
**9.20** |Load modules and run |B
**9.45** |Break |.
**10:00**|Packages |R
**10.45**|Break |.
**11.00**|Isolated environments |R
**11:45**|Break or informal chat |R
**12.00**|Lunch |.
**13.00**|Batch |B
**13:20**|GPU |B
**13.30**|Kebnekaise: Jupyter |B
. |Rackham: Jupyter |R
**13.45**|Break |.
**14.00**|Parallel and multithreaded functions|Pedro
. |Bianca: Conda, interactive|R
**14.30**|Conclusion |B
**14.45**|Evaluation |Anon [1]
**15.00**|END |.

* [1] A person that the students have never met yet, to ensure a proper anonymous evaluation. R will recruit
* Zoom: 1 silent room

OLD:

|Topic|Suggested Timing|Comment|Teacher|
|-----|--------|-|-|
|**9.00** Index intro|15|+ thinlinc |Richel|
|**9.15** Intro to python|10|Keep detailed schedule and link to HackMD|Birgitte|
|**9.25** Load and run|20|(can be shorten to 15m perhaps)|Richel|
**9.45** Break| 15
|**10:00** Packages|25| |Richel|
**10.45** break| 15
|**11.00** Isolated environments| 30| | Richel
|**11:30** Catching up and exercises|30|
|**12.00** Lunch| 60
|**13.00** Batch|25|| Birgitte
|**13.25** Jupyter with remote desktop|25| |Richel and Birgitte in parallel session
|**13.50** break| 15
**14.05** Parallel function intro|15|**NEW**|
|END Interactive session on command-line (UPPMAX only)|25| |Richel
|END Packages: Conda|20| |Richel
|**14.20** Summary|10||Richel|
|**14.30** Catching up, questions and exercises| 30||All
**15.00** END

## Next meeting Fri 15 Sep 11.00

- [x] Richel assumes to teach Python on Oct 17th
- [x] Richel added spellcheck and linkcheck GH-actions
- https://github.com/UPPMAX/R-python-julia-HPC/tree/main/.github/workflows
* Status
- project is active

- Todo:s
- Reservation [name=Diana]
- change start date to asap, but keep end date as Nov 1 [name=Diana]

## Next meeting Wed 6 Sep 10-11
- keep version?
- py3.9.5 or newer
- check packages until next meeting
- 3.10.X?
- same for R and Julia
- R/4.0.4
- 4.1.2 Keb
- 4.1.1 Rac
- julia/1.8.5
- stick with this, not very old
- 'til next time
- add GPU R??
- start thinking of the parallel intro material
- projects
- rackham
- snowy + reservation
- kebne

## Timings
### Python

- Table is based on 9.00-12.30 and 1x15 + 2x5 min breaks
- modified to 9-15

|Topic|Suggested Timing|Comment|Teacher|
|-----|--------|-|-|
|Index intro|15|+ thinlinc |Richel|
|Intro|10|Keep detailed schedule and link to HackMD|Birgitte|
|Load and run|20|(can be shorten to 15m perhaps)|Richel|
|Packages|25| (buffert time)|Richel|
|Packages: Conda|20| |Richel
|Isolated| 30| Plan for extra time as we do it together| Richel
|Catching up and exercises|30|
|Lunch
|Batch|25|slightly more effective| Birgitte
|Interactive, more jupyter|25| |Richel
|Parallel intro|15||
|Summary|10||Richel|
|Questions|15||All
|Catching up and exercises| 20||All

### Julia

- Table is based on 9.00-12.30 and 1 15 +2 5 min breaks
- short intro to parallel
- include jupyter

|Topic|Suggested shortening|Comment|Teacher|
|-----|--------|-|-
|Index intro|5|10|Move some practical to overview|Björn
|Intro|10||Pedro
|Load and run|20|(can be shorten to 15m perhaps)|Pedro
|Isolated| 30| Plan for extra time as we do it together| Pedro
|Packages at UPPMAX|20| (buffert time)| Björn
|Batch|25|slightly more effective|Pedro
|Interactive|15||Björn
|Exercises| 20||All
|Summary| 10| |Pedro + Björn
|Questions|?|10 ||All

### R
- End already at 12?
- proposed R scheddule:
https://uppmax.github.io/R-python-julia-HPC/R/intro.html
- Table is based on 9.00-12.00 and 1x15 + 1x5 min breaks

|Topic|HPC-Python|Suggested timing|Comment|Teacher|
|-----|----------|--------|-|-
|Index intro|10|Move some practical to overview|Matias|
|Intro|10|Keep detailed schedule and link to HackMD|Matias
|Load and run|20|(can be shorten to 15m perhaps)|Birgitte
|Packages|20| (buffert time)|Birgitte
|Isolated|5-10|renv + mention Conda| Matias
|Batch incl GPU|25|slightly more effective|Birgitte
|Interactive| 15?| |Matias
|ThinLinc+RStudio| 10-15 | | Matias
|Exercises| 20||All
|Summary| 10| |Matias
|Questions|10 ||All

## Meeting 11 Aug
- Changes
- more HPC?
- not parallel
- mention gpu also for R
- more exercises!
- extra time in the end (1 hr or so)
- assume students attend prereqs
- basic programming Python
- CodeRefinery https://coderefinery.github.io/data-visualization-python/python-basics/
- UPPMAX https://uppmax.github.io/uppmax_intro_python/
- Scicomp Aalto intermediate https://scicomp.aalto.fi/training/scip/python-for-scicomp-2022/
- Julia
- aalto https://github.com/AaltoRSE/julia-introduction
- enccs
- caRPENTRY https://carpentries-incubator.github.io/julia-novice/
- R
- carpentry https://swcarpentry.github.io/r-novice-gapminder/index.html
- intro course text material (recordings for Kebnekaise?)
- linux
- slurm

- how to do on other Naiss systems
- tabs
- or cheat sheets
- web site
- UNIVERSITY
- GITHUB: INCLUDE REGISTRATION LINKS

- Last course 45 in total
- py: c. 40
- jl: c. 15 --> 8
- R: c. 30
# Spring 2023

## Dates
* 8-10 February
* half-days

## Meetings
https://umu.zoom.us/j/65560989451
- [x] 11 Jan 13-14.
- [x] 18 Jan 13-14
- [x] 25 Jan 13-14
- [X] 1 Feb 13-14
- [ ] 6 Feb 10.15
- [ ]

## Status
- Python
- minor fixes
- mention Conda problems with interaction to other modules
- summary
- Julia
- Some new boxes/images(?)
- move `iJulia` to interactive @bclaremar
- mention `-l` in batch scripts UPPMAX @bclaremar
- interactive @bclaremar
- summary

- R
- load_run
- mention R_packages
- packages
- example
- R_packages on UPPMAX
- isolated section (short)
- renv
- rewrite somewhat (now more or less a plagiarism)
- Batch
- examples
- UPPMAX Parallel
- UPPMAX Gpu

- exercise
- interactive+RStudio
- restructure
- example?
- exercise
- summary

## Exercises on Zoom

- breakout rooms for specific help
- silent room

## Teachers (at least one per site)
- Python 8 Feb
- HPC2N: [name=Birgitte, Pedro]
- UPPMAX: [name=Björn]
- helper: [name=Diana]
- sum: 4 people
- Julia 9 Feb
- HPC2N: [name=Pedro]
- UPPMAX: [name=Björn]
- helper: [name=Birgitte]
- sum: 3 people

- R 10 Feb
- HPC2N: [name=Birgitte, Pedro]
- UPPMAX: [name= Matias]
- helper: [name=Björn]
- sum: 4 people

## Helpers

- Avail everyday
- Birgitte
- Pedro
- Björn
- Pavlin

- Python
- Diana
- Pavlin
- Julia
- Pavlin
- Birgitte
- R
- Matias
- Pavlin
- Björn
- Pedro

## Timings
### Python

- Table is based on 9.00-12.30 and 1x15 + 2x5 min breaks

|Topic|HPC-Python|Suggested timing|Comment|Teacher|
|-----|----------|--------|-|-|
|Index intro|5|15|+ thinlinc |Björn|
|Intro|30|10|Keep detailed schedule and link to HackMD|Birgitte|
|Load and run|20|20|(can be shorten to 15m perhaps)|Birgitte|
|Packages|16|20| (buffert time)|Björn|
|Packages: Conda|15|15| |Björn
|Isolated|12+15(site)=27| 30| Plan for extra time as we do it together| Björn
|Batch|28|25|slightly more effective| Birgitte
|Interactive|15 |15| |Björn
|Exercises| ?| 20||All
|Summary| 7| 10| Don't hurry too much|Björn|
|Questions|?|10||All

### Julia

- Table is based on 9.00-12.30 and 1 15 +2 5 min breaks

|Topic|HPC-Python|Suggested shortening|Comment|Teacher|
|-----|----------|--------|-|-
|Index intro|5|10|Move some practical to overview|Björn
|Intro|30|10||Pedro
|Load and run|20|20|(can be shorten to 15m perhaps)|Pedro
|Isolated|12+15(site)=27| 30| Plan for extra time as we do it together| Pedro
|Packages at UPPMAX|16|20| (buffert time)| Björn
|Batch|28|25|slightly more effective|Pedro
|Interactive|15 |15||Björn
|Exercises| ?| 20||All
|Summary| 7| 10| Don't hurry too much|Pedro + Björn
|Questions|?|10 ||All

### R
- End already at 12?
- proposed R scheddule:
https://uppmax.github.io/R-python-julia-HPC/R/intro.html
- Table is based on 9.00-12.00 and 1x15 + 1x5 min breaks

|Topic|HPC-Python|Suggested timing|Comment|Teacher|
|-----|----------|--------|-|-
|Index intro|5|10|Move some practical to overview|Matias|
|Intro|30|10|Keep detailed schedule and link to HackMD|Matias
|Load and run|20|20|(can be shorten to 15m perhaps)|Birgitte
|Packages|16|20| (buffert time)|Matias
|Isolated|| 5-10|renv + mention Conda| Matias
|Batch|28|25|slightly more effective|Birgitte
|Interactive|| 15?| |Matias
|ThinLinc+RStudio| |10-15 | | Matias
|Exercises| ?| 20||All
|Summary| 7| 10| |Matias
|Questions|?|10 ||All

## Zoom

- Lecture recorded in one room
- Exercises splitted into

## Microsoft-365
- 3 seperate pages on Monday

## Exercises
- 1-3 per topic

## Homepages
- HPC2N: https://www.hpc2n.umu.se/events/courses/2023/r-python-julia
- UPPMAX: https://www.uppmax.uu.se/support/courses-and-workshops/r-python-julia

## Material type
- Choose one or several
- **GH pages** at UPPMAX [name=Björn]
- sphinx
- MkDocs
- MD on GH
- PPT/PDF
- Like Git course??
- two repos
- scripts for creating material
- Exercises...

## Homepage for material
- logo: remove SNIC

## Structure of the days 9.00-12.30 (2 breaks)
- 2 hours lessons?
- c. 30-45 m with exercises split?
- ending summary with questions

## GH issues
- Python
- merge Conda into packages [name=Björn]
- merge UPPMAX ad HPC2N sections
- Julia
- load_run
- packages
- R
- intro
- load_run
- packages

## Material
* generally
* 10-15 m intro to the languages
* load module + run (interactive+scripts)
* working with packages
* py
* virt env
* pip
* conda (users may come to UPPMAX in future) (not splitted)
* very briefly Bianca (advertise Bianca course)
* batch jobs
* serial vs parallel running
* (not scripting)
* examples integrated
* (type-along examples kept)
* *break*
* exercises 30
* in the end
* same as code-along but other package
* summary 5
* questions 10

* Day 1
* python/3.9.5
* Python (settled)
* Based on material from HPC Python (https://www.hpc2n.umu.se/events/courses/2022/python-in-hpc)
* https://uppmax.github.io/HPC-python/
* modules/packages
* installing methods
* pip
* virtualenv
* conda (UPPMAX)
* Template for batch script + running
* HPC2N Python package install docco: https://www.hpc2n.umu.se/resources/software/user_installed/python
* UPPMAX background: https://www.uppmax.uu.se/support/user-guides/python-user-guide/

* Day 2
* Julia 1.8.5
* Björn installerar
* Julia (settled)
* UPPMAX background: https://www.uppmax.uu.se/support/user-guides/julia-user-guide/
* HPC2N background: https://www.hpc2n.umu.se/resources/software/julia
* Day 3
* R/4.0.4 (4.1.1 UPPMAX)
* R (settled)
* Maybe use some from HPC2N/PDC course on R in HPC (14-15 December 2022): https://www.hpc2n.umu.se/events/courses/2022/R-in-HPC
* UPPMAX background: https://www.uppmax.uu.se/support/user-guides/r-user-guide/
* HPC2N R package install docco: https://www.hpc2n.umu.se/resources/software/user_installed/r

## Misc
- in the future
- rust
- next HPC-python
- 12 April
- Refer back to the package-installation in this current course