Python for Sci Comp 2023

Useful links

Internal HackMD with our notes (this doc) https://hackmd.io/@coderefinery/pfsc2023internal
Course webpage with schedule: https://scicomp.aalto.fi/training/scip/python-for-scicomp-2023/
Course materials: https://aaltoscicomp.github.io/python-for-scicomp/
Notes doc with the learners: https://notes.coderefinery.org/python2023
Notes archive: https://hackmd.io/@coderefinery/python2023archive

2023-10-31 10:00 CET (11:00 EET) meeting

~~Let's choose a date, 1h slot~~ Date chosen.

Garage zoom: https://aalto.zoom.us/j/61322268370

Agenda

Pretasks
a. Make sure your name is listed in the instructors this page (next section)
b. Make sure you have taken the lead role in at least one episode, try to find a co-instructor. Add your initials.
c. Make sure that the webpage reflects what is done here (it is good if everyone does this check for their own data as well as everything else)
Briefly on registrations and communication
- 286 registrations so far. Last push needed if we want to reach 500. I can take care of Aalto, but ideally others have time to convince your people that it is worth registering and coming even for the favourite episodes.
Revising the schedule
Revising the webpage
- https://scicomp.aalto.fi/training/scip/python-for-scicomp-2023/
Practicalities during the event7. Practicalities during the event
- studio zoom + director (most likely RD)
Pre-event for installation help. Pre-event for installation help
- only for "our" unis?
- At least Aalto people will send an email to Aalto users.
Discussing the homework assignments since many asked for 1ECTS. Discussing the homework assignments since many asked for 1ECTS
10. seems good as it was/is
11. It can be added: if you use libraries that are not part of the course environment, please add etc etc. Would it be worth to have an exercise: explore a library that is not part of the course env and provide instructions on how others can rerun it
Working on materials: this is not a discussion items, but just a wish that those presenting a section can take the freedom to adapt it to their goals. Please do ask for help to others if needed. 10. Working on materials: this is not a discussion items, but just a wish that those presenting a section can take the freedom to adapt it to their goals. Please do ask for help to others if needed.

TODO for Webpage and other things

Update affiliation for Diana and Johan (remove SNIC) to NAISS
Make clear that advanced numpy is not covered
In the teaching materials depend mang can be just after libraries
Update website schedule for HackMD plan
Re-check and eventually update installation. For your teaching part, check your code snippets.
Check if your episode depends on previous episodes
https://scicomp.aalto.fi/training/scip/python-for-scicomp-2023/ and schedule do not match.
- e.g. Thu 9.nov, 9:50-13:00
it still says SNIC, remove.

Maybe for next year:

auto evaluate submissions e.g. with CI, autograder and things like that

Instructors

…and their initials for the next section.

Radovan (7 and 8 November) [RB]
Richard (when needed) [RD]
Enrico (when needed) [EG]
Johan [JH]
Sabry Razick(away on the 10th) [SR]
Simo [ST]
Teemu [TR]
Diana [DI]
Jarno [JR]
Thomas [TP]

Topics

Day 1 - Tue 7.nov, 9:50-13:00 (EET)
- 10:00 (15m) Intro (???)
  - Remember to mention about credit
- 10:15 (35m) Jupyter
  - DI
  - JR
    - 10:15 (15m) Teaching
    - 10:30 (15m) Exercises 1-2 (3 optional)
    - 10:45 (5m) Wrap-up
- 10:50 break
- 11:00 (60m) Numpy (basic version, one advanced numpy thing + link to videos). Advanced numpy material can still be mentioned and the viewer can decide themselves if the basic is too basic or do those later as homeworks
  - JH
  - EG
    - 11:00 (15m) Intro; What is an array; Creating an array [JH narrator EG typing]
    - 11:15 (10m) Exercise 1 learners alone
    - 11:30 (10m) Array math + indexing and slicing [JH narrator EG typing]
    - 11:40 (15m) Exercise 2 and 3 learners alone
    - 11:55 (5m) Mention solution of exercise 3 that is important and also check if something important is hackmd. Wrapping up quickly covering why we have those other things in the basic numpy page and where to go from here with advanced numpy [EG narrator no typing, we can improvise on passing turns explicitly]
- 12:00 (10m) break
- 12:10 (50m) pandas…
  - RD
  - JR
- 12:55 (5m) outro
  - RD
  - JR
Day 2 Wed 8.nov, 9:50-13:00 (EET)
- 10:00 (30m) pandas continued
  - RD
  - JR
    - 10:00 (10m): Merge (from working with dataframes) + Time series superpowers
    - 10:10 (15m): Exercise 3
    - 10:25 (0m): Wrap-up
- 10:30 (80m) matplotlib
  - RB: prepare a discussion on connection to pandas
  - JH: general overhaul
  - 15 min: intro (including pandas link) RB leads
  - 20 min: exercise 1
  - 10 min break
  - 5 min: Matplotlib har two interfaces
  - 15 min: customization (JH leads)
  - 25 min: exercise 2 and 3
  - 10 min: wrap-up
- 12:10 (20m) data formats # could be shortened? (ST's opinion: could be shortened a lot and combined with productivity tools)
  - RD
  - ST [RD's proposal… work together on these two]
- 12:20 (30m) productivity tools
  - ST
  - RD
- 12:55 (5m) outro
  - ST
  - RD
Thu 9.nov, 9:50-13:00
- 10:00 (50m) scripts # exercises important
  - TP
  - RD
  - xx:10 first exercise (1, 5min)
  - xx:15 talk
  - xx:30 second exercise (2-3, 15min)
  - xx:45 wrap-up
  - xx:50 done, to break
- 11:00 (10m) library ecosystem (it can be fast, the page is a collection of links to explore for the learners, and get in mindset for "dependency management")
  - SR
  - ST
- 11:10 (50m) dependency management (Sabry, Simo)
  - ST
  - SR
- 12:00 (10m) break
- 12:10 (45m) binder
  - RD
  - TP
- 12:55 (5m) outro (RD+TP)
Fri 10.nov, 9:50-13:00
- 10:00 (60m) parallel python # important to keep time
  - JR
  - ST
    - 10:00 Intro, multithreading, multiprocessing
    - 10:20 Exercises 1 (15m)
    - 10:35 MPI + demo
    - 10:45 Coupling to other languages + other libraries
  - For next year:
    - Simpler MPI example, just print "Hello", then add rank
    - List of libraries for simplifying parallellism
      - joblib, jax, dask, ray
      - Maybe with examples
- 11:00 (10m) break
- 11:10 (40m) packaging
  - JR
  - TP
    - 11:10 Packaging demo
    - 11:20 Exercise 1, 15 minutes
    - 11:35 PyPi + Wrapup
- 11:50 (10m) break
- 12:00 (30m) web APIs # still hands on exercise but less time than last year?
  - EG screen shares and asks questions to Richard
  - RD interacts
    - 12:00 (13m)
      - Enrico explains What is a server? what is an API? Why are APIs important?
      - Enrico asks: "Richard why does it say "http" for humans? What is http and https?"
      - Richard types and does the demo
      - Enrico asks: "Richard, why do we get b'{…jsonstuff}? and what is JSON?"
      - Sometimes we also want to send something rather than just requesting, parameters richard does demo
    - 12:13 (10m) Exercise 1 with the bored API
    - 12:23 (10m) Briefly on headers and sraping. If time, then demo.
      - Enrico asks: "Richard, did you ever had to modify the default header?"
- 12:30 (20m) panel discussion or buffer time?
  - EG, RD, TP, JR, add your initials!
- 12:50 (10m) Outro

OLD CONTENT

Hidden below the table we used to pick the date.

Times in Stockholm zone CEST

	StartTimes
Day	9:00	10:00	11:00	12:00	13:00
Tu24	EG, SR,RD	EG,SR	EG,SR	RD,	RD
We25	RD	RD	RD	EG,	EG,
Th26	SR,RD	EG, SR,RD	SR,RD	RD,RD
Fr27	EG, RB,RD	RB,RD	EG	EG,SR	EG,RD
Mo30	EG, SRRD,	EG,RD	RD	RB,RD	RB,RD
Tu31	EG,SR,RD	EG, RB,SR,RD	EG, RB,SR,RD	EG,RD	RD
We01	EG, RB,SR,RD	EG, RB,SR,RD	EG, RB,SR,RD	EG, RB,RD	EG,RD
Th02	EG, RB,RD	EG,RD	RB,RD	EG, RB,SR	EG, RB,RD
Fr03	EG, RB,RD	EG, RB,RD	EG, RB	EG, RB	EG, RB,RD

2023-10-02 meeting

Agenda:

measure of success: number of partners and registrations
- Depending on what we decide, to go beyond last year we might need to postpone. (Last year registrations started at end of September and course was in last week of November)
  - Having more committed partners would improve other aspects of future workshops / other activities.
Course structure:
- 4 days x 3h/day.
- Online only
- Maybe no zoom this time? Or zoom only at the end for interactive Q&A/mentoring? Or just the zoom afterparty on last day?
  - Also uppsala no zoom
Feedback from last year
- https://hackmd.io/@coderefinery/python2022archive & https://hackmd.io/@coderefinery/python2022archive2
- day 1: numpy under the hood had positive and negative feedback (people missed a basic numpy intro). Maybe people come without pre-checking things, even though they were told to?
- day 2: more hands-on, less talk (e.g. data formats could be shrinked), running out of time
- day 3: too packed with topics… maybe some reshuffling?
- day 4: more hands on
- on avg: about 30% found it too fast
Ideas for schedule changes
- Move jupyter and binder closer together?
  - on second thought it makes sense to have binder next to dependency management. jupyter is taught as basis to all lessons.
- dependency management earlier? (everybody needs it for any Python development and if it is at the end some learners might have left?)
  - Sabry taught last year, but gone last day Teach second to last day?
  - go after library ecosystem.
- numpy: both basic and advanced lessons are in the materials, the instructor could balance between the two. Better to cover few sections with more details and leave things for self study.
  - RD: +1 to change, alternate years of basic and advanced numpy (basic this year).
  - Decision: yes, do basic numpy. Tease advanced a bit. If it's too basic people cat mute stread and go to advanced.
- data formats
  - Don't teach this year. Integrate to pandas/etc and point to the episode.
- web APIs
  - No burning desire to keep it, can be special topic.
- Add "productivity tools"
  - Added

Partners

A partner is an organisation that has to satisfy A OR B:

create an event page in their "official" webpages/calendars (can link to main page)
email their local users to advertise the event
receive very few emails related to installation issues that can be organization specific

Has an affiliated instructor joining the event

Optional:

partner is also welcome to help with the notes document during the event, but this is not a requirement. (You'll learn a lot and can join as needed)
partner can help with social media communication (very appreciated!)
co-teach (or teach) - co-teaching is a low threshold way to get started!

What does a partner get?

Logo in the advertisement
explicit mention that this is a joint effort
impact for their key performance metrics (e.g. to show supervisors/deans/etc how successfull the course was)
impact for their users (better skills, more advanced output, less people coming and ask about python)

End of the day

Feedback, day 1

Tomorrow, we continue with Pandas. Then, visualization (matplotlib), data formats and productivity tools! See you tomorrow!

Today was:

too fast: o
too slow: o
just right: o
too simple: o
too advanced: o
was worth attending: o
I will recommend to others: o

Mention one good thing about today (any lessons):

Mention one thing to be improved for next time:

Other comments:

Email end of day 2

Hello again!

We hoped you enjoyed some Pandas, plots, and other discussions on best practices today. If you did not have time to join, you can already re-watch the streaming at https://www.twitch.tv/coderefinery. Tomorrow, we will go deeper into how to efficiently use Python in your research with scripts and reproducible virtual environments. It is ok if you did not have time to join today or yesterday since these episodes are independent from what we have covered so far.

For preparation: tomorrow we will use the command line more. Reviewing the shell crash course would be useful if this is new to you: https://scicomp.aalto.fi/scicomp/shell/ (video at https://youtu.be/56p6xX0aToI)

The interactive questions from today have been archived at
https://hackmd.io/@coderefinery/python2023archive

Today's videos with subtitles will appear later today at our YouTube channel https://www.youtube.com/channel/UCNErdFO1_GzSkDx0bLKWXOA

See you tomorrow at 8:50(CET)/9:50(EET)!
Aalto Scientific Computing/CodeRefinery

–
ps: privacy! you receive these emails because you registered. If your plans have changed, we can remove your email; just let us know by answering to this email.
pps: if you need to get in touch directly, email scip@aalto.fi

Email end of day 3

Hello again!

We hoped you enjoyed how to make python scripts and make your python code more reproducible with virtual environments. If you did not have time to join, you can already re-watch the streaming at https://www.twitch.tv/coderefinery.

Tomorrow is our last day together:

we will dive into more advanced topics, something for everyone! Parallelization with python, packaging, and basics of web APIs to connect your code with external services.
One lesson will use the command line, so you might want to revise the Linux Shell crash course https://scicomp.aalto.fi/scicomp/shell/ (video at https://youtu.be/56p6xX0aToI)
We also have an exciting panel discussion where you can "ask us anything."

It is ok if you did not have time to join so far, you can still join tomorrow since the next episodes are mostly independent from what we have covered.

The interactive questions from the live sessions have been archived at
https://hackmd.io/@coderefinery/python2023archive
https://hackmd.io/@coderefinery/python2023archive2

Today's videos with subtitles will appear later today at our YouTube channel https://www.youtube.com/channel/UCNErdFO1_GzSkDx0bLKWXOA

See you tomorrow at 8:50(CET)/9:50(EET)!
Aalto Scientific Computing/CodeRefinery

Last email

Hello!

Thank you for joining us last week for "Python for Scientific Computing 2023". This is the last email from us (although many of you explicitly requested to receive more updates about future courses).

Please spend a few minutes reading this email, even if you did not have time to join the course.

Course survey: it is important that we hear from you even if you did not have time to join the course. This anonymous survey helps us deliver a better course next year, and adapt to your needs. Take the survey at this link: https://forms.gle/kR4K9qWTqNSiXhHv7
Videos and notes from the lessons: All videos with subtitles are available on Aalto Scientific Computing YouTube channel https://www.youtube.com/playlist?list=PLZLVmS9rf3nNI3oQEqSJW6yXltOAZnkpa The notes from the sessions are archived at: https://hackmd.io/@coderefinery/python2023archive
Credit / certificates: for those who need the credit/certificate there is a "fast track" submission deadline this Friday 17/November/2023. Read more about the required homework at https://scicomp.aalto.fi/training/scip/python-for-scicomp-2023/#credits
Where to go from here? The best way to learn a programming language is to have a reason for writing code, possibly daily. A nice way to engage on daily programming tasks is the https://adventofcode.com/ daily challenges in an advent calendar format which will start on 1/December! We have a discussion group on CodeRefinery chat: join us! https://coderefinery.org/join/chat/
Join our mailing lists, if you haven't yet, to hear more about future similar workshops on related topics (linux shell scripting, git version control, reproducibility, python, R, containers…)
- https://coderefinery.org/#newsletter
- https://scicomp.aalto.fi/training/scicomp-announcements-maillist/

Have a great winter!
Aalto Scientific Computing / CodeRefinery