# "Software Engineering Practices with Python" Course Preparation

"loose guideline": Follow the order in which concerns pop up when starting a new project.

## Submitted Repos
- https://github.com/GridTools/gt4py, standard reasonably organized big pure python project, uses uv
- https://github.com/GridTools/icon4py, monorepo with all complications, uses uv
- https://git.cscs.ch/ws-dwdi/job_reporting, mixed levels of static checking, python primarily as glue code (?)
- https://github.com/spcl/dace, all kinds of wild stuff in there, good old setup.py
- https://github.com/GridTools/jace
- https://github.com/C2SM/Sirocco, young project, uses hatch
- https://github.com/arbor-sim/arbor/blob/master/pyproject.toml, example for scikit-build usage

NOTE: keep in sync with published agenda in Confluence! https://confluence.cscs.ch/x/0JKYMw

## DAY 1 (Foundational Breadth) (Red / Orange)

### MORNING (9:30 - 12:30 w/ 15 mins break)

- Basic Python environment configuration (30 mins) (enrique, rico, tomas)
- Python runtime
* Configuration: env variables, `sys.path`, site-packages, `.pth` files, ... PYTHONPATH
* Running scripts vs modules (-m, effects on interpreter and sys variables)
- Environment management (seek for hands-on / interactive stuff for after lunch)
* example of version clash if there's no venvs
* can you run different python/lib versions
- **EXERCISE**
- Create and activate an environment (to use in later exercises)
- Create 2 different environments with 1 python versions
- Quick introduction to other options available in the ecosystem (but we'll not covered them in detail)
- Conda (why? differences?)
- uv
- spack (why)

- Setting common Python ground (1h45) (enrique, tomas, rico) rotate enrique / tomas
- The Object Model
* Mention: Annotations have no effect at runtime
* What is an object? Everything!
* Mental model for variables (with handout) w.r.t memory model
* Scopes w.r.t variables usage
* `__magic_methods__`
- Attribute access
* Nothing is actually private, embrace it!
* attributes & properties vs variables
- **EXERCISE** `instance.attr` is really running code, not simple data access
- [Imports](https://hackmd.io/Hm08vRY6QmSiQd8oLGleVQ)
* Robust mental model for what import does (with handout)
* Remember imports have side effects (don't import *)
- **EXERCISE**
- spot the problem with broken import?
- import cycle
- monkey patching (import foo vs. from foo import bar)

- 30 mins buffer ??

### AFTERNOON (13:30 - 17:00 w/ 15 mins break)

- Advanced configuration of Python Environments (1h15) (ricoh, enrique, tomas)
- Installing and using Packages
* git clone + pip install
* External packages installation: `pip` and other frontends, editable installation, installation from VCS, version constraints, freezing dependencies
- Basic package creation ??
- **EXERCISE**

- Collaboration (2h) (rico, enrique, tomas) rotate presenter / co-presenter

* organize a project (lib vs apps?)
* IDE-agnostic dev setting sync
* Linting and typing (static analysis)
* Packaging for collaboration
* Project Management
* **EXERCISE**
* Setting up a pipeline with linting? or
* Pushing to a repo with a pipeline without breaking (or fixing) typical collaboration issues
* Typing exercises -> maybe run mypy on code developed without it or show off some pros for collab ?

## DAY 2 (Engineering & Collaboration Depth) (Orange and Up)

### MORNING (9:00 - 12:30 w/ 15 mins break)

- Testing (1h45) (tomas, rico, enrique)

* Creating Testing Environment consistency (i.e. local dev / CI parity)
* Writing testable code: dependency injection principle, single-purpose
* Writing test best practices with Pytest examples
* Mocking and stubbing (connecting to import mental model)
* **EXERCISE**
* Remove repetitions with pytest fixtures? or
* Test a composed function mocking its dependency?
* Adding coverage to a project with tests?
* Adding tests to a project with no tests?
* Paralellization of tests execution

- Pythonic Patterns Thought-provoking Demo + Q&A (1h30) (rico, tomas, enrique)

<!-- find cheat-seet of patter people can look -->

* open-closed dos and don'ts (e.g. respecting public interfaces)
* list comprehensions instead of c-like fors
* generators (iterables) vs lists (yield!)
* dataclasses instead of structs (and extensions with runtime checking)
* attr vs class attr vs descriptors vs methods ??
* decorators & contextmanagers
* exceptions handling (raise&catch) for good citizens

### AFTERNOON (13:30 - 16:30 w/ 15 mins break)

- Advanced Python Features a-la-carte (45 min)
* metaclasses
* alternatives: __init_subclass__()
* code in class bodies (e.g. spack)
* generated code
* pdb + object inspecting with dir et. al
* buffer protocol
* optional: DLPack
* optional: how to use with custom C/C++ cpython extensions

- **EXERCISE** (??)
- Find improvements to submitted code repos where patterns/advanced features can be applied?
- https://github.com/C2SM/Sirocco + guidance where to look for improvements
- work in groups, if available with BYO code

- Advanced Use-cases break-outs
- AI "copilot" tips exchange
- async, multithreading, multiprocessing?
- compiled extensions
- benchmarking??

---

## Topics for next one!

- Profiling
- Debuging