# 2024-11-14

## Agenda

- `componentize-py` topics
- Discuss logistics of [importing libpython3XX.so](https://github.com/bytecodealliance/componentize-py/issues/28) rather than bundling it in `componentize-py`-generated components
- [Downloading dependencies automatically](https://github.com/bytecodealliance/componentize-py/issues/25)
- Ongoing maintenance of [wasi-wheels](https://github.com/dicej/wasi-wheels/)
- Updating submodules to more recent versions
- Possibly moving it from Joel's GitHub account to somewhere more visible and "official"
- Next steps to publish WASI wheels to PyPI? Is true dynamic linking a blocker for this?

## Notes

- Challenges when generating components that import `libpython3XX.so`, etc.:
- `componentize-py` currently uses `component-init` to load Python modules and their dependencies during pre-init time, which has the effect of editing `libpython3XX.so` and others, customizing them for use in that specific component. If we started importing those modules instead, we wouldn't be able to customize them, except possibly by applying a memory diff during instantiation.
- [This issue](https://github.com/bytecodealliance/componentize-py/issues/23) describes how we might use `wasi-virt` instead of pre-initialization to bundle Python modules, but that could lead to code bloat by including Python code which the application will never actually use. Also, as of this writing, `wasmtime-py` can't handle composed components, so `wasi-virt` would not be compatible with it (i.e. the `sandbox` example wouldn't work). We could certainly support both approaches, though.
- Even putting aside the question of how to bundle Python stdlib and dependency code, the pre-init step also allows us to minimize runtime cold-start time by doing as much initialization at build time as possible, so we'll want to retain that advantage if possible. Perhaps we could pair each imported `.so` file with a pre-init snapshot in the form of a module which include data segments, global values, and a `start` function which contain and apply the snapshot, respectively.
- Thoughts on downloading dependencies automatically:
- Could just shell out to `pip` to grab pure Python wheels from pypi.org, but would need to redirect to `wasi-wheels` repo for anything that has native extensions (and report an error if a WASI build is unavailable or known to be broken).
- If we wanted to avoid a dependency on `pip` being installed, could use e.g. [uv](https://docs.astral.sh/uv/) as a library (although maybe it's a CLI only?)
- This is somewhat dependent on what the future of `wasi-wheels` might be.
- Brett: publish wheels using simple api; content-type munging can be tricky with a generic static content server. See packaging.python.org for details on simple api; html and json versions. Serve as custom mime type(s) (not application/json). Html version doesn't require custom mime types, though, so might be good first step for a generic static website.
- Joel: can you tell pip to use a different platform spec than the current platform?
- Brett: can point to python. Uv more strict; pip more flexible. Can also specify platform. Might have [a blog post](https://snarky.ca/testing-a-project-using-the-wasi-build-of-cpython-with-pytest/) about getting pytest working with wasi and using pip there. Can tell pip you support pure python with specific python version plus wasi.
- Brett: can tell pip about multiple indexes, but can't specify priority
- Brett: recently added support for declaring that and index trusts an external index on a per-project basis
- Brett: CPython now updated to use `wasm32-wasip1` name
- Brett: green threaded pthreads would be fine

# 2024-10-10

## Agenda

- Status updates
- `dlopen` updates: https://github.com/WebAssembly/component-model/issues/401
- Anything else?

## Notes

Meeting canceled in favor of async discussion on Bytecode Alliance Zulip.

# 2024-09-12

## Agenda

- Status updates
- Brett's [WASI TODO list](https://opensource.snarky.ca/Python/WASI)
- Recent and in-progress `componentize-py` contributions

## Notes

- Brett's TODO list
- 3.13 is rc; final on October 4; too late for WASI changes (but can be added to minor release)
- need to switch to `wasm32-wasip1` since `wasm32-wasi` is being reserved for WASI 1.0
- Will discuss all this at BA plumbers summit
- Python tier support modeled after Rust; p1 will (continue to) be tier 2, but p2 may start at tier 3; will need to to more work to move p2 to tier 2
- Cleaning up emscripten stuff since no longer supported
- Making dev container that includes wasi-sdk, wasmtime, etc. to make it easier for other core devs to debug
- Might help simplify CI
- Mac clean up
- Try to support e.g. Pandas and Numpy with native extentions as static builds
- Joel: note that Pandas has Cython/C++/exception handling challenges
- Brett: Micropython using setjmp/longjmp based on Wasm exception handling proposal
- Zlib, bzip2, etc.
- currently have Windows repo with source code for these libs
- may need adapting for wasi-libc
- Want to create standalone python.wasm that has python bytecode embedded
- Using "freezing": bytecode array in header file; import machinery can use this instead of loading from filesystem
- Joel: could be useful for componentize-py
- wasip2 networking
- not a priority yet since main customer is VS Code
- VS Code wants threads first
- also want threads because most sockets tests in CPython required
- WASIX
- Binary wheels
- Big task; not a high priority
- Joel: what do we need to do to get to an offical WASI builds?
- Brett: true dynamic loading would make it way easier
- Brett: clear guidance on ABI compatibility
- Brett: without dynamic linking hard to explain that WASI is different and the wheels must be used differently
- Lack of runtime code loading also means no JIT; weval and AOT instruction caches may or may not be enough
- Brett: also limited bandwidth, so have to pick priorities
- Brett: with JIT and freethreading, maybe there will be a shift away from native extensions in favor of pure Python.
- Joel: once we have threading, Fortran, and exception handling, I think we'll have a good story for native extensions

# 2024-08-08

Cancelled (because Joel forgot about it until after it was supposed to start; sorry!)

# 2024-07-11

## Agenda

- Status updates
- Whatever Brett wants to talk about (Welcome back!)

Notes

- Joel's status:
- mostly working on Component Model async ABI implementation and .NET stuff lately
- This involved fixing a few things in `wasi-sdk`, which will might making porting CPython to WASIp2 easier once `wasi-sdk` 23 is released
- `wasi-sockets` support has been upstreamed into `wasi-libc` and is available as part of `wasi-sdk` 22 (but only for the `wasm32-wasip2` target)
- published some minor releases of `componentize-py` with bug fixes
- otherwise, no Python related updates
- Brett:
- Had a kid
- Verified Wasmtime 22 works fine with CPython
- Cloudflare decided to do their own thing for dynamic libraries using Pyodide/Emscripten
- Using V8 anyway
- As mentioned before, dropped Emscripten support in CPython and got WASI to tier 2
- Kevin: stuck on old version of Wasmtime, so can't play with components yet
- The person working on it at SingleStore has had other priorities so far
- Wasm not a popular option yet, but components might help
- Brief discussion of logistics of moving CPython to wasm32-wasip2
- (Brett has to leave early due to urgent matter)

# 2024-02-29 - Happy Leap Day!

## Agenda

- Status updates
- [CPython WASI issue list](https://github.com/python/cpython/issues?q=is%3Aopen+is%3Aissue+label%3AOS-wasi)
- Anything else?

## Notes

- Brett: can I turn on `wasi-sockets` in CPython now?
- Joel: not until it's upstreamed into `wasi-libc` (working on it). `componentize-py` is currently using a fork.
- Brett status: shared a link to open issues (see above) updating to WASI 0.2.0. Everything now building, but test failures (e.g. dlopen). Will this be supported without a shim in wasi-libc?
- Joel: _could_ make the tests pass by pulling in `componentize-py` and adding a bunch of test boilerplate
- Brett: will probably just disable tests, but can also upstream `dl.so` code to CPython
- Joel: but it would be useless without `wasm-tools component link`
- Joel: could also propose e.g. `wasi-dynamic-linking` standard
- Brett: need to communicate dependency on external code to make dlopen work
- Brett: wasi-sdk 21 is biggish; prioritizing wasi 0.2.0; leaving on paternity leave; back June 9th

## Action Items

# 2024-02-15

## Agenda

- Status updates
- What to do about SSL/TLS
- Why not just use OpenSSL?
- side channel attacks (no constant time ops in Wasm)
- OpenSSL devs not interested in supporting Wasm anyway
- One idea: `wasi:sockets/tls` (+ emulation layer to make Python's `ssl` module work?)
- Next steps for WASI wheels
- Native extensions -- how do they work?
- Anything else?

## Notes

- Joel: WASI 0.2.0 has been released (yay!) and `componentize-py` is using it (including sockets)
- Joel: [wasi-wheels](https://github.com/dicej/wasi-wheels/) supports a lot more packages courtesy of @chrisdickinson
- Joel: won't have bandwidth for IC work on Python for a while going forward
- Brett: WASI is now tier 2 CPython platform. Test suite not passing with Wasmtime 17 -- fixes coming. Need to disable some POSIXy tests. Also problems with wasi-sdk 21 due to dlopen; looking into it. Meanwhile, have to use Wasmtime flags to run tests correctly.
- Brett: todo list includes wasmtime 18 working and likewise wasi-sdk 18; other ideas: self-contained wasi component/module; single-file dists
- Jordan: building Python env for decentralized, wasm-based cloud. Interested in C extensions. Also working on JS.
- Brett: there's a CPython script for setting up WASI build env which allows you to specify external dependencies. Whether we distribute officially will need to be determined among core CPython devs. Want to make it at least option. Core team hates ssl module. Brett would prefer to use http fetch api. i.e. ssl/tls no longer part of std lib, and make e.g. db drivers use third-party package to do TLS. That would be e.g. 5 years away, though.
- Brett regarding host tls library: the ssl module has a lot of futures and more have been requested.
- Jordan: fetch api sounds great
- Brett: moving to where stdlib has everything pip needs, including fetch (based on libcurl on linux perhaps)
- Brett on WASI binary wheels:
- Figure out platform tag for wasi
- How to deal with non-backwards compatible wasi-libc
- Project called `packaging` which calculates wheel tags which interpreter is compatible with
- How do we coordinate community around specific wasi-libc ABI
- Will need a PEP for all that
- Work with PyPI to enable uploading them
- Tooling to use .so files (i.e. more than just pip install)
- Ralph says money coming to work on wasi-libc maintenance and address compatibility problem
- cf. manylinux using super old, super compatible glibc
- Joel: wasi-lib maintainership has been a challenge for getting PRs merged, too
- Brett: can get started now and just choose wasi-libc.
- Joel: can we use wasi-libc 22 for all of WASI 0.2.0's lifetime
- Brett: if we don't even have binary compatibility for bugfix releases that's no good.
- Brett: need to know wasi-libc version via e.g. header directive

## Action Items

- Joel: figure out wasi-libc maintainership plan

# 2023-12-07

## Agenda

- Status updates
- `wasi-sockets`
- Emscripten/WASI convergence
- Anything else?

## Notes

- Brett: Tacit approval from steering committee to make WASI tier 2 for CPython. Looking to make a dev container for devs.
- Joel's status:
- Updating `componentize-py` to latest WASI Preview 2 RC (will release when Wasmtime 16 is released, i.e. late December)
- Also updating to Python 3.12 (pretty easy)
- Implementing `wasi-sockets` support in `wasi-libc`, Rust `std`, and CPython
- Check this out: https://github.com/bytecodealliance/componentize-py/tree/update-all-the-things/examples/tcp
- Brett: Python's network test suite is awesome, so use that for `wasi-sockets`
- Not sure if the tests are turned off for WASI (and thus need to be turned on); need to find out
- Brett: tools/wasm/wasi.py makes compilation and running test way easier. Read the readme. Uses host-runner env variable, etc.
- see also devguide.python.org
- Dan G. floated idea of converging Emscripten and WASI
- Brett: may be getting MS help on wasi-libc to help with ABI compatibility, etc. Addresses concerns in Python community regarding forward/backward compatibility.
- Robin: MS is showing up a lot, which is awesome
- Brett: Ralph pushing cloud, VSCode and .NET doing their things independently, each with their own budgets
- Robin: even if it takes a year before we get help, we'll love to have it
- Brett: coming back as an IC after paternity leave -> more time to contribute directly

## Action Items

# 2023-11-09

## Agenda

- Status updates
- Best way to control resource lifetimes in `componentize-py` (AKA please educate Joel about Context Manager and `with` statements)
- SciPy and Flang 17: An answer to our Fortran woes?
- Anything else?

## Notes

- Joel update: new version of `componentize-py` supports WASI Preview 2 snapshot 0.2.0-rc-2023-10-18. Working to update `wasi-virt` to use same so we can use it to provide e.g. a virtual filesystem etc. Planning to add sockets support to `wasi-libc` starting next week or the week after.
- Kevin: haven't gotten around to adding WASI stubs to `wasi-libc` yet; will be out most of November
- `with` statements: must provide `__enter__` and `__exit__` methods in class you want to use. Not clear how to match up the `__enter__` part with resources; can we just do nothing there?
- SciPy and Flang 17: nobody knows of a reason why it couldn't work for Wasm -- just need to try and see what happens

## Action Items