Joel Dice
    • Create new note
    • Create a note from template
      • Sharing URL Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Customize slides
      • Note Permission
      • Read
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Write
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Engagement control Commenting, Suggest edit, Emoji Reply
    • Invite by email
      Invitee

      This note has no invitees

    • Publish Note

      Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

      Your note will be visible on your profile and discoverable by anyone.
      Your note is now live.
      This note is visible on your profile and discoverable online.
      Everyone on the web can find and read all notes of this public team.
      See published notes
      Unpublish note
      Please check the box to agree to the Community Guidelines.
      View profile
    • Commenting
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
      • Everyone
    • Suggest edit
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
    • Emoji Reply
    • Enable
    • Versions and GitHub Sync
    • Note settings
    • Note Insights
    • Engagement control
    • Transfer ownership
    • Delete this note
    • Save as template
    • Insert from template
    • Import from
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
    • Export to
      • Dropbox
      • Google Drive
      • Gist
    • Download
      • Markdown
      • HTML
      • Raw HTML
Menu Note settings Versions and GitHub Sync Note Insights Sharing URL Create Help
Create Create new note Create a note from template
Menu
Options
Engagement control Transfer ownership Delete this note
Import from
Dropbox Google Drive Gist Clipboard
Export to
Dropbox Google Drive Gist
Download
Markdown HTML Raw HTML
Back
Sharing URL Link copied
/edit
View mode
  • Edit mode
  • View mode
  • Book mode
  • Slide mode
Edit mode View mode Book mode Slide mode
Customize slides
Note Permission
Read
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Write
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Engagement control Commenting, Suggest edit, Emoji Reply
  • Invite by email
    Invitee

    This note has no invitees

  • Publish Note

    Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

    Your note will be visible on your profile and discoverable by anyone.
    Your note is now live.
    This note is visible on your profile and discoverable online.
    Everyone on the web can find and read all notes of this public team.
    See published notes
    Unpublish note
    Please check the box to agree to the Community Guidelines.
    View profile
    Engagement control
    Commenting
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Suggest edit
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    Emoji Reply
    Enable
    Import from Dropbox Google Drive Gist Clipboard
       owned this note    owned this note      
    Published Linked with GitHub
    Subscribed
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    Subscribe
    # 2025-11-13 ## Agenda - WASIp3 support in `componentize-py` 0.19.x :tada: - Thoughts/question/concerns about the [pre-PEP for WASI support for CPython](https://discuss.python.org/t/draft-pep-on-defining-wasi-support/104758) ? - Semver stuff - Anything else? ## Notes - Joel: `componentize-py` 0.19.x supports component model async and WASIp3 - Note that WASI-SDK itself doesn't yet have WASIp3 support, so anything you do through the standard library will still use WASIp2 - To actually use WASIp3 interfaces, you'll need to use the generated bindings directly for now - Brett: regarding PEP: just read Specification section -- the rest is for the steering committee - Aim to get WASI-SDK version chosen by first beta of CPython release, with some wiggle room - alpha1 of next release happens soon after final previous release - beta1 around mid-May; that's when version branch created - rc1 in August -- no ABI breaks in general - final in October - 2 month cadence for minor releases - upshot: need WASIp3 support done by next May - could ask for exception if we miss that - threading will need to be turned on in CPython once wasi-libc has it - Joel: configure option? - Brett: ideally auto-detect - Erik regarding semver: virtualizing wasi:filesystem, etc. - wac was confused by mix of interface versions - hit resource conflicts; each version of each interface's resource types are distinct from other versions of that interface - https://github.com/WebAssembly/component-model/issues/534 - Brett's recent work: - dev container available - above PEP - build script (and thus container) now more version-independent - got a couple more people interested in helping out - trying to make updating wasi-sdk easier and have multiple installed without conflicts - after PEP is in, will discuss binary releases # 2025-10-09 ## Agenda - Recent `componentize-py` updates and plan for WASIp3 support - Wanted: [wasi-wheels](https://github.com/benbrandt/wasi-wheels) champion/maintainer - wac, componentize-py, and resource-type conflicts - Anything else? ## Notes - Joel: compentize-py updated to latest CPython RC and latest wasi-sdk - Brett: can't upgrade from wasi-sdk 25 until pthread stub bug fixed. Seems to be fixed in main; should be part of wasi-sdk 28. Could make 28 the official sdk for 3.14. - Joel: wasi-wheels needs help! Ben has limited cycles for it going forward. - Brett: planning to write Peps for official build toolchain and for an official wasi platform tag - Brett: hopefully get it into ci-buildwheel to make it easy for community - Joel: wasi-sdk doesn't support c++ exception yet; issue opened; blocker for Cython (e.g. runtime) - Brett: go bluejays - Paul: tried numpy 2.x also; same issues - Brett: freethreading should lead to fewer native extensions :crossed_fingers: - Paul: Fastly working on projects which will eventually rely on pthread pseudo-threads - Sy and Luke working on threading impl - Erik: starting with simple, limited wit world; can't provide all the stuff a normal componentize-py-generated component wants (e.g. filesystem I/O). - Paul: like the idea of stubbing via composing. - Joel: wasi-virt? Erik: haven't looked closely; no maintainer? - Paul: semver related? Joel: could be - Erik: manually getting around semver stuff by being explicit about matching up - Paul: will look at wasi-wheels - Ramon: caching would help with CI - Paul: eventually upstream will take over, hopefully - Maxim: also hit resource id mismatch issue during composition - https://github.com/bytecodealliance/wac/issues/149 - https://github.com/golemcloud/golem/blob/cd9c4f9ce739ab88c111cdd8b6c212c4c702bbb9/golem-common/src/model/agent/extraction.rs#L15 - Roman: setting hash seed explicitly might skip seeding from host? - Brett: maybe? - Paul: could also use custom adapter for stubbing - Joel: hopefully adapters going away soonish - Brett: Recent Py dev sprint; presented on wasi; got approval to make wasi binaries part of release process - Once WASIp3 out and have a PEP, will start doing the release binaries - Also add libz, etc. (everything but ssl) - Luke suggested skipping p2 and going to p3. Also suggested building component instead of module. - Created a container image that has all relevant wasi-sdks in it - TODO list if you want to help: https://opensource.snarky.ca/Python/WASI/Plans - p3 should have sockets because it will have threading? - Joel: 0.3.0 probably _won't_ have threading, but 0.3.x - Joel: still hoping for ssl-on-wasi-tls maybe? - Maxim: no updates on urlib-on-wasi-http patch - Brett: will get there - Ramon: might be held up by all the generated code - Joel: need an official wasi wheel with pre-generated bindgs - Brett: need a fetch api in standard library, really - # 2025-09-11 ## Agenda - ??? ## Notes - Erik: interested in maybe updating componentize-py to 3.13 or 3.14 - Joel: reach out to Ramon Klass and/or Ben Brandt who I believe have experimented with that - Joel: currently using forks of CPython and WASI-SDK; should be able to switch to stock WASI-SDK, but will need to port the (tiny) CPython patch # 2025-08-14 (Cancelled due to no agenda) # 2025-07-10 ## Agenda - Should we have a canonical place for WASI bindings to live (e.g. a package on pypi.org)? - See [this urllib3 PR](https://github.com/urllib3/urllib3/pull/3593) for example, which assumes the bindings live in `wit_world` (i.e. the default place `componentize-py` puts them in) - In Rust, we have [a `wasi` crate](https://docs.rs/wasi/latest/wasi/), which packages use to share types rather than generate their own bindings ## Notes - Brett will be at WebAssembly summit at EuroPython - Joel: propose to create official wasi wheel packages can depend on - Maxim: note that componentize-py currently adjusts names to avoid conflicts - Brett: if BA published an official crate, people would use it - Should be an optional dep - Python is pretty grassroots; nobody will balk at this idea - Brett: would like to add a fetch feature to std lib - Nobody uses existing std lib http feature; use `requests`, `urllib3` etc. - Maxim: no feedback on PR yet; using fork internally now - Brett: not a priority for maintainers, but they'll get to it - Maxim: focusing on JS/TS ecosystem for now # 2025-06-12 (Cancelled due to no agenda) # 2025-05-08 ## Agenda - Joel: updates on action items from last meeting - platform tags - https://bytecodealliance.zulipchat.com/#narrow/channel/219900-wasi/topic/Platform.20tags.20for.20packages.20targeting.20WASI - sdk version header - https://github.com/WebAssembly/wasi-libc/issues/490 - https://github.com/WebAssembly/wasi-sdk/pull/487 - https://github.com/WebAssembly/wasi-libc/pull/534 - bytecodealliance/wasi-wheels - https://github.com/bytecodealliance/governance/issues/41 ## Notes - Joel's action items - platform tags: see Zulip discussion linked above - do want wasm32/wasm64, wasi version, wasi-sdk version - do _not_ want world name (will address that with optional imports) - e.g. `wasm32_wasi_0_2_2_sdk24` - sdk version header: see issue and PRs linked above and comment if you have thoughts (especially related to how Python would use it) - Joel will add comment to issue and CC Brett - donating wasi-wheels to bytecodealliance - See link above for example of issue to open on governance - Ben will do this when the project is mature enough - Joel would like to deprecate his version of the project when we're ready - NumPy 2 has switched to different build system; will require different approach - Brett: probably SciKit; CI build wheel should help - Ben: looks like Meson - Ben: will try forking CI build wheel and try to get something working - Brett: ideally add wasi to CI build wheel's matrix - Brett: Meson folks are easy to work with - Working on supporting other projects - Brett will start PEP when time allows - Coauthors welcome - Ben might have time to make a draft, too; no specific timeline - Brett: spent yesterday putting out CPython 3.15 fires - There is now a 3.14 branch. There is now a tools/wasm/wasi directory; refactoring build to separate emscripten and wasi. `Wasi.py` file will go away in five years. - Ben: already have different code paths for 3.12, 3.13, 3.14, etc. - Brett: working on self-contained python.wasm binaries using freezing - Brett: now doing WASI Wednesdays (devoting these days to WASI-related work) ## Action items - Joel: comment on wasi-sdk version issue or PR and cc Brett # 2025-04-10 ## Agenda - Ben's items - Where / what to document around wasi-sdk versions for each Python version, and what our decision is here - If it makes sense to target wasi-wheels as a more "official" project and what it would take to get there? - Concrete detail: noticing some (surprising) large wasm sizes from componentize-py, and curious to know where this is coming from ## Notes - `componentize-py`-generated components size and runtime memory footprint - Ben: currently loading a bunch of these components from a registry and using an in-memory cache populated with InstancePre + wasmtime-wit-bindgen-generated `*Pre` items - Seeing 1GB for first instance and +130MB for each additional - Benchmarking and profiling how each step contributes to memory usage -- still in progress - See memory being given back eventually - Using Graphana dashboard to view memory usage at container level - Starting to look at in-process metrics - Joel: will need fine-grained look to see if cranelift is leaking vs. just not giving memory back to OS - Want to document which WASI-SDK versions to use with which CPython - And should this be part of the wheel tag name - Brett: should be a PEP marked as active; will define WASI-SDK compatibility story - Brett could write it or someone else; he'll need to sign off - PEP can be updated for each CPython release - e.g. whatever latest WASI-SDK release is as of CPython first alpha - Just need to decide ourselves - Brett: wheel tags can be defined in seperate PEP - Main idea is to avoid accidentally downloading wrong binaries - Tooling needs to be able to do this automatically - Consider that people might choose to use non-standard WASI-SDK for a given CPython - Do we want to use world(s) as part of tag? - Not sure if such wheels would be accepted on PyPI given specialized workflow with `componentize-py` - Emscripten is going through this process as well and trailblazing - CPython release schedule is annual (final for new release and first alpha of next released in October) - Brett: minor version of WASI-SDK could be problematic for existing automation - 3.12 and earlier: no official WASI-SDK version - 3.13: WASI-SDK 24 - 3.14: WASI-SDK 24 - 3.15 and later: whatever is latest WASI-SDK when first alpha released - Joel: do we want to support building with non-standard WASI-SDK for a given CPython - Brett: might be up to PyPI; CPython policy doesn't affect e.g. PyPI - Joel: ok, sounds like we _should_ put WASI-SDK version in wheel tag - Ben: comparing to manylinux-glibc-version - Brett: tooling does straight string matching for platform tags; no parsing - i.e. can do whatever we want - Brett: need a way to query CPython build for which WASI-SDK used - If we need to add that in future WASI-SDK version, can assume undefined mean WASI-SDK 24 - Ben: do we need to include p1 vs. p2 vs. p3 in wheel tag - Brett: emscripten doing PEP for wheel tags and tier 3 support; can read PEP(s) for inspiration - https://peps.python.org/pep-0776/ - https://peps.python.org/pep-0783/ - Brett: will need to support multiple build bots or multiple SDKs per build bot - Brett can do first PEP in a month or two ## Action Items - Joel will find out if WASI-SDK has C header definition to get WASI-SDK version or add it if not - Joel will investigate whether p2/p3/etc. and/or world name should be in wheel tag - Joel will look at making wasi-wheels an official bytecodealliance project # 2025-02-13 ## Agenda - Ben Brandt `wasi-wheels` update and demo - Anything else? ## Notes - Status updates - Joel: just working on WASIp3 implementation stuff - Ben: `wasi-wheels` work, including 3.12 and 3.13 - Mostly supporting both to prove it's possible - Can PR stuff; will figure out where it should live - Ramon: not a priority for employer currently, but interested in progress - Brett: calendars are hard; PyCon US is coming up - Heads down on standardizing python lock files - Then will catch up GitHub notifications - May have internal user who wants WASIp2 - Continuing to work on DX, e.g. wheel tags - Ben demoing end-to-end wheel index workflow - Looked at GitHub actions for CPython build - Set up CI to build and publish to github.io index - https://benbrandt.github.io/wasi-wheels - Generate hashes for each version for index - https://github.com/benbrandt/wasi-wheels - using repository-version 1.0 for now - see packaging.python.org for repository version history/features - Brett look at github action which uploads to pypi for examples of other types of metadata - README.md shows how to use wheels from e.g. `pip` - See tests/integration.rs for working example of using `pip` with `componentize-py`, and `wasmtime` - Easy to build/publish new versions as they're released - Had trouble with redirects, but not necessary for modern tools - JSON support would require custom server with content-type support, but HTML works for most purposes - pulling sdists from PyPI rather than using submodules - Still carrying same patches to CPython as componentize-py - Joel and Brett: might not need them for building wheels - Ben will try without them - Brett: might end up upstreaming socket support anyway; and maybe (optional) dynamic linking - Brett: need to choose an official WASI-SDK/wasi-libc version to use for Python 3.14 and then encode that in wheel names (and maybe `.so` names) - Joel also need to differentiate WASIp2/WASIp3/WASI 1.0 builds # 2025-01-09 ## Agenda - Ben Brandt update on improving `wasi-wheels` - Anything else? ## Notes - Ben: Created new repo for experimentation (and figure out which forks still needed): benbrandt/wasi-wheels - New `wasi-wheels` command - Using sdist wheel instead of submodules - Can build pydantic and publish whl to github - Can use pip install with appropriate flags (see README.md) to install from github - Then point `componentize-py` to it - Currently file ends with `-any.whl` but should be `-wasi_0_0_0_wasm32.whl`; might need to rename from what Maturin emits - Likewise, that platform tag should appear in WHEEL file inside `.whl` - Built as many versions of pydantic that would work with build script - Want CI to publish package and index updates automatically - Will try other packages besides pydantic once we've figured all the - Brett: explains what elements of `.whl` filename mean (e.g. package name, package version, target Python version, platform) - Must tell pip to allow `any` or `wasi_0_0_0_wasm32` since we're cross-installing - Aside: See Brett's blog post on venvs - `wasm32-wasi` still correct for `.so` file names - Will come up with better tag once we have a clearer idea about different wasi platforms, worlds, etc. we want to support and if/when we have a stable wasi-libc ABI. Need to discuss and make decisions. - Should be able to publish index on GitHub pages using HTML-style rather than MIME-style serving - Brett: should be able to build CPython for wasm32-wasip2 now - Joel: should be able to stop pretending p2 is p1, then (but might need to pass linker flag to bypass `wasm-component-ld`) - Ben: tried to use upstream CPython and WASI-SDK, but hit issues (e.g. needed OpenSSL 1.1 for build Python, but Homebrew doesn't have it anymore) - Brett: should be able to disable that (`--without-openssl`, e.g. `wasi.py configure-build-python -- --without-openssl`) - Let Brett know if `wasi.py` is inflexible or doesn't work or just have questions - Brett: - https://devguide.python.org/getting-started/setup-building/#wasi - https://packaging.python.org/en/latest/specifications/simple-repository-api/ - https://github.com/python/cpython/blob/087bb48acac997c06e69dae25bae2dd75194b980/Tools/wasm/wasi.py#L347-L348 - Joel: does `wasi-wheels` have built-in knowledge of Pydantic build idiosyncrancies? - Ben: yes, took bash script and migrated to Rust, plus shelling out to do venv stuff - Brett: why shell for venv? Can use venv without activating. Can also use `python -c` to avoid shell stuff. - Ben: hit issues with maturin doing cross build stuff; forced to use venv - Ben: for us, just need pydantic as part of sdk so far, so already valuable - Just going from `.tar.gz` to `.whl` is helpful, e.g. pip installing typing extensions - Ben: wondering what ultimate app developer experience should look like - Brett: depends on which tool you want to use (e.g. Poetry, etc.) - Joel: fat wheels (e.g. multiple sets of `.so` in single `.whl`)? - Brett: Yes, can do that with compressed tags, and pip understands that - But tooling and community not generally structured around that - Brett: Pylance might start using wasi stuff for dynamic inspection, which could justify more time to work on it # 2024-11-14 ## Agenda - `componentize-py` topics - Discuss logistics of [importing libpython3XX.so](https://github.com/bytecodealliance/componentize-py/issues/28) rather than bundling it in `componentize-py`-generated components - [Downloading dependencies automatically](https://github.com/bytecodealliance/componentize-py/issues/25) - Ongoing maintenance of [wasi-wheels](https://github.com/dicej/wasi-wheels/) - Updating submodules to more recent versions - Possibly moving it from Joel's GitHub account to somewhere more visible and "official" - Next steps to publish WASI wheels to PyPI? Is true dynamic linking a blocker for this? ## Notes - Challenges when generating components that import `libpython3XX.so`, etc.: - `componentize-py` currently uses `component-init` to load Python modules and their dependencies during pre-init time, which has the effect of editing `libpython3XX.so` and others, customizing them for use in that specific component. If we started importing those modules instead, we wouldn't be able to customize them, except possibly by applying a memory diff during instantiation. - [This issue](https://github.com/bytecodealliance/componentize-py/issues/23) describes how we might use `wasi-virt` instead of pre-initialization to bundle Python modules, but that could lead to code bloat by including Python code which the application will never actually use. Also, as of this writing, `wasmtime-py` can't handle composed components, so `wasi-virt` would not be compatible with it (i.e. the `sandbox` example wouldn't work). We could certainly support both approaches, though. - Even putting aside the question of how to bundle Python stdlib and dependency code, the pre-init step also allows us to minimize runtime cold-start time by doing as much initialization at build time as possible, so we'll want to retain that advantage if possible. Perhaps we could pair each imported `.so` file with a pre-init snapshot in the form of a module which include data segments, global values, and a `start` function which contain and apply the snapshot, respectively. - Thoughts on downloading dependencies automatically: - Could just shell out to `pip` to grab pure Python wheels from pypi.org, but would need to redirect to `wasi-wheels` repo for anything that has native extensions (and report an error if a WASI build is unavailable or known to be broken). - If we wanted to avoid a dependency on `pip` being installed, could use e.g. [uv](https://docs.astral.sh/uv/) as a library (although maybe it's a CLI only?) - This is somewhat dependent on what the future of `wasi-wheels` might be. - Brett: publish wheels using simple api; content-type munging can be tricky with a generic static content server. See packaging.python.org for details on simple api; html and json versions. Serve as custom mime type(s) (not application/json). Html version doesn't require custom mime types, though, so might be good first step for a generic static website. - Joel: can you tell pip to use a different platform spec than the current platform? - Brett: can point to python. Uv more strict; pip more flexible. Can also specify platform. Might have [a blog post](https://snarky.ca/testing-a-project-using-the-wasi-build-of-cpython-with-pytest/) about getting pytest working with wasi and using pip there. Can tell pip you support pure python with specific python version plus wasi. - Brett: can tell pip about multiple indexes, but can't specify priority - Brett: recently added support for declaring that and index trusts an external index on a per-project basis - Brett: CPython now updated to use `wasm32-wasip1` name - Brett: green threaded pthreads would be fine # 2024-10-10 ## Agenda - Status updates - `dlopen` updates: https://github.com/WebAssembly/component-model/issues/401 - Anything else? ## Notes Meeting canceled in favor of async discussion on Bytecode Alliance Zulip. # 2024-09-12 ## Agenda - Status updates - Brett's [WASI TODO list](https://opensource.snarky.ca/Python/WASI) - Recent and in-progress `componentize-py` contributions ## Notes - Brett's TODO list - 3.13 is rc; final on October 4; too late for WASI changes (but can be added to minor release) - need to switch to `wasm32-wasip1` since `wasm32-wasi` is being reserved for WASI 1.0 - Will discuss all this at BA plumbers summit - Python tier support modeled after Rust; p1 will (continue to) be tier 2, but p2 may start at tier 3; will need to to more work to move p2 to tier 2 - Cleaning up emscripten stuff since no longer supported - Making dev container that includes wasi-sdk, wasmtime, etc. to make it easier for other core devs to debug - Might help simplify CI - Mac clean up - Try to support e.g. Pandas and Numpy with native extentions as static builds - Joel: note that Pandas has Cython/C++/exception handling challenges - Brett: Micropython using setjmp/longjmp based on Wasm exception handling proposal - Zlib, bzip2, etc. - currently have Windows repo with source code for these libs - may need adapting for wasi-libc - Want to create standalone python.wasm that has python bytecode embedded - Using "freezing": bytecode array in header file; import machinery can use this instead of loading from filesystem - Joel: could be useful for componentize-py - wasip2 networking - not a priority yet since main customer is VS Code - VS Code wants threads first - also want threads because most sockets tests in CPython required - WASIX - Binary wheels - Big task; not a high priority - Joel: what do we need to do to get to an offical WASI builds? - Brett: true dynamic loading would make it way easier - Brett: clear guidance on ABI compatibility - Brett: without dynamic linking hard to explain that WASI is different and the wheels must be used differently - Lack of runtime code loading also means no JIT; weval and AOT instruction caches may or may not be enough - Brett: also limited bandwidth, so have to pick priorities - Brett: with JIT and freethreading, maybe there will be a shift away from native extensions in favor of pure Python. - Joel: once we have threading, Fortran, and exception handling, I think we'll have a good story for native extensions # 2024-08-08 Cancelled (because Joel forgot about it until after it was supposed to start; sorry!) # 2024-07-11 ## Agenda - Status updates - Whatever Brett wants to talk about (Welcome back!) Notes - Joel's status: - mostly working on Component Model async ABI implementation and .NET stuff lately - This involved fixing a few things in `wasi-sdk`, which will might making porting CPython to WASIp2 easier once `wasi-sdk` 23 is released - `wasi-sockets` support has been upstreamed into `wasi-libc` and is available as part of `wasi-sdk` 22 (but only for the `wasm32-wasip2` target) - published some minor releases of `componentize-py` with bug fixes - otherwise, no Python related updates - Brett: - Had a kid - Verified Wasmtime 22 works fine with CPython - Cloudflare decided to do their own thing for dynamic libraries using Pyodide/Emscripten - Using V8 anyway - As mentioned before, dropped Emscripten support in CPython and got WASI to tier 2 - Kevin: stuck on old version of Wasmtime, so can't play with components yet - The person working on it at SingleStore has had other priorities so far - Wasm not a popular option yet, but components might help - Brief discussion of logistics of moving CPython to wasm32-wasip2 - (Brett has to leave early due to urgent matter) # 2024-02-29 - Happy Leap Day! ## Agenda - Status updates - [CPython WASI issue list](https://github.com/python/cpython/issues?q=is%3Aopen+is%3Aissue+label%3AOS-wasi) - Anything else? ## Notes - Brett: can I turn on `wasi-sockets` in CPython now? - Joel: not until it's upstreamed into `wasi-libc` (working on it). `componentize-py` is currently using a fork. - Brett status: shared a link to open issues (see above) updating to WASI 0.2.0. Everything now building, but test failures (e.g. dlopen). Will this be supported without a shim in wasi-libc? - Joel: _could_ make the tests pass by pulling in `componentize-py` and adding a bunch of test boilerplate - Brett: will probably just disable tests, but can also upstream `dl.so` code to CPython - Joel: but it would be useless without `wasm-tools component link` - Joel: could also propose e.g. `wasi-dynamic-linking` standard - Brett: need to communicate dependency on external code to make dlopen work - Brett: wasi-sdk 21 is biggish; prioritizing wasi 0.2.0; leaving on paternity leave; back June 9th ## Action Items # 2024-02-15 ## Agenda - Status updates - What to do about SSL/TLS - Why not just use OpenSSL? - side channel attacks (no constant time ops in Wasm) - OpenSSL devs not interested in supporting Wasm anyway - One idea: `wasi:sockets/tls` (+ emulation layer to make Python's `ssl` module work?) - Next steps for WASI wheels - Native extensions -- how do they work? - Anything else? ## Notes - Joel: WASI 0.2.0 has been released (yay!) and `componentize-py` is using it (including sockets) - Joel: [wasi-wheels](https://github.com/dicej/wasi-wheels/) supports a lot more packages courtesy of @chrisdickinson - Joel: won't have bandwidth for IC work on Python for a while going forward - Brett: WASI is now tier 2 CPython platform. Test suite not passing with Wasmtime 17 -- fixes coming. Need to disable some POSIXy tests. Also problems with wasi-sdk 21 due to dlopen; looking into it. Meanwhile, have to use Wasmtime flags to run tests correctly. - Brett: todo list includes wasmtime 18 working and likewise wasi-sdk 18; other ideas: self-contained wasi component/module; single-file dists - Jordan: building Python env for decentralized, wasm-based cloud. Interested in C extensions. Also working on JS. - Brett: there's a CPython script for setting up WASI build env which allows you to specify external dependencies. Whether we distribute officially will need to be determined among core CPython devs. Want to make it at least option. Core team hates ssl module. Brett would prefer to use http fetch api. i.e. ssl/tls no longer part of std lib, and make e.g. db drivers use third-party package to do TLS. That would be e.g. 5 years away, though. - Brett regarding host tls library: the ssl module has a lot of futures and more have been requested. - Jordan: fetch api sounds great - Brett: moving to where stdlib has everything pip needs, including fetch (based on libcurl on linux perhaps) - Brett on WASI binary wheels: - Figure out platform tag for wasi - How to deal with non-backwards compatible wasi-libc - Project called `packaging` which calculates wheel tags which interpreter is compatible with - How do we coordinate community around specific wasi-libc ABI - Will need a PEP for all that - Work with PyPI to enable uploading them - Tooling to use .so files (i.e. more than just pip install) - Ralph says money coming to work on wasi-libc maintenance and address compatibility problem - cf. manylinux using super old, super compatible glibc - Joel: wasi-lib maintainership has been a challenge for getting PRs merged, too - Brett: can get started now and just choose wasi-libc. - Joel: can we use wasi-libc 22 for all of WASI 0.2.0's lifetime - Brett: if we don't even have binary compatibility for bugfix releases that's no good. - Brett: need to know wasi-libc version via e.g. header directive ## Action Items - Joel: figure out wasi-libc maintainership plan # 2023-12-07 ## Agenda - Status updates - `wasi-sockets` - Emscripten/WASI convergence - Anything else? ## Notes - Brett: Tacit approval from steering committee to make WASI tier 2 for CPython. Looking to make a dev container for devs. - Joel's status: - Updating `componentize-py` to latest WASI Preview 2 RC (will release when Wasmtime 16 is released, i.e. late December) - Also updating to Python 3.12 (pretty easy) - Implementing `wasi-sockets` support in `wasi-libc`, Rust `std`, and CPython - Check this out: https://github.com/bytecodealliance/componentize-py/tree/update-all-the-things/examples/tcp - Brett: Python's network test suite is awesome, so use that for `wasi-sockets` - Not sure if the tests are turned off for WASI (and thus need to be turned on); need to find out - Brett: tools/wasm/wasi.py makes compilation and running test way easier. Read the readme. Uses host-runner env variable, etc. - see also devguide.python.org - Dan G. floated idea of converging Emscripten and WASI - Brett: may be getting MS help on wasi-libc to help with ABI compatibility, etc. Addresses concerns in Python community regarding forward/backward compatibility. - Robin: MS is showing up a lot, which is awesome - Brett: Ralph pushing cloud, VSCode and .NET doing their things independently, each with their own budgets - Robin: even if it takes a year before we get help, we'll love to have it - Brett: coming back as an IC after paternity leave -> more time to contribute directly ## Action Items # 2023-11-09 ## Agenda - Status updates - Best way to control resource lifetimes in `componentize-py` (AKA please educate Joel about Context Manager and `with` statements) - SciPy and Flang 17: An answer to our Fortran woes? - Anything else? ## Notes - Joel update: new version of `componentize-py` supports WASI Preview 2 snapshot 0.2.0-rc-2023-10-18. Working to update `wasi-virt` to use same so we can use it to provide e.g. a virtual filesystem etc. Planning to add sockets support to `wasi-libc` starting next week or the week after. - Kevin: haven't gotten around to adding WASI stubs to `wasi-libc` yet; will be out most of November - `with` statements: must provide `__enter__` and `__exit__` methods in class you want to use. Not clear how to match up the `__enter__` part with resources; can we just do nothing there? - SciPy and Flang 17: nobody knows of a reason why it couldn't work for Wasm -- just need to try and see what happens ## Action Items

    Import from clipboard

    Paste your markdown or webpage here...

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lose their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template has been removed or transferred.
    Upgrade
    All
    • All
    • Team
    No template.

    Create a template

    Upgrade

    Delete template

    Do you really want to delete this template?
    Turn this template into a regular note and keep its content, versions, and comments.

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password

    or

    By clicking below, you agree to our terms of service.

    Sign in via Facebook Sign in via Twitter Sign in via GitHub Sign in via Dropbox Sign in with Wallet
    Wallet ( )
    Connect another wallet

    New to HackMD? Sign up

    Help

    • English
    • 中文
    • Français
    • Deutsch
    • 日本語
    • Español
    • Català
    • Ελληνικά
    • Português
    • italiano
    • Türkçe
    • Русский
    • Nederlands
    • hrvatski jezik
    • język polski
    • Українська
    • हिन्दी
    • svenska
    • Esperanto
    • dansk

    Documents

    Help & Tutorial

    How to use Book mode

    Slide Example

    API Docs

    Edit in VSCode

    Install browser extension

    Contacts

    Feedback

    Discord

    Send us email

    Resources

    Releases

    Pricing

    Blog

    Policy

    Terms

    Privacy

    Cheatsheet

    Syntax Example Reference
    # Header Header 基本排版
    - Unordered List
    • Unordered List
    1. Ordered List
    1. Ordered List
    - [ ] Todo List
    • Todo List
    > Blockquote
    Blockquote
    **Bold font** Bold font
    *Italics font* Italics font
    ~~Strikethrough~~ Strikethrough
    19^th^ 19th
    H~2~O H2O
    ++Inserted text++ Inserted text
    ==Marked text== Marked text
    [link text](https:// "title") Link
    ![image alt](https:// "title") Image
    `Code` Code 在筆記中貼入程式碼
    ```javascript
    var i = 0;
    ```
    var i = 0;
    :smile: :smile: Emoji list
    {%youtube youtube_id %} Externals
    $L^aT_eX$ LaTeX
    :::info
    This is a alert area.
    :::

    This is a alert area.

    Versions and GitHub Sync
    Get Full History Access

    • Edit version name
    • Delete

    revision author avatar     named on  

    More Less

    Note content is identical to the latest version.
    Compare
      Choose a version
      No search result
      Version not found
    Sign in to link this note to GitHub
    Learn more
    This note is not linked with GitHub
     

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub
        • Please sign in to GitHub and install the HackMD app on your GitHub repo.
        • HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.
        Learn more  Sign in to GitHub

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions
        Include title and tags
        Available push count

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch
        Available push count

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully