# JupyterLite Community Workshop 2022 [Paris]

## December 7, 8 and 9 2022

This shared document will be used to take notes during the workshop.
We can also use it to link to external resources and work happening in respective repositories or elsewhere.

At the end of the workshop we will be posting these notes publicly:

- on the Jupyter Discourse: https://discourse.jupyter.org/
- as a discussion on the JupyterLite repository: https://github.com/jupyterlite/jupyterlite/discussions
- feel free to share it elsewhere as well

There will likely be a blog post summarizing the event on the Jupyter blog as well.

## Remote Attendees

- You are also welcome to join the Discord server during the workshop: https://discord.gg/hdkg4df2
- We are also using the `jovyan` Zoom channel for now, but might switch to another one at some point during the workshop: https://zoom.us/my/jovyan?pwd=c0JZTHlNdS9Sek9vdzR3aTJ4SzFTQT09

### Topics (ideas)

Here is a list of topics that can be discussed and worked on during the workshop.
We'll have 3 rooms at our disposal, so we make sub-groups during the workshop to focus on particular topics.

- State of the different projects: JupyterLite, Pyodide, Emscripten Forge, PyScript, Basthon…
- Demos of use cases: Try Jupyter, NumPy, …
- What can be shared between Pyodide and Emscripten Forge? Discuss the overlaps and use cases
- Integrate JupyterLite with Thebe. There has already been some work on the following PRs:
- https://github.com/executablebooks/thebe/pull/569
- https://github.com/executablebooks/thebe/pull/569
See also: https://github.com/executablebooks/thebe/labels/jlite-workshop
- Run Voila in-browser (Voilite). Initial PR for this in Voila: https://github.com/voila-dashboards/voila/pull/1187
- Discuss deployment scenarios and integrations with other services (for example Capytale)
- Improve usability w.r.t loading times and bundle size
mamba in the browser: https://github.com/mamba-org/picomamba
- More seamless creation of environments for xeus-python: https://github.com/jupyterlite/xeus-python-kernel/issues/53
- Tutorial to create JupyterLab extensions that can be used in JupyterLite
- Tools to make JupyterLite deployments easier:
- Ideas for a BinderLite: https://github.com/yuvipanda/repo2jupyterlite-action
- Maybe start thinking about a “hybrid Binder” that can leverage JupyterLite for some repos
- Better error messages for unsupported functionality in WASM
- RTC:
- Update to the 3.6 JupyterLab packages which have improved the state of RTC
- Prototype a note-taking app a la hackmd
- Using JupyterLite as a frontend for software forges
https://github.com/jupyterlite/jupyterlite/discussions/874
- JEP idea: WASM-based architecture for server-side Jupyter server, Hub, kernel
- Language Server Protocol in JupyterLite
- Wheels for emscrtipten/wasm32 platform
- Get a DOI for downstreams to cite
https://github.com/jupyterlite/jupyterlite/discussions/890
- A terminal for emscripten like env (e.g. to explore the file system)
- In browser and out of browser compatibility of WASM specific tooling
- WASM as 1st class platform for the Python ecosystem discussion

### Wednesday 07 December

#### Morning

**Introductions**

- Round of table:
- attendees introduced themselves: who they are, what they are working on

**Presentations**

- Thorsten on Emscripten Forge
- Pyodide
- came out of mozilla (and emscripten)
- started with conda-like approach
- now building wasm32.whl
- can't upload to pypi
- can be installed with `micropip`
- working to unvendor things (`micropip`, `pytest-pyodide`)
- pypi wants... once there is usage
- complimentary to emscripten-forge
- install a few missing packages
- building complicated packages is difficult
- note: conda-build can also generate wheels
- emf: if there is a package with a lot of syscalls
- same problems
- taking pyodide recipes, using same patches
- in teaching environments
- use conda packages when possible
- a few packages from pypi/git
- as in mamba, use `pip:` section
- make pyodide-build versions binary compatible
- working together
- jupyter kernels
- reduced versions in the browser
- "browser api to get queue in the worker"
- thebe
- lightweight connector for any webpage vs a jupyter backend
- originally jquery
- find code cells, make them into code cells
- connect to binder
- also codemirror
- now a core TypeScript library
- anything on the UI frontend
- intentionally simple examples
- activate, gets some status
- textarea, run code
- works with ipywidgets
- connects to jupyterlite server (almost server in browser)
- doesn't work with the serviceworker
- `disabledExtensions?`
- @jtpio: Agree it should be able to work without the serviceworker plugin activated (need to check if this is an issue)
- @bollwyvl: this [PR](https://github.com/jupyterlite/jupyterlite/pull/871) firms that up
- thebe for folk that don't know anything about jupyter
- don't need service worker?
- > bollwyvl: THIS IS WHY WE DON'T PRELOAD PACAKGES
- yuvi started some initial work on using JupyterLite for some Binder scenarios:
- binderlite
- repo2jupyterlite
- detects binder-like files, builds site
- would save deploying full binder
- https://github.com/yuvipanda/repo2jupyterlite/
- voilite?
- hard-coded for pyodide?
- integration with jupyterbook
- put things in yml, get site
- sphinx-thebe wraps thebe
- what is the ux of switching between edit/view
- a site might not be _about_ code, but use code for displays
- wouldn't show you the code
- some curvenote fourier demo
- has tanglejs-like variables and inline widgets, updates variables
- sphinx/myst
- bringing myst, etc into js
- MyST JavaScript
- [Some documentation here](https://js.myst.tools/)
- MyST parsing / rendering / referencing functionality in JS
- Complementary / parallel effort to Sphinx
- Will focus more around scholarly / scientific publishing for now, Jupyter Book focused more on multi-page books.
- in shadow mode for 7 years
- lots of work to rebuild with modern conventions/opinions
- nick
- https://github.com/deathbeds/jupikchr
- try it easily: https://jupikchr.readthedocs.io/en/latest/_static/lab/index.html?path=Pikchr%20in%20Notebooks.ipynb
- extension works in JupyterLab and JupyterLite the same
- made of components built for JupyterLab
- need to show that it works once inside a Jupyter environment
- https://github.com/jupyrdf/ipyelk
- heavy widget
- Try it with (link is from a PR, might not work anymore later): https://ipyelk--107.org.readthedocs.build/en/107/_static/lab/index.html?path=_index.ipynb
- use Jupyter plugins via mimebundle renderers
- maybe something to do with https://github.com/jupyterlab/richoutput-js?
- split JS / Python? nobody wants that
- chris
- executable books
- some cloud thing
- bringing lightweight stuff into books
- wouldn't have to go to a new interface
- binder
- currently launches UI on cloud infra
- not needed for most cases
- spinning up multiple containers is a ridiculous use of resources for a single plot
- need to do that in a way that uses the same protocols and files already supported by repo2docker
- similar interfaces
- save carbon
- building the images is usually the most expensive
- might spend an hour in a session
- mostly doing human stuff (reading)
- using jupyterlite
- the way binderhub works, requests nodes, puts users on them
- fewer nodes would be cheaper
- we switched try.jupyter.org to lite
- dropped traffic by 30%
- proof of principle for avoiding full build experience
- or just keep spending cloud
- Antonio from anaconda
- working on pypy
- pypy.js was a thing (not official)
- compiled to asm.js
- nobody's working on it right now
- [williamstein] I tried pretty hard (and failed) to get this to work last year, and a nontrivial difficulty is it assumes "32 bit Linux", which isn't well supported (a lot in that world bitrotted).
- bundle size
- can we reduce the bundle sizes
- cached by domain
- serviceworker caches by top-level app path (basically) `manifest.json`
- can cache stuff from other sites
- open question: can the service worker be used to serve files out from the kernel? i.e. bokeh generates an html page, can that be served?
- this would allow for self-hosting jupyterlite , e.g. `jupyter lite build` inside a `pyodide` kernel
- single bundle can deploy multiple apps (e.g. retro, repl, lab)
- scipy metapackage
- environment impact of downloading python to do a plot
- CNN is 50mb
- thundering herd of students
- getting students to instll
- browser extension
- switching devices
- cloud providers for content persistence
- learning management systems
- what to do about something that doesn't work in the browser
- user expects stuff to work
- easier to have a crowdsourcing web page?
- different distributions, where would it go?
- worker/non-worker
- people **aren't** going to read the docs anyway
- _you can't use X in Y because Z_
- python is a lot of things to users
- provide nice error messages
- `warnings.warn`
- google results will show up for jupyterlite/etc
- informing developers
- if you have a fixed `numba` dep
- what happens in the browser?
- hide code that's running that's...
- in the beginning, pypy tried to send upstream patches
- when a new technology becomes widespread
- bootstrapping
- dependabot for wasm?
- support needs to be opt-in for library authors
- easier for authors to support wasm
- nice way to **test on github**
- test suites
- pyodide mostly just tries to import
- numpy now has a CI for running the test suite within Pyodide
- better
- having a way to run WASM on the command line to enable testing away from the browser, inc. CI, gh actions, etc...
- see https://github.com/lesteve/scipy-tests-pyodide and https://github.com/lesteve/scikit-learn-tests-pyodide for our current way of running scipy and scikit-learn test suites within pyodide, plenty of issues (Pyodide fatal errors, test failures) to debug ;-)
- wasmer/whatever wrapper for pyolite/jupyterlite-xeus-python
- test kernel functionality without browser
- speak jupyter kernel message
- some wasm is in js
- so you get other errors inside your call stack
- too-old-of browser: `webassembly has tag XYZ`
- pyscript
- presentation by Antonio
- slides: http://antocuni.eu/misc/pyberlin/slides.html
- most of the projects demoed here would really benefit from having a proper loading bar
- emscripten has an undocumented hook/fn which could help @thorsten
- there are some open issues on e.g. pyodide for structured message callbacks
- e.g. in `micropip.install`
- move to webworker
- bidirectional proxy with `comlink` for DOM apis
- https://ashleyscirra.github.io/via.js/
- fake "shadow" (not really shadow) DOM in browser
- RustPython: https://github.com/RustPython/RustPython
- it's difficult to implement an alternative Python that fully works
- usually the first 90% are easy
- also sometimes CPython bugs need to be ported to get the "correct" behavior

#### Afternoon

- JupyterLite architecture @jtpio
- jupyter front end
- e.g. lab/notebook "untouched" (ha!)
- reuse extensions
- overload `ServerConnection`
- patch `fetch` and `websocket`
- these prevent talking to the server
- `mock-socket`
- `JupyterLiteServer`
- plugin-based `@lumino/application:Application`
- "serverlite" extensions for
- contents
- kernel sessions and specs
- settings
- kernels _themselves_ (in place of `subprocess.POpen`)
- js
- pyodide
- basthon?
- worked in lite?, will find out more
- think it runs in main thread
- ruby kernel :tada:
- link: https://github.com/kateinoigakukun/jupyterlite-ruby-kernel
- "serverlite" extension configuration
- where to look for wheels?
- captured in [schema](https://github.com/jupyterlite/jupyterlite/blob/main/app/jupyterlite.schema.v0.json#L171)
- launched in `index.js`
- boilerplate which configures plugins
- mocking `zmq`
- all done over websocket
- doesn't really support client/session ids
- `config-utils.js`
- merges `jupyter-lite.json`, `jupyter-lite.ipynb`, `index.html`
- and parents
- updates `jupyter-config-data`
- which is ready everywhere by `@jupyterlab/apputils:PageConfig`
- WebRTC (in 3.5)
- `@jupyterlab/docprovider:IDocumentProviderFactory`
- overloaded by `jupyterlab-webrtc-docprovider`
- uses a public signaling server
- one _can_ deploy their own
- `@jupyterlab/services:ServiceManager`
- get hooks upstream for
- `addFetchProvider(prefix, ...)`
- `addWebSocketProvider(prefix, ...)`

**Pyodide and lower-level WebAssembly subgroup**

Antonio, Roman, Mariana, Gyeongjae,

**Real Time Collaboration**

Carlos, David, Jeremy

- state of the `jupyterlab-webrtc-provider` extension: https://github.com/jupyterlite/jupyterlab-webrtc-docprovider
- how to handle distributed content since each user has its own copy of the content
- looking into updating `jupyterlab-webrtc-provider` so it builds with the JupyterLab 3.6 alpha packages and make sure there are no breaking changes

**Thebe-lite**

Martin, Steve

- thebe-lite now working against latest jupyterlite version, updated method for packaging wheels
- Short term there going to continue to use requriejs method over federatd modules, but need to move to federated modules, probably along with or after changes to the widget manager to bring it in line with what voile is currently doing (and removing the hacked, dummy context)

### Thursday 08 December

#### Morning

**Recap**

- ways to improve startup time of pyodide
- precompiling?
- placeholders, fetch the js
- progress on optional service worker
- https://github.com/jupyterlite/jupyterlite/pull/892
- updating [DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped) for [`@types/emscripten`](https://www.npmjs.com/package/@types/emscripten)
- thebe
- webpack config for "sensitive" filenames (`.whl`)
- stay with "putting the wheels in a place"
- work with federated build pattern
- serviceworker for kernel filesystem
- loading (from pyodide, etc)
- > nb: this may be on the jupyterlite site
- WebRTC provider
- reconcile the 3.6+ with jupyterlite
- API of document provider has changed?
- This is tracked in https://github.com/jupyterlab/jupyterlab/issues/13551
- need to know the contents through ContentsAPI?
- needs something for Lite to load the contents
- > nb: should be pretty easy: contents are hoisted

**Presentations**

- Capytale and Basthon
- Slides: https://github.com/jupyterlite/jupyterlite/files/10202654/CAPYTALE_presentation.pdf

#### Lunch

Lunch at "Les Fils à Maman Batignolles": https://goo.gl/maps/2jxdiare4mAseMrC6

#### Afternoon

⚠️ On Thursday 08 afternoon we will be at the Hugging Face offices:

**Spaces - Paris, Spaces Réaumur**: https://goo.gl/maps/kQooUksNS8ZjLWLs5

#### Chris work ideas

I (Chris H) found a few issues/PRs where the Jupyter Book and JupyterLite/pyodide ecosystems interact - sharing a few links below in case others would like to work with me on resolving any of these:

- [Support JupyterLite natively in the pydata theme](https://github.com/pydata/pydata-sphinx-theme/pull/1023)
- Follow-up would be documenting how it works in Jupyter Book
- Might also be improvements to sphinx-jupyterlite docs?
- [Basthon integration in the Book Theme](https://github.com/executablebooks/sphinx-book-theme/pull/595)
- Is there a Sphinx PyScript extension?
- Sphinx Thebe / Thebe documentation improvements w/ the latest updates.
- Binder - can we provide guidance to users about when they should use JupyterLite vs. Binder?

#### jupyterlite-lsp

@bollwyvl will be hacking on a [proof-of-concept LSP server](https://github.com/jupyterlite/jupyterlite/discussions/895) for JupyterLite

#### jupyterlab-webrtc-provider

Update the extension to the JupyterLab 3.6 packages: https://github.com/jupyterlite/jupyterlab-webrtc-docprovider/pull/17

### Friday 09 December

#### Morning

**Presentations**

- Basthon by Romain Casati
- https://basthon.fr/
- https://capytale.basthon.fr/pres/jlcw22/
- module whishlist: https://capytale2.ac-paris.fr/wiki/doku.php?id=demandes_de_modules_python

- Round Table: WASM as 1st class platform for the Python ecosystem
- CPython now compiles to WebAssenbly by default on the `main` branch
- How can we avoid making libraries that are only compatible with 1 setup / environment?
- write a PEP?
- write a message on the Python WebAssembly subtopic on Discourse
- wasm extension for 64bits?
- psycopg and psycopg-cffi
- requests in Pyodide: should it keep the same name even if it's not the real `requests`? [williamstein: A reason not to use the same name is that it *is* possible to support the real requests on servers by directly supporting sockets (e.g., this already works in cowasm on nodejs and mac/linux), and in the browser by proxying sockets.]
- issues with renaming
- does it make sense to do it in the same package if the wasm specfic code does not share anything with the original library?
- wasm is considered as a different OS
- it should be easy to run test in Wasm on CI
- `pip install pyodide`, then run test from the CLI with pyodide?
- how to make users aware of the different projects, so they avoid doing the same thing multiple times
- how to search for wasm specific issues on google or stack overflow
- tool to make it easier to run webassembly from the CLI https://github.com/emscripten-forge/pyjs-code-runner
- useful for testing things with a quicker feedback loop
- have tooling that is compatible cross distributions
- wasi, wasmer, wasmedge, Python needs a standard for now we need to choose one
- for Ruby you can just import Wasm
- maybe need the wasm component model?

- thebe-lite
- Now loading the service worker successfully in demo code and added scripts to bundle this for site deployers

#### Lunch

Lunch will be provided at the venue.

#### Afternoon

Attendees split into subgroups to work on focus on specific topics.

Some discussions started on Discourse:
- https://discuss.python.org/t/make-wasm-a-1st-class-platform-in-the-python-ecosystem/21798
- https://discourse.jupyter.org/t/idea-jupyterlite-in-a-usb-stick/17177

There were also some discussions on the Discord server, posting here as reference:

**JupyterLab MySTjs**

New `jupyterlab-mystjs` prototyped in https://github.com/executablebooks/jupyterlab-mystjs

**Pyodide tests**

Loic: Attempts to better get an idea of what is not working in Pyodide (scipy and scikit-learn):

- https://github.com/lesteve/scipy-tests-pyodide
- https://github.com/lesteve/scikit-learn-tests-pyodide

**JupyterLite tutorial**

Pierre and Nicolas worked on creating content for an introductory course on using Jupyter, using JupyterLite.