# Jupyter Community Workshop - The Future of Jupyter Widgets
## Tuesday 18th of October - Friday 21st of October, London

## Week Agenda:

[Google Docs Agenda](https://docs.google.com/spreadsheets/d/1KMB3Mda1MNVXzjSm30eNO7qIW7Oj2z3nYBFY5l-sZ0A/edit#gid=998190500)

## Friday 2022-10-21
Zoom link for remote attendees: https://bloomberg.zoom.us/j/95137727255?pwd=OGpmYmI0RkJZelk1V2ErUWoyc2w4QT09

* 8:30-9:30: Breakfast
* 9:30-12:00: Knowledge sharing / Hacking
* 12:00-13:00: Lunch
* 13:00-18:00: Socials / Hacking!

ipywidgets overview
- package overview:
- js
- base/widget.ts -> basic widget class
- base-manager -> inherts from base
- html manager -> based on base manager
- schema -> useful documenation around
- python
-

## Thursday 2022-10-20
Zoom link for remote attendees: https://bloomberg.zoom.us/j/92527562243?pwd=ODFuR2VlZjVia0xLcCtPemRhUkFPUT09

* 8:30-9:30: Breakfast
* 9:30-12:00: Building larger applications using ipywidgets (with discussion of performance and developer experience)
* 12:00-13:00: Lunch
* 13:00-17:00: Hacking!
* 17:00-18:00: Demos
* 18:30 (optional): Impromptu drinks/dinner

## Wednesday 2022-10-19
Zoom link for remote attendees: https://bloomberg.zoom.us/j/97006618705?pwd=SnM3OXdWYUtDbWUwbFZZenY5S2ZWZz09

* 8:30-9:30: Breakfast
* 9:30-12:00: Reviewing the entire ipywidgets stack
* 12:00-13:00: Lunch
* 13:00-17:00: Hacking!
* 17:00-18:00: Demos
* 18:30 (optional): Dinner @ Apulia (https://apuliarestaurant.co.uk/)

## Tuesday 2022-10-18
Zoom link for remote attendees: https://bloomberg.zoom.us/j/93920480098?pwd=SnkvNGlCWTZiUGY4b3EzWlNMMHEydz09

15 Attendees representing academia and industry, with core developers, third-party widget developers, widget users, etc.

* 8:30-9:30: Breakfast
* 9:30-12:00: Introductions and demos
* 12:00-13:00: Lunch
* 13:00-17:00: Discussion about widgets in various frontends
* 17:00-18:00: Demos
* 18:30 (optional): Drinks at The Last Talisman

### Demos

* Maarten Breddels on [Reacton](https://github.com/widgetti/reacton)
* Provides an answer for how to build large applications in ipywidgets, basically a port of React to Python, which uses ipywidgets, etc., as the equivalent of the DOM elements
* Wrapped ipywidgets, and can autogenerate wrapper as long as constructor arguments match the traits. Currently ipywidgets, ipycanvas, bqplot, wrappers are shipped with reacton
* Compare with idom: idom is driven by react. Reacton does everything in Python
* How to create a new wrapped react component? Create a widget, then generate the wrapper in Reacton.
* How do we embed an ipywidget into a React application?
* Maarten does embed into Vue
* We could have better examples for embedding an ipywidgets into existing web applications
* You can embed reacton inside an ipywidgets app, and embed ipywidgets inside a reacton app
* [Max Fordham](https://www.maxfordham.com/): [ipyautoui](https://github.com/maxfordham/ipyautoui) project
* Generates a ui based on pydantic types
* Nate Rush on [mito](https://www.trymito.io/)
* Jeremy Tuloup, Martin Renou:
* JupyterLite
* Jupyter Notebook 7 (based on JupyterLab components)
* [JupyterLab App Cookiecutter](https://github.com/jupyterlab-contrib/jupyterlab-app-cookiecutter)
* https://xeus-python-kernel.readthedocs.io/en/latest

### Discussion: Widgets in Other Locations

* Don Jayamanne on ipywidgets in VS Code
* https://github.com/microsoft/vscode-jupyter/wiki/IPyWidget-Architecture-Challenges
* Details of how VS code handles message passing from frontend to kernel
* Widget echos: https://github.com/jupyter-widgets/ipywidgets/pull/3195
* Pete Blois on colab:
* How colab handles this: https://github.com/googlecolab/colabtools/blob/main/google/colab/output/_widgets.py
* [Finding all dependencies of a set of widgets including jslink](https://github.com/jupyter-widgets/ipywidgets/pull/3114/files#diff-346aff842b126a18502cf33a55d587dbfe6c963c9991cf9f9bb56e205b88e41eR299)
* Issue in ipycanvas where there was a missing ref between widgets https://github.com/martinRenou/ipycanvas/pull/290 resulting in iframed canvases failing to render.

### Issues for working groups

* iframe isolation for widgets
* lightweight kernel manager that can support multiple versions of the widget manager
* output widgets, and how outputs are updated (kernel vs frontend)
* ipywidgets 7+8
* Want to support ipywidgets 7 core controls in ipywidgets 8 widget manager
* Layering the apis in the frontend to separate concerns:
* synchronization
* model instantiation
* view rendering
* Clearly documenting the entry points for AMD modules (like in JLab has metadata in package.json)
* Provide a RESTful end point to expose the list of widgets and their corresponding entry points in JLab and Jupyter Notebook.
* Rich outputs without Widgets (HTML & Comms)
* Freezing part of the stack to give predictability and prevent breakageshttps://github.com/jpmorganchase/jupyterlab_templates
* Testing
* currently have unit tests and some Galata ui tests
* expand to more comprehensive protocol end-to-end protocol tests
*
* ipywidgets in other non-notebook contexts: streamlit, solara, shiny
* ipywidgets as a protocol?
* Moving the documentation (which exists in an npm package somewhere) to the docs
* Should the kernel handle the Output widgets redirection?
* error handling for comm messages that don't get through
* Handling the "comm target does not exist". Should we [improve the jupyter kernel protocol](https://github.com/jupyter/enhancement-proposals/issues/86)?
* [Remove the ipykernel and IPython dependencies](https://github.com/jupyter-widgets/ipywidgets/issues/3514):
* Rely on a Comm modules that's separated from ipykernel: https://github.com/martinRenou/comm / https://github.com/jupyter-widgets/ipywidgets/pull/3533 / https://github.com/ipython/ipykernel/pull/973 / https://github.com/jupyter-xeus/xeus-python/issues/342
* ipywidget author education:
* Finding documentation in the first place
* Custom widget dev tutorials
* Checking for broken links / tutorials
* Run codespell on the docs/code-base
* Linting: https://github.com/jupyter-widgets/ipywidgets/pull/3566
* Packaging: https://github.com/jupyter-widgets/ipywidgets/pull/3587
* async/await: https://github.com/jupyter-widgets/ipywidgets/pull/2779
* What problems are we solving with widgets vs. what we set out to solve:
* devs who rely on the packaging aspect: widgets as a ways to distribute to multiple UIs
* users who rely on widgets: quick and dirty prototypes vs. full applications (or, as a gateway from one to the other)

### End of day Demos

* Matt Craig: Demoed
* Templating extensions:
* https://pypi.org/project/jupyterlab-multicontents-templates/
* https://github.com/jpmorganchase/jupyterlab_templates ([pypi](https://pypi.org/project/jupyterlab-templates/))
* https://github.com/trungleduc/jupyter_app_launcher
* https://jupyterstarters.readthedocs.io/en/stable/

## Wednesday, Oct 19

### Full ipywidgets stack discussion

#### Traitlets discussion
- pydantic (typing) -> value: Syncable[Int]
- Can we rebuild trailets on top of pydantic?
- Outside our ecosystem, so is it worth the risk?
- Perhaps the best bet is just improving traitlets
- We don't have to abondon trailets for widget authors; we can let widget users use pydantic
- Refactor traitlets:
- Performance:
- Creating traitlets with static default values can be improved 20x (maarten has working PR)
- Event processing: a lot of time appears to be spent in `bunch`
- Exposing typing to static typers
- Removing the metaclass that attaches the name to each traitlet
- [pep 487](https://peps.python.org/pep-0487/)
- [Adding typing using dataclass transform](https://peps.python.org/pep-0681/)

#### JS Stack

#### Backbone
- Uses:
- Eventing
- Framework for instantiating and syncing
- Equality comparison (deciding if you need to sync something); _.isEqual
- Lifycycle
- State of Backbone as of October 2022:
- There is a pinned issue on the repo stating the project is still maintained: https://github.com/jashkenas/backbone/issues/4244#issuecomment-991381707
- There was a release on February 26th 2022: https://backbonejs.org/#changelog
- So there *might* not be a need to fork and maintain a custom Backbone *for now*
- Issues with backbone:
- jquery: exposed as part of the API for backwards compatibilty
- Would be nice to remove this as it's not used by jupyter
- How to remove this without breaking custom widgets, unclear
- $ as a property, first time you access it, it loads?
- We don't fit the syncronization model, have had to strip a lot of this
- Maintainablity: it's fossilized, hard to get changes merged.
- Maarten had issue getting equality checking for typed arrays merged
- The goal perhaps is removing jquery
- It's very hard to do a lazy-load of jquery, as these are syncronous APIs
- Removing jquery is far less of a breaking change than ipywidgets 7 -> ipywidgets 8
- It's like 50kb for 2022. The modern browser versions are super small, as they punt to the browser for everything
- What's even the goal of getting rid of jquery?
- The biggest thing you get is the a simpler interface - reimplementing widgets is easier
- Options:
- Fork backbone, have more control, and we can get rid of jquery
- Leave in all the syncing stuff, etc - in case people rely on this
- Could remove the collections, etc
- What would it take to drop backbone?
- If we had lumino widget class from the start, would we have used backbone?
- Backbone has a more nuianced event system: e.g. the value change events, there's more advanced logic here
- Changing the event handling is high-risk
- e.g. ipythreejs relies on specific event ordering
- Forking the portion of backbone that does this event handling?
- Issues with typing? Only for when widget authors upgrade, so not the worst sort of breaking change
- Could we just have a strategy for when backbone fails us?
- E.g. if you could put in a custom equals, this typed array equality comparison would be ok
- Deprication messages?
- When you use $ on the page
- In webpack, we could log a warning
- On ipywidgets 7 -> 8:
- This could be a test case for if widget authors are willing to upgrade
- Potentially, the upgrade fatigue will just build up
- Maybe we should insist on being future compatible and backward compatible

#### Lumino

- lumino widget
- processLuminoMessage
- Can we rely on the browser APIs?
- before attach, after attach
- onresize, presumably you could shim this to the the browsers resize event w/ ResizeObserver
- Do widgets need to use lumino?
- Not necessarily
- Questions around keeping the API backward compatible + a smaller subset stable API?
- A working group looks at the API, decides what a minimal API looks like for building widgets. Can we build a large portion of the widgets going forward just with this smaller API?
- Two approaches:
- Slimming down the widgets API
- Starting from scratch
- Lumino is going to be around for as long as we use it to build the frontends ; there's less concern than JQuery

- Working group on widget API changes:
- Option 1: removing things (e.g. jquery, Lumino) from the API
- Option 2: reccomending a specific subset of the API that will be long-term maintained
- Option 3: keeping the API the same, and then adding a new API that is long-term maintained
- Etc.
- Dicsussion on the above working group:
- removing things from existing API gets less pressing if we introduce a new minimal subset API that users can rely on
- although one of the things we're loosing is composability of widgets if we just have this minimal API
- How much composition of indivigual outputs really occurs?
- The layout widgets are extremly common
- if different people do it different ways, then there will be libraries that effectively do this widget management
- The hard part of the minimal API is the composability of the widgets:
- But if both widgets are syncing back to Python, then you can still do the side-by-side widgets
- But this is where jslink becomes an issue, but maybe this can be exposed as an API
- How do widgets communicate on the frontend:
- Pete argues that this different frameworks do this differently, and trying to be opinioned on this in a long term contract is a loosing game.
- Don points out that this means that widgets interoperating requires widget authors to opt-in in ways that they don't
- Plan: punt the discussion to this afternoon, and come up with a concrete proposal
- A group of people have a more concrete proposal:
- Here's the API
- Here's how we handle embeding a widget inside another widget
- Here's how they talk to eacother on the front-end / backend
- Here's how a widget would work with the current stack
- Here's how a widget that doesn't work with the current stack would work (and here's how it communicates with the current stack)

#### Webpack

- This is an internal detail of widgets:
- We could benefit from using another technology
- esbuild:
- Does es6 module output
- We put a checkmark on this, and move on. We're not gonna consider this one now

#### AMD modules

- Current cookiecutter, there are currently a federated module for jupyterlab, and then two amd modules (for notebooks and for cdn) that look pretty much the same.
- Options and benefits:
- es6 modules are the modern one, most things are moving to this. The primary benefit is that it's a bit more standarized.
- When loading as an es6 module, async dependencies, this is standardized.
- This is the fully standardized API.
- Potentially prefering this becuase this is the browser standard
- What would a managed transition to es6 look like?
- What steps are the breaking changes, how to make them less painful?
- If we're going to go to the minimal API, then we can leave AMD modules in the old API
- Aka, one option is just adding the es6 as another module to the webpack
- What is the carrot for widget authors? This allows them to load these modules from a CDN (jlab would need to support this?)
- This question is about: how do widget managers pull in the widgets?
- Don, pete agree that the important this is that we all agree on how to load the widgets
- What non-jupyter notebooks do:
- Collab has their own impl of require js that avoids polluting the global namespace
- Microsoft used to have their own require js for .ipynb, but moved to just having standard require js in the webview last week
- Working group for looking into ES6
- Answering two questions:
- What are the benefits, if any?
- Is it feasible to do it?
- Major question around external modules - need some external configuration in the bundler or something
- If there are benefits, and it is feasible, then could consider adding es6 modules as another bundle?
- Grant Nestor had a demo of widget modules as es6 modules

- What would you love to see in widgets?

### Minimal Interactive Outputs API
- Previous brainstorming: https://github.com/Quansight-Labs/jupyter-output-spec
- Colab's stable API: https://github.com/googlecolab/colabtools/blob/main/packages/outputframe/lib/index.d.ts
- How can this be integrated into an improved `%%javascript` magic? https://github.com/ipython/ipython/issues/13376#issuecomment-989040562
- Examples: https://github.com/blois/js-module-renderer/blob/master/tests.ipynb

### Working Groups

#### Minimal API

Attendee: Pete, Vidar, Paul, Jeremy, Jason

An es6 module that exports a render method taking a context argument.

A context has a discovery function and functions for:
- comms
- update output state
-

Colab has a comms api:
- `open("target"): Promise<Comm>`

An example mimebundle:

{
  'application/vnd.jupyter.es6-rich-output': 'a string that will be the src of a script tag, i.e., a url, data url, etc.',
  'application/vnd.jupyter.datagrid+json': {'data': ['some', 'data', 'in', 'a', 'structure', 'which', 'the', 'js', 'can', 'access']},
  'text/html': 'fallback rendering of your data',
  'text/plain': 'fallback rendering of your data'
}

The mimebundle renderer for `application/vnd.jupyter.es6-rich-output` does a System.import of the url as an object, then calls the render method if it exists with a context object, awaiting the returned `Promise<void>` before moving to the next output.

The context object has mandatory fields:

- `output`: {'data': {...}, 'metadata': {...}} (which is the display_data or execute_result message data/metadata fields). Changes are not respected - treat it as immutable.
- Should the `transient` data also be passed in???
- `element`: HTMLDivElement

Optional fields:
- `render(output, element): Promise<void>`:
- `output`: a kernel iopub stream, display_data, execute_result, error message (i.e., inputs into the rendering system)
- Should these be formatted for nbformat (e.g., transient field is not included)??? Should they include the trusted field that is passed in from JLab???
- `element`: an HTMLDivElement
- Promise resolves when the rendering callback returns
- `comms`: See Colab interface...
- `Comm` interface
- `open("target")`:
- `sharedState`: Map([@@Symbol]: Any) - shared state that might span render calls or contexts. This is considered mutable, in that a renderer can share things with other render calls, replace implementations, etc.
- Let's explore exactly the scope of shared state (e.g., per kernel, per document, etc.). Does there need to be different scopes, or a way to introspect the scopes, etc.???
-

JupyterLab branch for prototype implementation: https://github.com/jupyterlab/jupyterlab/tree/es6-rich-output

#### Traitlet improvements

- Integration with more natural typing
- ?
- Removing the metaclass
- ?
- Performance improvements

https://github.com/ipython/traitlets/pull/788

### Thursday

#### Building larger applications with ipywidgets

- Initial load performance?
- There are hard limits on the
- Server-side rendering
- Doing some things client-side with a wasm runtime, doing what you need on the backend
- Could we statically figure out which is which?
- Performance:
- Users sometimes end up with hundreds to thousands of widgets, pushing memory usage (mostly on Python?)
- Where are the performance bottlenecks with hundreds of widgets?
- Where is the time spent with creating composition lots of widgets?
- How does creating 100 button widgets compare to creating 100 normal html buttons -- what is the overhead?
- Solara:
- Viola slower than and solara on initial load, due to starting a different Python process per
- Potential benchmarks:
- 100 widget buttons vs. 100 HTML buttons
- Where is memory used for hudreds/thousands of widgets that are all visible?
- On scaling application complexity:
- Why do applications get complex?
- No seperation of concerns
- n^2 communication problem
- Callback hell
- There are no state management libraries for Python
- It's also not clear that these Python users are looking for state management solutions
- What are the user profiles who are using ipywidgets?
- Recent college grad: goes to a bank, starts writing Pythonm uses ipywidgets to create an application. Not a software engineer, etc - it grows organically.
- This is an iterative proces. Takes week if not months to work through issues.
- E.g. memory that hangs around, this is really slow. There is no notion of lifecycle, etc.
- More typical quants: they are using Jupyter for more prototyping stuff
- This conversation is very similar to notebook vs. raw Pyhton files. It's very easy to get to a place in your code development where it's extremly hard to figure out what state you have vs. not, and then u.r.done....
- Potential documentation changes:
- Should we have a section on the application page like: you built your first app.
- Perhaps we should think about it as "how can we allow users to just copy and paste and it just works"
- Potentially a clever opinionated way of a way of building applications:
- Sometimes this gets built internally within companies -
- This could be something that we distribute, and allow users to use
- Ideas:
- Class with inheritance?
- Reacton
- Providing a visual programming interface? We could give a GUI builder
- Concerns about degree of customizability
- Doc changes:
- More complex examples of building complex widget applications
- Testing and typing:
- Good practices for testing (although people might not use it)
- Static typing wherever possible
- Working groups:
- Proper state management for
- Perhaps form input, simple output, that you regenerate
- Show an example of building small components that then transitions to composing them together
- Create a `contrib` organization on GitHub, similar to `jupyterlab-contrib`:
- There could be a list of custom widgets like on the `jupyterlab-contrib` website: https://jupyterlab-contrib.github.io/extensions.html
- The [Jovyan](https://github.com/jovyan) github org is also a place for Jupyter community projects. See https://github.com/jupyter/docker-stacks/issues/358 for more info about the name "Jovyan".
- The name `jupyter-widgets-contrib` could make it more explicit about what it is about? And easier to find via search engines. It would also be similar to the existing `jupyterlab-contrib`.

### Friday

#### Contributing

+ Walk-through of fixing a [bug in ipywidgets](https://github.com/jupyter-widgets/ipywidgets/issues/3597).
+ Installation along the way: wrong (old) version of jupyterlab installed when creating conda environment. Known issues, see: https://github.com/mamba-org/mamba/issues/1581

#### Yjs investigations

+ https://github.com/jupyter-widgets/ipywidgets/pull/3195 already helped quite significantly to keep the state synced between the kernel and multiple frontends
+ *Enabled* by default in 8.x: https://github.com/jupyter-widgets/ipywidgets/blob/e3d8fe576334dc84d24ba8b20b68423e9e72ffe7/python/ipywidgets/ipywidgets/widgets/widget.py#L42
+ *Disabled* by default in 7.x: https://github.com/jupyter-widgets/ipywidgets/blob/892166dd0524e93b912d8826a6da26c09afd4366/ipywidgets/widgets/widget.py#L49
+ Looking into using `yjs` / `ypy` for widgets:
+ probably the simplest for now would be to start a completely separate project rather than trying to replace the internals of ipywidgets directly
+ There could be an ephemeral `YDoc` created in the kernel
+ A first step could be to create a new `Websocket` connection between the web clients (JS) and the Python kernel. This would bypass the current contract of message ordering but could be fine for now for testing. There could be a simple JupyterLab extensions exposing a simple widget like the `IntSlider` via a mimerender extension
+ Widgets state are managed in the kernel via the `YDoc`
+ Ideally use a custom websocket provider over the `comms`?