Jupyter Real Time Collaboration Public Meeting

About

What: Meeting to discuss on going work on Real Time Collaboration in Jupyter, taking place mostly in this repository: github.com/jupyterlab/rtc

When: Every other Monday, 12:30pm EST

Saul Shanabrook is inviting you to a scheduled Zoom meeting.

Topic: Jupyter Real Time Collaboration Public Meeting
Time: Jun 1, 2020 12:30 PM Eastern Time (US and Canada)
        Every 2 weeks on Mon, until Apr 19, 2021, 24 occurrence(s)
        Jun 1, 2020 12:30 PM
        Jun 15, 2020 12:30 PM
        Jun 29, 2020 12:30 PM
        Jul 13, 2020 12:30 PM
        Jul 27, 2020 12:30 PM
        Aug 10, 2020 12:30 PM
        Aug 24, 2020 12:30 PM
        Sep 7, 2020 12:30 PM
        Sep 21, 2020 12:30 PM
        Oct 5, 2020 12:30 PM
        Oct 19, 2020 12:30 PM
        Nov 2, 2020 12:30 PM
        Nov 16, 2020 12:30 PM
        Nov 30, 2020 12:30 PM
        Dec 14, 2020 12:30 PM
        Dec 28, 2020 12:30 PM
        Jan 11, 2021 12:30 PM
        Jan 25, 2021 12:30 PM
        Feb 8, 2021 12:30 PM
        Feb 22, 2021 12:30 PM
        Mar 8, 2021 12:30 PM
        Mar 22, 2021 12:30 PM
        Apr 5, 2021 12:30 PM
        Apr 19, 2021 12:00 PM
        May 3, 2021 12:30 PM
        May 17, 2021 12:30 PM
        May 31, 2021 12:30 PM
		June 14, 2021 12:30 PM
        June 28, 2021 12:30 PM
        
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/tJAqd-utpzIsEtc2-eGEq4Gyg942YaIdQDCn/ics?icsToken=98tyKuCsqTsoH92cthGERowIB4iga-_xmFhdj7dezlLgJAVCNiTvMbsWYIt-Odzh

Join Zoom Meeting
https://zoom.us/j/94702599683

Meeting ID: 947 0259 9683
One tap mobile
+13017158592,,94702599683# US (Germantown)
+13126266799,,94702599683# US (Chicago)

Dial by your location
        +1 301 715 8592 US (Germantown)
        +1 312 626 6799 US (Chicago)
        +1 346 248 7799 US (Houston)
        +1 669 900 6833 US (San Jose)
        +1 929 205 6099 US (New York)
        +1 253 215 8782 US (Tacoma)
Meeting ID: 947 0259 9683
Find your local number: https://zoom.us/u/aLW5p3hT6

Anyone and everyone is welcome at these meetings. They are a place to form community, keep each other up to date on what we have been up to, and discuss topics that require a higher bandwidth than Github issues. They are sometimes recorded and uploaded to this YouTube playlist.

We use this document both to keep track of the planned agenda for the meeting and to fill in notes during the meeting. These notes are then posted to Github after the fact for reference. Please feel free to add items to any of the sections during or before the next planned meeting.

See the agenda

October 18th

August 20th

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Updates:
  • Kevin Jahns, rust port of YJs! You can do everything in Rust as you can with normal YJs
  • Carlos Herrero. JupyterLab issues addressed:
    • https://github.com/jupyterlab/jupyterlab/pull/11092
    • https://github.com/jupyterlab/jupyterlab/pull/11064
    • JupyterLab now has different users within the same server, and Carlos intend to create a plugin to handle user information that other packages can depend on.
    • Sylvain raises a question about refactoring work, is it worth doing part of 3.x or should it wait? Kevin notes it will be a breaking change.
• Kevin Jahns, a suggestion to set –collaborative=true by default. It has been found to be performant besides one issue about using locks which can be avoided if there is a server component that can handle what previously only have been handled via JavaScript in on the client side.
• A clarification about jupyter server and the two jupyter-server extensions used by Jupyterlab
  • REST endpoints jupyterlab-server package, settings, workspaces – pure json
  • /lab handler and adds to the /static endpoint - jupyterlab package

August 9th

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Where to store the Yjs data for notebooks:
  • Database – Central and transparent to users
  • File – Simple to reason about but makes additional files
• How to authenticate WebSocket connections for RTC

July 26

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
  • Jupyter Community call is tomorrow
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• RTC Issues
• Standing PRs
  • Persist RTC color and names
  • https://github.com/jupyterlab/jupyterlab/pull/10490
  • Jess will continue on this and Calros will also provide a review
• Undo/redo issue
  • https://github.com/jupyterlab/jupyterlab/issues/10694
  • Outpus should probably not be part of Yjs history so that undo/redo do not afffect outputs
  • Carlos will take a look.
• Triaging discussion
  • Have dedicated groups in the jupyterab org teams?
  • "Assign"/promote issue triaging to core members
  • Trying a proposal to do this

July 12

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• RTC with JupyterLite:
  • Docs: https://jupyterlite.readthedocs.io/en/latest/rtc/index.html
• Persistence in JupyterLab with SQLite - Kevin
  • https://github.com/jupyterlab/jupyterlab/issues/10544
• Discussing a clear vision about authentication in JupyterLab. Demo of VSCode - Kevin
• Notification viz. RTC

June 28

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

Gonzalo: I cannot make it to the meeting today.

What's next for RTC? What do people want the next focus to be on (near future)? JupyterLab label.

• Commenting/chat (CodeMirror 6 might block)
  • This JEP may help with a basic commenting implementation
• Enabling offline running
• Rust port (this is being worked on now)
• Persistent user name and color
• What UI-focused work can be done?

Carlos shared https://github.com/jupyterlab/extension-examples/pull/163

June 14

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
  • Jupyter community call is in two weeks!
  • Blog on RTC history released
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Gonzalo:
  • State of docs: (Finally) Working on a PR
• Kevin:
  • RTC color and names PR
  • Rust efforts.
    • Randomized test suite
    • Making progress!
• Carlos:
  • Demo small example
  • State XML model

Standing items

Actions

• TODO: Ask for QHub resources for a persistent JLab for fun.
• Isabela
  • Change RTC Community meeting to the new Zoom link.
  • Rename AMA with Kevin Jahns to "Y Community Meetings".

May 31

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
  • Jupyterhub news: JupyterHub Helm chart 1.0.0-beta.1 is out!
    • https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/main/CHANGELOG.md
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Gonzalo:
  • State of docs?
  • Repo merging: Holding CZI.
• Kevin
  • Progress update
  • David is working on jupyter server to have a POC for simple Github auth.
  • State of Yrs
    • More sutainable.
    • Updates are working. Sync client.
    • New data structures. Ground work. Very performant.
    • Follow progress on: https://github.com/yjs/y-crdt
    • Rust Crate: Alpha with all features 2.5 months.
• Carlos
  • State of drawio work.
  • docs
  • Tony mentioned: https://ipydrawio–46.org.readthedocs.build/en/46/tutorials/designing-jupyter-extensions/index.html
• Jeremy
  • Initial RTC implementation in JupyterLite: https://github.com/jtpio/jupyterlite/pull/109
  • Users can share URLs that look like this: https://jupyterlite–109.org.readthedocs.build/en/109/_static/lab/index.html?room=the-coolest-room
  • Still a couple of issues (empty cell created)
• Tony
  • follow with jupyterlab meetings switch to rtc for notetaking

Standing items

• Jessica
  • State of cursor work
• Kevin:
  • State of RJs?
• Carlos:
  • Demo for the DrawIO plugin?
• Gonzalo:
  • State of repos: Waiting to atchive repos.
  • Documentation status: On hold

Actions

• Ask for QHub resources for a persistent JLab for fun.

May 17

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Jeremy

  • Binder includes the collaborative work by default
  • Docprovider package PR#10256

• Kevin

  • JupyterLab RTC PR merged
  • JupyterLab 3.1 alpha with RTC

• Erik

  • Work on JupyterHub: RBAC features introduced in JupyterHub 2.0 will enable more fine grained access control to its own API. Currently we only have user/admin permissions. you will have rules. Interaction will now be permission controlled.

• Sylvain

  • https://github.com/jupyter-rtc <- should we delete the lab fork?
    • Gonzalo: Yes. Will add to the action items.
  • What about other jupyter-rtc repositories?
    • YES. BUT we might need to hold it while the CZI final document (on the grant that was used to do that work initially) is made.

• Carlos

  • JupyterLab/extensions-examples: creating new documents
  • Jupyterlab-drawio: new yjs xml model for collaboration in the diagram editing extension.

Standing items

• Kevin:
  • State of Automerge work
• Carlos:
  • Demo for the DrawIO plugin
• Gonzalo:
  • State of repos
  • Documentation status

Actions

• Kevin:

  • Continue the rust port. Upate the API in the next 2 weeks. (if time allows for it!)

• Gonzalo:

  • Move to use the Jupyter Zoom Meeting
  • Update the RTC Docs.
  • Archive RTC repo
  • Archive jupyter-rtc organzation?

• Carlos

  • Continue work on DrawIO

• Docs:

  • Update an extension examples to include. See drawio.
  • Add an example using something some else than just text. (XML in drawio).
  • Update extension points on the docs: https://jupyterlab.readthedocs.io/en/stable/extension/extension_points.html

May 17

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• Jeremy

  • Binder includes the collaborative work by default
  • Docprovider package PR#10256

• Kevin

  • JupyterLab RTC PR merged
  • JupyterLab 3.1 alpha with RTC

• Erik

  • Work on JupyterHub: RBAC features introduced in JupyterHub 2.0 will enable more fine grained access control to its own API. Currently we only have user/admin permissions. you will have rules. Interaction will now be permission controlled.

• Sylvain

  • https://github.com/jupyter-rtc <- should we delete the lab fork?
    • Gonzalo: Yes. Will add to the action items.
  • What about other jupyter-rtc repositories?
    • YES. BUT we might need to hold it while the CZI final document (on the grant that was used to do that work initially) is made.

• Carlos

  • JupyterLab/extensions-examples: creating new documents
  • Jupyterlab-drawio: new yjs xml model for collaboration in the diagram editing extension.

Standing items

• Kevin:
  • State of Automerge work
• Carlos:
  • Demo for the DrawIO plugin
• Gonzalo:
  • State of repos
  • Documentation status

Actions

• Kevin:

  • Continue the rust port. Upate the API in the next 2 weeks. (if time allows for it!)

• Gonzalo:

  • Move to use the Jupyter Zoom Meeting
  • Update the RTC Docs.
  • Archive RTC repo
  • Archive jupyter-rtc organzation?

• Carlos

  • Continue work on DrawIO

• Docs:

  • Update an extension examples to include. See drawio.
  • Add an example using something some else than just text. (XML in drawio).
  • Update extension points on the docs: https://jupyterlab.readthedocs.io/en/stable/extension/extension_points.html

May 3

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
• Important announcements (2 minutes)
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• yjs updates
  • nbmodel -> shared models - different collaborative models documents.
  • open binder -> https://github.com/jupyterlab-contrib/jupyterlab-link-share
• PSA: Hiding or showing my video on my display on Zoom
• Move to standard Jupyter Zoom room?
  • change hackpad link
  • change the community calendar link

Standing items

Actions

April 19: RTC Workshop Agenda

This is special two hours workshop

When: Monday 4/19, 12:00pm -2:00 EST

Participants:

20-25 participants

• A. T. Darian
• Isabela Presedo-Floyd
• William Stein
• Gonzalo
• Tony
• Nick Bollweg
• Erik Sundell
• Fernando Pérez
• Carol Willing
• Kevin Jahns
• Pierre
• Eric Charles
• Carlos
• David Brochart
• Matthias
• Jeremy
• Matthew
• Brian
• Sylvain
• Adam
• Bert Robberechts
• Harald Schilly
• Tania Allard
• Chris Holdgraf
• Jessica Xu

Agenda

• [:05-:10] Goal Setting

• [:10-:20] Introduction (Fernando Perez)

• [:20-:40] What is RTC?

  • Jupyterlab Y.js (demo by Kevin Jahns)

  • Architectures (Eric Charles)

• How does this effect ipywidgets? Have we thought about collaborative widgets?

  • Not so far, but it seems like it might need its own work.
  • William Stein: I implemented collaborative ipywidgets in CoCalc Jupyter notebooks, so you can try them there. It's basically separate from the main document sync.

• [:40-:60] Breakout 1 (15 min): How does RTC affect your users? the good, the bad, the ugly…

  • Rooms are randomly assigned, split into groups of 3-4

• [:15-1:45] Topic Focused Breakout 2: We need to ask who wants to be in each subgroup. Ideally 1 person of JLab, Jupyter_server, Jupyter HUB.

  • Rooms are predefined by topic

    • Jupyter Frontend (UI interactions): Collaboration, Commenting and Sharing

      • Background and notes document

    • Authentication and Authorization

      • Background and notes document
      • Add some background on existing options so people can read about them prior to this meeting.

    • Outputs sharing: CRDT and/or Kernel message?

      • Background and notes document

    • Notebook Storage

      • Background and notes document
      • nbformat impact with CRDT history?
      • Store system, retrieval, history snapshnots…
      • Client vs Server Model

• Comments on breakouts and closing

Notes

Introduction

2015 first grant. Some demos using google docs API in 2016. In 2018 Google deactivated that and the whole work got scrapped.

We are really just beginning to consider different things:

• Permissions
• Sharing
• Execution state

Current solutions

• Cocalc (William Stein)
• Colab (Google)

How do we make this usable in classes and pedagogical settings?

How the experience with CoCalc and its users can help in shape the effort of RTC.

It is really interesting to look back in github issues around the notebook. Summer 2011, the idea was. to have a single github issue to ask for requests and bugs. RTC came up almost inmediately, it has been one of the longest standing issues requested by users. USer perspective the main driver: What are they trying to do. What are their needs and their pain points.

Even just starting from this perspective ends up showing a lot of technical issues.

In 2020 with the new algorithms, the technical challenges are still there.

Some users do not want RTC but some of the problms they discuss are related to RTC. It is important to keep this in mind.

Current state of affairs (Kevin)

What is the state of Yjs and what is it by the way?

Please make comments on the current PR

Higher level on architecture (Eric)

What is the state of things?

Breakout Room 1:

Please assign a representative to share highlights from your groups discussion after the break out room session.

How does RTC affect your users? the good, the bad, the ugly…

Group 1

Members:

• Matthew Seal
• Afshin Darian
• David Brochart

Representative: Afshin Darian

Notes

• Depends on how flexible and secure the interface is.
  • Authorization permissions
    • Should be fully enumerated and clearly defined
    • Can arbitrary execution allow anyone to act as anyone else (e.g. steal identity token)?
  • What kernel should each user be calling?
    • Depends on organization (some would want same and some require separate)

Group 2

Members:

• Brian Granger
• Carlos Herrero
• Erik Sundell

Representative: Erik Sundell

Notes

• Brian: git collaboration is painful, so RTC becomes wanted. But what about the need for both?
  • Pure git is reasonable, pure RTC is reasonable, but what about the git+rtc compelxity.
  • Git and other applications are not RTC identity-aware
  • [william stein] we hit this problem in cocalc so i have some experience with it.
    • [erik] william explained that they have a time travel system, both in the RTC realm of changes of a notebook, but also in order to handle snapshots of the filesystem.
• Identity provider for the RTC collaborative spaces
  • Situation: JupyterHub, external Identity provider available
  • Situation: A single Jupyter server

Group 3

Members:

• Isabela
• Kevin
• Fernando

Representative: Isabela

Notes

• Problems of concurrent editing, permission control, are more specific issues beyond what Jupyter already might have.
• The real concern of both accidental and malicious errors that might have huge consequences. We are working in the real file system and that is an intentional choice we need to be accountable for.
• Learning lessons from people that have already done this kind of work. Just because we have some complexity that others do not does not mean that we can't build on more broad foundations.
• You may not be collaborating only with others, but also yourself.
• How might this fit with user requests of better Jupytext support? Could they be different views like Hackmd's edit and preview side by side.

User issues raised:

• Will bring a lot of new users who may not be used to all Jupyter technology/terms/model.
• Dangers of having people who might not know what they are doing accidently causing harm.

Group 4

Members:

• Pierre-Olivier Simonard
• Ivan Gomes
• Carol Willing
• Nick Bollweg

Representative: Pierre

Notes

• "know your audience"

  • Pierre : Students, teachers
  • Carol : same + training in enterprise, people used to the google doc workflow (as opposed to the github workflow)
  • Ivan : echos what Carol said, with a segment of users familiar to github, in the engineering/scientific domain.
  • Nick : mentoring with students, scientific journalists, makers, individual user with multiple screens, librarians and archivists

• How does RTC would impact these personas ?

  • Ivan : Lower the barriers of entry for a class of users very right for being empowered with such tools, lower the overhead of collaboration.
  • Carol : Challenges An increase in complexity of dynamic changes when multiple people are changing the notebook and output artifacts.
  • Nick
  • Pierre : ease the process of teaching, allow 1 to 1 teaching sessions with real time "show and tell", remotely or not.

Group 5

Members:

• Jeremy touloup
• Tony Fast
• William Stein

Representative: William Stein

Notes

• tracking changes over time to mitigate bugs. lots and lots of versions because of bugs and problems
• debugging is hard, "things went weird at revision…"
• the security issues are important to consider.
• bugs amplified
• working with code is different, than narrative
• cocalc has a different systems model
• purely client-side will provide more usage without setup needed.

Group 6

Members:

• Tony Fast
• Eric Charles
• Chris Holdgraf
• Matthias Bussonnier

Representative:

• Matthias

Notes

• RTC in or Outside the HUB.
  • Maybe multiple step, first in hub, then upgrade.
  • Determine what feature we want eventually with RTC between hub and outside world, and make sure we don't make it impossible with a current implementation, but don't force this to be a first step.
  • Most business SaaS services offer the option to create a temporary collaboration point w others outside the org (e.g,. a read-only view, or some other view w/ limited permissions)
• Need to be user centric.
• If you look at enterprise software, ability to break out of the authenticated context and share with the public. Extra challenge to share across authentication boundary.
• Eric see issues in UI on how to send invites.
• What functionality exists for authorization in Hub?
  • Hub had often this discussion about permissions and groups and how to decompose those.
  • There is a group table but it's not used.
  • There are some implementations of "user group permission" (e.g., QHub), but not built into JupyterHub to our knowledge
• Do not depend on hub entirely for authorization
  • Many companies want RTC, but rely on other systems for auth
  • They cannot use an authorization system for this if it depends entirely on JupyterHub services.
  • Hook into existing permission system (Active directory mentioned by Eric)

Breakout Room 2

Focussed topics

Group 1: Collaboration, commenting and sharing

https://docs.google.com/document/d/1CnrwCaX2zqhOdN2Co2mrGuLNmLFLs2UgAlCXwlJEvO4/edit#

members:

Group 2: Authentication and Authorization

https://docs.google.com/document/d/1U9TRpDzkv7IIY3l13rw5Vd3xq0d4DMxpWS5jjjzVovI/edit

members:

Group 3: Output sharing: CRDT/Kernels

https://docs.google.com/document/d/1SxC1ctfJHue5kK9oRyafPZ-2zY2IX7mTke3Xz7V9XoQ/edit#

members:

Group 4: Notebook storage

https://docs.google.com/document/d/1rGqYPrSBWIRLxU4uRRHl2OLOWkVmnYwHoHBI-15ZTKo/edit#heading=h.wumt6j7imwcg

members:

April 5 - Skipped due to holidays

Discussion items

March 22

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda

• Welcome (1 minute)
• Introductions for new folks (2 minutes)
  • Ian Roth
  • Sheyras
  • Chris
  • Jessica
• Important announcements (2 minutes)
  • Next Tuesday 30th communicty call
• Fill in discussion items. Include name or initials on the item to discuss. (5 minutes)
• Discussion items (30-35 minutes)
• Standing items (5-10 minutes)
• Action items (5 minutes)

Discussion items

• [Kevin] Yrs status
  • R: Carlos Bryan, and whoever wants to join. Creating a new PR that is bnackawards compatible. Being able to sync with clients using feature that allows you to populate differences on the YJS document format.
  • Rust Coding help (anyone?) https://github.com/yjs/yrs -> subscribe
  • Python binding work with Pierre? Pending meeting.
  • If you want to help or are interested in Yjs follow Kevin on twitter. He is going to announce community meetings soon https://twitter.com/kevin_jahns
• [G] Current work on https://github.com/jupyter-rtc/jupyterlab
  • We decided to use a different repository, a separated org to do the initial work that has been discussed to allow a less breaking and generic way to implement CRDT into JLab.
  • [Eric] NBModel
    • NBModel package to define a single place to define all definitions and interface using NBFormat.
  • [Kevin] YModel
    *
  • [Kevin] nbmodel integration
    *
• [G] Cross projects Athentication and Authorisation
  • Authentication & Authorization
    • Fernando issue on the general conversation
      • Early implementations of the notebook (10 years ago) sage notebook was out already and their model of execution was very different. Every cell would be run in a separate directory.
      • Jupyter idea is to give the user the usual environment, allowing normal write system.
      • Cocalc (the google model) that create "abstract" filesystem might not work for the Jupyter ecosystem. Documents and execution and how do we handle it.
      • Use cases:
        • Education workflow. Teaching clases with hub. More restricted.
        • High performance computing: They expect a normal computing.
          • Side note: There may also be a notion of underlying group accounts, where users authenticate as themselves but are authorized to use common account.
        • Sensistive government data: Collaboration and security boundaries have to match.
        • Resources that need protection:
          • Notebook files.
          • Differential permitions. Who gets to execute on this kernel? Running your own?, sharing one? Who gets to do the work.
    • https://github.com/jupyterhub/team-compass/issues/380#issuecomment-803262897
    • https://github.com/jupyterlab/rtc/issues/132
  • Jupyterhub discussions:
    • https://github.com/jupyterhub/team-compass/issues/382
    • CZI grant?
  • WIP to document on https://jupyter-auth.readthedocs.io
• [Tony] Goal Setting for RTC workshop
  • who needs to attend (~20-25)
    • It would be good to find a couple of hours to
    • Get the hub community and RTC community together.
      • Jupyter Server
      • Jupyter Hub
      • Jupyter Lab
  • [G] Estimated date?
• [G] Centralized point for RTC questions
  • https://github.com/jupyter-rtc
  • https://github.com/jupyterlab/rtc
  • Other?
  • Enable discussions on a specific repository?
    • Study how it might work.
  • Disbale issue trackers?
  • Create a home page.
  • Move jupyterlab/rtc to jupyter-rtc?
• [G] Dual model for RTC
  • We want to be backwards compatible, by default we are gonna release RTC of bits of RTc that will be available in 3.1 and 3.2 and what we do now is a dual model. ModelDB and the new shared NBModel.
    • We will try to keep them in sync (update sugnals on both)
    • Dual model will stay for 3.x and 4.x, 5.x but we will need to do deprecations method.
  • Deprecation route
    • Once we merge in Jlab we 2will have mor ecvoisibility and propose to de3precate modelDB observable. The signals and extension authors are going to have a headache.
    • Contact extension authors.
      • Discuss on the Jupyterlab Dev meeting
• [Chris] Text-based notebooks - in-scope for this group?
  • Text-based representations of notebooks. Jupytext is great and used a lot.
  • I believe Marc Wouts is interested in upstreaming Jupytext to the Jupyter communitg (https://github.com/mwouts/jupytext)
    • How can he contribute this upstream or make it official?>
  • I don't think this is strictly RTC, but seems related, and wonder if others think this would be a welcome addition
    • How can we bring this topic into the community.
    • Some user interface changes on the JLab side, are related to the idea of full synchronization of different representations of the notebook.
    • How RTC might be the right UI/UX layer to back this.
      • Use case: several notebook opened to be able to edit parts
        • How does Jupytext plug into this? Challenging.
        • Keep all things in sync in all the different displays.

Standing items

• Discuss how to process with PRs on jupyter-rtc to jupyterlab.
• Yrs state: The Yjs port to rust.

Actions

• Add pipes without spaces! on this document.
• Role for hire Fernando.
• Ping Kevin if interested in working with Yjs and Yrs
• Define an structure for the workshop and an agenda to send them to everyone
• March/22: How do we want things to look by then?
• Once nbmodel is working as expected, we can move the JLab work on jupyter-rtc to jupyterlab/jupyterlab. A simple server can be created to test things.
• Contact extension authors.
  • Discuss on the Jupyterlab Dev meeting

March 8

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Discussions between the two teams to join forces
  • Since last meeting 2 weeks ago, Sylvain, Kevin and Eric had calls to define a common path to move forward:
    • The result of the brainstormings is to create a shared-model package (name can be changed) that would expose the collaborative models to be used by other packages. So a single package will be responsible for the notebook, cells, file editor, commenting… models.
    • There is a consensus around using Y.js as the underlying implementation.
    • The goal is to be as generic as possible, but still be specific if it makes sense and is easier. For example, to leverage the current Y.js codemirror binding, we may have a specific code block that would instanciate and maintain that Y.js binding.
    • How the shared-model will be used by the current packages is still under definition/discussion. We have options for that, we just need to experiment and pick the best way.
    • The current work done by Pierre and Eric on Automerge will continue until a first milestone.
• Dicusssion around release cycle:
  • It sounds good to have small steps to ensure backwards compatibility with JupyterLab 3.x releases.
  • We can break the API and make the implementations simpler (removing uneeded components/layers) by JupyterLab 4.
• Identity is transversal to CRDT
  • Definitions across JupyterLab, Jupyter Server and JupyterHub are needed.
  • Sylvain proposes to organize a meeting with a few participants to already discuss those aspects.
  • Tony proposes to organize a 2 hours workshop within one month.

February 22

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Eric: RTC Demo with following features
  • Support multiple collaboration protocols:
    • none and automerge for now.
    • yjs on its way.
    • Any volunteer for microsoft-fluid?
  • Does not break the extensions and partners ecosystem so that e.g. jupyterlab-lsp, Kite… continue working.
  • Deliver faster: We can ship sooner in JupyterLab 3.1 and not wait on 4.0.
  • Good to know:
    • Cell Outputs via Kernel Messages instead of CRDT.
    • Multiuser with Custom authentification (e.g. GitHub): "user messages", "authenticated cursors"…
    • Kernel Permissions on its way.
  • Try out and review?
    • Pull Request this week on JupyterLab repository
    • Online permanent server to test
• Pierre
  • Rust / Python binding Demo for RTC Server
• Jan
  • Microsoft Fluid benchmark
  • Volunteer for the demo with Fluid
• Kevin
  • Progress collaborative editing using Yjs: https://github.com/jupyterlab/jupyterlab/pull/9785
• Discussions
  • Join forces

February 08

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Demo of Collaborative File Editor with shared Cursors ( Eric)

• Demo of Collaborative Editing in Lab by Kevin

January 25

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Demo
  • Demo of Collaborative File Editor ( Pierre)
    • JupyterLab with a Jupyter Server Extension
    • Backend in rust compiled to python
    • Integrated with Contents service on the server.
    • Multi-user with Authentication (e.g. github auth for the demo)
    • Users Status
    • Users Presence
  • Demo of Collaborative Jupyter Notebooks (Kevin)
    • Replace ModelDB with Yjs
    • https://github.com/QuantStack/jupyterlab/pull/1
    • Ask questions in the Quantstack gitter channel or in our Jupyter fork

January 11

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Demo
  • Automerge server in Rust, compiled to Python as server extension, with simple UI TextArea
• JupyterLab Extension
  • WASM loading exception
• Repo folders
  • Lumino, Automerge, yjs sub-folders
• Next steps in short run
  • Base document model with cells, outputs, extensible for comments, commands
  • Authentication (aka identity) and authorisation (aka permissions) in juypyter server
  • Document snapshot to lower the history
• Medium term
  • Need for common api (client and server) to allow different CRDT protocols implementations (automerge, yjs…)

December 14

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Eric

  • Gave CZI demo during EOSS workshops
    • Commenting as feature has been asked
    • See e.g. https://github.com/jupyterlab/jupyterlab-commenting
    • Where to store the comments (cfr. notebook format)
    • Big work done by Tim Georges on the UI design
  • RTC roadmap
    • Details on https://jupyter-rtc.readthedocs.io/en/latest/organisation/deliverables.html
    • WIP Step B : text-area with Automerge, Y.js and Fluid with node.js server
    • WIP Step C to bind Rust with Python to remove node.js requirement
    • WIP Step D Benchmark : Fluid being begin added to the existing Y.js benchmark
    • Feedbacks
      • Authentication is needed (at least segregate anonymous users)
      • If running on JupyterHub, authentication could be delegated to jupyterhub
      • Permissions not in scope for now, however could be useful when "sharing" notebooks
      • Existing PR on permission https://github.com/jupyter-server/jupyter_server/pull/165

• Sylvain

  • YJS and coming QuantStack funding on RTC.

November 30

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

• Eric
  • Wrap-up of JupyterLab RTC implementation based on Lumino Datastore (2020 port of 2019 flavor) - see https://www.youtube.com/watch?v=fRlVawMDVMk
  • Python version of existing libs? Going to rust?
    • Automerge https://github.com/automerge/automerge/issues/285
    • Y.js https://github.com/yjs/yjs/issues/261
    • Fluid https://github.com/microsoft/FluidFramework/issues/4449
• Discussions on making the Jupyter Server able to spwan services (content…), a bit like JupyterHub does
  • Would need to add to the traitlets the ability to create API specs.

November 16th

If you are joining the Jupyter RTC Call, sign in below so we know who was here.

Agenda + Minutes

• [eric]

  • Working on Server side Authentication
    • Pluggable user token creation/validation https://github.com/jupyter/jupyter_server/issues/50
    • Add a Session Management infrastructure for extension developers https://github.com/jupyter/jupyter_server/issues/122
    • Brief discussion of Yjs, CRDTs, and server
    • Discussion of services and hub and if there is an architecture with server to do something similar (possibilities for auth to be a service)
  • Demo at CZI begin December, date to be confirmed.
  • Open Discussion: Idea is to re-implement server extension to collect, distribute (to the other the participants and to the core server content/kernel) the messages

• [nick]

  • leverging existing backhauls codimd, ejabberd, dat, ipfs
  • One-codebase-to-rule-them-all e.g. wasmer, enarx

• jupyter server

  • traitlets, tornado… offer opportunity to depend on other things
  • frankenstein: build into the server a thin layer of logic to allow extensions to tornado to proxy other things as services
    • this is basically the jupyterhub model
    • swap out server to do other thing
    • could rewrite jupyter server in nodejs, go, rust, etc.
    • could make contents, kernels, etc. provided remotely

November 2nd

• Discussions
  • Some participants raise concerns about the recording for themselves but also for other potential participants that may not join because of that.
  • Potential Master Thesis on Jupyter RTC
  • Work on Jupyter Server to bring a multi-user "aware" server
    • Authentication and Authorizations layers are needed
    • Parallels to JupyterHub with a focus on an asset (file) and API to query re: user activity
    • Prototype:
      • import JHub db and added a column for project (i.e. asset);
      • authenticator (who the user is);
      • authorizer (who can access an asset)
    • Can use JupyterHub libraries but should not need a running JupyterHub instance
  • User perspective
    • User mental model is Google Doc.
    • Actually, users want to be able to come into a shared workspace with configuration and setup.
    • Another model Every user would get a NFS share, but permissions is needed (as discussed above).
    • User stories can be shared in 2020
  • Y.js or Microsoft Fluid?
    • GraphQL work paused for now.
    • Experiment with both.
    • https://github.com/jupyterlab/rtc/issues/57 to add Fluid to the Y.js benchmark

October 19th

• Recording

October 5th

• Notes

September 21st

Cancelled

September 7th

• Notes
• Recording

August 22nd

Canceled

August 10th

• Notes

July 27th

• Notes
• Recording: To upload

July 13th

• Notes
• Recording: Forgot!

June 29th

• Notes
• Recording: To be uploaded

June 15th

• Notes
• Recording

June 1st

• Notes
• Recording: To be uploaded