Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Meeting Notes 2022 #15

Closed
Zsailer opened this issue Jan 6, 2022 · 57 comments
Closed

Meeting Notes 2022 #15

Zsailer opened this issue Jan 6, 2022 · 57 comments
Labels
documentation Improvements or additions to documentation

Comments

@Zsailer
Copy link
Member

Zsailer commented Jan 6, 2022 •
edited by kevin-bates
Loading

Hello everyone,

Gitter

Welcome to the Jupyter Server Team!

You can find us in our Gitter chat room or visit us at our weekly meetings!

We meet on Thursdays at 8:00am, Pacific Standard Time on Jupyter's Zoom Channel. You can add yourself to the weekly agenda here. Everyone is welcome!

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

Meeting Notes
@Zsailer Zsailer pinned this issue Jan 6, 2022
@Zsailer
Copy link
Member Author

Zsailer commented Jan 6, 2022 •
edited by kevin-bates
Loading

January 6th, 2022

Attendees

Name affiliation username
Zach Sailer Apple @Zsailer
Jason Weill AWS @jweill-aws
Vidar T Fauske JP Morgan Chase @vidartf
David Brochart QuantStack @davidbrochart
Eric Charles Datalayer @echarles
Kevin Bates IBM @kevin-bates
A. T. Darian Two Sigma @afshin
Mariko Wakabayashi OpenZeppelin @mwakaba2
Sylvain Corlay QuantStack @SylvainCorlay
Rahul Goyal AWS @rahul26goyal
Frederic Collonval QuantStack @fcollonval

Agenda + Minutes

  • Terminals as a server extension (initial work):
  • Jupyter Server Team
    • Name initial core team and add documentation about roles #14
    • An important note for how the Server team is being formed:
      • We (accidently, 😅) side-stepped the official approach, defined by Jupyter Governance, to form a decision making body.
      • This process requires a current steering council member to seed (i.e. nominate an initial member). Together, the gradually growing team continues nominating new members.
      • Large, diverse, and inclusive teams is encouraged.
      • We, instead, self nominated ourselves, proposed by @Zsailer.
      • @afshin mentioned in the meeting that this is might be okay
        • we made a good-faith effort at nominating people
        • the current group that's actively participating is relatively small
        • we arrived at the same results anyways.
      • For other teams who might be observing this process, do not follow our process 😆
    • Changes needed in the PR before merging:
      • Absolve the Team Lead position.
      • Define Software Steering Council Representative (we can vote on this person later).
      • Eliminate team colors.
        • General consensus is that we shouldn't distinguish between red and blue.
      • We just have an "active" team member (voting ability) and a "not currently active" team members (no voting ability).
  • Filtering request using OpenAPI spec: Filtering request using OpenAPI spec jupyter_server#645
    • All handlers are added to the server, but requests from the user can be blocked/allowed by a defined OpenAPI Spec.
    • There is some overlap with the current work happening around "Authorization" in Jupyter Server
    • Is there a way to merge these two efforts so there aren't two ways to filter requests?
    • Not immediate urgency to merge this work; we'll take some time to compare with authorization work.
  • Kernel Parameterization needs a reboot.

@Zsailer Zsailer added the documentation Improvements or additions to documentation label Jan 6, 2022
@Zsailer
Copy link
Member Author

Zsailer commented Jan 13, 2022

January 13th, 2022

Attendees

Name affiliation username
Zach Sailer Apple @Zsailer
Jason Weill AWS @jweill-aws
Vidar T Fauske JP Morgan Chase @vidartf
David Brochart QuantStack @davidbrochart
Sylvain Corlay QuantStack @SylvainCorlay
A. T. Darian Two Sigma @afshin
Kevin Bates IBM @kevin-bates
Max Klein JP Morgan Chase @telamonian
Mariko Wakabayashi OpenZeppelin @mwakaba2
Eric Charles Datalayer @echarles

Agenda + Minutes

@Zsailer Zsailer mentioned this issue Jan 14, 2022
Merged
@Zsailer
Copy link
Member Author

Zsailer commented Jan 20, 2022

January 20th, 2022

Attendees

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
A. T. Darian Two Sigma @afshin
Steven Silvester MongoDB @blink1073
David Brochart QuantStack @davidbrochart
Sylvain Corlay QuantStack @SylvainCorlay
Vidar T Fauske JP Morgan Chase @vidartf
Craig Citro Google @craigcitro

Agenda + Minutes

  • Windows tests are failing from pending kernels work
    • Really flaky on Windows.
    • Might (temporarily) skip these tests on Windows to prevent other PRs from failing.
    • Will open an issue to track this.
  • 7.1.0: this version makes JupyterLab very slow
    • Windows-only and unique to user (so far), possible b/c issue with < 1.12 server
    • Conda defaults channel version-lag - what updates defaults? This channel is periodically updated - so version-lags are expected.
    • See also: JupyterLab takes seconds to execute simple code -- It's too SLOW jupyterlab/jupyterlab#11811
    • Actions: Vidar is going to spend some time attempting to reproduce the issue and narrow down the server versions. The current hypothesis is that this may be related to the "nudge" logic that was recently extended in 1.13 to utilize two channels. Once we have more information, we'll move/wrap-up the issues.
  • Team Compass is Live!
  • Adding SC members who requested to be on the Jupyter Server decision-making body if they aren't already there: @ellisonbg, @ivanov, @damianavila
  • Software Steering Council Representative
    • An issue will be opened asking for nominations, after which a vote will be called

@Zsailer
Copy link
Member Author

Zsailer commented Jan 27, 2022 •
edited by kevin-bates
Loading

January 27th, 2022

Name affiliation username
Jason Weill AWS @jweill-aws
Zach Sailer Apple @Zsailer
Damián Avila 2i2c & MADS @damianavila
Vidar T Fauske JP Morgan Chase @vidartf
Steven Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Eric Charles Datalayer @echarles
Piyush Jain AWS @3coins
Akshay Chitneni Apple @akshaychitneni
Craig Citro Google @craigcitro
Jason Grout Databricks @jasongrout

Agenda + Minutes

  • SSC Representative
    • Nominations open for 1 more day
    • Vote next week.
  • Any feedback?
  • Moving Gateway repos (Jupyter Kernel Gateway, Jupyter Enterprise Gateway) into jupyter-server organization?
  • Discussion: how to onboard new contributors
    • "Good first issue" label might deter some contributions. If you've already contributed once, you might feel guilty picking up another one.
    • Let's not treat "good first issues" as so precious.
    • Often times, someone "claims" a good first issue but never finish. This prevents others from picking up that issue.
    • Host a weekly hack-a-thon/office hour meeting regularly?
      • Interested potential contributors can join and get help with contributing to Jupyter Server
      • Jupyter Server Team members help these contributors get their development environment setup, discuss areas where contributions are needed, teach them about the codebase, etc.
      • If no new contributors show up, Jupyter team members spend the time pair programming on some new software feature in Jupyter Server.

@Zsailer
Copy link
Member Author

Zsailer commented Feb 3, 2022

February 3rd, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Kevin Bates IBM @kevin-bates
Luciano Resende Apple @lresende
Jason Weill AWS @jweill-aws
Steven Silvester MongoDB @blink1073
Eric Charles Datalayer @echarles
Rahul Goyal AWS @rahul26goyal
Damián Avila 2i2c & MADS @damianavila
Akshay Chitneni Apple @akshaychitneni
Paul Ivanov Noteable @ivanov
Craig Citro Google @craigcitro
A. T. Darian Two Sigma @afshin
Jessica Xu @jess-x

Agenda + Minutes

  • Authorization PR ready for final review
    • Final review between now and next Tuesday
    • If nothing major shows up in the review, we'll plan to merge on Tuesday! (Thank you, Min)!
  • Test Flakiness
    • Possible conflict between pytest-tornasync, pytest-timeout, and asyncio.
  • Moving Gateways to Server org
    • EG Update:
      • 2.6.0 release is out
      • Major doc re-org coming, along with 3.0 (Spark 3 support)
      • 4.0 will be based on kernel provisioners
  • Discussion of starting experimental repos within jupyter-server Github org
    • Ok to use org as umbrella for experiments (instead of personal accounts)
    • communicate explicitly in the README that it is experimental
  • Auto pre-commit - stick with the default GitHub Access Token for now?
  • Jupyter Server Office/Hacking Hour.
    • Thoughts on this idea?
    • If we like it, how often?
    • Naming this hour session matters.
      • Calling it "office" hours may lead people to come with specific questions about their deployments.
      • Calling it "hacking hour" may lead to people doing a bunch of exploratory work
      • Calling it a "work party" may be more explicit in capturing the intent
  • SSC Representative
    • Open an issue to vote.

@Zsailer
Copy link
Member Author

Zsailer commented Feb 10, 2022

February 10th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Max Klein JPM @telamonian
Kevin Bates IBM @kevin-bates
Jason Weill AWS @jweill-aws
A. T. Darian Two Sigma @afshin
Mariko W. OpenZeppelin @mwakaba2
Rahul G AWS @rahul26goyal
Eric Charles Datalayer @echarles
Piyush Jain AWS @3coins
Craig Citro Google @craigcitro

Agenda + Minutes

@davidbrochart davidbrochart mentioned this issue Feb 15, 2022
Merged
@Zsailer
Copy link
Member Author

Zsailer commented Feb 23, 2022

February 17th, 2022

Name affiliation username
Jason Weill AWS @jweill-aws
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
A. T. Darian Two Sigma @afshin
Rahul Goyal AWS @rahul26goyal
Mariko W OpenZeppelin @mwakaba2
Piyush Jain AWS @3coins
Craig Citro Google @craigcitro

Agenda + Minutes

@Zsailer
Copy link
Member Author

Zsailer commented Feb 24, 2022

February 24th, 2022

Name affiliation username
Jason Weill AWS @jweill-aws
Kevin Bates IBM @kevin-bates
Zach Sailer Apple @Zsailer
Eric Charles Datalayer @echarles
Vidar T Fauske JP Morgan Chase @vidartf
Rahul Goyal AWS @rahul26goyal
Akshay Chitneni Apple @akshaychitneni
A. T. Darian Two Sigma @afshin
Andrey Velichkevich Apple @andreyvelich
Piyush Jain AWS @3coins

Agenda + Minutes

  • Making sense of the new world: Notebook, RetroLab, JupyterLab, NBClassic, Notebook Shim, and Jupyter Server
    • Notebook v6 —> Tornado web server and legacy Notebook Javascript
    • Notebook v7 —> JupyterLab components with classic Notebook UX (used to be RetroLab) + Jupyter Server
    • JupyterLab —> JupyterLab UX + Jupyter Server
    • NBClassic used to have 2 tasks
      • Adds classic Notebook JS and endpoints to Jupyter Server
      • Handle shimming traits and config (moved to notebook_shim)
    • notebook_shim
      • Handle shimming traits and config
    • What options will users have to run "side-by-side"?
      • Notebook v6, NBClassic, JupyterLab
      • Notebook v7 + JupyterLab
  • Current state of custom messages on the control channel?
  • "Contributing" Hour right after this meeting!
    • Introduced traitlets
    • Introduced server extensions
    • cover Jupyter Server unit tests next week.

@jess-x
Copy link
Contributor

jess-x commented Mar 6, 2022

March 3, 2022

Name affiliation username
Jason Weill AWS @jweill-aws
Vidar T Fauske JP Morgan Chase @vidartf
Zach Sailer Apple @Zsailer
Steven Silvester MongoDB @blink1073
Rahul Goyal AWS @rahul26goyal
Kevin Bates IBM @kevin-bates
Kiersten Stokes IBM @kiersten-stokes
Akshay Chitneni Apple @akshaychitneni
A. T. Darian Two Sigma @afshin
Mariko W OpenZeppelin @mwakaba2
Andrey Velichkevich Apple @andreyvelich

Agenda + Minutes

  • Issue #312 - Improving Network Performance opened September 2020

    • In looking at things, @goanpeca found that the server is validating notebooks twice - potential 50% savings per notebook!
      • Nbformat PR #249 adds the ability to capture validation errors]
      • TODO: On next nbformat release, update server to add the additional parameter, remove second validation, and bump dependency floor on nbformat
      • TODO: We should also enable the ability to leverage fastjsonschema that @goanpeca added to nbformat, exposed via env.
        • Should this be configurable, or fastjsonschema is auto-detected and used unconditionally? (leaning toward configurable in case issues arise or other options appear)
          • If configurable, should it expose (and track) available options or a boolean? (leaning toward enum, default = jsonschema)
    • We should probably talk about static asset management posted in the issue as well.

  • Maintenence-related cleanups

    • Terminal culling works again on Windows!
    • Testing against min dependencies and pre-releases
    • Removed pinning up upper version deps
    • Working on cleaning up jupyter_client as well so we can handle warnings as errors as well
    • Flit-based backend in jupyter-packaging coming soon

  • pre/post_save_hook PR: https://github.com/jupyter-server/jupyter_server/pull/696

    • Urgent, because of issues in JupyterLab
    • Not ready to merge in it’s current state.
    • Current state/question: should there be a way to order these post_save_hooks?
    • Thoughts:
      • The single value trait, post_save_hook, should be deprecated
      • Replace with a List trait, post_save_hooks, where order matters.
      • What happens if the same post_save_hook is given by an extension and by the user configuration? It would add the hook twice, which probably isn’t intended.
      • Need to put more thought into this.

  • Identity API /api/me

    • This will likely require a new major version. It may be about time to do this.
    • Adding this API is one of the critical building blocks for RTC

  • Websocket subprotocol bug: https://github.com/jupyter-server/jupyter_server/pull/706

    • Causing issues with pending kernels
    • Thank you, David, for quickly fixing!

  • “Contributing” hour right after this!

@jess-x
Copy link
Contributor

jess-x commented Mar 14, 2022

March 10, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Vidar Tonaas Fauske JP Morgan Chase @vidartf
A. T. Darian Two Sigma @afshin
Kiersten Stokes IBM @kiersten-stokes
Rahul Goyal AWS @rahul26goyal
Jason Weill AWS @jweill-aws
Steven Silvester MongoDB @blink1073
Carlos Herrero QuantStack @hbcarlos
Akshay Chitneni Apple @akshaychitneni
Andrey Velichkevich Apple @andreyvelich
Craig Citro Google @craigcitro

Agenda + Minutes

@Zsailer
Copy link
Member Author

Zsailer commented Mar 17, 2022

March 17th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Jason Weill AWS @jweill-aws
Kiersten Stokes IBM @kiersten-stokes
A. T. Darian Two Sigma @afshin
Jessica Xu Jupyter community @jess-x
Rahul AWS @rahul26goyal
Andrey Velichkevich Apple @andreyvelich
Piyush Jain AWS @3coins

Agenda + Minutes

@Zsailer
Copy link
Member Author

Zsailer commented Mar 24, 2022

March 24th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Jason Weill AWS @jweill-aws
A. T. Darian Two Sigma @afshin
Steven Silvester MongoDB @blink1073
Marc Udoff @mlucool
Kiersten Stokes IBM @kiersten-stokes
Andrey Velichkevich Apple @andreyvelich
Kevin Bates IBM @kevin-bates
Frederic Collonval QuantStack @fcollonval
Piyush Jain AWS @3coins
Jessica Xu Jupyter @jess-x
Rahul Goyal AWS @rahul26goyal

Agenda + Minutes

@mlucool mlucool mentioned this issue Mar 30, 2022
Open
@mwakaba2
Copy link
Member

mwakaba2 commented Apr 6, 2022 •
edited by Zsailer
Loading

March 31st, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
David Brochart QuantStack @davidbrochart
Jason Weill AWS @jweill-aws
Mariko W. OpenZeppelin @mwakaba2
Kevin Bates IBM @kevin-bates
Andrey Velichkevich Apple @andreyvelich
Kiersten Stokes IBM @kiersten-stokes
Piyush Jain AWS @3coins

Agenda + Minutes

@blink1073 blink1073 mentioned this issue Apr 6, 2022
Open
@Zsailer
Copy link
Member Author

Zsailer commented Apr 7, 2022

April 7th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Jason Weill AWS @jweill-aws
Steven Silvester MongoDB @blink1073
David Brochart QuantStack @davidbrochart
A. T. Darian QuantStack @afshin
Kevin Bates IBM @kevin-bates
Andrey Velichkevich Apple @andreyvelich
Akshay Chitneni Apple @akshaychitneni
Piyush Jain AWS @3coins
Kiersten Stokes IBM @kiersten-stokes
Rahul Goyal AWS @rahul26goyal

Agenda + Minutes

  • Darian's affiliation

  • Big "shout out" to Steve (@blink1073) for updating the Jupyter repos (including those outside the Jupyter Server org) to have a consistent CI experience. This is really nice from a "maintainer-familiarity" perspective and much appreciated! Thank you Steve! 🎉 ❤️

  • Events: An event system for Jupyter jupyter_server#780

    • Proposing to add 4 REST endpoints and Websocket to core Jupyter Server
    • Darian's goal is to get some form of this in by early summer—even if it's a simple/lightweight version.

  • Jupyter Server 2.0 Discussion

    • New features for 2.0
      • Identity
        • Is this needed for JupyterLab 4.0, i.e. before June? No.
      • Events
      • Remove Terminals
    • Do we need JupyterLab 4 to require Jupyter Server 2.0?
    • Next Steps:
      • Create a milestone
      • Create 1.x branch and make main the 2.x.
      • Merge Identity PR
      • Cut a 2.0 pre-release

  • Released [EXPERIMENTAL] Synchronizer extension

    • Requires an addition in the GatewayClient API in (core) Jupyter Server—a new method called list_kernels
    • Can we use the list_kernels from the MultiKernelManager.

  • Extensions configuring other extensions?

  • "Contributing" hour right after this!

@fcollonval
Copy link
Member

Is needed for JupyterLab 4.0, i.e. before June? No.

I disagree on that point. Identity is needed for RTC and we certainly hope to get a first decent RTC experience in 4.0.

@echarles echarles mentioned this issue Apr 8, 2022
Closed
@jess-x
Copy link
Contributor

jess-x commented Apr 21, 2022

April 14th, 2022

Name affiliation username
Frederic Collonval QuantStack @fcollonval
Afshin T. Darian QuantStack @afshin
Jason Weill AWS @jweill-aws
Andrey Velichkevich Apple @andreyvelich
Kevin Bates IBM @kevin-bates
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Piyush Jain AWS @3coins
Akshay Chitneni Apple @akshaychitneni
Craig Citro Google @craigcitro

Agenda + Minutes

  • Notification/Telemetry endpoints should be available prior to Lab 4.0 (Server 2.0)
  • main branch of jupyter_server needs to target 2.x before we merge backward-incompatible things like identity API
  • (Andrey) Share few Jupyter Server diagrams
  • Demo of Jupyter Server Synchronizer.
  • Contributing Hour right after this!

@jess-x
Copy link
Contributor

jess-x commented Apr 21, 2022

April 21st, 2022

Name affiliation username
Steven Silvester MongoDB @blink1073
Vidar T Fauske JP Morgan Chase @vidartf
Jason Weill AWS @jweill-aws
Kevin Bates IBM @kevin-bates
Kiersten Stokes IBM @kiersten-stokes
Rahul Goyal AWS @rahul26goyal
A. T. Darian QuantStack @afshin
Andrey Velichkevich Apple @andreyvelich
Jessica Xu

Agenda + Minutes

@afshin
Copy link
Contributor

afshin commented Apr 29, 2022

April 28th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Steven Silvester MongoDB @blink1073
Rahul Goyal AWS @rahul26goyal
Brian Granger AWS/CalPoly @ellisonbg
Kiersten Stokes IBM @kiersten-stokes
A. T. Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart
Kevin Bates IBM @kevin-bates
Kien Dang NUS @kiendang
Adam Patterson UM adpatter
Andrey Velichkevich Apple @andreyvelich
Craig Citro Google @craigcitro

Agenda + Minutes

  • Welcome, Rahul, to the Jupyter Server Team 🎉
  • Jupyter Event System and Telemetry
  • PyCON Packaging Summit tomorrow - Steve will be presenting about External Data - a replacement for the now deprecated data_files.
  • "Contributing" hour right after this!

@choldgraf choldgraf mentioned this issue May 5, 2022
Open
@Zsailer
Copy link
Member Author

Zsailer commented May 12, 2022

May 5th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Steven Silvester MongoDB @blink1073
Piyush Jain AWS @3coins
Vidar T Fauske JP Morgan Chase @vidartf
Jason Weill AWS @jweill-aws
Andrii Ieroshenko Cal Poly Jupyter @andrii-i
Afshin T. Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart
Kevin Bates IBM @kevin-bates
Kiersten Stokes IBM @kiersten-stokes
Adam Patterson US adpatter
Matt Henderson LBNL @mlhenderson
Rahul Goyal AWS @rahul26goyal
Andrey Velichkevich Apple @andreyvelich

Agenda + Minutes

  • Zach

  • Steve

  • Piyush

    • Discuss tasks for Event Notification System, intent would be to open PRs for each of the numbered items.

      1. Basic Event Bus - #820

        • A central event bus to relay events
        • /api/events/subscribe - Websocket for subscribing to events
        • A default handler for consuming events

      2. Rest API Endpoints

        • POST /api/events - Rest api to create new events
        • Do we need API for listing topics/schemas? May be add this is a separate PR.
          • /api/events/schemas?

      3. JupyterLab 4 Event Client (jupyterlab-events)

        • Rename jupyterlab-telemetry repo to jupyterlab-events
        • Remove server endpoints, any redundant server code
        • Update client handlers to use the rest api endpoints
        • Add websocket handler to enable subscribtion to events

      4. Default events in server

        • Add default events e.g., content handler, kernel events in jupyter server

      5. JupyterLab 4 Updates

        • Add jupyterlab-events to JupyterLab
        • Subscribe to default events

      6. Event Notification UI (JupyterLab)

        • UI updates for event notification

      7. Event buffer (Move to Jupyter Server Roadmap #2 or Move nbresuse into jupyter-server organization #3)

        • A queue/buffer to store event messages if not delivered by the websocket

      8. Jupyter Notebook 7 Updates

      9. Telemetry?

  • David

@Zsailer
Copy link
Member Author

Zsailer commented May 19, 2022

May 12, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Jason Weill AWS @jweill-aws
Piyush Jain AWS @3coins
David Qiu AWS @dlqqq
A. T. Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart
Andrii Ieroshenko Cal Poly Jupyter @andrii-i
Kiersten Stokes IBM @kiersten-stokes
Kevin Bates IBM @kevin-bates
Jessica Xu n/a @jess-x
Kien Dang NUS @kiendang
Andrey Velichkevich Apple @andreyvelich
Rahul Goyal AWS @rahul26goyal
Carlos Herrero QuantStack @hbcarlos
Rosio Reyes Anaconda @RRosio

Agenda + Minutes

  • Events Talk™ (notifications edition)
    • First pass: sqlite + labextension
    • Second pass: Yjs? + labextension
    • Third pass: sqlite + labextension
    • Questions mentioned during the meeting:
      • Are we broadcasting all events?
      • Are we doing some topic/subtopic matching?
      • How does identity play into routing events?
      • How do we handle multiple clients (same identity)?
    • Create jupyter_events package and move most of telemetry into that package.
    • Event Bus
    • Filtering subscriptions from the client
      • e.g. /api/events/subscribe/notifications
        • Creates WebsocketHandler with a filter that accepts only "notification" events.
    • Let's create a HackMD as a working doc for the Event System:

@Zsailer
Copy link
Member Author

Zsailer commented Jun 2, 2022

May 19, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Andrii Ieroshenko Cal Poly Jupyter @andrii-i
Jason Weill AWS @jweill-aws
A. T. Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart
Vidar T Fauske JP Morgan Chase @vidartf
Kien Dang NUS @kiendang
Steven Silvester MongoDB @blink1073
Piyush Jain AWS @3coins
Akshay Chitneni Apple @akshaychitneni
Rahul Goyal AWS @rahul26goyal

Agenda + Minutes

  • Zach

    • Basic Event Bus PR ready for (hopefully final) review
    • In current implementation, the /api/events/subscribe endpoint sends all events across the websocket.
    • We discussed that we need to add a way to open a websocket that only streams a subset of events.
    • Proposed plan:
      • Client hits /api/events/subscribe to get a websocket connection
      • The server responds with a READY or IDLE message to notify client that the connection is available to stream data.
      • The client sends a list of filters to apply (or empty list for all events)

  • Vidar

  • Steve (will be 15 min late)

  • "Contributing" hour right after this!

@Zsailer
Copy link
Member Author

Zsailer commented Jun 2, 2022

May 26, 2022

Name affiliation username
Vidar T Fauske JP Morgan Chase @vidartf
Steven Silvester MongoDB @blink1073
Rick Wagner UCSD @rpwagner
Kiersten Stokes IBM @kiersten-stokes
Jason Weill AWS @jweill-aws
A. T. Darian QuantStack @afshin
Zach Sailer Apple @Zsailer
Rahul Goyal AWS @rahul26goyal
Andrey Velichkevich Apple @andreyvelich
Piyush Jain AWS @3coins
Rosio Reyes Anaconda @RRosio

  • Rick & Jason W

  • Vidar

  • Steve (will be 15-20 min late)

    • Update on hatch_jupyter_builder
    • Zach will add himself as a maintainer of conda-forge

  • Darian

    • Jupyter Kernels Subproject question (team-compass)
    • Question about downstream development that relies on the main branch:
      • notebook_shim
      • jupyterlab_server
      • jupyterlab

  • Piyush

    • Binder doesn't work for serving api endpoints in a server extension. This might be due to binder not using the jupyter-server. How can we update this?
    • Contributing hour questions:
      • How to setup resources e.g., a database for a extension application that is available to the API handlers?

@kevin-bates
Copy link
Member

September 1st, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Piyush Jain AWS @3coins
Andrii Ieroshenko AWS @andrii-i
Vidar T Fauske JPMorgan Chase @vidartf
Kiersten Stokes IBM @kiersten-stokes
AT Darian QuantStack @afshin
Rahul G AWS @rahul26goyal
Luciano Resende Apple @lresende

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Sep 8, 2022

September 8th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
David Qiu @dlqqq
Kevin Bates IBM @kevin-bates
Frederic Collonval QuantStack @fcollonval
Kiersten Stokes IBM @kiersten-stokes
Piyush Jain AWS @3coins
Andrii Ieroshenko AWS @andrii-i
AT Darian QuantStack @afshin
Steven Silvester MongoDB @blink1073
Vidar T Fauske JP Morgan Chase @vidartf

Agenda

  • Pagination support in the contents service

    • Practical issue for filebrowser in JupyterLab: Handling large number of files jupyter_server#538
      • In that issue, the calls of os.lstat can be a bottleneck due to the large number of files.
    • What is the problem definition? (memory, rendering time, both, other?)
      • Primarily memory - only want "the view" in memory.
      • Secondary is rendering time due to the large increase of DOM nodes to render the full list.
      • Implemented pagination + filtering raises questions of how do you know how many pages there are for a filter query?
      • Can query results be kept in a server-side cache?
      • David Q suggestion: only run query (e.g. call os.stat, sort, filter, etc.) on the first page, cache it in a persistent store (e.g. SQLite), and then every subsequent page is a LIMIT OFFSET query on the cached results
        - Would need to distinguish a fresh GET vs. a traversal GET when the latter targets page=1 (noted here).
    • Live trial of Vidar:
      • 3k+ entries in contents manager, with gzip: 800 Kb -> 91 kB
      • We should probably consider if JupyterLab can use browser caching (etags should be reliable?). Current it uses no-cache header. i.e. so that we don't resend it on polls if there are no server-side changes (most calls).

  • Zach

    • Jupyter Server Council Updates
      • Vote status on "New repo for Headless/scheduled Notebook Execution"
      • New Team member!
    • Jupyter Server RC
      • Bump Jupyter Server version in JupyterLab Server and JupyterLab

  • David Q

  • Piyush

    • Why is JL4 not using JServer 2.0?
      - [name=frederic] Feel free to open the PR - but yes it will 😉 👍
      - We should look to add an item for releasing 3.5 with JServer 2
      The login handler change is probably the only one that can bring some issue for user overriding it
      We need to check that no deprecated APIs are used by JLab 3.x - just to be sure.

  • Contributing Hour Topics

    • Publishing server extension to readthedocs
    • Publishing server extension to PyPI.

This was referenced Sep 8, 2022
Merged
Merged
@blink1073 blink1073 mentioned this issue Sep 12, 2022
Closed
@Zsailer
Copy link
Member Author

Zsailer commented Sep 15, 2022 •
edited
Loading

September 15th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Vidar T Fauske JP Morgan Chase @vidartf
David L. Qiu @dlqqq
AT Darian QuantStack @afshin
Piyush Jain AWS @3coins
Sylvain Corlay QuantStack @SylvainCorlay
Steven Silvester MongoDB @blink1073
Kiersten Stokes IBM @kiersten-stokes
Rahul Goyal AWS @rahul26goyal
Matt Henderson LBNL @mlhenderson
Andrii Ieroshenko AWS @andrii-i

Agenda

@jasongrout
Copy link

  • New governance goes into effect by October 2022

I'm sure there was more context in the actual meeting, but just to clarify for those reading the notes: the overall new Jupyter governance (Software Steering Council, Executive Council) transition is planned for before the end of the year (probably in December). For October, we are planning for subproject councils to be in place and functioning.

@Zsailer
Copy link
Member Author

Zsailer commented Sep 22, 2022

September 22nd, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Steven Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Kiersten Stokes IBM @kiersten-stokes
AT Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart
Piyush Jain AWS @3coins
Chia-Wei Chen Pinterest @chiawchen
Rahul Goyal AWS @rahul26goyal

Agenda

  • Steve

    • Working on removing nest-asyncio from the stack, for compliance with Python 3.12 changes in asyncio.
      • Client PR
      • Server PR

  • Kevin

    • Nearing EG 3.0 GA
    • Updating remote provisioners with "recent" EG fixes, would like to move into jupyter-server once CI/build is updated
    • Thinking lots about kernel parameterization, discovery in particular

  • Zach

  • Sylvain

@Zsailer Zsailer mentioned this issue Sep 22, 2022
Open
@kevin-bates
Copy link
Member

September 29th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Steven Silvester MongoDB @blink1073
Kiersten Stokes IBM @kiersten-stokes
Andrii Ieroshenko AWS @andrii-i
Rahul Goyal AWS @rahul26goyal
David Brochart QuantStack @davidbrochart

Agenda

  • Zach
    • SSC Nominations. Finish nominations this week.
    • Question about extensions that span multiple layers. Where do they go?
      • We've tried "contrib" repos—more orgs aren't great...
      • Hypothetical "Jupyter-plugins" org.
        • Not a Jupyter subproject; i.e. no SSC rep
        • Not officially governed by Jupyter
        • Has representation from people with Jupyter subprojects team membership.
        • Inherits some processes from Jupyter governance, i.e. team-compass, team-membership, voting to accept outside repos.
        • Set expectations for plugin repos.
          • Only accept repos that show long term support.
          • Must inherit Jupyter's BSD-3 license.
      • Why use a separate non-Jupyter governed repo?
        • One layer of separation from Jupyter core components; sets the users expectations.
        • Still reviewed and evaluated by representation from Jupyter subproject members.
          • Users of the plugins can "trust" it more than plugins under a single user's namespace
  • Steve
    • Removing pending-kernels from client 8?
      • We can provide the same behavior by wrapping the Kernel Manager and tracing method calls, and using our own MultiKernelManager in Server v2. Server v1 would pin to client<8.
    • Releaser changes - experimental support for Releaser v2 that is run from Jupyter Server itself using shared credentials - requires admin write access to the repository. The intent is to repeat this pattern across most of the Jupyter repos.

@ellisonbg
Copy link

ellisonbg commented Sep 29, 2022 via email

@Zsailer
Copy link
Member Author

Zsailer commented Sep 29, 2022

Hey @ellisonbg, in the discussion today, we were thinking about extensions that have both server-side and client-side components/extension that are closely coupled together. This could be expanded to plugins that work in a JupyterHub context—where it requires single-user server extension, client UI extension, and a JupyterHub-managed "service" to coordinate the UX.

I can open a separate thread for future discussion.

@Zsailer
Copy link
Member Author

Zsailer commented Sep 29, 2022

@ellisonbg I opened this issue for further discussion: #38

@kevin-bates
Copy link
Member

October 6th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Piyush Jain AWS @3coins
Frederic Collonval QuantStack @fcollonval
Kiersten Stokes IBM @kiersten-stokes
Andrii Ieroshenko AWS @andrii-i
Jessica Xu Cal Poly @jess-x

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Oct 20, 2022

October 13th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Andrii Ieroshenko AWS @andrii-i
Sylvain Corlay QuantStack @SylvainCorlay
David Brochart QuantStack @davidbrochart
Thorsten Beier QuantStack @DerThorsten
AT Darian QuantStack @afshin
Johan Mabille QuantStack @JohanMabille
Kiersten Stokes IBM @kiersten-stokes
Jessica Xu Cal Poly @jess-x
Antoine Prouvost QuantStack @AntoinePrv
Frederic Collonval QuantStack @fcollonval
Michal Krassowski (Quansight) @krassowski

Agenda

  • Frederic (need to drop after 15min):
    • JupyterLab 3.5.0rc0 has been released (lift Jupyter Server version constrain) - ETA for final is Oct. 20th.
    • JupyterLab server 2.16.0 has been released (lift Jupyter Server version constrain)

Jupyter kernels

  • Antoine : xeus-octave
    • Binder link
    • Octave Jupyter Kernel build in C++
    • Highlights
      • Plotly integration
      • Latex-based rich output of matrices
      • Widget integration
  • Thorsten & Sylvain : long-running kernels
    • Connecting to kernels that are not started/stopped by the main Jupyter instance
    • The kernels persist beyond the lifetime of the Jupyter application.
    • We would connect to them by just knowing their connection info.
    • Ideally they would show up in the "running kernels" section of JupyterLab
    • Ideas discussed:
      • Use some of the existing APIs in Jupyter client to connect to these kernels. Look into kernel clients
      • Use custom provisioner to discover a kernel, but maybe no-op some of the lifecycle calls (e.g. start/stop)?
      • There are some rehydration work happening in enterprise gateway that might be useful for connecting to a kernel that already exists.

Jupyter FileID:

Jupyter SSC Voting

  • Voting happening this week. Check your inboxes

Jupyter Releaser

  • Created a Jupyter Server Bot with write credentials
  • Shared PyPI repo-scoped access token for publishing jupyter server
  • Releaser will pull the Github API to check if the user doing the release has admin permission for publishing.

Clean up old issues on team-compass

  • Did you create / answer an issue on team-compass? Can it be closed? It will be easier to check for votes etc if things are kept up to date 👍

@kevin-bates
Copy link
Member

October 20th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Steve Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Kiersten Stokes IBM @kiersten-stokes
Piyush Jain AWS @3coins
AT Darian QuantStack @afshin
Andrii Ieroshenko AWS @andrii-i
Rahul Goyal AWS @rahul26goyal
Jessica Xu Cal Poly @jess-x
Theodore Aptekarev OpenBB @piiq

Agenda

  • Zach

    • 2.0 release plans
    • SSC stuff
    • (Non-breaking changes to) Kernel Websocket Architecture
      • Add a new configurable "kernel websocket client" class to ServerApp that owns all the logic underneath the ZMQChannelsHandler (websocket handler).
        • The websocket handler would only include the public Tornado methods. Everything else belongs under the configurable class.
        • In my current design (not public yet), the client class includes the following public methods:
        •   class KernelWebsocketClient(...):
  
      session: jupyter_client.session.Session
  
      def connect():
          ... 
          
      def disconnect():
          ... 
          
      def handle_incoming_message(ws_msg):
          ... 
          
      def handle_outgoing_message(zmq_msg):
          ...
          
      def send_status_message(state):
      • Why?
        • Enables folks to customize the logic under the websocket handler.
        • Today, you can't remove the websocket handler in favor of your own.
        • You can write a new websocket handler and inject it before the core websocket handler, but that's a hack.
      • Name is hard. What should we call this (sorry to bikeshed here)?
        • Jessica suggested: Kernel Message Broker
      • Finish before 2.0 release.

  • Steve

    • Releaser Updates

      • We are now using releaser on server repo itself. We have bot accounts for GitHub, PyPI, and npm. We're using a scoped PyPI token for server. The npm account has access to the @jupyter-server npm org, which right now only affects resource-usage (which is not yet using releaser).

    • Asyncio Cleanup

      • Incorporate changes Min had made earlier
      • Required a fork of pytest-tornasync to handle new asyncio
      • Most of the fixtures are now async
      • Since it is a breaking change to the fixtures (and a couple utility functions), it should be in 2.0
      • Deprecate MappingKernelManager and all sync managers (non-asyncio) in 2.0?
      • Zach will work on async KernelSpecManager

  • Darian

@kevin-bates
Copy link
Member

October 27th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Steve Silvester MongoDB @blink1073
Brian Granger AWS @ellisonbg
Kevin Bates IBM @kevin-bates
Andrii Ieroshenko AWS @andrii-i
A. T. Darian QuantStack @afshin
Kiersten Stokes IBM @kiersten-stokes

Agenda

  • Zach

    • Jupyter Executive Council (self) nominations. If you're interested, please fill out the form.

  • Brian

    • Public API of IdentityProvider, how to extend?
      • It's not clear which methods in the Identity provider are necessary for working with the rest of Jupyter Server.
      • We should define an Abstract Base Class to set this interface in stone to avoid future breaking changes.
    • Packaging and defaults for file id service.

  • Steve

    • Asyncio refactor - managed to keep backward compat with the pytest_plugin. Found a race condition in client. Client 8 will have a slight change to Provisioner where the provisioner will be in charge of handling the connection file/info.
      • Make the async managers the default, and have the sync managers subclass those and be deprecated? This would allow the default trait class to be the async one.
    • Move to support Python 3.8 - 3.11 in all new minor versions?

@kevin-bates
Copy link
Member

November 3rd, 2022

Name affiliation username
Steve Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Piyush Jain AWS @3coins
Kiersten Stokes IBM @kiersten-stokes
Andrii Ieroshenko AWS @andrii-i
Zach Sailer Apple @Zsailer
Frederic Collonval QuantStack @fcollonval
Rahul Goyal AWS @rahul26goyal
Vidar T Fauske JP Morgan Chase @vidartf
A. T. Darian QuantStack @afshin

Agenda

@kevin-bates
Copy link
Member

kevin-bates commented Nov 10, 2022 •
edited
Loading

November 10th, 2022

Name affiliation username
Steve Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Zach Sailer Apple @Zsailer
Luciano Resende Apple @lresende
AT Darian QuantStack @afshin
Kiersten Stokes IBM @kiersten-stokes
Andrii Ieroshenko AWS @andrii-i
Piyush Jain AWS @3coins
Frederic Collonval QuantStack @fcollonval

Agenda

  • Steve

    • Jupyter Core 5.0 released. We're now using the platform dirs in our test suite.
    • One more PR for Jupyter Client 8
    • Add authorization to AuthenticatedFileHandler
    • Writing API errors to console
      • Discussion during the meeting: will this expose any security vulnerabilities? i.e. is there anything in the traceback that shouldn't be exposed on the console.
      • Previously, we caused a security issue by printing the access token to the console, so we're a little cautious here.

  • Kevin

    • Should we inject a SQLAlchemy layer per this issue or wait and do this in jupyverse?
      • Kevin is planning to open an issue to discuss further (see Issue 1063)
      • Can we build a "Jupyter Server Database API" that's abstract enough to work with both jupyter_server and jupyverse?
      • Can we have an object store as well (e.g. Yjs/Ypy)?

  • Darian

    • EventManager front-end: subscribing to the event bus emissions

  • Vidar

    • preferred_dir PR: Needed to drop validation of preferred_dir due to async/sync.

  • Frederic

    • 🤔⁇🧐❔🤨❓ (unable to join maybe toward the end of the meeting)
    • Question about the responsibility of the ContentManager.
      I saw that the event emission is done within the contents managers like for get. Would it makes sense to do that in the handler to remove the event emission responsibility from the content manager (that may be custom)?
      The point was discussed and the decision was to place it in the manager because the manager may want to extend/add events (maybe more true for the kernel manager).
      To ease migration to v2 it may be interested to move the event emission to a base class or hooks that developer can use when migrating.
      We could also provide tests re-usable by developer to entire the managers contract.
      An alternative is to define event at the REST API level to be more reliable that emission by uncontrolled managers.

@kevin-bates kevin-bates mentioned this issue Nov 10, 2022
Open
@kevin-bates
Copy link
Member

November 17th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Kevin Bates IBM @kevin-bates
Andrii Iersohenko AWS @andrii-i
Kiersten Stokes IBM @kiersten-stokes
Rahul Goyal AWS @rahul26goyal
Afshin T. Darian QuantStack @afshin
David Brochart QuantStack @davidbrochart

Agenda

  • Zach

    • Did everyone see the email? Vote for EC by December 4th.
    • Holiday season
      • Thanksgiving Holiday in the US. No Jupyter Server or Contributing Hour meeting next week (11/24)?
      • Christmas/New Years Holiday. No Jupyter Server + Contributing Hour meetings on December 12/29?
    • I'll be going out on parent leave 12/22-2/3
    • Discussed Websocket connection abstraction layer
      • New configurable/overridable kernel ZMQ+Websocket connection API jupyter_server#1047
      • One more unit test to fix; please review when folks have time.
      • Defines an ABC for connecting kernel websockets to their ZMQ sockets
      • Allows easy overriding of the websocket API. Decouples the business logic from the server framework choice (e.g. tornado, fastapi, etc.)
      • Might be helpful for transitioning folks to jupyverse in the future.

  • Kevin

    • Move jupyter/nb2kg to jupyter_server and archive repo
      • Steve suggested to archive in place for future discovery (makes sense)
      • Kevin will open an issue in NG2KG referencing the team compass issue and call for a jupyter-server-council "vote" on the team-compass issue.

  • Steve

  • Darian

    • Question re: events schemas
      • How do we check if the events API exists? e.g. JupyterLab 3.6.1 can use Jupyter Server 1.0 and 2.0.

This was referenced Nov 17, 2022
Closed
Closed
@kevin-bates
Copy link
Member

December 1st, 2022

Name affiliation username
Kevin Bates IBM @kevin-bates
Zach Sailer Apple @Zsailer
Andrii Ieroshenko AWS @andrii-i
Frederic Collonval QuantStack @fcollonval
David Brochart QuantStack @davidbrochart
Steven Silvester MongoDB @blink1073
Kiersten Stokes IBM @kiersten-stokes
Luciano Resende Apple @lresende
Johan Mabille QuantStack @JohanMabille

Agenda

  • Reminder call for proposal for JupyterCon 2023 is December 15th.

  • Jupyter Executive Council election is going on right now: December 4 AoE deadline. If you are on a Jupyter Council, please vote.

  • Extract out kernels service to a separate package.

  • Frederic (need to drop quickly but I'll try to come back before the end)

    • Issue seen on JupyterLab that may be related to the Websocket change in v2rc
      failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header

    • Pull request to add sponsor badge on jupyter-server repositories

    We did it for example on the GitHub jupyterlab organization. IPython has some sponsor badge too.

  • Zach

  • Steve

    • pytest-jupyter is now available to replace jupyter_server.pytest_plugin
    • Coverage is now above 80%. We also added coverage and downstream to auto-merge criteria. Coverage will fail if it drops by 1%.

  • Johan

@Zsailer
Copy link
Member Author

Zsailer commented Dec 15, 2022

December 8th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Johan Mabille QuantStack @JohanMabille
Steven Silvester MongoDB @blink1073
Kevin Bates IBM @kevin-bates
Andrii Ieroshenko AWS @andrii-i
Afshin T. Darian QuantStack @afshin
Piyush Jain AWS @3coins

Agenda

@jess-x
Copy link
Contributor

jess-x commented Jan 5, 2023

December 15th, 2022

Name affiliation username
Zach Sailer Apple @Zsailer
Andrii Ieroshenko AWS @andrii-i
Steven Silvester MongoDB @blink1073
David Brochart QuantStack @davidbrochart
Afshin T. Darian QuantStack @afshin
Johan Mabille QuantStack @JohanMabille
Jessica Xu Cal Poly @jess-x

Agenda

  • Zach
    • Holiday schedule
      • We're planning to cancel both meetings.
      • Zach will remove from the Jupyter calendar.
    • JupyterCon Proposal
      • I'll be working on this proposal today. If anyone would like to review it, I'll share a hackmd later.
      • David and Zach will be authors. Possibly Kevin?
  • David
    • Jupyter kernel sub-shells JEP
    • Looking for a shepherd. If anyone is interested, please reach out!
    • Probably wait until after the holidays to review more closely.
    • Question: Does the Jupyter messaging protcol define a "message not known" response for the control channel?
      • If it doesn't, we should add this to the protocol to make it easier to add new message types.
      • New JEP? <= YES ;)
  • Steve

@jess-x jess-x closed this as completed Jan 5, 2023
@blink1073 blink1073 unpinned this issue Jan 5, 2023
@kevin-bates kevin-bates mentioned this issue Feb 1, 2023
Closed
@Zsailer Zsailer mentioned this issue Jan 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests