Kubeflow Notebooks 2.0 (Kubeflow Workspaces)

[thesuperzapper]

/kind feature

Here is my proposal for Kubeflow Notebooks 2.0, which can be called "Kubeflow Workspaces".

Frequently Asked Questions

---

What is Notebooks 2.0?

Kubeflow Notebook 2.0 is the next evolution of Kubeflow Notebooks.
It makes running IDEs on Kubernetes for ML/AI significantly better for users and cluster admins alike.

---

Where can I find Notebooks 2.0?

We are developing Notebooks 2.0 in kubeflow/notebooks on the notebooks-v2 branch.

Here are direct links to each component's code:

• ./workspaces/controller/: new Kubernetes controller (Golang, Kubebuilder)
• ./workspaces/backend/: new REST API "backend for the frontend" (Golang)
• ./workspaces/frontend/: new frontend UI (NodeJS, React)

---

How can I contribute to Notebooks 2.0?

We welcome code contributions and ideas! The best way to get involved is participating on the #kubeflow-notebooks Slack channel, and joining our weekly notebooks community calls (currently at 8AM PT each Thursday).

If you don't see any unassigned tasks, take a look at the current state of the code and make some suggestions (by creating issues on the kubeflow/notebooks repo or starting a thread in Slack).

---

When will Notebooks 2.0 be ready?

We hope to ship a widely available alpha version as an optional component of Kubeflow 1.11 (ETA mid 2025).

However, we will release a testing version as soon as we have an end-to-end working platform (ETA late 2025), subscribe to this issue for updates and to help us test.

---

Motivation

The main idea has always been to make the Notebook CRD not just a wrapper around PodSpec, with the goal of abstracting away the Kubernetes resources from end users, while also giving cluster admins the ability to define a set of "templates" that end users can choose from.

The main benefits of this approach are:

• end-users can create Workspaces without needing to know anything about Kubernetes, because the UI literally becomes 3 dropdowns and 2 volume mounts:
  • Choose a WorkspaceKind (e.g. "jupyter-lab", "vs-code", "rstudio")
  • Choose an image from the approved list
  • Choose a pod-config (e.g. "small_cpu", "big_gpu")
  • Create/Mount a home volume (optional)
  • Create/Mount data volumes (optional)
• cluster-admins can update the definitions of WorkspaceKinds without breaking existing Workspaces:
  • they simply add a new image/pod-config option, and redirect the old one to the new one
  • we can even provide a config to make the controller wait for a Workspace to restart before applying the redirect
    (we can also make the Spawner UI display a warning that the Workspace needs to be restarted to get the new config)
• scheduling Pods with GPU resources often requires a number of Pod configs to be correctly aligned (e.g. tolerating a taint, and setting a resource limit), collecting all these configs into a single "pod-config" allows cluster-admins to provide drop-down options, rather than requiring users to understand the structure of their Kubernetes cluster.
• resizing an existing notebook is effectively impossible right now, because all the nice "spawner configs" are only available at the time you create the Notebook, once it's spawned you have a PodSpec.
  • With WorkspaceKinds, we can make an "edit" button on the UI which allows you to pick from the current "pod-configs" and "images" available for the WorkspaceKind.

Implementation

For more detailed information about the design, please see:

• Notebooks 2.0 - Design Document.

New Components

• a new Workspace Controller:
  • Will manage and reconcile the new CRDs
  • Will provide webhooks (for validation of CRD patches)
  • Will be written in GoLang with Kubebuilder
• a new Workspace Backend API
  • Will be the interface between the frontend and Kubernetes
  • Will allow programmatic management of Notebooks, and allow easier replacement of frontend.
  • Will be a REST API written in Golang
• a new Workspace Frontend:
  • Will let users overview the workspaces in a namespace
  • Will let users spawn, edit and connect to Workspaces
  • Will be written in JS (React)

New CRDs

The high-level overview is to split the Notebook CRD into these two CRDs:

• Workspace (namespaced resource)
  • this is the resource that end-users create via the "Workspace Spawner UI" or kubectl
  • it is NOT a wrapper around PodSpec
• WorkspaceKind (cluster resource)
  • this is the resource that cluster-admins create
  • it specifies the template for a Workspace (e.g. "JupyterLab", "VSCode", "RStudio")
  • initially, we would only support a "podTemplate" kind, which is very similar to the existing Notebook CRD,
    but in the future, we could support other types of templates (e.g. "helmTemplate")

[thesuperzapper]

@kubeflow/wg-notebooks-leads I would appreciate any feedback or ideas on this proposal.

[thesuperzapper]

@kimwnasptd we (that is, @juliusvonkohout, @tzstoyanov, and me) discussed this in the NB meeting today and are thinking that we can target this for Kubeflow 1.9.

I would love to get your general feedback on this proposal, and if you think it looks ok, then we can discuss some specifics and get a finalized design doc.

I think the general approach would be that we keep "kubeflow notebooks" as a "deprecated" component for 1.9, and add a new component called "kubeflow workspaces", which will have:

1. A new controller for the Workspace and WorkspaceKind CRDS
2. A new UI to manage spawning/editing/pausing/etc them
3. A new "component" section on the docs website

[kimwnasptd]

@thesuperzapper @juliusvonkohout @tzstoyanov I'm good with the way of implementing this as a new CRD, and web app since this will be the least intrusive implementation. Users will be able to pick and choose which one they'll want, until we are fully confident on using Workspaces by default.

What I'd like to confirm before committing to this effort would be what are the criteria for for considering this as the default?
* Have culling?
* Have a web app that allows us to have the same functionality as the current one?

[juliusvonkohout]

@mishraprafful and kubeflow/kubeflow#6734

[kabartay]

Thanks for this new changes. Looks good.

[thesuperzapper]

Calling anyone who wants to help move this feature forward!

Please reply if you are willing/able to help with any of the following areas:

• Frontend (JS - Angular) (Python - Flask):
  • We will need to add 2 new crud-web-apps, to manage and view the new resources.
    • The Workspace app will behave similarly to the existing Notebooks one, and will mostly be about spawning and managing the state of Workspace instances in each profile namespace.
    • The WorkspaceKind app can start with the goal of only showing the current WorkspaceKinds registered in the cluster, and in the future can allow admins to change them graphically.
• Backend (Golang - Kubebuilder) (Istio):
  • We will need to build a new controller for the Workspace and WorkspaceKind resources.
    • We should use Kubebuilder like we already do in the notebook-controller
    • The Workspace controller will also need to create the Istio VirtualServices/etc to enable access to the notebook web interfaces.
  • We will also probably need to create Webhooks to achieve some of what we are doing, which can also be created using Kubebuilder.
• Docs (Markdown and Images):
  • Docs are just as important as the code! Otherwise, people won't know about the awesome new features!
  • We will need to add a new "component" section of the Kubeflow website for "Kubeflow Workspace".

Once we have a few people willing to help, we can organize a specific call to flesh out a doc with:

• User Stories
• Controller Architecture/Behaviour
• UI Mockups
• Alternatives / Problems

---

@mishraprafful and @guohaoyu110 mentioned that they might be interested in contributing to this, also, there are some others who might be like @punkerpunker @helloericsf @wjhhuizi @wadhah101

[guohaoyu110]

• Backend (Golang - Kubebuilder) (Istio)

I am interested in this part: Backend (Golang - Kubebuilder) (Istio)

[thesuperzapper]

Also, @vvnpan @apo-ger may be interested in helping this effort, given they have raised large PRs on the old Notebooks.