[TRACKING] Kubeflow 1.10.0 (Notebooks WG Components)

[thesuperzapper]

This issue is to track the 1.10.0 release for the kubeflow/kubeflow repo, which includes Kubeflow Notebooks and other core components like Central Dashboard and Profile Controller.

---

Changes for 1.10.0 (GH Milestone)

These issues are what we are actually targeting for 1.10.0.

Repo Cleanup

• Splitting kubeflow/kubeflow repo into kubeflow/notebooks and kubeflow/dashboard:
  • [TRACKING] discussion & planning for future of kubeflow/kubeflow repo #7549
  • NOTES:
    • It's unlikely that we will get this done for 1.10, as we plan to cut the 1.10.0-rc.0 from the kubeflow/kubeflow repo.
    • After we do the initial 1.10 cut @juliusvonkohout and @kimwnasptd have said they will start migrating the dashboard components to kubeflow/dashboard

Notebooks

• ...

Dashboard

• ...

---

Consider for 1.10.0 (GH Milestone)

This content comes from the 1.9.0 tracking issue that were labeled as 1.10.0.

NOTE: as we are so close to the 1.10.0 release, many of these things may not make their way in, so we need to prune this list.

CI/CD

• ci: read-only token-permissions for all workflows #7278

Notebook Images

• Add kubeflow-katib and kserve packages to default notebook images notebooks#122

Notebooks Controller

• tensorboard fails to connect if there is a notebook with the same name dashboard#50
  • Fixed By: fix: resolve service name conflict for notebooks and tensorboards #7468
  • WARNING: this needs a lot of testing and work to ensure it is safe to do in-place upgrades
• notebook-controller: Update dependency versions #7152
  • WARNING: this can be deferred to a future update as it will need to be rebased and retested.
• feat: always set a Pod status in Notebook CRD #7015
  • WARNING: needs to be reviewed
• fix: add lease RBAC to notebook controller role (for leader election) #6942
  • WARNING: needs to be reviewed
• feat: expose runningCount and totalCount in notebook controller metrics #6838
  • WARNING: needs to be reviewed
• feat: expose NotebookCullingCount and NotebookCullingTimestamp in prometheus metrics #6488
  • WARNING: needs to be reviewed
• fix: unnecessary notebook updates caused by compare issues #6474
  • WARNING: needs to be reviewed
• fix: add lease RBAC to notebook controller role (for leader election) #6942
  • WARNING: needs to be reviewed

Profile Controller

• profile controller fails to update profile owner dashboard#33
  • Fixed by: Fix (profile controller): Profile controller fails to update profile owner, fixes #6576 and #5449 #7276
  • Fixed By: feat: allow updating profile owners #7547
    • WARNING: this needs a bit of work, it is NOT ready to merge
• feat(profile-controller): propagate labels from CR to namespace #7481
  • THIS SHOULD NOT BE INCLUDED
  • WARNING: including this introduces issues for ArgoCD and possible security issues by allowing users to set arbitrary namespace labels by updating profile labels.
• fix: extend profile AuthorizationPolicy for KServe models #6627
  • WARNING: needs to be updated
• fix: stop unnecessary updates of profiles #6435
  • WARNING: needs to be reviewed
• feat: make profile controller error logs more descriptive #6231
  • WARNING: needs to be reviewed

Tensorboard Controller

• Stop using gcr.io/kubebuilder/kube-rbac-proxy in TensorBoards before March 18, 2025 dashboard#58
  • NEEDS A PR
• fix: add error handling to tensorboard controller #7067
  • WARNING: needs to be reviewed
• fix: use "required" in tensorboard node affinity, rather than "preferred" #7024
  • WARNING: needs to be reviewed
• feat: make TensorBoard image configurable in CRD #6783
  • WARNING: needs to be reviewed

Notebook Spawner UI

• chore: remove unused jupyter-notebook-role and jupyter-notebook-role-binding #7616
  • This looks mostly ok to me, as I cant see any usage of these roles/rolebinding.
• feat: add gpusCount to spawner_ui_config.yaml #7295
  • WARNING: in its current state, this is a breaking change, we need to find a way to make it backwards compatible.
• feat: add memoryVendors and arbitrary counts for notebook GPU resource limits #6657
  • WARNING: there are a bunch of PRs which do similar things, we should only merge one of them
• feat(notebooks): only display allocatable GPU configuration when creating notebook #7124
  • WARNING: needs to be reviewed
• feat(notebook): dynamically display the allocatable GPU type/num in notebook server #7134
  • WARNING: needs to be reviewed
• web-apps: Implement watch to avoid constant polling from backend to K8s #7080
  • WARNING: needs to be reviewed
• feat: support selector.matchExpression in PodDefaults #7551
  • WARNING: needs to be reviewed
• fix: notebooks ui should support PodDefaults with matchExpressions #6983
  • WARNING: needs to be reviewed
• fix: typo when setting server_type for jupyter notebook (no in app affect) #6666
  • this change does nothing, but is fine to fix a typo
• fix: correctly display notebook images from registries with port #6351
  • WARNING: needs to be reviewed

PVC Viewer UI / Controller

• volume app list pvcs facing 404 error after upgrade kubeflow from v1.7 to v1.8 #7414
  • Caused by: Add PVCViewer Frontend Integration #7179 (see this part)
  • Fixed By: (NEED A PR)
• pvcviewer-controller-manager creates pvcviewer pod on different node from notebook server pod dashboard#60:
  • Fixed By: (NEED A PR)

Central Dashboard

• fix: show correct BUILD_VERSION in centraldashboard #6978
  • THIS SHOULD NOT BE INCLUDED
  • WARNING: we can probably just remove the build version section of the UI instead
• fix: double URL encoding in central-dashboard links #6564
  • WARNING: needs to be reviewed
• feat(dashboard): include fonts as assets #6009
  • WARNING: we need to confirm we are allowed to include these fonts in Apache 2.0 licenced code
• fix: show error logs for YAML parsing in notebooks ui #7497
  • WARNING: needs to be reviewed

Admission Webhook (PodDefaults)

• jupyter-web-app's PodDefault configurations are keyed by their label selector's key, not their name notebooks#121
  • (Need PR)
• feat: allow same volume to be mounted multiple times in single PodDefault #6847
  • WARNING: needs to be reviewed

KFAM

• feat: use native RBAC subject to find RoleBindings in KFAM, rather than annotations #6839
  • WARNING: needs to be reviewed

[kimwnasptd]

I'd suggest that we still cut a branch from what we have (happy to help with the PRs and cut the branches/tags)

Specifically those are some small items we worked on from Canonical's side

• feat(central-dashboard): Add Prometheus metrics with prom-client #7639
• feat(crud-web-apps): Add Prometheus metrics #7634

Plus a couple of other small nice ones like:

• fix: logout button for oauth2-proxy #7654
• fix: return notebook events error message #7649

@thesuperzapper could you help with the reviews of PRs I'll make for updating the branch etc?