Kubeflow Platform (Manifests & Security WG) roadmap for KF 1.10.0

[juliusvonkohout]

Describe your issue

Tracker for 1.10 @rimolive @kimwnasptd
Follow up of #2598 and #2592

We are looking for volunteers and I already mentored over 20 people, so please reach out if you want to help.

Organizational:

• Kubeflow 1.9 follow up: Upstream our authorizationpolicy and oauth2-proxy changes to kubeflow/pipelines and kubeflow/kubeflow #2804
• RENAME /contrib to /experimental #2922
• Use the new CNCF Zoom account for the platform meeting #2783
• Merge security and manifests WG meeting in a cal entry community#725
• Disable issue creation and update README kubeflow#7618
• Expose from /contrib ray and seldon in the example.yaml (commented out)
• Kserve web application: can you help with rebasing/forking the 5 PRs here and merging them https://github.com/kserve/models-web-app/pulls. We also need someone to go trough https://github.com/kserve/models-web-app/issues and close or confirm whether they are still valid.
• SPARK / KSERVE promote from contribution to application #2912

documentation:

• update oauth2-proxy readme #2930
• Dex,oauth2-proxy and k8s token documentation #2911

chore:

• update cert-manager to 1.16.1 and move the upstream manifests to the upstream folder #2919
• Synchronize knative manifests v1.16.0 #2917
• Update Dex and oauth2-proxy to 2.41.1 and 7.7.1 #2916
• Synchronize kserve manifests v0.14.0 #2908 --server-side --force-conflicts since CRDs became too big in Kserve 0.14.0
• Upgrade istio-cni to v.1.23.2 #2903
• Upgrade istio to v.1.23.2 #2890
• kustomize 5.4.3 and documentation updates #2929 (cilium compatibility)
• remove seldon, bentoml and metacontroller (included in upstream KFP) #2920
• Dex,oauth2-proxy and k8s token documentation #2911

Security external:

• WIP: feat(frontend, backend): Delete python visualization server pipelines#11134
• [backend] Security exploit in mlpipeline-UI pipelines#9889
• [feature] Seek alternative for object store solution other than MinIO pipelines#7878
• feat(backend): isolate artifacts per namespace/profile/user using only one bucket pipelines#7725
• [Multi User] Support separate metadata for each namespace pipelines#4790
• chore: Set Python 3.9 as the Minimum Supported Version pipelines#11159 as follow up of chore(backend): update python from 3.7 to 3.12 pipelines#10950
• [backend] Unable to create directory in Minio when using Artifacts: Permission denied pipelines#10397
• kserve/models-web-app v0.10.0 using kubeflow/kubeflow#2b4cc42e (not v1.7.0) #2433

Performance / Denial of service:

• web-apps: Implement watch to avoid constant polling from backend to K8s kubeflow#7080
• Kubeflow Notebooks 2.0 (Kubeflow Workspaces) notebooks#85 a single bad CR can kill the controller of v1 Workbenches
• Notebook Culling for VSCode/CodeServer style notebooks notebooks#93
• NOT SOLVED Support big (5+ GB) s3 artifacts in pipelines EOF when many steps using same big input artifact argoproj/argo-workflows#9525 and Download large file from S3 cause memory rise in the init container argoproj/argo-workflows#1322 (they are not solved)

Security JWTs:

• proposal: Uniform way for handling JWTs #2748
• Getting Istio RBAC issues when submitting a pipeline from a Notebook #2747
• authorize requests to ml-pipeline endpoint if contains any trusted principals #2753

Rootless Kubeflow & Istio:

• Rootless Kubeflow #2528
• Make istio CNI the default and set the kserve-needed annotation across all Pods. We should use the new "Values.gateways.seccompProfile" and securitycontext to get rid of the manual patch at

  manifests/.github/workflows/pss_test.yaml

  Lines 35 to 45 in 26d970d

  	- name: Configure istio init container with seccompProfile attribute
  	run: |
  	kubectl get cm istio-sidecar-injector -n istio-system -o yaml > temporary_patch.yaml
  	sed -i '0,/runAsNonRoot: true/{s//&\n seccompProfile:\n type: RuntimeDefault/}' temporary_patch.yaml
  	sed -i '/runAsNonRoot: true/{N; /runAsUser: {{ .ProxyUID | default "1337" }}/a\
  	seccompProfile:\n type: RuntimeDefault
  	}' temporary_patch.yaml
  	kubectl apply -f temporary_patch.yaml
  	rm temporary_patch.yaml

  . First PR: ISTIO CNI by default #2907
• Istio Ambient support #2676

Rest:

• [bug] v1 Metrics not working in KFP 2.2.0 pipelines#10988
• feat: allow updating profile owners kubeflow#7547
• Kubeflow 1.10 Enterprise grade Agenda #2727
• Remove the old tekton version #2765
• Add extra namespace labels option kubeflow#7325
• Update the ray example cluster to the stable v1 version and integrate istio #2742
• To many open issues and PRs and no stalebot #2514
• Deprecate optional Knative Eventing components from common manifests #2685
• Add the networkpolicies to cicd tests #2718
• Fix view edit cluster roles pipelines#11067

[kimwnasptd]

(Orthogonal question, but why is this called Kubeflow Platform and includes Manifests and Security WGs in it?

Especially once we'll have the repos broken up after the discussion in kubeflow/kubeflow#7549 (comment)

[thesuperzapper]

Similar to what @kimwnasptd said, it's hard to imagine that "Kubeflow Platform" should not also involve the distribution owners? I am not sure we should be calling the meetings "Kubeflow Platform" if they are only referring to Manifests/Security WGs.

[andreyvelich]

(Orthogonal question, but why is this called Kubeflow Platform and includes Manifests and Security WGs in it?

Especially once we'll have the repos broken up after the discussion in kubeflow/kubeflow#7549 (comment)

@kimwnasptd The main motivation is to identify contributors who can maintain Kubeflow Platform components. E.g. Kubeflow Manifests, Profile Controller, KFAM, Central Dashboard, etc. So if users have any questions about these components they can join the Kubeflow Platform community calls.
Since @juliusvonkohout was driving the security effort around Kubeflow ecosystem, he included the security roadmap to the Kubeflow Platform discussion.

From my perspective, distributions should certainly join those meetings since they are main consumers of Kubeflow Manifests.

[juliusvonkohout]

@kimwnasptd it is in line with kubeflow/community#725 (comment)