The choices of the number of GPUs in Kubeflow Jupyter Notebook is not enough.

[ahmed1alqulayti]

Hello all, good work as usual, I have a simple question, which is if I have 3 GPUs in my cluster and I can't use them once for Jupyter Notebook since option 3 does not exist in Kubeflow UI, how can I solve this problem, please?

[ReggieCarey]

If your GPUs are on different nodes, you will not be able to create a single pod to consume them all. That is a limitation of Kubernetes, pods don’t span nodes. You can create a kf pipeline that can create multiple pods each which consumes a GPU. Sent from my iPhoneOn Mar 21, 2023, at 01:37, Ahmed Alqulayti ***@***.***> wrote: /kind question Question: [You can ask any question about this project.] Hello all, good work as usual, I have a simple question, which is if I have 3 GPUs in my cluster and I can't use them once for Jupyter Notebook since option 3 does not exist in Kubeflow UI, how can I solve this problem, please? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Jeet-14]

@ahmed1alqulayti
Dependes on installation type, but setting value to configmap would help to choose desired number of GPU on UI side.

https://github.com/kubeflow/manifests/blob/master/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml#L114

[TobiasGoerke]

@ahmed1alqulayti Dependes on installation type, but setting value to configmap would help to choose desired number of GPU on UI side.

https://github.com/kubeflow/manifests/blob/master/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml#L114

This doesn't work (anymore?), even though the comment in the yaml file suggests it. The gpuCount is hardcoded in the jupyter web app

[andreyvelich]

/transfer notebooks

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed if no further activity occurs.
Thank you for your contributions.

Members may comment /lifecycle frozen to prevent this issue from being marked as stale.