Description
related #85
Now that #34 is merged, there are only a small number of remaining tasks for the "Kubeflow Workspaces Controller to be finished.
One task to finish the workspace controller loop, so it also reconciles the Istio VirtualServices required to make the Workspace Pods accessible on the Istio mesh
Implementation
Here is the TODO in the code:
notebooks/workspaces/controller/internal/controller/workspace_controller.go
Lines 345 to 348 in bc4e445
VirtualService Definition
The structure of the HTTP paths exposed by each Workspace should be: /workspace/<NAMESPACE_NAME>/<WORKSPACE_NAME>/<PORT_ID>/
Where PORT_ID
is the id of a port defined in the currently selected spec.podTemplate.options.imageConfig
, which is defined in the WorkspaceKind under spec.podTemplate.options.imageConfig.values[].spec.ports
Note, this path structure is already reflected in the httpPathPrefix
function for the spec.podTemplate.extraEnv[0].value
go templates (which should probably be factored into a helper):
notebooks/workspaces/controller/internal/controller/workspace_controller.go
Lines 594 to 604 in bc4e445
Example VirtualService
Here is a templated example of the VirtualService we should reconcile:
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
## for reference, see the existing `generateNamePrefix()` usage for `Service` and `StatefulSet` generation
name: "ws-<WORKSPACE_NAME>-<RANDOM_SUFFIX>"
namespace: "<PROFILE_NAME>"
## NOTE: see the full VirtualService spec here:
## https://istio.io/latest/docs/reference/config/networking/virtual-service/
spec:
gateways:
## read this from an environment variable called `ISTIO_GATEWAY`, set on the controller
## DEFAULT: "kubeflow/kubeflow-gateway"
- "<ISTIO_GATEWAY>"
hosts:
## read this from an environment variable called `ISTIO_HOST`, set on the controller
## DEFAULT: "*"
- "<ISTIO_HOST>"
http:
## there should be an element for EACH of the `ports` in the Workspace currently selected `imageConfig`
- match:
- uri:
prefix: "/workspace/<PROFILE_NAME>/<WORKSPACE_NAME>/<PORT_ID>/"
## if `httpProxy.removePathPrefix` is true, include this block. otherwise, remove it
rewrite:
uri: "/"
headers:
## implement any `httpProxy.requestHeaders` manipulation here
## DOCS: https://istio.io/latest/docs/reference/config/networking/virtual-service/#Headers-HeaderOperations
request:
## NOTE: the default is {}, this is just an example
## NOTE: this turns into "/workspace/{PROFILE_NAME}/{WORKSPACE_NAME}/{PORT_ID}/" when rendered
## NOTE: implement go templates the same as `httpPathPrefix` used in the `spec.podTemplate.extraEnv[].value`
## WARNING: remember to add a new Webhook validation for this field (and `add`) to ensure it's a valid Go template
set:
X-RStudio-Root-Path: |-
{{ httpPathPrefix "PORT_ID" }}
add: {}
remove: []
route:
- destination:
## read CLUSTER_DOMAIN from an environment variable called `CLUSTER_DOMAIN`, set on the controller
## DEFAULT: "cluster.local"
host: "<SERVICE_NAME>.<PROFILE_NAME>.svc.<CLUSTER_DOMAIN>"
port:
number: <PORT_NUMBER>
CRDs
Besides making the Pod available, the VirtualService will also implement the spec.podTemplate.ports[].httpProxy
part of the WorkspaceKind spec:
spec:
podTemplate:
## port configs (MUTABLE)
##
ports:
## configs to apply to a specific port
## - ports are identified by their `ports[].id` from the `imageConfig` spec
##
- portId: "jupyterlab"
## httpProxy configs for the port
## - note, only "HTTP" protocol ports are supported
##
httpProxy:
## if the HTTP path prefix should be removed
## - if true, the '/workspace/{PROFILE_NAME}/{WORKSPACE_NAME}/{PORT_ID}' path prefix
## will be stripped from incoming requests
## - that is, the application sees the request as if it was made to '/...'
## - this only works if the application serves RELATIVE URLs for its assets
##
removePathPrefix: false
## header manipulation rules for incoming HTTP requests
## - sets the `spec.http[].headers.request` of the Istio VirtualService
## https://istio.io/latest/docs/reference/config/networking/virtual-service/#Headers-HeaderOperations
## - the following go template functions are available:
## - `httpPathPrefix(portId string)`: returns the HTTP path prefix of the specified port (regardless of `removePathPrefix`)
##
requestHeaders: {}
#set: { "X-RStudio-Root-Path": "{{ httpPathPrefix 'rstudio' }}" } # example for RStudio
#add: {}
#remove: []
Changes from current CRDs
After designing and discussing this implementation, we figured that we need to make some small additions to the existing Notebooks 2.0 CRDs:
- WorkspaceKind:
- Remove the
spec.podTemplate.httpProxy
map, and replace it withspec.podTemplate.ports
list. - Each element of
spec.podTemplate.ports
will be a map with the following keys:portId
: the map-index of this array and determines which port the configs are applied tohttpProxy
: the same as the old ones, but now scoped to a specific port.
- Remove the
Metadata
Metadata
Type
Projects
Status