feat(controller): introduce inferenceservice controller

[Al-Pragliola]

Description

closes #577

How Has This Been Tested?

Tilt testing

# if using podman
DOCKER_BUILDKIT=0 make -C devenv tilt-up
# with docker
make -C devenv tilt-up

## wait for model registry proxy to be up and running

export MR_HOSTNAME=localhost:8080

curl --silent -X 'POST' \
    "$MR_HOSTNAME/api/model_registry/v1alpha3/registered_models" \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
    "description": "Iris scikit-learn model",
    "name": "iris"
}'

curl --silent -X 'POST' \
      "$MR_HOSTNAME/api/model_registry/v1alpha3/model_versions" \
      -H 'accept: application/json' \
      -H 'Content-Type: application/json' \
      -d '{
      "description": "Iris model version v1",
      "name": "v1",
      "registeredModelID": "1"
}'

curl --silent -X 'POST' \
      "$MR_HOSTNAME/api/model_registry/v1alpha3/model_versions/2/artifacts" \
      -H 'accept: application/json' \
      -H 'Content-Type: application/json' \
      -d '{
      "description": "Model artifact for Iris v1",
      "uri": "gs://kfserving-examples/models/sklearn/1.0/model",
      "state": "UNKNOWN",
      "name": "iris-model-v1",
      "modelFormatName": "sklearn",
      "modelFormatVersion": "1",
      "artifactType": "model-artifact"
}'

kubectl apply -n kubeflow -f - <<EOF
    apiVersion: "serving.kserve.io/v1beta1"
    kind: "InferenceService"
    metadata:
      name: "iris-model"
      labels:
        modelregistry.kubeflow.org/registered-model-id: "1"
        modelregistry.kubeflow.org/model-version-id: "2"
      annotations:
        sidecar.istio.io/inject: "false"
    spec:
      predictor:
        model:
          modelFormat:
            name: sklearn
            version: "1"
          storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
    EOF

kubectl get inferenceservices.serving.kserve.io -n kubeflow iris-model -o yaml | grep inference-service-id

Manual testing

kubectl create ns kubeflow

MODEL_REGISTRY_VERSION=0.2.15
kubectl apply -k "https://github.com/kubeflow/model-registry/manifests/kustomize/overlays/db?ref=v${MODEL_REGISTRY_VERSION}"

# port forward in another shell 
kubectl port-forward --namespace kubeflow svc/model-registry-service 8080:8080

export MR_HOSTNAME=localhost:8080

curl --silent -X 'POST' \
    "$MR_HOSTNAME/api/model_registry/v1alpha3/registered_models" \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
    "description": "Iris scikit-learn model",
    "name": "iris"
}'

curl --silent -X 'POST' \
      "$MR_HOSTNAME/api/model_registry/v1alpha3/model_versions" \
      -H 'accept: application/json' \
      -H 'Content-Type: application/json' \
      -d '{
      "description": "Iris model version v1",
      "name": "v1",
      "registeredModelID": "1"
}'

curl --silent -X 'POST' \
      "$MR_HOSTNAME/api/model_registry/v1alpha3/model_versions/2/artifacts" \
      -H 'accept: application/json' \
      -H 'Content-Type: application/json' \
      -d '{
      "description": "Model artifact for Iris v1",
      "uri": "gs://kfserving-examples/models/sklearn/1.0/model",
      "state": "UNKNOWN",
      "name": "iris-model-v1",
      "modelFormatName": "sklearn",
      "modelFormatVersion": "1",
      "artifactType": "model-artifact"
}'

helm install kserve-crd oci://ghcr.io/kserve/charts/kserve-crd --version v0.15.0

docker build -t docker.io/controller/controller:0.1.0 -f ./cmd/controller/Dockerfile.controller .

kind load docker-image docker.io/controller/controller:0.1.0

IMG_REPO=controller/controller make controller/deploy

kubectl patch deployment -n kubeflow controller-controller-manager --patch '{"spec": {"template": {"spec": {"containers": [{"name": "manager", "image": "controller/controller:0.1.0", "imagePullPolicy": "IfNotPresent"}]}}}}'

kubectl apply -n kubeflow -f - <<EOF
    apiVersion: "serving.kserve.io/v1beta1"
    kind: "InferenceService"
    metadata:
      name: "iris-model"
      labels:
        modelregistry.kubeflow.org/registered-model-id: "1"
        modelregistry.kubeflow.org/model-version-id: "2"
      annotations:
        sidecar.istio.io/inject: "false"
    spec:
      predictor:
        model:
          modelFormat:
            name: sklearn
            version: "1"
          storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
    EOF

kubectl get inferenceservices.serving.kserve.io -n kubeflow iris-model -o yaml | grep inference-service-id

output of grep should be

modelregistry.kubeflow.org/inference-service-id: "4"

Merge criteria:

• All the commits have been signed-off (To pass the DCO check)

• The commits have meaningful messages; the author will squash them after approval or in case of manual merges will ask to merge with squash.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work.
• Code changes follow the kubeflow contribution guidelines.
• For first time contributors: Please reach out to the Reviewers to ensure all tests are being run, ensuring the label ok-to-test has been added to the PR.

[Al-Pragliola]

/cc @pboyd @dhirajsb @adysenrothman

[google-oss-prow]

@Al-Pragliola: GitHub didn't allow me to request PR reviews from the following users: dhirajsb, adysenrothman.

Note that only kubeflow members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

/cc @pboyd @dhirajsb @adysenrothman

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[pboyd]

I had few minor comments, but nothing major.

/lgtm

[pboyd]

What do you think about starting with ubi9? I understand that we'll have to upgrade soon.

[Al-Pragliola]

We are stuck in ubi8 until we create an alternative to mlmd, I would like to wait until then and move everything to ubi9

[Al-Pragliola]

thanks for your suggestions @pboyd 🙏🏼

[pboyd]

/lgtm

[adysenrothman]

good work on this! /lgtm

[pboyd]

/lgtm

[tarilabs]

I've restarted the Test CSI GHA.

[tarilabs]

@pboyd can you kindly reapply lgtm, from my side
/approve

[pboyd]

/lgtm

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pboyd, tarilabs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [tarilabs]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment