Skip to content

Commit e6dbfb1

Browse files
authored
Improve Canary guide and add prerequisites (#225)
1 parent 86c6a1f commit e6dbfb1

File tree

1 file changed

+44
-26
lines changed

1 file changed

+44
-26
lines changed

docs/canary_ab_deployment.md

Lines changed: 44 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,56 @@
1-
## Model deployment to AKS cluster with Canary deployment
1+
# Model deployment to AKS cluster with Canary deployment <!-- omit in toc -->
22

33
[![Build Status](https://aidemos.visualstudio.com/MLOps/_apis/build/status/microsoft.MLOpsPython-Canary?branchName=master)](https://aidemos.visualstudio.com/MLOps/_build/latest?definitionId=133&branchName=master)
44

5-
If your target deployment environment is a K8s cluster and you want to implement [Canary and/or A/B testing deployemnt strategies](http://adfpractice-fedor.blogspot.com/2019/04/deployment-strategies-with-kubernetes.html) you can follow this sample guidance.
5+
If your target deployment environment is a Kubernetes cluster and you want to implement [Canary and/or A/B testing deployment strategies](http://adfpractice-fedor.blogspot.com/2019/04/deployment-strategies-with-kubernetes.html) you can follow this sample guide.
66

7-
**Note:** It is assumed that you have an AKS instance and configured ***kubectl*** to communicate with the cluster.
7+
- [Prerequisites](#prerequisites)
8+
- [Install Istio on a K8s cluster](#install-istio-on-a-k8s-cluster)
9+
- [Set up variables](#set-up-variables)
10+
- [Configure a pipeline to build and deploy a scoring Image](#configure-a-pipeline-to-build-and-deploy-a-scoring-image)
11+
- [Build a new Scoring Image](#build-a-new-scoring-image)
812

9-
#### 1. Install Istio on a K8s cluster.
13+
## Prerequisites
1014

11-
This guidance uses [Istio](https://istio.io) service mesh implememtation to control traffic routing between model versions. The instruction on installing Istio is available [here](https://docs.microsoft.com/en-us/azure/aks/servicemesh-istio-install?pivots=client-operating-system-linux).
15+
Before continuing with this guide, you will need:
1216

13-
Having the Istio installed, figure out the Istio gateway endpoint on your K8s cluster:
17+
* An [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service) cluster
18+
* This does **not** have to be the same cluster as the example in [Getting Started: Deploy the model to Azure Kubernetes Service](/docs/getting_started.md#deploy-the-model-to-azure-kubernetes-service)
19+
* The cluster does not have to be connected to Azure Machine Learning.
20+
* If you want to deploy a new cluster, see [Quickstart: Deploy an Azure Kubernetes Service cluster using the Azure CLI](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough)
21+
* An Azure Container Registry instance that is authenticated with your Azure Kubernetes Service cluster.
22+
* The chart you will deploy is assuming you are authenticated using a service principal.
23+
* See [Authenticate with Azure Container Registry from Azure Kubernetes Service](https://docs.microsoft.com/en-us/azure/aks/cluster-container-registry-integration#configure-acr-integration-for-existing-aks-clusters) for an authentication guide.
24+
* In Azure DevOps, a service connection to your Kubernetes cluster.
25+
* If you do not currently have a namespace, create one named 'abtesting'.
26+
27+
## Install Istio on a K8s cluster
28+
29+
You'll be using the [Istio](https://istio.io) service mesh implementation to control traffic routing between model versions. Follow the instructions at [Install and use Istio in Azure Kubernetes Service (AKS)](https://docs.microsoft.com/azure/aks/servicemesh-istio-install?pivots=client-operating-system-linux).
30+
31+
After Istio is installed, figure out the Istio gateway endpoint on your K8s cluster:
1432

1533
```bash
1634
GATEWAY_IP=$(kubectl get svc istio-ingressgateway -n istio-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
1735
```
1836

1937
You don't need to create any Istio resources (e.g. Gateway or VirtualService) at this point. It will be handled by the AzDo pipeline that builds and deploys a scoring image.
2038

21-
#### 2. Set up variables
39+
## Set up variables
2240

2341
There are some extra variables that you need to setup in ***devopsforai-aml-vg*** variable group (see [getting started](./getting_started.md)):
2442

25-
| Variable Name | Suggested Value |
26-
| --------------------------- | -----------------------------------------------------|
27-
| K8S_AB_SERVICE_CONNECTION | AzDo service connection to a K8s cluster |
28-
| K8S_AB_NAMESPACE | Namespace in a K8s cluster to deploy the model |
29-
| IMAGE_REPO_NAME | Image reposiory name (e.g. mlopspyciamlcr.azurecr.io)|
43+
| Variable Name | Suggested Value | Short Description |
44+
|---------------------------|-----------------------|-----------------------------------------------------------|
45+
| K8S_AB_SERVICE_CONNECTION | mlops-aks | Name of the service connection to your Kubernetes cluster |
46+
| K8S_AB_NAMESPACE | abtesting | Kubernetes namespace for model deployment |
47+
| IMAGE_REPO_NAME | [Your ACR's DNS name] | Image reposiory name (e.g. mlopspyciamlcr.azurecr.io) |
3048

31-
#### 3. Configure a pipeline to build and deploy a scoring Image
49+
## Configure a pipeline to build and deploy a scoring Image
3250

3351
Import and run the [abtest.yml](./.pipelines/abtest.yml) multistage deployment pipeline.
3452

35-
The result of the pipeline will be a registered Docker image in the ACR repository attached to the AML Service:
53+
After the pipeline completes successfully, you will see a registered Docker image in the ACR repository attached to the Azure ML Service:
3654

3755
![scoring image](./images/scoring_image.png)
3856

@@ -44,21 +62,21 @@ NAME READY UP-TO-DATE AVAILABLE AGE
4462
model-green 1/1 1 1 19h
4563
```
4664

47-
#### 4. Build a new Scoring Image
65+
## Build a new Scoring Image
4866

4967
Change value of the ***SCORE_SCRIPT*** variable in the [abtest.yml](./.pipelines/abtest.yml) to point to ***scoring/scoreA.py*** and merge it to the master branch.
5068

51-
**Note:** ***scoreA.py*** and ***scoreB.py*** files used in this tutorial are just mockups returning either "New Model A" or "New Model B" respectively. They are used to demonstrate the concept of testing two scoring images with different models or scoring code. In real life you would implement a scoring file similar to [score.py](./../code/scoring/score.py) (see [getting started](./getting_started.md)).
69+
**Note:** ***scoreA.py*** and ***scoreB.py*** files used in this tutorial are just mockups returning either "New Model A" or "New Model B" respectively. They are used to demonstrate the concept of testing two scoring images with different models or scoring code. In real life you would implement a scoring file similar to [score.py](./../code/scoring/score.py) (see the [Getting Started](./getting_started.md) guide).
5270

5371
It will automatically trigger the pipeline and deploy a new scoring image with the following stages implementing ***Canary*** deployment strategy:
5472

55-
| Stage | Green Weight| Blue Weight| Description |
56-
| ------------------- |-------------|------------|-----------------------------------------------------------------|
57-
| Blue_0 |100 |0 |New image (blue) is deployed.<br>But all traffic (100%) is still routed to the old (green) image.|
58-
| Blue_50 |50 |50 |Traffic is split between old (green) and new (blue) images 50/50.|
59-
| Blue_100 |0 |100 |All traffic (100%) is routed to the blue image.|
60-
| Blue_Green |0 |100 |Old green image is removed. The new blue image is copied as green.<br>Blue and Green images are equal.<br>All traffic (100%) is routed to the blue image.|
61-
| Green_100 |100 |0 |All traffic (100%) is routed to the green image.<br>The blue image is removed
73+
| Stage | Green Weight | Blue Weight | Description |
74+
|------------|--------------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
75+
| Blue_0 | 100 | 0 | New image (blue) is deployed.<br>But all traffic (100%) is still routed to the old (green) image. |
76+
| Blue_50 | 50 | 50 | Traffic is split between old (green) and new (blue) images 50/50. |
77+
| Blue_100 | 0 | 100 | All traffic (100%) is routed to the blue image. |
78+
| Blue_Green | 0 | 100 | Old green image is removed. The new blue image is copied as green.<br>Blue and Green images are equal.<br>All traffic (100%) is routed to the blue image. |
79+
| Green_100 | 100 | 0 | All traffic (100%) is routed to the green image.<br>The blue image is removed. |
6280

6381
**Note:** The pipeline performs the rollout without any pausing. You may want to configure [Approvals and Checks](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass) for the stages on your environment for better experience of the model testing. The environment ***abtestenv*** will be added automatically to your AzDo project after the first pipeline run.
6482

@@ -74,7 +92,7 @@ You can also emulate a simple load test on the gateway with the ***load_test.sh*
7492
./charts/load_test.sh 10 $GATEWAY_IP/score
7593
```
7694

77-
The command above sends 10 requests to the gateway. So if the pipeline has completted stage Blue_50, the result will look like this:
95+
The command above sends 10 requests to the gateway. So if the pipeline has completed stage Blue_50, the result will look like this:
7896

7997
```bash
8098
"New Model A"
@@ -89,14 +107,14 @@ The command above sends 10 requests to the gateway. So if the pipeline has compl
89107
"New Model A"
90108
```
91109

92-
Despite what blue/green weights are configured now on the cluster, you can perform ***A/B testing*** and send requests directly to either blue or green images:
110+
Regardless of the blue/green weight values set on the cluster, you can perform ***A/B testing*** and send requests directly to either blue or green images:
93111

94112
```bash
95113
curl --header "x-api-version: blue" $GATEWAY_IP/score
96114
curl --header "x-api-version: green" $GATEWAY_IP/score
97115
```
98116

99-
or with the load_test.sh:
117+
or with a load_test.sh script:
100118

101119
```bash
102120
./charts/load_test.sh 10 $GATEWAY_IP/score blue

0 commit comments

Comments
 (0)