Skip to content

Commit 668a3ec

Browse files
Yao XiaoBobgy
Yao Xiao
authored and
Bobgy
committed
feat(manifest): add support for Standalone KFP on AWS. Fixes #4337 (#4350)
1 parent 135715d commit 668a3ec

10 files changed

+255
-1
lines changed

‎manifests/kustomize/README.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ To install Kubeflow Pipelines, you have several options.
88
- Via [GCP AI Platform UI](http://console.cloud.google.com/ai-platform/pipelines).
99
- Via an upcoming commandline tool.
1010
- Via Kubectl with Kustomize, it's detailed here.
11-
- Community maintains a repo [here](https://github.com/e2fyi/kubeflow-aws/tree/master/pipelines) for AWS.
1211

1312
## Install via Kustomize
1413

@@ -51,6 +50,13 @@ Its storage is based on CloudSQL & GCS. It's better than others for production u
5150

5251
Please following [sample](sample/README.md) for a customized installation.
5352

53+
### Option-4 Install it to AWS with S3 and RDS MySQL
54+
Its storage is based on S3 & AWS RDS. It's more natural for AWS users to use this option.
55+
56+
Please following [AWS Instructions](env/aws/README.md) for installation.
57+
58+
Note: Community maintains a repo [e2fyi/kubeflow-aws](https://github.com/e2fyi/kubeflow-aws/tree/master/pipelines) for AWS.
59+
5460
## Uninstall
5561

5662
If the installation is based on CloudSQL/GCS, after the uninstall, the data is still there,

‎manifests/kustomize/env/aws/OWNERS

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
approvers:
2+
- Jeffwan
3+
- PatrickXYS

‎manifests/kustomize/env/aws/README.md

+82
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Sample installation
2+
3+
1. Create an EKS cluster and setup kubectl context
4+
5+
Using configuration file to simplify EKS cluster creation process:
6+
```
7+
apiVersion: eksctl.io/v1alpha5
8+
kind: ClusterConfig
9+
metadata:
10+
name: kfworkshop
11+
region: us-west-2
12+
version: '1.17'
13+
# If your region has multiple availability zones, you can specify 3 of them.
14+
availabilityZones: ["us-west-2b", "us-west-2c", "us-west-2d"]
15+
16+
# NodeGroup holds all configuration attributes that are specific to a nodegroup
17+
# You can have several node group in your cluster.
18+
nodeGroups:
19+
- name: cpu-nodegroup
20+
instanceType: m5.xlarge
21+
desiredCapacity: 2
22+
minSize: 0
23+
maxSize: 4
24+
volumeSize: 50
25+
# ssh:
26+
# allow: true
27+
# publicKeyPath: '~/.ssh/id_rsa.pub'
28+
29+
# Example of GPU node group
30+
- name: Tesla-V100
31+
instanceType: p3.8xlarge
32+
# Make sure the availability zone here is one of cluster availability zones.
33+
availabilityZones: ["us-west-2b"]
34+
desiredCapacity: 0
35+
minSize: 0
36+
maxSize: 4
37+
volumeSize: 50
38+
# ssh:
39+
# allow: true
40+
# publicKeyPath: '~/.ssh/id_rsa.pub'
41+
```
42+
Run this command to create EKS cluster
43+
```
44+
eksctl create cluster -f cluster.yaml
45+
```
46+
47+
2. Prepare S3
48+
49+
Create S3 bucket. [Console](https://console.aws.amazon.com/s3/home).
50+
51+
Run this command to create S3 bucket by changing `<YOUR_S3_BUCKET_NAME>` to your prefer s3 bucket name.
52+
53+
```
54+
export S3_BUCKET=<YOUR_S3_BUCKET_NAME>
55+
export AWS_REGION=us-west-2
56+
aws s3 mb s3://$S3_BUCKET --region $AWS_REGION
57+
```
58+
59+
3. Prepare RDS
60+
61+
Follow this [doc](https://www.kubeflow.org/docs/aws/rds/#deploy-amazon-rds-mysql-in-your-environment) to set up AWS RDS instance.
62+
63+
4. Customize your values
64+
- Edit [params.env](params.env), [secret.env](secret.env) and [minio-artifact-secret-patch.env](minio-artifact-secret-patch.env)
65+
66+
5. Install
67+
68+
```
69+
kubectl apply -k ../../cluster-scoped-resources
70+
71+
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
72+
73+
kubectl apply -k ./
74+
# If upper one action got failed, e.x. you used wrong value, try delete, fix and apply again
75+
# kubectl delete -k ./
76+
77+
kubectl wait applications/mypipeline -n kubeflow --for condition=Ready --timeout=1800s
78+
79+
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
80+
```
81+
82+
Now you can access via `localhost:8080`

‎manifests/kustomize/env/aws/aws-configuration-patch.yaml

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ml-pipeline-ui
5+
spec:
6+
template:
7+
metadata:
8+
labels:
9+
app: ml-pipeline-ui
10+
spec:
11+
volumes:
12+
- name: config-volume
13+
configMap:
14+
name: ml-pipeline-ui-configmap
15+
containers:
16+
- name: ml-pipeline-ui
17+
env:
18+
- name: AWS_ACCESS_KEY_ID
19+
valueFrom:
20+
secretKeyRef:
21+
name: mlpipeline-minio-artifact
22+
key: accesskey
23+
- name: AWS_SECRET_ACCESS_KEY
24+
valueFrom:
25+
secretKeyRef:
26+
name: mlpipeline-minio-artifact
27+
key: secretkey
28+
29+
---
30+
apiVersion: apps/v1
31+
kind: Deployment
32+
metadata:
33+
name: ml-pipeline
34+
spec:
35+
template:
36+
metadata:
37+
labels:
38+
app: ml-pipeline
39+
spec:
40+
containers:
41+
- env:
42+
- name: OBJECTSTORECONFIG_SECURE
43+
value: "true"
44+
- name: OBJECTSTORECONFIG_BUCKETNAME
45+
valueFrom:
46+
configMapKeyRef:
47+
name: pipeline-install-config
48+
key: bucketName
49+
- name: OBJECTSTORECONFIG_HOST
50+
valueFrom:
51+
configMapKeyRef:
52+
name: pipeline-install-config
53+
key: minioServiceHost
54+
- name: OBJECTSTORECONFIG_REGION
55+
valueFrom:
56+
configMapKeyRef:
57+
name: pipeline-install-config
58+
key: minioServiceRegion
59+
- name: OBJECTSTORECONFIG_PORT
60+
value: ""
61+
name: ml-pipeline-api-server

‎manifests/kustomize/env/aws/config

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
namespace: $(kfp-namespace),
3+
executorImage: gcr.io/ml-pipeline/argoexec:v2.7.5-license-compliance,
4+
containerRuntimeExecutor: $(kfp-container-runtime-executor),
5+
artifactRepository:
6+
{
7+
s3: {
8+
bucket: $(kfp-artifact-bucket-name),
9+
keyPrefix: artifacts,
10+
endpoint: s3.amazonaws.com,
11+
insecure: true,
12+
accessKeySecret: {
13+
name: mlpipeline-minio-artifact,
14+
key: accesskey
15+
},
16+
secretKeySecret: {
17+
name: mlpipeline-minio-artifact,
18+
key: secretkey
19+
}
20+
},
21+
archiveLogs: true
22+
}
23+
}

‎manifests/kustomize/env/aws/kustomization.yaml

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
namespace: kubeflow
4+
bases:
5+
- ../../env/platform-agnostic
6+
configMapGenerator:
7+
- name: pipeline-install-config
8+
env: params.env
9+
behavior: merge
10+
- name: workflow-controller-configmap
11+
behavior: replace
12+
files:
13+
- config
14+
- name: ml-pipeline-ui-configmap
15+
behavior: replace
16+
files:
17+
- viewer-pod-template.json
18+
secretGenerator:
19+
- name: mysql-secret
20+
env: secret.env
21+
behavior: merge
22+
- name: mlpipeline-minio-artifact
23+
env: minio-artifact-secret-patch.env
24+
behavior: merge
25+
generatorOptions:
26+
disableNameSuffixHash: true
27+
patchesStrategicMerge:
28+
- aws-configuration-patch.yaml
29+
# Identifier for application manager to apply ownerReference.
30+
# The ownerReference ensures the resources get garbage collected
31+
# when application is deleted.
32+
commonLabels:
33+
application-crd-id: kubeflow-pipelines

‎manifests/kustomize/env/aws/minio-artifact-secret-patch.env

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
accesskey=YOUR_AWS_ACCESS_ID
2+
secretkey=YOUR_AWS_SECRET_KEY

‎manifests/kustomize/env/aws/params.env

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
dbHost=YOUR_RDS_ENDPOINT
2+
3+
bucketName=YOUR_S3_BUCKET_NAME
4+
minioServiceHost=s3.amazonaws.com
5+
minioServiceRegion=YOUR_AWS_REGION

‎manifests/kustomize/env/aws/secret.env

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
username=YOUR_RDS_USERNAME
2+
password=YOUR_RDS_PASSWORD

‎manifests/kustomize/env/aws/viewer-pod-template.json

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"spec": {
3+
"containers": [
4+
{
5+
"env": [
6+
{
7+
"name": "AWS_ACCESS_KEY_ID",
8+
"valueFrom": {
9+
"secretKeyRef": {
10+
"name": "mlpipeline-minio-artifact",
11+
"key": "accesskey"
12+
}
13+
}
14+
},
15+
{
16+
"name": "AWS_SECRET_ACCESS_KEY",
17+
"valueFrom": {
18+
"secretKeyRef": {
19+
"name": "mlpipeline-minio-artifact",
20+
"key": "secretkey"
21+
}
22+
}
23+
},
24+
{
25+
"name": "AWS_REGION",
26+
"valueFrom": {
27+
"configMapKeyRef": {
28+
"name": "pipeline-install-config",
29+
"key": "minioServiceRegion"
30+
}
31+
}
32+
}
33+
]
34+
}
35+
]
36+
}
37+
}

0 commit comments

Comments
 (0)