Skip to content

Commit 8e7ac9e

Browse files
sgaistleafty
sgaist
and
leafty
authored
feat: add configuration for building images from private repositories (#4428)
* feat: add configuration options for building image from private repositories * feat: add configuration for session private image pull secret This supplements the building part allowing to use independent secrets for pushing during build and pulling for session creation. This secret is not accessible by the user. * refactor: move private repo build support under a feature flag Renku admins thus have the choice to provide builds only for public repositories or for both. * chore: add documentation for admins * chore: add changelog entries for values file --------- Co-authored-by: Flora Thiebaut <flora.thiebaut@sdsc.ethz.ch>
1 parent 709c63f commit 8e7ac9e

5 files changed

Lines changed: 62 additions & 10 deletions

File tree

‎docs/docs/20-admins/20-installation/20-configuration.md‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -241,7 +241,35 @@ The file above is just an example you will have to modify the options shown as f
241241
The example in the yaml snippet above is for a secret called `renku-build-docker-secret` located
242242
in the same namespace as where Renku is installed.
243243

244-
- Label the node(s) you want to use for the builds with `renku.io/node-purpose: image-build`
244+
By default, only builds from public repositories are enabled. Building from
245+
internal or private repositories requires more work:
246+
247+
- An dedicated registry to hold the images
248+
- Two additional secrets that will hold the credentials to access that
249+
registry.
250+
251+
Here is an example on how to configure that:
252+
253+
```yaml
254+
dataService:
255+
imageBuilders:
256+
enabled: true
257+
privateRepositoryBuilds:
258+
enabled: true
259+
outputPrivateImagePrefix: "harbor.dev.renku.ch/renku-private-build/"
260+
pushPrivateSecretName: "renku-build-private-docker-secret"
261+
pullPrivateSecretName: "renku-pull-private-docker-secret"
262+
```
263+
264+
The same rules applies as for the public builds:
265+
266+
- `outputPrivateImagePrefix` contains the Harbor domain and project name.
267+
The prefix **must** be different from `outputImagePrefix`
268+
- `pushPrivateSecretName` is the secret to push the image created to the
269+
dedicated registry
270+
- `pullPrivateSecretName` is the secret the pod will need to load the image.
271+
272+
3. Label the node(s) you want to use for the builds with `renku.io/node-purpose: image-build`
245273

246274
### Build strategy
247275

‎helm-chart/renku/templates/data-service/deployment.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,12 @@ spec:
130130
value: {{ .Values.dataService.trustedProxies.realIpHeader | default "" | quote }}
131131
- name: IMAGE_BUILDERS_ENABLED
132132
value: {{ .Values.dataService.imageBuilders.enabled | quote }}
133+
- name: BUILD_PRIVATE_REPO_BUILDS_ENABLED
134+
value: {{ .Values.dataService.imageBuilders.privateRepositoryBuilds.enabled | quote }}
133135
- name: BUILD_OUTPUT_IMAGE_PREFIX
134136
value: {{ .Values.dataService.imageBuilders.outputImagePrefix | default "" | quote }}
137+
- name: BUILD_OUTPUT_PRIVATE_IMAGE_PREFIX
138+
value: {{ .Values.dataService.imageBuilders.privateRepositoryBuilds.outputPrivateImagePrefix | default "" | quote }}
135139
- name: BUILD_BUILDER_IMAGE
136140
value: {{ .Values.dataService.imageBuilders.builderImage | default "" | quote }}
137141
- name: BUILD_RUN_IMAGE
@@ -140,6 +144,10 @@ spec:
140144
value: {{ .Values.dataService.imageBuilders.strategyName | default "" | quote }}
141145
- name: BUILD_PUSH_SECRET_NAME
142146
value: {{ .Values.dataService.imageBuilders.pushSecretName | default "" | quote }}
147+
- name: BUILD_PUSH_PRIVATE_SECRET_NAME
148+
value: {{ .Values.dataService.imageBuilders.privateRepositoryBuilds.pushPrivateSecretName | default "" | quote }}
149+
- name: BUILD_PULL_PRIVATE_SECRET_NAME
150+
value: {{ .Values.dataService.imageBuilders.privateRepositoryBuilds.pullPrivateSecretName | default "" | quote }}
143151
- name: BUILD_RUN_RETENTION_AFTER_FAILED_SECONDS
144152
value: {{ .Values.dataService.imageBuilders.buildRunRetentionAfterFailedSeconds | default "" | quote }}
145153
- name: BUILD_RUN_RETENTION_AFTER_SUCCEEDED_SECONDS

‎helm-chart/renku/templates/ui/ui-client-deployment-template.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,8 @@ spec:
107107
value: {{ toJson .Values.ui.client.sessionClassEmailUs | quote }}
108108
- name: IMAGE_BUILDERS_ENABLED
109109
value: {{ .Values.dataService.imageBuilders.enabled | quote }}
110+
- name: BUILD_PRIVATE_REPO_BUILDS_ENABLED
111+
value: {{ .Values.dataService.imageBuilders.privateRepositoryBuilds.enabled | quote }}
110112
- name: METRICS_ENABLED
111113
value: {{ .Values.ui.client.prometheus.enabled | quote }}
112114
- name: LEGACY_SUPPORT

‎helm-chart/renku/values.yaml‎

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -555,7 +555,7 @@ ui:
555555
replicaCount: 1
556556
image:
557557
repository: renku/renku-ui
558-
tag: "4.25.0"
558+
tag: "4.26.0"
559559
pullPolicy: IfNotPresent
560560
## Optionally specify an array of imagePullSecrets.
561561
## Secrets must be manually created in the namespace.
@@ -747,7 +747,7 @@ ui:
747747
keepCookies: []
748748
image:
749749
repository: renku/renku-ui-server
750-
tag: "4.25.0"
750+
tag: "4.26.0"
751751
pullPolicy: IfNotPresent
752752
imagePullSecrets: []
753753
nameOverride: ""
@@ -829,7 +829,8 @@ dlf-chart:
829829
enabled: false
830830
dataset-operator-chart:
831831
enabled: true
832-
csi-rclone: {}
832+
csi-rclone:
833+
{}
833834
# This section is only relevant if you are installing csi-rclone as part of Renku
834835
## Name of the csi storage class to use for RClone/Cloudstorage. Should be unique per cluster.
835836
# storageClassName: csi-rclone
@@ -1199,12 +1200,12 @@ dataService:
11991200
existingPriorityClass: ""
12001201
image:
12011202
repository: renku/renku-data-service
1202-
tag: "0.76.0"
1203+
tag: "0.77.0"
12031204
pullPolicy: IfNotPresent
12041205
k8sWatcher:
12051206
image:
12061207
repository: renku/data-service-k8s-watcher
1207-
tag: "0.76.0"
1208+
tag: "0.77.0"
12081209
pullPolicy: IfNotPresent
12091210
resources: {}
12101211
sentry:
@@ -1215,7 +1216,7 @@ dataService:
12151216
dataTasks:
12161217
image:
12171218
repository: renku/data-service-data-tasks
1218-
tag: "0.76.0"
1219+
tag: "0.77.0"
12191220
pullPolicy: IfNotPresent
12201221
resources: {}
12211222
enableResourceRequestTracking: false
@@ -1254,14 +1255,25 @@ dataService:
12541255
enabled: false
12551256
## The container image prefix for images built from code
12561257
outputImagePrefix: harbor.dev.renku.ch/renku-build/
1258+
## Private repository support requires a dedicated registry and is a separate feature.
1259+
privateRepositoryBuilds:
1260+
enabled: false
1261+
## The container image prefix for images built from private code
1262+
outputPrivateImagePrefix: "harbor.dev.renku.ch/renku-private-build/"
1263+
## The name of the secret used to push images built from private code.
1264+
pushPrivateSecretName: "renku-build-private-docker-secret"
1265+
## The name of the secret used to pull images built from private code.
1266+
pullPrivateSecretName: "renku-pull-private-docker-secret"
1267+
12571268
## The builder image (see https://buildpacks.io/docs/for-platform-operators/concepts/builder/)
12581269
builderImage: "ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/selector:0.5.1"
12591270
## The run image (see https://buildpacks.io/docs/for-platform-operators/concepts/base-images/)
12601271
runImage: "ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/run-image:0.5.1"
12611272
## The name of the BuildStrategy to use for image builds.
12621273
strategyName: renku-buildpacks-v3
12631274
## Configuration overrides for specific target platforms
1264-
platformOverrides: {}
1275+
platformOverrides:
1276+
{}
12651277
# linux/arm64:
12661278
# builderImage: "ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/cuda-selector:0.5.1"
12671279
# runImage: "ghcr.io/swissdatasciencecenter/renku-frontend-buildpacks/cuda-run-image:0.5.1"
@@ -1345,7 +1357,7 @@ authz:
13451357
secretsStorage:
13461358
image:
13471359
repository: renku/secrets-storage
1348-
tag: "0.76.0"
1360+
tag: "0.77.0"
13491361
pullPolicy: IfNotPresent
13501362
service:
13511363
type: ClusterIP

‎helm-chart/values.yaml.changelog.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@ For changes that require manual steps other than changing values, please check o
55
Please follow this convention when adding a new row
66
* `<type: NEW|EDIT|DELETE> - *<resource name>*: <details>`
77

8-
## Upgrading to Renku 2.8.0
8+
## Upgrading to Renku 2.18.0
99

1010
* DELETE `enableInternalGitlab`, it is now not possible to configure Renku to use an "internal" GitLab instance. Admins can set up a GitLab integration instead.
11+
* NEW `dataService.imageBuilders.privateRepositoryBuilds.enabled` feature flag to enable building images from private code repositories (defaults to `false`)
12+
* NEW `dataService.imageBuilders.privateRepositoryBuilds` other configuration options for building images from private code repositories.
1113

1214
## Upgrading to Renku 2.15.0
1315

0 commit comments

Comments
 (0)