Merge pull request #2 from opencrvs/add-secrets

Add secrets

Author: adskyiproger

README.md

@@ -7,6 +7,7 @@ Please note that not all features from the Docker Swarm solution are supported y
 - Manual Helm installation and upgrade only
 - Manual initial user configuration for MinIO, MongoDB, Elasticsearch
 - No data reset feature available
+- Any kind of secrets (Logins and passwords, etc) should be created manually 
 
 ---
 

charts/opencrvs-services/README.md

@@ -0,0 +1,120 @@
+# 🚧 Work in Progress
+
+# General documentation
+
+Helm chart to deploy all OpenCRVS services on Kubernetes cluster.
+
+# Dependencies Configuration
+
+<table>
+    <thead>
+        <tr>
+            <th>Name</th>
+            <th>Default</th>
+            <th>Description</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td>elasticsearch_host</td>
+            <td>elasticsearch.opencrvs-deps-dev.svc.cluster.local:9200</td>
+            <td>Elasticsearch configuration, including the hostname and port. TODO: Consider defining the port as a separate variable.</td>
+        </tr>
+        <tr>
+            <td>influxdb.host</td>
+            <td>influxdb-0.influxdb.opencrvs-deps-dev.svc.cluster.local</td>
+            <td>InfluxDB hostname configuration.</td>
+        </tr>
+        <tr>
+            <td>influxdb.port</td>
+            <td>8086</td>
+            <td>InfluxDB port configuration.</td>
+        </tr>
+        <tr>
+            <td>influxdb.db</td>
+            <td>ocrvs</td>
+            <td>InfluxDB database name.</td>
+        </tr>
+        <tr>
+            <td>fhir_url</td>
+            <td>http://hearth.opencrvs-deps-dev.svc.cluster.local:3447/fhir</td>
+            <td>FHIR URL Endpoint. TODO: Add description for FHIR URL configuration.</td>
+        </tr>
+        <tr>
+            <td>minio.host</td>
+            <td>minio-0.minio.opencrvs-deps-dev.svc.cluster.local</td>
+            <td>MinIO hostname configuration.</td>
+        </tr>
+        <tr>
+            <td>minio.port</td>
+            <td>3535</td>
+            <td>MinIO port configuration.</td>
+        </tr>
+        <tr>
+            <td>mongodb_host</td>
+            <td>mongodb-0.mongodb.opencrvs-deps-dev.svc.cluster.local</td>
+            <td>MongoDB hostname configuration.</td>
+        </tr>
+        <tr>
+            <td>hostname</td>
+            <td>farajaland.com</td>
+            <td>Hostname for OpenCRVS application, without wildcard or subdomain. Example: hostname: opencrvs.localhost</td>
+        </tr>
+        <tr>
+            <td>dev_mode</td>
+            <td>false</td>
+            <td>Developer mode flag. TODO: Check the usage and purpose of this variable.</td>
+        </tr>
+        <tr>
+            <td>env</td>
+            <td>{}</td>
+            <td>Global environment variables, each variable defined here is available to all workloads (service) deployed by helm chart. See example at [values.yaml](values.yaml)</td>
+        </tr>
+        <tr>
+            <td>&ltservice_name&gt.env</td>
+            <td>{}</td>
+            <td>Service level environment variables, each variable defined here is available to particular workload (service) only. See example for `config` microservice at [values.yaml](values.yaml)</td>
+        </tr>
+        <tr>
+            <td>&ltservice_name&gt.secrets</td>
+            <td>{}</td>
+            <td>Mapping kubernetes secrets as environment variables. For more information see [Mapping secrets](#mapping-secrets)</td>
+        </tr>
+    </tbody>
+</table>
+
+# Microservice environment variables configuration
+
+<pre>Do we need this section?</pre>
+
+Helm chart allows to define environment variables in following scopes:
+- **Global variables** are defined at top level of values file and is added to all containers. See `env` key in [values.yaml](values.yaml)
+- **Service level variables** are defined for each particular service. See `<service_name>.env` key in [values.yaml](values.yaml)
+- **Secret environment variables** are defined at service level as `<service_name>.secrets` key, see [values.yaml](values.yaml).
+
+# Mapping secrets
+
+Suppose we need to store ES_HOST variable as a secret since it contains url with login and password for Elastic search.
+Kubernetes secret is key/value object usually created from `.env` file, for example:
+```
+ES_HOST=user:randompass@elasticsearch:9200
+```
+
+Mapping needs to be added for particular service to access variable inside workload (service), e/g for `search` service to access ES_HOST following configuration is needed:
+```
+search:
+    secrets:
+        elasticsearch-secret:
+            - ES_HOST
+```
+
+In some cases variable name (key) stored in kubernetes secret doesn't match with environment variable
+```
+secrets:
+  <secret_name>:
+     - <secret_key>:<environment_variable>
+```
+Summary:
+- `secret_name`, name of Kubernetes secret object
+- `secret_key`, key (variable name) inside Kubernetes secret data property
+- `environment_variable`, environment variable name inside container. If `secret_key` value `environment_variable` are the same, last one can be omitted.

charts/opencrvs-services/TODO.md

@@ -48,6 +48,7 @@ We are starting development, but helm charts already exist and once we setup ser
    - check if it's possible to build post-deploy job for opencrvs-services chart
 3. Add workflow for creating users in ELK and mongo
    - check if it's possible to build post-deploy job for dependencies chart
+
 # Monitoring
 
 1. Review option of replacing ELK with something more simple
@@ -58,12 +59,19 @@ Automatically issue SSL secret for traefix, check possibility to issue valid SSL
 
 # Fixes
 
-1. Fix events
-2. Fix clients:
-   - [16:11:44.470] ERROR: Failed to connect to MongoDB. Retrying...
-   - HTTP 500 https://config.opencrvs.localhost/publicConfig
-
+1. Fix events variables
 
 # Check
 
-- https://kubernetes.io/docs/concepts/storage/volumes/#image
+- https://kubernetes.io/docs/concepts/storage/volumes/#image
+
+# Secrets
+
+There is a need to store secrets for Mongo, Elastic search, etc.
+
+Easiest option is to store values as kubernetes secrets
+```
+kubectl create secret generic opencrvs-shared-secrets --from-env-file=.env.qa
+```
+
+And then pass as shared secret name

charts/opencrvs-services/templates/_helpers.tpl

@@ -0,0 +1,39 @@
+{{/*
+
+render-env-vars
+---
+This is a helper template that dynamically generates environment variables and secret references for Kubernetes Deployments.
+It accounts for both global and service-specific environment variables and secrets.
+
+Parameters:
+- .ServiceName: The name of the microservice, which is used to access service-specific values.
+- .Values: The top-level Values object for the Helm chart.
+*/}}
+
+{{- define "render-env-vars" -}}
+  {{- $service_name := .service_name }}
+  {{/* Loop through and generate global environment variables */}}
+  {{- range $k, $v := .Values.env }}
+            - name: {{ $k }}
+              value: {{ $v | quote }}
+  {{- end }}
+  {{/* Access the service-specific values using the service name */}}
+  {{- with index .Values $service_name }}
+    {{/* Loop through and generate service-specific environment variables */}}
+    {{- range $k, $v := .env }}
+            - name: {{ $k }}
+              value: {{ $v | quote }}
+    {{- end }}
+    {{/* Loop through and generate secret references for service-specific secrets */}}
+    {{- range $secret_name, $secret_values := .secrets }}
+      {{- range $secret_value := $secret_values }}
+        {{- $secret := split ":" $secret_value }}
+            - name: {{ $secret._1 | default $secret._0 }}
+              valueFrom:
+                secretKeyRef:
+                name: {{ $secret_name }}
+                key: {{ $secret._0 | quote}}
+      {{- end }}
+    {{- end }}
+  {{- end }}
+{{- end }}

charts/opencrvs-services/templates/auth-deployment.yaml

@@ -66,25 +66,18 @@ spec:
             - name: CONFIG_TOKEN_EXPIRY_SECONDS
               value: '604800'
             - name: METRICS_URL
-              value: http://metrics.{{ .Release.Namespace }}.svc.cluster.local:1050 # FIXME: harcoded
+              value: http://metrics.{{ .Release.Namespace }}.svc.cluster.local:1050
             - name: NOTIFICATION_SERVICE_URL
               value: http://notification.{{ .Release.Namespace }}.svc.cluster.local:2020/
             - name: USER_MANAGEMENT_URL
-              value: http://user-mgnt.{{ .Release.Namespace }}.svc.cluster.local:3030/ # FIXME: hardcoded
+              value: http://user-mgnt.{{ .Release.Namespace }}.svc.cluster.local:3030/
             - name: AUTH_PORT
               value: "4040"
             - name: COUNTRY_CONFIG_URL_INTERNAL
-              value: http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040 # FIXME: harcoded
+              value: http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040
             - name: DOMAIN
               value: "{{ .Release.Namespace }}.svc.cluster.local"
-          {{- range $k, $v := .Values.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
-          {{- range $k, $v := .Values.auth.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
+          {{- include "render-env-vars" (dict "service_name" "auth" "Values" .Values) }}
           ports:
             - containerPort: 4040
               protocol: TCP

charts/opencrvs-services/templates/client-deployment.yaml

@@ -52,13 +52,14 @@ spec:
             - name: CONTENT_SECURITY_POLICY_WILDCARD
               value: "*.{{ .Values.hostname }}"
             - name: COUNTRY_CONFIG_URL_INTERNAL
-              value: http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040 # FIXME: harcoded
+              value: http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040
             - name: GATEWAY_URL_INTERNAL
-              value: http://gateway.{{ .Release.Namespace }}.svc.cluster.local:7070 # FIXME: harcoded
+              value: http://gateway.{{ .Release.Namespace }}.svc.cluster.local:7070
             # CHECK: Following variables are present on Dev environment
             # "COUNTRY_CONFIG_URL
             # "DECLARED_DECLARATION_SEARCH_QUERY_COUNT
             # "MINIO_URL
+          {{- include "render-env-vars" (dict "service_name" "client" "Values" .Values) }}
           name: client
           ports:
             - containerPort: 80

charts/opencrvs-services/templates/config-deployment.yaml

@@ -49,6 +49,8 @@ spec:
         - name: config
           image: "ghcr.io/opencrvs/ocrvs-config:{{ .Values.image.tag }}"
           env:
+            - name: CERT_PUBLIC_KEY_PATH
+              value: /secrets/public-key.pem
             - name: FHIR_URL
               value: {{ .Values.fhir_url | quote }}
             - name: CLIENT_APP_URL
@@ -72,19 +74,10 @@ spec:
             - name: USER_MANAGEMENT_URL
               value: http://user-mgnt.{{ .Release.Namespace }}.svc.cluster.local:3030/
             - name: HEARTH_MONGO_URL
-              value: "mongodb://{{ .Values.mongodb.mongodb_host }}/hearth-dev"
+              value: "mongodb://{{ .Values.mongodb_host }}/hearth-dev"
             - name: MONGO_URL
-              value: mongodb://{{ .Values.mongodb.mongodb_host }}/application-config
-          {{- range $k, $v := .Values.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
-          {{- range $k, $v := .Values.config.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
-            - name: CERT_PUBLIC_KEY_PATH
-              value: /secrets/public-key.pem
+              value: mongodb://{{ .Values.mongodb_host }}/application-config
+          {{- include "render-env-vars" (dict "service_name" "config" "Values" .Values) }}
           ports:
             - containerPort: 2021
               protocol: TCP

charts/opencrvs-services/templates/countryconfig-deployment.yaml

@@ -100,17 +100,10 @@ spec:
             - name: CONFIRM_REGISTRATION_URL
               value: http://workflow.{{ .Release.Namespace }}.svc.cluster.local:5050/confirm/registration
             - name: CONFIG_MONGO_URL
-              value: mongodb://{{ .Values.mongodb.mongodb_host }}/application-config
+              value: mongodb://{{ .Values.mongodb_host }}/application-config
             - name: MONGO_URL
-              value: mongodb://{{ .Values.mongodb.mongodb_host }}/user-mgnt
-          {{- range $k, $v := .Values.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
-          {{- range $k, $v := .Values.countryconfig.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
+              value: mongodb://{{ .Values.mongodb_host }}/user-mgnt
+          {{- include "render-env-vars" (dict "service_name" "countryconfig" "Values" .Values) }}
           ports:
             - containerPort: 3040
               protocol: TCP

charts/opencrvs-services/templates/dashboards-deployment.yaml

@@ -53,9 +53,6 @@ spec:
             - name: OPENCRVS_METABASE_MAP_URL
               value: "http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040/content/map.geojson"
             - name: OPENCRVS_METABASE_DB_HOST
-              value: {{ .Values.mongodb.mongodb_host | quote }}
-          {{- range $k, $v := .Values.dashboards.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-          {{- end }}
+              value: {{ .Values.mongodb_host | quote }}
+          {{- include "render-env-vars" (dict "service_name" "dashboards" "Values" .Values) }}
       restartPolicy: Always

charts/opencrvs-services/templates/data-seeder.yaml

@@ -23,14 +23,7 @@ spec:
               value: "http://gateway.{{ .Release.Namespace }}.svc.cluster.local:7070"
             - name: COUNTRY_CONFIG_HOST
               value: http://countryconfig.{{ .Release.Namespace }}.svc.cluster.local:3040
-            {{- range $k, $v := .Values.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-            {{- end }}
-            {{- range $k, $v := .Values.data_seeder.env }}
-            - name: {{ $k }}
-              value: {{ $v | quote }}
-            {{- end }}
+          {{- include "render-env-vars" (dict "service_name" "data_seeder" "Values" .Values) }}
       completions: 1 # Run the job once
       backoffLimit: 0 # Don't retry if the job fails
       restartPolicy: "OnFailure"