castdatamart Helm chart

Deploys the CAST Imaging Datamart (castimaging/datamart) on Kubernetes. Since the Datamart is a batch ETL (not a server), the chart creates:

• a ConfigMap with the non-sensitive .env settings (REST API URLs, target database, extraction scope)
• a Secret with CREDENTIALS or APIUSER/APIKEY, plus _DB_USER and PGPASSWORD
• a one-time Job running install (Helm post-install hook)
• a CronJob running refresh (or update) on a schedule
• a CLI pod
• an optional PVC mounted at /home/datamart/outputdir to persist logs and intermediate files, as recommended by the Datamart docs

Quick start (single data source)

helm install --set auth.apiKey=... --set database.user=... --set database.password=... leaks those secrets into your shell history and into every process's argv (visible to any user on the node via ps aux / /proc/*/cmdline) for as long as the command runs. The recommended flow instead pre-creates the Kubernetes Secret out-of-band. Values only ever travel over stdin, never as a command-line argument, and points the chart at it with auth.existingSecret.

In CI/CD, populate the Secret from your pipeline's own secret store (Vault, Azure Key Vault, sealed-secrets, External Secrets Operator, etc.) instead of read/Read-Host, then run the same helm install referencing auth.existingSecret, no cleartext credentials touch the helm invocation either way.

In the helm install command below, replace restApi.defaultRoot and database.host, among the other --set placeholder values, with your actual settings. The database.host shown (console-postgres.castimaging-v3.svc.cluster.local) is just an example of the in-cluster Kubernetes DNS name pattern used when the target is the CAST Imaging embedded PostgreSQL. It won't resolve for any other setup.

Linux/macOS:

kubectl create namespace castdatamart --dry-run=client -o yaml | kubectl apply -f -

# APIUSER can be any value (Imaging Console does not check it, but it is required)
read -s -p "API user: " API_USER; echo
read -s -p "API key: " API_KEY; echo
read -s -p "DB user: " DB_USER; echo
read -s -p "DB password: " DB_PASSWORD; echo

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: mydatamart-castdatamart
  namespace: castdatamart
type: Opaque
stringData:
  APIUSER: ${API_USER}
  APIKEY: ${API_KEY}
  _DB_USER: ${DB_USER}
  PGPASSWORD: ${DB_PASSWORD}
EOF
unset API_USER API_KEY DB_USER DB_PASSWORD

helm install mydatamart ./castdatamart \
  --namespace castdatamart \
  --set restApi.defaultRoot=https://console.mycompany.com/dashboards/rest \
  --set restApi.defaultDomain=AAD \
  --set auth.existingSecret=mydatamart-castdatamart \
  --set database.host=console-postgres.castimaging-v3.svc.cluster.local \
  --set database.port=2285 \
  --set database.name=postgres \
  --set database.schema=datamart

Windows PowerShell:

kubectl create namespace castdatamart --dry-run=client -o yaml | kubectl apply -f -

# APIUSER can be any value (Imaging Console does not check it, but it is required)
$ApiUser = Read-Host -AsSecureString "API user"
$ApiKey = Read-Host -AsSecureString "API key"
$DbUser = Read-Host -AsSecureString "DB user"
$DbPassword = Read-Host -AsSecureString "DB password"
$ApiUserPlain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($ApiUser))
$ApiKeyPlain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($ApiKey))
$DbUserPlain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($DbUser))
$DbPasswordPlain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($DbPassword))

@"
apiVersion: v1
kind: Secret
metadata:
  name: mydatamart-castdatamart
  namespace: castdatamart
type: Opaque
stringData:
  APIUSER: $ApiUserPlain
  APIKEY: $ApiKeyPlain
  _DB_USER: $DbUserPlain
  PGPASSWORD: $DbPasswordPlain
"@ | kubectl apply -f -
Remove-Variable ApiUser, ApiKey, DbUser, DbPassword, ApiUserPlain, ApiKeyPlain, DbUserPlain, DbPasswordPlain

helm install mydatamart .\castdatamart `
  --namespace castdatamart `
  --set restApi.defaultRoot=https://console.mycompany.com/dashboards/rest `
  --set restApi.defaultDomain=AAD `
  --set auth.existingSecret=mydatamart-castdatamart `
  --set database.host=console-postgres.castimaging-v3.svc.cluster.local `
  --set database.port=2285 `
  --set database.name=postgres `
  --set database.schema=datamart

Quick local test only (not for production, puts secrets in shell history and process listing)

Again, in the helm install command below, replace restApi.defaultRoot and database.host, among the other --set placeholder values, with your actual settings. database.host here is just an example DNS name for the embedded PostgreSQL.

Linux/macOS:

helm install mydatamart ./castdatamart \
  --namespace castdatamart --create-namespace \
  --set restApi.defaultRoot=https://console.mycompany.com/dashboards/rest \
  --set restApi.defaultDomain=AAD \
  --set auth.apiUser=api_user \
  --set auth.apiKey=zil1wN4m.x... \
  --set database.host=console-postgres.castimaging-v3.svc.cluster.local \
  --set database.port=2285 \
  --set database.name=postgres \
  --set database.schema=datamart \
  --set database.user=operator \
  --set database.password=CastAIP

Windows command:

helm install mydatamart .\castdatamart ^
  --namespace castdatamart --create-namespace ^
  --set restApi.defaultRoot=https://console.mycompany.com/dashboards/rest ^
  --set restApi.defaultDomain=AAD ^
  --set auth.apiUser=api_user ^
  --set auth.apiKey=zil1wN4m.x... ^
  --set database.host=console-postgres.castimaging-v3.svc.cluster.local ^
  --set database.port=2285 ^
  --set database.name=postgres ^
  --set database.schema=datamart ^
  --set database.user=operator ^
  --set database.password=CastAIP

Watch the initial load:

kubectl logs -n castdatamart -f -l app.kubernetes.io/instance=mydatamart,app.kubernetes.io/component=install --tail=-1

If helm install fails (e.g. the install Job errors out or times out), Helm marks the release as failed but leaves it in place; it does not roll back or retry on its own. The install Job runs as a post-install hook only, so a plain helm upgrade won't re-run it. You need to remove the failed release and install again:

helm uninstall mydatamart -n castdatamart
# fix the underlying issue (bad REST URL/credentials, unreachable DB, etc.),
# then re-run the same helm install command from above

helm uninstall only removes the Helm-managed resources (ConfigMap, Secret if auto-created, Job, CronJob, CLI pod). It does not touch the namespace or a pre-created auth.existingSecret, so if you followed the secure flow above you don't need to redo those steps before retrying.

Multiple data sources (HD + ED domains)

# my-values.yaml
mode: multiple
restApi:
  hdRoot: https://console.mycompany.com/dashboards/rest
  edRoots:
    - https://console.mycompany.com/dashboards/rest
  jobs: 2          # do not exceed REST API max DB connections (10 by default)
refreshCronJob:
  command: update  # sync only new snapshots (multiple mode)
  schedule: "0 3 * * *"

As above, pre-create the Secret (with CREDENTIALS: user:password instead of APIUSER/APIKEY, plus _DB_USER/PGPASSWORD) and reference it instead of passing credentials via --set:

helm install mydatamart ./castdatamart -n castdatamart -f my-values.yaml \
  --set auth.existingSecret=mydatamart-castdatamart

Notes

• Compatibility: pin image.tag to a version compatible with your Dashboards release (see the Datamart release notes); avoid latest in prod.
• Credentials: use either auth.credentials (user:password) or auth.apiUser + auth.apiKey. For Imaging Console, the API key mode is required. HEX:-obfuscated values (from utilities/encode.py) are accepted. database.user and database.password are also stored in the Secret (never the ConfigMap), since a DB username is sensitive too. Prefer auth.existingSecret referencing a Secret you manage (keys: CREDENTIALS or APIUSER/APIKEY, plus _DB_USER and PGPASSWORD) over --set or values files, so credentials never appear in shell history, process listings, or helm history/helm get values output.
• Scheduling: concurrencyPolicy: Forbid prevents overlapping runs. Run extractions outside of source-code analysis / metrics calculation windows.
• To trigger the refresh job manually: spawn a one-off Job from the CronJob instead of waiting for its schedule

  kubectl create job -n castdatamart --from=cronjob/mydatamart-castdatamart-refresh manual-refresh-sometimestamp

• To follow the latest cronjob/job run (scheduled or manual) without knowing its generated name:

  kubectl logs -n castdatamart -f -l app.kubernetes.io/instance=mydatamart,app.kubernetes.io/component=refresh --tail=-1

• CLI pod: when cli.enabled (default true), the chart deploys a <release>-castdatamart-cli Deployment running the same image with the same ConfigMap/Secret env, but it stays up indefinitely. Use it to debug or run scripts manually, e.g. re-running an extraction/refresh command by hand, inspecting outputdir logs, or testing REST/DB connectivity. Connect to the CLI pod in a shell using this command:

  kubectl exec -it -n castdatamart deploy/mydatamart-castdatamart-cli -- bash

• To run schema upgrade: after upgrading the Datamart image, if you use refresh/update, exec into the CLI pod from the same image (with the same envFrom) and run upgrade_schema.sh, then recreate views if needed.