Skip to content

Commit dc82561

Browse files
yu-teoYuriy Teodorovych
andauthored
refactor: allow custom db connection in deploy.sh (#649)
https://redhat.atlassian.net/browse/RHOAIENG-52044 ## Description Added `create_maas_db_config_secret()` helper function that creates the `maas-db-config` **Secret** with a given connection URL. All of those changes tie into updated `scripts/deploy.sh` to allow `POSTGRES_CONNECTION` config variable with `--postgres-connection` flag. Additionally, I have added validation of the postgras URL to fail early. For clarity, I have added a warning banner like so: ``` "" "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓" "┃ ⚠️ WARNING FOR PRODUCTION USE. ⚠️ ┃" "┃ This deploys PostgreSQL with ephemeral storage (emptyDir). ┃" "┃ Data WILL be lost on pod restart. ┃" "┃ For production, use an external database: ┃" "┃ deploy.sh --postgres-connection postgresql://... ┃" "┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛" "" ``` ## How Has This Been Tested? Missing flag value fails ``` ./scripts/deploy.sh --postgres-connection [INFO] =================================================== [INFO] Models-as-a-Service Deployment [INFO] =================================================== [ERROR] Flag --postgres-connection requires a value [ERROR] Use --help for usage information ``` With --postgres-connection ``` ./scripts/deploy.sh --postgres-connection 'postgresql://user:pass@rds.example.com:5432/maas?sslmode=require' ... [INFO] Waiting for DataScienceCluster to be ready... * Waiting for DataScienceCluster 'default-dsc' KServe and ModelsAsService components to be ready... - KServe state: , KserveReady: , ModelsAsServiceReady: , ModelControllerReady: - KServe state: Managed, KserveReady: False, ModelsAsServiceReady: False, ModelControllerReady: True - KServe state: Managed, KserveReady: False, ModelsAsServiceReady: False, ModelControllerReady: True * KServe and ModelsAsService are ready in DataScienceCluster 'default-dsc' [INFO] Using external PostgreSQL connection secret/maas-db-config created [INFO] Created maas-db-config secret with external connection * MAAS_API_IMAGE not set, using operator default [INFO] Configuring TLS backend for Authorino and MaaS API... ... ``` Without flag ``` ./scripts/deploy.sh ... [INFO] Waiting for DataScienceCluster to be ready... * Waiting for DataScienceCluster 'default-dsc' KServe and ModelsAsService components to be ready... - KServe state: , KserveReady: , ModelsAsServiceReady: , ModelControllerReady: - KServe state: Managed, KserveReady: False, ModelsAsServiceReady: False, ModelControllerReady: False - KServe state: Managed, KserveReady: False, ModelsAsServiceReady: False, ModelControllerReady: True * KServe and ModelsAsService are ready in DataScienceCluster 'default-dsc' [WARN] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ [WARN] DEPLOYING POC POSTGRESQL — NOT INTENDED FOR PRODUCTION USE [WARN] Data is stored in ephemeral storage and will be lost on pod restart. [WARN] For production, use --postgres-connection with an external database [WARN] (AWS RDS, Crunchy Operator, Azure Database, etc.) [WARN] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ⚠️ WARNING FOR PRODUCTION USE. ⚠️ ┃ ┃ This deploys PostgreSQL with ephemeral storage (emptyDir). ┃ ┃ Data WILL be lost on pod restart. ┃ ┃ For production, use an external database: ┃ ┃ deploy.sh --postgres-connection postgresql://... ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ 🔧 Deploying PostgreSQL for API key storage in namespace 'opendatahub'... Generated random PostgreSQL password (stored in secret postgres-creds) Creating PostgreSQL deployment... ⚠️ Using POC configuration (ephemeral storage) secret/postgres-creds created deployment.apps/postgres created service/postgres created secret/maas-db-config created Waiting for PostgreSQL to be ready... deployment.apps/postgres condition met ✅ PostgreSQL deployed successfully Database: maas User: maas Secret: maas-db-config (contains DB_CONNECTION_URL) ⚠️ For production, use AWS RDS, Crunchy Operator, or Azure Database Note: Schema migrations run automatically when maas-api starts * MAAS_API_IMAGE not set, using operator default [INFO] Configuring TLS backend for Authorino and MaaS API... ... ``` ## Merge criteria: <!--- This PR will be merged by any repository approver when it meets all the points in the checklist --> <!--- Go over all the following points, and put an `x` in all the boxes that apply. --> - [x] The commits are squashed in a cohesive manner and have meaningful messages. - [x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious). - [x] The developer has manually tested the changes and verified that the changes work <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Option to deploy using an external PostgreSQL via a connection string; this skips the bundled proof-of-concept DB. * When an external DB is provided, the system automatically creates/updates the database configuration secret. * Connection string is validated to ensure proper format and will be rejected if invalid. * **Documentation** * Install guide updated with external DB usage, an example connection string, and a warning that the bundled DB uses ephemeral storage (data not persisted). <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Yuriy Teodorovych <Yuriy@ibm.com>
1 parent 2a424d5 commit dc82561

4 files changed

Lines changed: 107 additions & 11 deletions

File tree

docs/content/install/maas-setup.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,13 @@ postgresql://USERNAME:PASSWORD@HOSTNAME:PORT/DATABASE?sslmode=require
4444

4545
The full `scripts/deploy.sh` script also creates PostgreSQL automatically when deploying MaaS.
4646

47+
!!! note "Using deploy.sh with an external database"
48+
If you use `scripts/deploy.sh`, you can supply your own PostgreSQL connection string with the `--postgres-connection` flag. This skips the built-in POC PostgreSQL deployment and creates the `maas-db-config` Secret automatically:
49+
50+
```bash
51+
./scripts/deploy.sh --postgres-connection 'postgresql://username:password@hostname:5432/database?sslmode=require'
52+
```
53+
4754
!!! note "Restarting maas-api"
4855
If you add or update the Secret after the DataScienceCluster already has modelsAsService in managed state, restart the maas-api deployment to pick up the config:
4956

scripts/deploy.sh

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@
6060
# # Test custom MaaS API image
6161
# MAAS_API_IMAGE=quay.io/myuser/maas-api:pr-123 ./scripts/deploy.sh
6262
#
63+
# # Use external PostgreSQL (production)
64+
# ./scripts/deploy.sh --postgres-connection 'postgresql://user:pass@db.example.com:5432/maas?sslmode=require'
65+
#
6366
# For detailed documentation, see:
6467
# https://opendatahub-io.github.io/models-as-a-service/latest/install/maas-setup/
6568
################################################################################
@@ -108,6 +111,7 @@ MAAS_API_IMAGE="${MAAS_API_IMAGE:-}"
108111
MAAS_CONTROLLER_IMAGE="${MAAS_CONTROLLER_IMAGE:-}"
109112
KUSTOMIZE_FORCE_CONFLICTS="${KUSTOMIZE_FORCE_CONFLICTS:-false}"
110113
EXTERNAL_OIDC="${EXTERNAL_OIDC:-false}"
114+
POSTGRES_CONNECTION="${POSTGRES_CONNECTION:-}"
111115

112116
#──────────────────────────────────────────────────────────────
113117
# HELP TEXT
@@ -144,6 +148,11 @@ OPTIONS:
144148
Creates keycloak-system namespace and deploys Keycloak operator
145149
See docs/samples/install/keycloak/ for configuration guide
146150
151+
--postgres-connection <connection-string>
152+
Use an external PostgreSQL database instead of deploying a POC instance.
153+
Format: postgresql://USER:PASSWORD@HOST:PORT/DATABASE?sslmode=require
154+
When set, skips the built-in PostgreSQL deployment entirely.
155+
147156
--namespace <namespace>
148157
Target namespace for deployment
149158
Default: redhat-ods-applications (RHOAI) or opendatahub (ODH)
@@ -195,6 +204,7 @@ ENVIRONMENT VARIABLES:
195204
OIDC_ISSUER_URL External OIDC issuer URL for maas-api AuthPolicy patching
196205
LOG_LEVEL Logging verbosity (DEBUG, INFO, WARN, ERROR)
197206
KUSTOMIZE_FORCE_CONFLICTS When true, pass --force-conflicts to kubectl apply in kustomize mode (default: false)
207+
POSTGRES_CONNECTION External PostgreSQL connection string (same as --postgres-connection)
198208
199209
TIMEOUT CONFIGURATION (all values in seconds):
200210
Customize timeouts for slow clusters or CI/CD environments:
@@ -229,6 +239,9 @@ EXAMPLES:
229239
--operator-catalog quay.io/opendatahub/opendatahub-operator-catalog:pr-456 \\
230240
--operator-image quay.io/opendatahub/opendatahub-operator:pr-456
231241
242+
# Use an external PostgreSQL database
243+
./scripts/deploy.sh --postgres-connection 'postgresql://user:pass@rds.example.com:5432/maas?sslmode=require'
244+
232245
For more information, see: https://github.com/opendatahub-io/models-as-a-service
233246
EOF
234247
}
@@ -314,6 +327,11 @@ parse_arguments() {
314327
OPERATOR_CHANNEL="$2"
315328
shift 2
316329
;;
330+
--postgres-connection)
331+
require_flag_value "$1" "${2:-}"
332+
POSTGRES_CONNECTION="$2"
333+
shift 2
334+
;;
317335
--external-oidc)
318336
EXTERNAL_OIDC="true"
319337
shift
@@ -697,8 +715,30 @@ deploy_via_kustomize() {
697715
# POSTGRESQL DEPLOYMENT
698716
#──────────────────────────────────────────────────────────────
699717

718+
validate_postgres_connection() {
719+
local conn="$1"
720+
if [[ ! "$conn" =~ ^postgres(ql)?:// ]]; then
721+
log_error "Invalid PostgreSQL connection string format"
722+
log_error "Expected: postgresql://USER:PASSWORD@HOST:PORT/DATABASE?sslmode=require"
723+
return 1
724+
fi
725+
}
726+
700727
deploy_postgresql() {
701-
NAMESPACE="$NAMESPACE" "${SCRIPT_DIR}/setup-database.sh"
728+
if [[ -n "$POSTGRES_CONNECTION" ]]; then
729+
validate_postgres_connection "$POSTGRES_CONNECTION" || exit 1
730+
log_info "Using external PostgreSQL connection"
731+
create_maas_db_config_secret "$NAMESPACE" "$POSTGRES_CONNECTION"
732+
log_info "Created maas-db-config secret with external connection"
733+
else
734+
log_warn "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
735+
log_warn " DEPLOYING POC POSTGRESQL — NOT INTENDED FOR PRODUCTION USE"
736+
log_warn " Data is stored in ephemeral storage and will be lost on pod restart."
737+
log_warn " For production, use --postgres-connection with an external database"
738+
log_warn " (AWS RDS, Crunchy Operator, Azure Database, etc.)"
739+
log_warn "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
740+
NAMESPACE="$NAMESPACE" "${SCRIPT_DIR}/setup-database.sh"
741+
fi
702742
}
703743

704744
#──────────────────────────────────────────────────────────────

scripts/deployment-helpers.sh

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1456,3 +1456,35 @@ wait_authorino_ready() {
14561456
echo " WARNING: Auth request verification timed out, continuing anyway"
14571457
return 0
14581458
}
1459+
1460+
# ==========================================
1461+
# Database Secret Helpers
1462+
# ==========================================
1463+
1464+
# create_maas_db_config_secret <namespace> <connection_url>
1465+
# Creates the maas-db-config Secret containing DB_CONNECTION_URL.
1466+
# This secret is read by maas-api at startup to connect to PostgreSQL.
1467+
#
1468+
# Usage:
1469+
# create_maas_db_config_secret "opendatahub" "postgresql://user:pass@host:5432/db?sslmode=require"
1470+
create_maas_db_config_secret() {
1471+
local namespace="$1"
1472+
local connection_url="$2"
1473+
1474+
if [[ -z "$namespace" ]]; then
1475+
log_error "create_maas_db_config_secret: namespace is required"
1476+
return 1
1477+
fi
1478+
if [[ -z "$connection_url" ]]; then
1479+
log_error "create_maas_db_config_secret: connection_url is required"
1480+
return 1
1481+
fi
1482+
1483+
# Pass the connection URL via stdin to avoid exposing credentials in process arguments
1484+
printf '%s' "$connection_url" | \
1485+
kubectl create secret generic maas-db-config \
1486+
--from-file=DB_CONNECTION_URL=/dev/stdin \
1487+
--dry-run=client -o yaml | \
1488+
kubectl label --local -f - app=maas-api --dry-run=client -o yaml | \
1489+
kubectl apply -n "$namespace" -f -
1490+
}

scripts/setup-database.sh

Lines changed: 27 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,11 @@
2525

2626
set -euo pipefail
2727

28+
# Source helpers
29+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
30+
# shellcheck source=deployment-helpers.sh
31+
source "${SCRIPT_DIR}/deployment-helpers.sh"
32+
2833
# Default namespace for ODH; use redhat-ods-applications for RHOAI
2934
: "${NAMESPACE:=opendatahub}"
3035

@@ -34,6 +39,15 @@ if ! kubectl get namespace "$NAMESPACE" >/dev/null 2>&1; then
3439
kubectl create namespace "$NAMESPACE"
3540
fi
3641

42+
echo ""
43+
echo "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓"
44+
echo "┃ ⚠️ WARNING FOR PRODUCTION USE. ⚠️ ┃"
45+
echo "┃ This deploys PostgreSQL with ephemeral storage (emptyDir). ┃"
46+
echo "┃ Data WILL be lost on pod restart. ┃"
47+
echo "┃ For production, use an external database: ┃"
48+
echo "┃ deploy.sh --postgres-connection postgresql://... ┃"
49+
echo "┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛"
50+
echo ""
3751
echo "🔧 Deploying PostgreSQL for API key storage in namespace '$NAMESPACE'..."
3852

3953
# Check if PostgreSQL already exists
@@ -142,18 +156,21 @@ spec:
142156
ports:
143157
- port: 5432
144158
targetPort: 5432
145-
---
146-
apiVersion: v1
147-
kind: Secret
148-
metadata:
149-
name: maas-db-config
150-
labels:
151-
app: maas-api
152-
purpose: poc
153-
stringData:
154-
DB_CONNECTION_URL: "postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable"
155159
EOF
156160

161+
# Create the maas-db-config secret used by maas-api
162+
# URL-encode the password in case it contains reserved characters (@, :, /, ?, etc.)
163+
# 1. printf '%s' outputs the raw password bytes (no trailing newline)
164+
# 2. od -An -tx1 converts each byte to space-separated two-digit hex (e.g., "a" -> " 61")
165+
# 3. tr -d ' \n' strips spaces and newlines to produce a continuous hex string
166+
# 4. sed inserts a "%" before every hex pair, producing percent-encoding (e.g., "61" -> "%61")
167+
# This encodes all characters (including safe ones like letters), which is more aggressive
168+
# than strictly necessary but is always correct per RFC 3986 — %61 is equivalent to "a".
169+
# Uses od (POSIX) instead of xxd which may not be available in all environments.
170+
ENCODED_PASSWORD=$(printf '%s' "$POSTGRES_PASSWORD" | od -An -tx1 | tr -d ' \n' | sed 's/../%&/g')
171+
DB_CONNECTION_URL="postgresql://${POSTGRES_USER}:${ENCODED_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable"
172+
create_maas_db_config_secret "$NAMESPACE" "$DB_CONNECTION_URL"
173+
157174
echo " Waiting for PostgreSQL to be ready..."
158175
if ! kubectl wait -n "$NAMESPACE" --for=condition=available deployment/postgres --timeout=120s; then
159176
echo "❌ PostgreSQL deployment failed to become ready" >&2

0 commit comments

Comments
 (0)