Ai 429 by kmandryk · Pull Request #20 · bcgov/ai-adoption-document-intelligence

kmandryk · 2026-01-06T22:26:52Z

AI-429: Develop and Deploy Simple UI

Summary

Changes

Add kustomize manifests for backend-services deployment
Introduces base, image-streams, and overlays for dev and prod environments to support backend-services deployment on OpenShift using kustomize. Includes ConfigMap, Deployment, Service, Route, PVC, NetworkPolicy, and environment-specific configuration for streamlined management and deployment.

Introduces base, image-streams, and overlays for dev and prod environments to support backend-services deployment on OpenShift using kustomize. Includes ConfigMap, Deployment, Service, Route, PVC, NetworkPolicy, and environment-specific configuration for streamlined management and deployment.

Deleted all kustomize configuration files for backend-services and related image streams in the OpenShift deployments director. redundant now with overlays.

AI-489 QA Workflows Fix

…nt-intelligence into develop

Introduces Dockerfile, .dockerignore, and nginx.conf for the frontend app. Adds OpenShift kustomize manifests for frontend deployment, service, route, network policy, and configmap. Updates build-apps workflow to support frontend context. Sets global API prefix in backend services and improves logging. Also adds package-lock.json files for demo backend and frontend.

+    runs-on: ubuntu-latest
+    environment:
+      name: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}
+    outputs:
+      backup-id: ${{ steps.set-backup-id.outputs.id }}
+      test-start-time: ${{ steps.set-time.outputs.time }}
+
+    steps:
+      - name: Set environment variables
+        run: |
+          echo "NAMESPACE=f3c07a-${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}" >> $GITHUB_ENV
+
+      - name: Set Cluster Name
+        id: set-cluster
+        run: |
+          CLUSTER_NAME='${{ github.event.inputs.cluster_name || 'api-postgres-clone' }}'
+          echo "cluster_name=$CLUSTER_NAME" >> $GITHUB_OUTPUT
+
+      - name: Set Test Start Time
+        id: set-time
+        run: |
+          START_TIME=$(date -u -d '5 minutes ago' "+%Y-%m-%d %H:%M:%S+00")
+          echo "time=$START_TIME" >> $GITHUB_OUTPUT
+
+      - name: Set Backup ID
+        id: set-backup-id
+        run: |
+          BACKUP_ID=$(date '+%Y%m%d-%H%M%S')
+          echo "id=$BACKUP_ID" >> $GITHUB_OUTPUT
+
+      - name: Display Backup ID
+        run: echo "Generated Backup ID is ${{ steps.set-backup-id.outputs.id }}"
+
+      - name: Login to OpenShift
+        run: |
+          oc login ${{ secrets.OPENSHIFT_SERVER }} \
+            --token=${{ secrets.OPENSHIFT_API_TOKEN }} \
+            --insecure-skip-tls-verify=true
+          oc project ${{ env.NAMESPACE }}
+
+      - name: Configure Manual Backup
+        run: |
+          oc patch postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} --type merge \
+            -p '{"spec":{"backups":{"pgbackrest":{"manual":{"repoName":"repo1","options":["--type=full","--annotation=source=${{ github.event.inputs.backup_annotation }}"]}}}}}'
+
+      - name: Trigger Backup
+        run: |
+          oc annotate postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} \
+            --overwrite \
+            postgres-operator.crunchydata.com/pgbackrest-backup="${{ steps.set-backup-id.outputs.id }}"
+
+      - name: Wait for backup completion
+        run: |
+          timeout=1200  # 20 minutes timeout
+          while [ $timeout -gt 0 ]; do
+            status=$(oc get postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} \
+              -o jsonpath='{.status.pgbackrest.manualBackup}')
+
+            if [ -n "$status" ]; then
+              echo "Current status: $status"
+              if echo "$status" | grep -q "${{ steps.set-backup-id.outputs.id }}"; then
+                if echo "$status" | grep -q '"finished":true'; then
+                  if echo "$status" | grep -q '"succeeded":1'; then
+                    echo "Backup completed successfully"
+                    exit 0
+                  else
+                    echo "Backup failed"
+                    exit 1
+                  fi
+                fi
+              fi
+            fi
+
+            echo "Backup in progress..."
+            sleep 10
+            timeout=$((timeout-10))
+          done
+
+          echo "Backup timed out"
+          exit 1


+    runs-on: ubuntu-latest
+    environment:
+      name: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}
+    env:
+      ENVIRONMENT: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}
+      BACKUP_ANNOTATION: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.backup_id }}
+      RESTORE_TIMESTAMP: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.restore_timestamp }}
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Validate Inputs
+        run: |
+          if [[ -z "${{ github.event.inputs.restore_timestamp }}" && -z "${{ github.event.inputs.backup_id }}" ]]; then
+            echo "Error: Either restore_timestamp or backup_id must be provided"
+            exit 1
+          fi
+
+      - name: Set environment variables
+        run: |
+          echo "NAMESPACE=f3c07a-${{ env.ENVIRONMENT }}" >> $GITHUB_ENV
+
+      - name: Set Cluster Name
+        id: set-cluster
+        run: |
+          CLUSTER_NAME='${{ github.event.inputs.cluster_name || 'api-postgres-clone' }}'
+          echo "cluster_name=$CLUSTER_NAME" >> $GITHUB_OUTPUT
+
+      - name: Debug - Show Environment Variables
+        run: |
+          echo "Current environment variables:"
+          echo "NAMESPACE: ${{ env.NAMESPACE }}"
+          # echo "RESTORE_TIMESTAMP: ${{ env.RESTORE_TIMESTAMP }}"
+          echo "CLUSTER_NAME: ${{ steps.set-cluster.outputs.cluster_name }}"
+
+      - name: Login to OpenShift
+        run: |
+          oc login ${{ secrets.OPENSHIFT_SERVER }} \
+            --token=${{ secrets.OPENSHIFT_API_TOKEN }} \
+            --insecure-skip-tls-verify=true
+          oc project ${{ env.NAMESPACE }}
+
+      - name: Debug - Show Current Cluster Configuration
+        run: |
+          echo "Current base postgrescluster.yml content:"
+          cat deployments/openshift/kustomize/base/crunchy/postgrescluster.yml
+          echo "\nCurrent patch.yml content (if exists):"
+          cat deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}/patch.yml || echo "patch.yml doesn't exist yet"
+          echo "\nCurrent kustomization.yml content:"
+          cat deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}/kustomization.yml
+
+      - name: Get Backup ID
+        if: github.event_name == 'workflow_dispatch' && github.event.inputs.backup_id != ''
+        run: |
+          # Get the pod name
+          POSTGRES_POD=$(oc get pods -l postgres-operator.crunchydata.com/role=master -o jsonpath='{.items[0].metadata.name}')
+
+          # Run pgbackrest info and filter for the backup with matching annotation
+          BACKUP_ID=$(oc exec $POSTGRES_POD -- pgbackrest info \
+            --output=json \
+            | jq -r '.[] | select(.backup[] != null) | .backup[] 
+            | select(.annotation.source == "${{ env.BACKUP_ANNOTATION }}") 
+            | .label' \
+            | tail -n1)
+
+          # Set the backup ID as an output
+          echo "BACKUP_ID=$BACKUP_ID" >> $GITHUB_ENV
+
+      - name: Update cluster names and patch (Backup ID)
+        if: github.event_name == 'workflow_dispatch' && github.event.inputs.backup_id != ''
+        run: |
+          # Update only metadata.name in base postgrescluster.yml
+          sed -i '0,/name:/{s/name: .*/name: '"${{ steps.set-cluster.outputs.cluster_name }}"'/}' \
+            deployments/openshift/kustomize/base/crunchy/postgrescluster.yml
+
+          # Use yq to append only the restore section
+          yq eval -i '.spec.backups.pgbackrest.restore = {
+            "enabled": true,
+            "repoName": "repo1",
+            "options": [
+              "--set=${{ env.BACKUP_ID }}",
+              "--type=immediate"
+            ]
+          }' deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}/patch.yml
+
+      - name: Update cluster names and patch (PITR)
+        if: github.event_name == 'workflow_dispatch' && github.event.inputs.restore_timestamp != ''
+        run: |
+          # Update only metadata.name in base postgrescluster.yml
+          sed -i '0,/name:/{s/name: .*/name: '"${{ steps.set-cluster.outputs.cluster_name }}"'/}' \
+            deployments/openshift/kustomize/base/crunchy/postgrescluster.yml
+
+          # Use yq to append only the restore section
+          yq eval -i '.spec.backups.pgbackrest.restore = {
+            "enabled": true,
+            "repoName": "repo1",
+            "options": [
+              "--type=time",
+              "--target=\"${{ env.RESTORE_TIMESTAMP }}\"",
+              "--target-timeline=current"
+            ]
+          }' deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}/patch.yml
+
+      - name: Debug - Show Updated Configuration
+        run: |
+          echo "Updated base postgrescluster.yml content:"
+          cat deployments/openshift/kustomize/base/crunchy/postgrescluster.yml
+          echo "\nUpdated patch.yml content:"
+          cat deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}/patch.yml
+          echo "\nDebug kustomize build output:"
+          oc kustomize deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }}
+
+      - name: Apply restore configuration
+        run: |
+          oc apply -k deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }} \
+            -n ${{ env.NAMESPACE }}
+
+      - name: Set Restore ID
+        id: set-restore-id
+        run: |
+          RESTORE_ID="restore-$(date '+%Y%m%d-%H%M%S')"
+          echo "id=$RESTORE_ID" >> $GITHUB_OUTPUT
+
+      - name: Trigger restore with annotation
+        run: |
+          oc annotate postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} \
+            --overwrite \
+            postgres-operator.crunchydata.com/pgbackrest-restore="${{ steps.set-restore-id.outputs.id }}" \
+            -n ${{ env.NAMESPACE }}
+
+      - name: Wait for specific restore completion
+        run: |
+          timeout=1200
+          while [ $timeout -gt 0 ]; do
+            status=$(oc get postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} \
+              -n ${{ env.NAMESPACE }} \
+              -o jsonpath='{.status.pgbackrest.restore}')
+
+            if [ -n "$status" ]; then
+              echo "Current restore status: $status"
+
+              # Parse JSON and check ID
+              restore_id=$(echo "$status" | jq -r '.id')
+              finished=$(echo "$status" | jq -r '.finished')
+
+              if [ "$restore_id" = "${{ steps.set-restore-id.outputs.id }}" ]; then
+                if [ "$finished" = "true" ]; then
+                  echo "Restore completed successfully"
+                  exit 0
+                fi
+              fi
+            fi
+
+            echo "Restore in progress..."
+            sleep 10
+            timeout=$((timeout-10))
+          done
+
+          echo "Restore timed out"
+          exit 1
+
+  cleanup:


+    needs: restore
+    if: always()
+    environment:
+      name: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}
+    env:
+      ENVIRONMENT: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.environment || (github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev') }}
+      BACKUP_ANNOTATION: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.backup_id || '20241125-163544' }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set environment variables
+        run: |
+          echo "NAMESPACE=f3c07a-${{ env.ENVIRONMENT }}" >> $GITHUB_ENV
+
+      - name: Set Cluster Name
+        id: set-cluster
+        run: |
+          CLUSTER_NAME=${{ github.event.inputs.cluster_name || 'api-postgres-clone' }}
+          echo "cluster_name=$CLUSTER_NAME" >> $GITHUB_OUTPUT
+
+      - name: Login to OpenShift
+        run: |
+          oc login ${{ secrets.OPENSHIFT_SERVER }} \
+            --token=${{ secrets.OPENSHIFT_API_TOKEN }} \
+            --insecure-skip-tls-verify=true
+          oc project ${{ env.NAMESPACE }}
+
+      - name: Remove restore annotation
+        continue-on-error: true
+        run: |
+          oc annotate postgrescluster ${{ steps.set-cluster.outputs.cluster_name }} \
+            postgres-operator.crunchydata.com/pgbackrest-restore- \
+            -n ${{ env.NAMESPACE }}
+
+      - name: Remove restore configuration
+        continue-on-error: true
+        run: |
+          # Update only metadata.name in base postgrescluster.yml
+          sed -i '0,/name:/{s/name: .*/name: '"${{ steps.set-cluster.outputs.cluster_name }}"'/}' \
+            deployments/openshift/kustomize/base/crunchy/postgrescluster.yml
+
+      - name: Apply configuration
+        run: |
+          oc apply -k deployments/openshift/kustomize/overlays/crunchy/${{ env.ENVIRONMENT }} \
+            -n ${{ env.NAMESPACE }}


+    name: Get Environment
+    runs-on: ubuntu-latest
+    outputs:
+      environment: ${{ steps.env.outputs.environment }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Get Environment
+        id: env
+        uses: ./.github/actions/get-environment
+  metadata:


+    name: Get Metadata
+    runs-on: ubuntu-latest
+
+    outputs:
+      needs-migration: ${{ steps.get-migrations.outputs.changes }}
+    steps:
+      - name: Debug Info
+        run: |
+          echo "Ref: ${{ github.ref }}"
+          echo "Ref Name: ${{ github.ref_name }}"
+          echo "SHA: ${{ inputs.sha }}"
+          echo "Full Ref: ${{ github.ref }}"
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Get Migrations
+        id: get-migrations
+        uses: dorny/paths-filter@v2
+        with:
+          base: ${{ github.ref }}
+          ref: ${{ github.ref }}
+          filters: |
+            migration:
+              - 'apps/backend/prisma/migrations/**'
+  migrate:


+    name: Migrate DB Schema
+    needs: [get-environment, metadata]
+    if: ${{ contains(needs.metadata.outputs.needs-migration, 'migration') }}
+    runs-on: ubuntu-latest
+    environment:
+      name: ${{ needs.get-environment.outputs.environment }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: 'migrate db'
+        uses: ./.github/actions/migrate-db
+        with:
+          environment: ${{ github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev' }}
+          openshift-server: ${{ secrets.OPENSHIFT_SERVER }}
+          api-token: ${{ secrets.OPENSHIFT_API_TOKEN }}
+      - name: Create Version Changeset
+        run: npm install @changesets/cli && npx changeset version
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Commit Version
+        if: ${{ github.ref_name == 'stage' }}
+        uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_message: 'Update version [skip ci]'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+  validate-inputs:


+    name: Validate Inputs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Validate changed_apps input
+        run: |
+          if [ -z "${{ inputs.changed_apps }}" ]; then
+            echo "Error: changed_apps input is required"
+            exit 1
+          fi
+          
+          # Validate JSON format
+          if ! echo '${{ inputs.changed_apps }}' | jq empty 2>/dev/null; then
+            echo "Error: changed_apps must be valid JSON array"
+            exit 1
+          fi
+          
+          # Check if array is empty
+          ARRAY_LENGTH=$(echo '${{ inputs.changed_apps }}' | jq 'length')
+          if [ "$ARRAY_LENGTH" -eq 0 ]; then
+            echo "Info: changed_apps is empty array, skipping deployment"
+            exit 0
+          fi
+          
+          echo "Validation passed: changed_apps contains $ARRAY_LENGTH item(s)"
+          
+  deploy:


+    name: Deploy
+    needs: [get-environment, metadata, validate-inputs]
+    if: ${{ needs.validate-inputs.result == 'success' && inputs.changed_apps != '[]' && inputs.changed_apps != '' }}
+    runs-on: ubuntu-latest
+    environment:
+      name: ${{ needs.get-environment.outputs.environment }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Trigger Deployments
+        uses: ./.github/actions/trigger-deploy
+        with:
+          openshift-server: ${{ secrets.OPENSHIFT_SERVER }}
+          api-token: ${{ secrets.OPENSHIFT_API_TOKEN }}
+          project: ${{ inputs.changed_apps }}
+          environment: ${{ github.ref_name == 'main' && 'prod' || github.ref_name == 'stage' && 'test' || 'dev' }}


      // Save file to filesystem
-      await writeFile(filePath, fileBuffer);
-      this.logger.debug(`File saved to: ${filePath}`);
+      await writeFile(fullFilePath, fileBuffer);


… source-producer picker Review findings #18, #19, #20, #21 (PR 3 — typed I/O): - #18: isAssignable compares array nesting depth, so T[][] no longer assigns to T[]. - #19: unrecognised kind strings fail closed (a typo no longer wildcards a port); only the intended wildcard/Artifact convention stays permissive. - #20: the variable picker enumerates source.upload/source.api producers, matching the backend validator so its compatibility grouping agrees with save-time checks. - #21: library-compat checks every declared input, not just inputs[0] (LibraryPortDescriptor has no `required` flag — documented in code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

kmandryk and others added 29 commits November 18, 2025 14:56

Remove backend-services OpenShift kustomize configs

d5923b8

Deleted all kustomize configuration files for backend-services and related image streams in the OpenShift deployments director. redundant now with overlays.

test separation with cache

fbbf521

adjust path

692417b

specify working directory

33c3e6f

make sure we're installing backend npm packages

44629bb

try built-in cach option

a8249c6

action for caching

367ead7

all dependencies, not just prod ones

1055a7f

remove prisma generate step.

182bd54

pass with no tests

72078af

do we need to install deps a second time with cache?

1544b58

update lint script and precommit

7d90030

add lint job

32bb131

frontend linting as well

14cf47c

fix endlines

edf73a7

Merge pull request #9 from bcgov/AI-489

b714a68

AI-489 QA Workflows Fix

add codeowners

32c1850

PR template

be737e8

add apache license

2984692

AI-426 OCR Service (#8)

b02111e

Merge branch 'develop' of https://github.com/bcgov/ai-adoption-docume…

88232a8

…nt-intelligence into develop

remove builds for all PRs. Only build for merge to develop and main

da3d295

Mantine UI and standard auth (#7)

a01f811

front end with upload functionality and backend with simple queue system

93d009b

formatting

e770834

bounding box overlay

685976b

Merge branch 'main' into AI-429

d407f3a

kmandryk requested a review from dbarkowsky as a code owner January 6, 2026 22:26

kmandryk requested review from NoorChasib, alex-struk and antsand as code owners January 6, 2026 22:26

github-advanced-security AI found potential problems Jan 6, 2026

View reviewed changes

kmandryk closed this Jan 6, 2026

kmandryk deleted the AI-429 branch February 27, 2026 17:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Ai 429 - #20

Ai 429#20
kmandryk wants to merge 29 commits into
mainfrom
AI-429

kmandryk commented Jan 6, 2026

Uh oh!

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check failure

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

kmandryk commented Jan 6, 2026

Summary

Changes

Uh oh!

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants