Skip to content

fix: Add Helm hooks for schema Job to fix deployment ordering#888

Merged
robholland merged 3 commits intomainfrom
jiechenz/add-helm-hook
Apr 14, 2026
Merged

fix: Add Helm hooks for schema Job to fix deployment ordering#888
robholland merged 3 commits intomainfrom
jiechenz/add-helm-hook

Conversation

@jiechenz
Copy link
Copy Markdown
Contributor

What was changed

Added Helm hooks (pre-install, pre-upgrade) to the schema Job and shims ConfigMap to enforce deployment ordering. The schema Job now completes before server pods are created.

Why?

When using Elasticsearch for visibility, a race condition exists between the schema Job and server Deployments. If a server pod writes to Elasticsearch before the schema Job creates the index with correct mappings, Elasticsearch auto-creates the index with wrong field types (e.g., WorkflowId as text instead of keyword). The schema Job then fails with mapper [WorkflowId] cannot be changed from type [text] to [keyword].

Helm hooks eliminate this race by ensuring the schema Job runs and completes before any server Deployments are created.

Checklist

  1. Closes [Bug] Error in temporal-schema pod creating mappings for Elasticsearch #882

  2. How was this tested:

  • helm unittest
  • helm template verified hook annotations present
  • Deployed on kind cluster with PostgreSQL + Elasticsearch:
    • Verified schema Job completes before server pods start
  • Deployed via ArgoCD on kind cluster:
    • verified argoCD correctly mapped Helm hooks to PreSync wave
Screenshot 2026-04-13 at 2 54 19 PM
  1. Any docs updates needed?

Updated README with a new section "Schema Setup and Deployment Ordering"

@jiechenz jiechenz requested a review from a team as a code owner April 13, 2026 23:18
Comment thread charts/temporal/values.yaml Outdated
podDisruptionBudget: {}
schema:
# Use Helm hooks to ensure schema setup completes before server pods start.
# Set to false if using ArgoCD, Flux, Rancher or Terraform.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We should soften this if the hooks can also help in some ArgoCD cases, which the README suggests is the case. Maybe call out the specific for ArgoCD if it's succinct enough, or refer to the README if not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed ArgoCD from the comments.

@robholland robholland merged commit 8c3297d into main Apr 14, 2026
4 checks passed
@robholland robholland deleted the jiechenz/add-helm-hook branch April 14, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Error in temporal-schema pod creating mappings for Elasticsearch

2 participants