Skip to content

Commit 706a9bf

Browse files
authored
fix: increase create-namespace retry attempts and make timeout configurable (#142)
Fixes #136 ## Problem `temporal-create-namespace` was hardcoded to 3 health check attempts (~45 seconds total). On slower machines or fresh setups, the Temporal server takes longer to fully initialize its membership ring — so the script exhausts its retries and exits before the server is ready. The existing `temporal` container healthcheck uses `nc -z localhost 7233`, which only checks TCP connectivity. The port opens before the membership ring stabilizes, so `service_healthy` fires too early and `temporal-create-namespace` starts against a server that isn't truly ready. ## Fix - Increase default health check attempts from 3 to 30 (2.5 minutes at 5s intervals) - Add `TEMPORAL_HEALTH_CHECK_MAX_ATTEMPTS` and `TEMPORAL_HEALTH_CHECK_SLEEP_SECONDS` env vars for overriding - Add retry loop to the port wait step (previously a single `nc` call with no retry) - Add retry loop to the namespace create/describe step (previously no retry) - Fix unquoted variable references throughout
1 parent efb48a1 commit 706a9bf

1 file changed

Lines changed: 49 additions & 11 deletions

File tree

compose/scripts/create-namespace.sh

Lines changed: 49 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,63 @@ set -eu
33

44
NAMESPACE=${DEFAULT_NAMESPACE:-default}
55
TEMPORAL_ADDRESS=${TEMPORAL_ADDRESS:-temporal:7233}
6+
MAX_ATTEMPTS=${TEMPORAL_HEALTH_CHECK_MAX_ATTEMPTS:-30}
7+
SLEEP_SECONDS=${TEMPORAL_HEALTH_CHECK_SLEEP_SECONDS:-5}
68

79
echo "Waiting for Temporal server port to be available..."
8-
nc -z -w 10 $(echo $TEMPORAL_ADDRESS | cut -d: -f1) $(echo $TEMPORAL_ADDRESS | cut -d: -f2)
10+
SERVER_HOST=$(echo "$TEMPORAL_ADDRESS" | cut -d: -f1)
11+
SERVER_PORT=$(echo "$TEMPORAL_ADDRESS" | cut -d: -f2)
12+
attempt=1
13+
while ! nc -z -w 10 "$SERVER_HOST" "$SERVER_PORT"; do
14+
if [ "$attempt" -ge "$MAX_ATTEMPTS" ]; then
15+
echo "Temporal server port did not become available after $MAX_ATTEMPTS attempts"
16+
exit 1
17+
fi
18+
19+
echo "Temporal server port not ready yet, waiting... (attempt $attempt/$MAX_ATTEMPTS)"
20+
attempt=$((attempt + 1))
21+
sleep "$SLEEP_SECONDS"
22+
done
923
echo 'Temporal server port is available'
1024

1125
echo 'Waiting for Temporal server to be healthy...'
12-
max_attempts=3
13-
attempt=0
26+
attempt=1
1427

15-
until temporal operator cluster health --address $TEMPORAL_ADDRESS; do
16-
attempt=$((attempt + 1))
17-
if [ $attempt -ge $max_attempts ]; then
18-
echo "Server did not become healthy after $max_attempts attempts"
28+
while :; do
29+
if temporal operator cluster health --address "$TEMPORAL_ADDRESS"; then
30+
break
31+
fi
32+
33+
if [ "$attempt" -ge "$MAX_ATTEMPTS" ]; then
34+
echo "Server did not become healthy after $MAX_ATTEMPTS attempts"
1935
exit 1
2036
fi
21-
echo "Server not ready yet, waiting... (attempt $attempt/$max_attempts)"
22-
sleep 5
37+
38+
echo "Server not ready yet, waiting... (attempt $attempt/$MAX_ATTEMPTS)"
39+
attempt=$((attempt + 1))
40+
sleep "$SLEEP_SECONDS"
2341
done
2442

2543
echo "Server is healthy, creating namespace '$NAMESPACE'..."
26-
temporal operator namespace describe -n $NAMESPACE --address $TEMPORAL_ADDRESS || temporal operator namespace create -n $NAMESPACE --address $TEMPORAL_ADDRESS
27-
echo "Namespace '$NAMESPACE' created"
44+
45+
attempt=1
46+
while :; do
47+
if temporal operator namespace describe -n "$NAMESPACE" --address "$TEMPORAL_ADDRESS" >/dev/null 2>&1; then
48+
echo "Namespace '$NAMESPACE' already exists"
49+
break
50+
fi
51+
52+
if temporal operator namespace create -n "$NAMESPACE" --address "$TEMPORAL_ADDRESS" >/dev/null 2>&1; then
53+
echo "Namespace '$NAMESPACE' created"
54+
break
55+
fi
56+
57+
if [ "$attempt" -ge "$MAX_ATTdMPTS" ]; then
58+
echo "Failed to create namespace '$NAMESPACE' after $MAX_ATTEMPTS attempts"
59+
exit 1
60+
fi
61+
62+
echo "Namespace operation not ready yet, waiting... (attempt $attempt/$MAX_ATTEMPTS)"
63+
attempt=$((attempt + 1))
64+
sleep "$SLEEP_SECONDS"
65+
done

0 commit comments

Comments
 (0)