Skip to content

Commit 1b162a0

Browse files
authored
SEP-1946: Order the side-car's API processes behind schema creation (#1448)
Orders the consolidated side-car's three API programs behind the creation of every schema they read. - **New `app/core/celery/bootstrap.py`.** `bootstrap_beat_schema()` drives `sqlalchemy_celery_beat`'s own `SessionManager.prepare_models`, resolving the store and schema exactly as `DatabaseScheduler.__init__` does, so beat and this step cannot disagree about where the tables belong. The tables stay the library's to define — no alembic revision is added. It lives beside `app/core/celery/db.py` rather than inside it so importing it does not construct that module's `asyncpg` engine, which `prepare_models` cannot drive. Its readiness wait is **unbounded**, matching the three alembic one-shots' `until nc -z` loops. Every one-shot is `autorestart=false`, so a step that gave up could never publish its sentinel and would hold the programs gated on it for the life of the container; waiting means the sentinel still lands whenever the store appears. Only a non-`OperationalError` failure — a broken driver, a malformed URL — is surfaced immediately, which is what bounds the loop in place of a deadline. The scheduler's pool options are deliberately **not** forwarded. On this non-forked path the library pins `NullPool` and drops every `pool`-prefixed key, so such an option configures nothing; `max_overflow` carries no such prefix, reaches `create_engine`, and is rejected with a `TypeError`. Forwarding could therefore only be a no-op or a hard failure of this step on a documented, validated setting. - **New `sidecar/wait_for_schema.sh`.** Polls for the named `/tmp/migrate-<step>.ok` sentinels under a 300s budget, logging what is still missing every 10s, then hands off to the app with `exec`. Joined with `&&` in the program table, so an exhausted gate never starts an app against tables nothing created — the inverse of `celery-beat`'s `;`, which exists so beat starts whatever its gate reports. - **`sidecar/supervisord.conf`.** Adds `[program:migrate-beat]` — a fourth priority-10 one-shot running `python -m app.core.celery.bootstrap`, carrying the same keywords as its three siblings. It alone has no `until nc -z %(ENV_SEP_DB_HOST)s` loop: `CELERY__BEAT_DBURI` is a documented input for pointing beat at a store other than the SEP database, so that loop would probe the wrong host. Readiness instead lives in `_wait_for_store`, keyed on the URL the process actually dials. The three API programs are wrapped in the gate and gain `stopasgroup`/`killasgroup` so a stop reaches the waiting loop rather than orphaning it. - **`sidecar/entrypoint.sh`.** Clears the four sentinels from PID 1, which runs strictly before any program is spawned. Each one-shot clears its own sentinel only *after* being spawned — concurrently with the programs now gated on it — so this is the only race-free point. The four paths are named rather than globbed; a test asserts the named set equals the `migrate-*` programs in the program table. - **`sidecar/healthcheck.sh`.** Asserts the fourth sentinel too. The HTTP probe is unchanged. - **`sidecar/wait_for_api.py`.** Docstring only: it stated that the `migrate-*` one-shots all carry `until nc -z` loops, which this change makes false. - **`sidecar/Containerfile.sidecar`.** Copies the new gate into the image at mode `550` — supervisord invokes it directly rather than through an interpreter, which is why it is executable where the python helpers beside it are `440`. - **`.github/workflows/ci.yml`.** Adds a `test -x` assertion for `wait_for_schema.sh` to the existing side-car binaries smoke test, beside the one for `entrypoint.sh`. Also adds `wait_for_schema.sh`, `entrypoint.sh` and `Containerfile.sidecar` to the `python` change-filter: it already lists `settings.yaml`, `settings-env.sh`, `supervisord.conf` and `healthcheck.sh` because `tests/sidecar/` asserts on their contents, and without the three additions a change to any of them skipped the pytest tier written to guard it (`sidecar/**` fires only the image build, whose smoke tests are static). - **`sidecar/README.md`.** Documents the fourth one-shot, the gate and its budget, the sentinel clearing, and that the process-state assertion is weaker for a gated program. - **Tests.** New `tests/app/core/celery/test_bootstrap.py` covers the bootstrap against a real SQLite store (table creation, idempotency, the scheduler-matching resolution, the sync-URL requirement, the readiness retry and its deadline, an overridden store, and that the store password never reaches the log). New `tests/sidecar/test_wait_for_schema.py` drives the gate as a subprocess and asserts the program table's shape; its step list is derived from the parsed `migrate-*` sections rather than listed, so a step added to the program table without being wired into the API gates, the healthcheck and the sentinel clearing fails the suites asserting those. The supervisord parser behind that moves to `tests/sidecar/conftest.py`, which `test_wait_for_api.py` and `test_entrypoint.py` now share instead of each carrying a copy. `tests/sidecar/test_entrypoint.py` gains the sentinel-clearing cases plus a guard that the cleared names equal the `migrate-*` programs. `autorestart=true` and `startsecs=8` / `startretries=30` are deliberately kept on the API programs. Making an exhausted gate terminal would need `autorestart=unexpected` plus a reserved exit code, which reintroduces the risk of parking a process in FATAL for good rather than merely late. A schema step that genuinely fails is legible where the failure actually is: its one-shot ends `EXITED` with no sentinel and the container reports unhealthy. A gated program reports `RUNNING` while it is still waiting, so `healthcheck.sh`'s process-state assertion is weaker than it reads for those three. Its HTTP probe on `:9000`/`:9001`/`:9002` is what keeps a healthy container meaning the APIs answer, and that probe is untouched. **This PR modifies `.github/workflows/ci.yml`, and the `sidecar` job it changes is gated on `label-gate`, which fails while `qa in progress` is set.** The added smoke-test line therefore has not executed on this head, and the skipped job is not evidence it works. The assertion was checked by hand against a built image — the file is present and executable at that path — but the job must be observed green before merge.
1 parent d35925f commit 1b162a0

15 files changed

Lines changed: 1267 additions & 44 deletions

File tree

.github/workflows/ci.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,9 @@ jobs:
7979
- 'sidecar/settings-env.sh'
8080
- 'sidecar/supervisord.conf'
8181
- 'sidecar/healthcheck.sh'
82+
- 'sidecar/entrypoint.sh'
83+
- 'sidecar/wait_for_schema.sh'
84+
- 'sidecar/Containerfile.sidecar'
8285
- 'Makefile'
8386
- 'frontend/packages/api/specs/**'
8487
precommit:
@@ -301,6 +304,7 @@ jobs:
301304
test -f /home/sep/app/supervisord.conf
302305
test -f /home/sep/app/healthcheck.sh
303306
test -x /home/sep/app/entrypoint.sh
307+
test -x /home/sep/app/wait_for_schema.sh
304308
test -f /home/sep/app/settings-env.sh
305309
test -f /home/sep/app/settings.yaml
306310
'

app/core/celery/bootstrap.py

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Copyright (C) 2026 Percona LLC
2+
#
3+
# This program is free software: you can redistribute it and/or modify
4+
# it under the terms of the GNU Affero General Public License as published by
5+
# the Free Software Foundation, either version 3 of the License, or
6+
# (at your option) any later version.
7+
#
8+
# This program is distributed in the hope that it will be useful,
9+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11+
# GNU Affero General Public License for more details.
12+
#
13+
# You should have received a copy of the GNU Affero General Public License
14+
# along with this program. If not, see <https://www.gnu.org/licenses/>.
15+
16+
"""Create the Celery beat schedule tables ahead of the processes that read them.
17+
18+
The ``sqlalchemy_celery_beat`` tables are created by no alembic revision in any of
19+
SEP's three migration tracks: the library builds them itself, from
20+
:meth:`sqlalchemy_celery_beat.session.SessionManager.prepare_models`, which
21+
:class:`~sqlalchemy_celery_beat.schedulers.DatabaseScheduler` reaches on beat's
22+
own startup. Every service that seeds periodic tasks during its lifespan
23+
(:func:`app.core.celery.utils.init_periodic_tasks_db`) therefore reads tables
24+
whose only creator is a process ordered *behind* it, and on a database with no
25+
schema the read fails.
26+
27+
Driving the library's own bootstrap from a step ordered ahead of those services
28+
breaks the cycle while leaving the tables the library's to define — nothing here
29+
declares their shape. This module is kept beside :mod:`app.core.celery.db` rather
30+
than inside it so importing it does not construct that module's asynchronous
31+
engine, which resolves the same setting through a driver
32+
:meth:`~sqlalchemy_celery_beat.session.SessionManager.prepare_models` cannot use.
33+
"""
34+
35+
import logging
36+
import logging.config
37+
from time import sleep
38+
39+
from sqlalchemy.engine import Engine
40+
from sqlalchemy.exc import OperationalError
41+
from sqlalchemy_celery_beat.session import SessionManager
42+
43+
from app.core.config import settings
44+
45+
logger = logging.getLogger(__name__)
46+
47+
STORE_READINESS_POLL_INTERVAL = 1.0
48+
"""Seconds between connection attempts while the beat store is unreachable."""
49+
50+
51+
def _wait_for_store(engine: Engine) -> None:
52+
"""Block until the beat store accepts a connection.
53+
54+
The side-car's three alembic one-shots wait on ``SEP_DB_HOST``/``SEP_DB_PORT``
55+
in the shell, because that is the database they upgrade. The beat store is
56+
whatever ``CELERY.beat_dburi`` resolves to, and a deployment may point it at a
57+
separate database, so readiness is probed against the URL this process will
58+
actually dial rather than against a host named in the program table.
59+
60+
The wait is unbounded, matching those three shell loops. A bounded one could
61+
expire while the store was merely slow, and the caller runs as a one-shot that
62+
is never re-run, so its sentinel could then never appear — leaving every
63+
program gated on it waiting for the life of the container. What bounds the
64+
observable behaviour instead is the gate in front of each API program, and the
65+
healthcheck, which reports the missing sentinel either way.
66+
67+
``prepare_models`` retries too, but only for the check-then-create race it was
68+
written for: ten attempts with sub-second backoff, which a database that has
69+
not finished starting outlasts.
70+
71+
:param engine: The synchronous engine for the resolved beat store.
72+
:raises DBAPIError: On a connection failure that is not an
73+
``OperationalError``, which is raised on the first attempt rather than
74+
retried — only an ``OperationalError`` is treated as "not up yet".
75+
"""
76+
while True:
77+
try:
78+
with engine.connect():
79+
return
80+
except OperationalError:
81+
# Host and port only: the resolved URL carries the store's password.
82+
logger.info(
83+
"Waiting for the Celery beat store at %s:%s",
84+
engine.url.host,
85+
engine.url.port,
86+
)
87+
sleep(STORE_READINESS_POLL_INTERVAL)
88+
89+
90+
def bootstrap_beat_schema() -> None:
91+
"""Create the ``sqlalchemy_celery_beat`` schedule tables if they are absent.
92+
93+
The store and schema are resolved exactly as
94+
:meth:`sqlalchemy_celery_beat.schedulers.DatabaseScheduler.__init__` resolves
95+
them, so beat and this step cannot disagree about where the tables belong.
96+
``prepare_models`` checks before it creates, so a store that already carries
97+
them is left alone.
98+
99+
The scheduler's pool options are deliberately **not** forwarded. On this
100+
non-forked path the library pins ``NullPool`` and drops every
101+
``pool``-prefixed key, so such an option is either ignored or — for a key
102+
outside that prefix, such as ``max_overflow`` — rejected outright by
103+
``create_engine``. Neither outcome can configure anything, and the second
104+
would fail this step on a documented, validated setting.
105+
106+
:raises DBAPIError: When the store refuses a connection for a reason other
107+
than not being up yet, or when creating the tables fails after the
108+
library has exhausted its own retries. The family is ``DBAPIError``
109+
rather than ``DatabaseError`` because the first case surfaces as
110+
``InterfaceError``, a sibling of ``DatabaseError`` rather than one of
111+
its subclasses.
112+
:raises ArgumentError: When the resolved URL is malformed, or names a dialect
113+
whose driver is not installed. The engine is built before the wait, so
114+
this surfaces immediately.
115+
"""
116+
manager = SessionManager()
117+
engine, _ = manager.create_session(
118+
settings.CELERY.beat_dburi,
119+
schema=settings.CELERY.beat_schema,
120+
)
121+
try:
122+
_wait_for_store(engine)
123+
manager.prepare_models(engine, schema=settings.CELERY.beat_schema)
124+
finally:
125+
engine.dispose()
126+
127+
128+
def main() -> None:
129+
"""Run the bootstrap, configuring logging for a freshly spawned process.
130+
131+
Supervisord starts this in a process that has run no ``dictConfig``, and the
132+
wait's own log lines are the only account an operator gets of why the schema
133+
step has not finished. A failure is deliberately left to propagate: the
134+
non-zero exit is what keeps the caller's sentinel unwritten.
135+
136+
:raises SQLAlchemyError: When the tables cannot be created, or the store
137+
refuses a connection for a reason other than not being up yet.
138+
"""
139+
logging.config.dictConfig(settings.LOGGING_CONFIG)
140+
bootstrap_beat_schema()
141+
logger.info("Celery beat schedule tables are present.")
142+
143+
144+
if __name__ == "__main__":
145+
main()

changelog.d/SEP-1946.fixed.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The consolidated side-car no longer crash-loops on a first boot against an empty database. Its API processes now wait for every schema step to finish first, including the Celery beat schedule tables, which nothing created ahead of them before. This affects any deployment whose database starts empty, a first install included; it takes effect from the next container start on the new image, and a deployment whose schema is already populated sees no change.

sidecar/Containerfile.sidecar

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,9 @@ COPY --chown=0:1001 --chmod=440 ./alembic.ini .
7575
COPY --chown=0:1001 --chmod=440 ./sidecar/supervisord.conf ./supervisord.conf
7676
COPY --chown=0:1001 --chmod=550 ./sidecar/healthcheck.sh ./healthcheck.sh
7777
COPY --chown=0:1001 --chmod=550 ./sidecar/entrypoint.sh ./entrypoint.sh
78+
# Invoked directly by each API program's command rather than through an
79+
# interpreter, hence 550 where the python helpers below are 440
80+
COPY --chown=0:1001 --chmod=550 ./sidecar/wait_for_schema.sh ./wait_for_schema.sh
7881
# Sourced by entrypoint.sh, never executed, hence 440 rather than 550
7982
COPY --chown=0:1001 --chmod=440 ./sidecar/settings-env.sh ./settings-env.sh
8083
# Run through the interpreter and never executed directly -- grafana_service_account.py

sidecar/README.md

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,9 @@ the app packages the settings profile activates — see [App set](#app-set).
1818
|---|---|
1919
| `Containerfile.sidecar` | Final stage; ships the backend only, with no frontend-builder stage, and reuses the shared `sep:builder` wheel image. |
2020
| `entrypoint.sh` | PID 1. Mints the broker credential for the container run, resolves SEP's Grafana service-account token, then hands off to `supervisord`. |
21-
| `supervisord.conf` | Runs `valkey`, three `migrate-*` one-shots, the `sep`/`inventory`/`tasks` APIs, and the Celery worker and beat. |
21+
| `supervisord.conf` | Runs `valkey`, four `migrate-*` one-shots, the `sep`/`inventory`/`tasks` APIs, and the Celery worker and beat. |
2222
| `wait_for_api.py` | Run by `supervisord` ahead of the beat command; holds beat until the three APIs answer `/health`, then starts it whatever the outcome. |
23+
| `wait_for_schema.sh` | Run by `supervisord` ahead of each API command; holds the API until all four schema one-shots have published their sentinel, and fails rather than starting it if they do not. |
2324
| `healthcheck.sh` | Aggregate probe wired as the image `HEALTHCHECK`. |
2425
| `settings-env.sh` | Sourced by `entrypoint.sh`; expands the per-deployment inputs into the canonical `__`-nested settings variables, leaving unexported any name a file under `SECRETS_DIR` already supplies. |
2526
| `grafana_service_account.py` | Run by `entrypoint.sh` before `supervisord`; resolves SEP's Grafana service-account token, minting one when no source supplies it. |
@@ -403,12 +404,37 @@ rather than only a restart:
403404
## Health
404405

405406
`healthcheck.sh` exits 0 only when every non-one-shot program is `RUNNING`, all
406-
three `migrate-*` one-shots have written their `/tmp/migrate-<svc>.ok` sentinel,
407+
four `migrate-*` one-shots have written their `/tmp/migrate-<step>.ok` sentinel,
407408
the three `/health` endpoints return 200, and the bundled Valkey answers `PING`.
408409
The sentinels matter because a failed `alembic upgrade` ends in `EXITED` — the
409410
same state a successful one reaches — so program state alone cannot distinguish
410411
them.
411412

413+
The same four sentinels gate the API programs: each runs `wait_for_schema.sh sep
414+
inventory tasks beat` before `exec`-ing its app, so no API starts against a
415+
database whose schema has not been applied. The gate is uniform — `inventory`
416+
reads no beat table, but container health already requires all three APIs to
417+
answer, so gating them alike costs nothing and cannot drift out of step with
418+
which service seeds what. A step that never completes holds the APIs for
419+
`WAIT_BUDGET_SECONDS` (300) and then fails them — and because they keep
420+
`autorestart=true`, each one restarts into a fresh gate rather than stopping, so
421+
`supervisorctl status` shows the three APIs cycling every five minutes for as
422+
long as the sentinel is missing. The container is reported unhealthy well before
423+
the first budget expires, because its one-shot's sentinel is already missing.
424+
425+
Each one-shot waits for its own store without a bound — the three alembic steps
426+
in the shell, `migrate-beat` inside its bootstrap, against whatever
427+
`CELERY__BEAT_DBURI` resolves to. None of them is re-run once it exits, so a
428+
step that gave up could never publish its sentinel and would hold every gated
429+
program for the life of the container; waiting instead means the sentinel still
430+
lands whenever the store appears.
431+
432+
An API still inside its gate reports `RUNNING`, so the program-state assertion
433+
is weaker than it reads for those three — the `/health` probe is what keeps a
434+
healthy container meaning the APIs answer. `entrypoint.sh` clears the four
435+
sentinels before any program is spawned, so a restart cannot release a gate on
436+
the previous run's markers.
437+
412438
`HEALTHCHECK` is configured `--interval=15s --timeout=15s --start-period=150s
413439
--retries=5`, so a program going down surfaces as an unhealthy container after
414440
roughly 75-80s.

sidecar/entrypoint.sh

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,4 +62,13 @@ fi
6262
# Grafana administrator credential for the life of the container.
6363
unset grafana_token GF_SECURITY_ADMIN_USER GF_SECURITY_ADMIN_PASSWORD
6464

65+
# /tmp survives a container restart, and each schema one-shot clears its own
66+
# sentinel only after being spawned -- concurrently with the API programs now
67+
# gated on it, which could therefore read a previous run's marker. PID 1 runs
68+
# before supervisord starts anything, so clearing here covers every container
69+
# start and restart. A `supervisorctl restart` does not re-enter this script and
70+
# still races; nothing in the documented operation of the image does that.
71+
rm -f /tmp/migrate-sep.ok /tmp/migrate-inventory.ok /tmp/migrate-tasks.ok \
72+
/tmp/migrate-beat.ok
73+
6574
exec supervisord -c "$app_dir/supervisord.conf" "$@"

sidecar/healthcheck.sh

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
11
#!/usr/bin/env bash
22
# Aggregate container health for the consolidated SEP side-car.
3-
# Exit 0 only if every non-one-shot supervisord program is RUNNING, all three
4-
# migrations completed, all three API /health endpoints return 200, and the
3+
# Exit 0 only if every non-one-shot supervisord program is RUNNING, all four
4+
# schema steps completed, all three API /health endpoints return 200, and the
55
# bundled Valkey broker answers PING.
6+
#
7+
# An API program still inside its wait_for_schema.sh gate reports RUNNING, so
8+
# the process-state loop below cannot distinguish waiting from serving: the HTTP
9+
# probe is what keeps a healthy container meaning the APIs answer.
610
set -o errexit -o nounset -o pipefail
711

812
conf=/home/sep/app/supervisord.conf
@@ -26,9 +30,9 @@ for prog in "${programs[@]}"; do
2630
fi
2731
done
2832

29-
for svc in sep inventory tasks; do
33+
for svc in sep inventory tasks beat; do
3034
if [[ ! -f /tmp/migrate-$svc.ok ]]; then
31-
echo "unhealthy: $svc migrations did not complete successfully" >&2
35+
echo "unhealthy: $svc schema step did not complete successfully" >&2
3236
exit 1
3337
fi
3438
done

sidecar/supervisord.conf

Lines changed: 55 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -35,12 +35,15 @@ stdout_logfile=/dev/stdout
3535
stdout_logfile_maxbytes=0
3636
redirect_stderr=true
3737

38-
; --- Migration one-shots ----------------------------------------------------
39-
; No depends_on in supervisord: each waits for Postgres, runs once, ends EXITED.
40-
; A failed upgrade also ends EXITED (startsecs=0 makes FATAL unreachable), so
41-
; success is published as a sentinel file for healthcheck.sh to assert on.
38+
; --- Schema one-shots -------------------------------------------------------
39+
; No depends_on in supervisord: each waits for its own store, runs once, ends
40+
; EXITED. A failed run also ends EXITED (startsecs=0 makes FATAL unreachable),
41+
; so success is published as a sentinel file -- asserted by healthcheck.sh, and
42+
; waited on by the wait_for_schema.sh gate every API program runs.
4243
; Each runs under `sh -c`, so signals go to the process group -- otherwise a
4344
; stop reaches only the wrapper and orphans the wait loop or the upgrade.
45+
; entrypoint.sh clears every sentinel before any of these is spawned, which is
46+
; the only point strictly ahead of the programs gated on them.
4447
[program:migrate-sep]
4548
command=sh -c 'rm -f /tmp/migrate-sep.ok; until nc -z %(ENV_SEP_DB_HOST)s %(ENV_SEP_DB_PORT)s; do echo "[migrate-sep] waiting for postgres %(ENV_SEP_DB_HOST)s:%(ENV_SEP_DB_PORT)s"; sleep 1; done; alembic --name=sep upgrade heads && touch /tmp/migrate-sep.ok'
4649
directory=/home/sep/app
@@ -83,40 +86,80 @@ stdout_logfile=/dev/stdout
8386
stdout_logfile_maxbytes=0
8487
redirect_stderr=true
8588

86-
; --- API services, tolerate migration/DB lag via autorestart -----------------
87-
; A start that fails before startsecs -- Postgres not up yet, or the migration
88-
; one-shots still running -- spends the startretries budget, and exhausting it
89-
; parks the program in FATAL, which autorestart never revisits. The default of
90-
; 3 is spent in about ten seconds, so it is raised to span a first-run upgrade.
89+
; The beat schedule tables belong to sqlalchemy_celery_beat -- no revision in any
90+
; of the three alembic tracks creates them -- and until this step existed their
91+
; only creator was celery-beat, which starts behind the very APIs that read them.
92+
; This drives the library's own prepare_models, so the tables stay its to define.
93+
;
94+
; Alone among the four it carries no `until nc -z` loop, and the asymmetry is
95+
; deliberate: CELERY__BEAT_DBURI is a documented input for pointing beat at a
96+
; store other than the SEP database, which is the only host the loop can name.
97+
; Readiness therefore lives in the bootstrap, keyed on the URL it actually dials,
98+
; and is unbounded exactly as those loops are -- autorestart=false means a step
99+
; that gave up could never publish its sentinel, stranding every gated program.
100+
[program:migrate-beat]
101+
command=sh -c 'rm -f /tmp/migrate-beat.ok; python -m app.core.celery.bootstrap && touch /tmp/migrate-beat.ok'
102+
directory=/home/sep/app
103+
priority=10
104+
startsecs=0
105+
autorestart=false
106+
exitcodes=0
107+
startretries=0
108+
stopasgroup=true
109+
killasgroup=true
110+
stdout_logfile=/dev/stdout
111+
stdout_logfile_maxbytes=0
112+
redirect_stderr=true
113+
114+
; --- API services, gated on the schema one-shots -----------------------------
115+
; Each waits for all four sentinels before exec'ing its app, joined with `&&` so
116+
; an exhausted gate never starts an app against tables nothing created. The gate
117+
; is uniform even though inventory reads no beat table: container health already
118+
; requires all three APIs to answer, so partial availability buys nothing, and a
119+
; uniform rule cannot drift out of step with which service seeds what.
120+
;
121+
; startsecs=8 / startretries=30 are kept but no longer cover the schema race the
122+
; gate now closes: they cover a transient failure after it. A schema step that
123+
; genuinely failed leaves its sentinel unwritten, which healthcheck.sh reports.
124+
;
125+
; A program still inside its gate reports RUNNING, so healthcheck.sh's RUNNING
126+
; assertion is weaker than it reads -- its HTTP probe is what keeps a healthy
127+
; container meaning the APIs answer.
91128
[program:sep]
92-
command=python -m app.sep.main
129+
command=sh -c './wait_for_schema.sh sep inventory tasks beat && exec python -m app.sep.main'
93130
directory=/home/sep/app
94131
priority=20
95132
autorestart=true
96133
startsecs=8
97134
startretries=30
135+
stopasgroup=true
136+
killasgroup=true
98137
stdout_logfile=/dev/stdout
99138
stdout_logfile_maxbytes=0
100139
redirect_stderr=true
101140

102141
[program:inventory]
103-
command=python -m app.inventory.main
142+
command=sh -c './wait_for_schema.sh sep inventory tasks beat && exec python -m app.inventory.main'
104143
directory=/home/sep/app
105144
priority=20
106145
autorestart=true
107146
startsecs=8
108147
startretries=30
148+
stopasgroup=true
149+
killasgroup=true
109150
stdout_logfile=/dev/stdout
110151
stdout_logfile_maxbytes=0
111152
redirect_stderr=true
112153

113154
[program:tasks]
114-
command=python -m app.tasks.main
155+
command=sh -c './wait_for_schema.sh sep inventory tasks beat && exec python -m app.tasks.main'
115156
directory=/home/sep/app
116157
priority=20
117158
autorestart=true
118159
startsecs=8
119160
startretries=30
161+
stopasgroup=true
162+
killasgroup=true
120163
stdout_logfile=/dev/stdout
121164
stdout_logfile_maxbytes=0
122165
redirect_stderr=true

sidecar/wait_for_api.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,9 @@
1616
1717
``supervisord.conf`` runs this immediately before the ``celery-beat`` command.
1818
supervisord orders its spawn calls by ``priority`` but never waits for readiness —
19-
it has no ``depends_on``, which is why the ``migrate-*`` one-shots carry their own
20-
``until nc -z`` loops. ``celery-beat`` is therefore spawned in the same tick as the
19+
it has no ``depends_on``, which is why the alembic one-shots carry their own
20+
``until nc -z`` loops and the API programs carry ``wait_for_schema.sh``.
21+
``celery-beat`` is therefore spawned in the same tick as the
2122
priority-20 API programs it is ordered behind, and those need seconds to accept
2223
connections. Beat's schedule is persisted by the ``sqlalchemy`` scheduler, so
2324
anything already overdue is dispatched about a second in, and a periodic task whose

0 commit comments

Comments
 (0)