Skip to content

Commit cf69a05

Browse files
refactor(infra-local): simplify + unify into one Python compose package (~50% smaller) (#796)
## What Simplifies `apps/infra-local` (the BoxLite-based local dev stack) and unifies its two-language orchestrator into one Python package. **~4,500 → ~2,190 lines (~50%)**, one entry point (`python -m compose`), flat structure. ## Commits 1. **simplify** — drop dead config (the `BOXLITE_*_HOST_PORT` knobs never moved a bound port; `pg_url`/`tcp_port`/`Severity.WARN`; dead `configs/minio/init.sh`), dedupe the SDK import shim, delete the stale `CONNECTIONS.md` (445 lines restating the code), rewrite the README, and remove the infra-local test suite (not in CI; mostly asserted dataclass defaults) — preserving the one unique capability as a native-gated SDK test. 2. **port L2 to Python + rename + flatten** — the 4 native host processes (API/runner/proxy/dashboard) move from ~800 lines of bash (`scripts/`) into `compose/native.py` (subprocess supervision: detached daemons, pidfiles, `killpg` teardown). Package `boxlite_local` → `compose`. `scripts/` + `configs/` deleted; `api.env` → root. CLI is 7 verbs (`up/down/status/logs/restart/reset/nuke`); Makefile ~19 → 9 thin aliases. 3. **adversarial fixes** — a 2-round adversarial-review loop found + fixed an `export KEY=val` parser bug in the `.env` reader; the SDK capability test was rewritten (`alpine` busybox lacks `httpd`) to a `python:3-alpine` http.server box and verified live. ## Final structure 4 root files (`api.env`, `Makefile`, `README.md`, `pyproject.toml`) + the `compose/` package (config / services / orchestrator [L1] / native [L2] / doctor / _sdk / __main__). No `scripts/`, `configs/`, or `tests/`. ## Verification - `python -m compose --help` + all 7 subcommands; `py_compile` + import all modules. - SDK volume-port test passes **live** (`pytest -m integration` → 1 passed, 30s): RW-volume persistence + host-port mapping. - 2-round adversarial review terminated at `ZERO_FINDINGS`; CLAUDE.md auditor PASS on every commit; pre-push integration suite green. ## ⚠️ Reviewer note — live-stack validation needed before merge `compose/native.py` is a faithful behavior-port of the deleted bash, but **process supervision is runtime-only behavior**. It compiles/imports/parses and the adversarial review cleared it against the bash spec, but a live **`make up && make status`** (then `make down` / `make restart COMPONENTS=runner`) on an Apple Silicon Mac is the final proof the L2 daemons spawn/reap correctly. Please run that before merging. No Cargo/Rust/core-SDK source changes; scope is `apps/infra-local/` + one `sdks/python/tests/` file. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit # Release Notes * **Refactor** * Simplified local infrastructure orchestration: replaced the legacy `boxlite_local` workflow with `python -m compose`; Makefile now exposes thin aliases for `up`, `down`, `status`, `logs`, `restart`, `reset`, and `nuke`. * Simplified configuration and standardized the persistent state directory to `.apps-local/`. * **Documentation** * Updated `apps/infra-local` README for the new `compose` workflow and endpoints. * Removed legacy connection-reference documentation. * **Tests** * Added a Python SDK integration test covering RW volume persistence and port forwarding across restarts. * Removed legacy infra-local integration/unit test suites. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: dorianzheng <8065637+DorianZheng@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
1 parent 757692a commit cf69a05

39 files changed

Lines changed: 1299 additions & 3678 deletions

apps/infra-local/CONNECTIONS.md

Lines changed: 0 additions & 450 deletions
This file was deleted.

apps/infra-local/Makefile

Lines changed: 22 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -1,99 +1,34 @@
1-
# BoxLite-based infra-local orchestrator
2-
#
3-
# All targets are thin wrappers around `python -m boxlite_local`. The Python
4-
# CLI is the source of truth — see `python -m boxlite_local --help` for the
5-
# full surface area. This Makefile exists for muscle memory + tab completion.
1+
# infra-local — thin aliases over `python -m compose <cmd>`.
2+
# The Python CLI is the source of truth: `python -m compose --help`.
3+
# This Makefile exists for muscle memory + tab completion.
64

75
PY ?= python
86

9-
# All recipes must agree with scripts/_stack-common.sh (and the python
10-
# orchestrator's own default) on where the L1 SDK home lives, so the
11-
# `boxlite` CLI here (stack-rebuild-l1-box) sees the same boxes.
12-
REPO_ROOT := $(abspath $(CURDIR)/../..)
13-
export BOXLITE_HOME ?= $(REPO_ROOT)/.apps-local/boxlite
7+
.PHONY: help install up down status logs restart reset nuke
148

15-
# Prefer a boxlite CLI built from THIS checkout over a stale global install
16-
# (an older PATH `boxlite` can fail to read this version's box-config rows).
17-
# Matches the resolver in scripts/_stack-common.sh.
18-
BOXLITE_CLI := $(firstword $(wildcard $(REPO_ROOT)/target/release/boxlite $(REPO_ROOT)/target/debug/boxlite) boxlite)
9+
help: ## show this help
10+
@awk -F':.*## ' '/^[a-zA-Z][a-zA-Z0-9_-]*:.*## / { printf " %-9s %s\n", $$1, $$2 }' $(MAKEFILE_LIST)
1911

20-
.PHONY: help up down ps doctor logs wipe itest test install migrate e2e itest-all \
21-
stack-build stack-up stack-down stack-restart stack-status stack-logs stack-reset stack-reset-hard stack-nuke \
22-
seed-init-data stack-rebuild-l1-box
12+
install: ## install the package (editable; pulls the boxlite SDK)
13+
$(PY) -m pip install -e .
2314

24-
help: ## show this help
25-
@awk -F':.*## ' '/^[a-zA-Z][a-zA-Z0-9_-]*:.*## / { printf " %-16s %s\n", $$1, $$2 }' $(MAKEFILE_LIST)
15+
up: ## ensure L1 boxes + start L2 (COMPONENTS="api runner")
16+
$(PY) -m compose up $(COMPONENTS)
2617

27-
install: ## install the package + test extras
28-
$(PY) -m pip install -e ".[test]"
18+
down: ## stop L2 procs (ARGS=--all also stops L1 boxes)
19+
$(PY) -m compose down $(ARGS) $(COMPONENTS)
2920

30-
up: ## bring all services up (runs doctor preflight first)
31-
$(PY) -m boxlite_local up
21+
status: ## one-screen L1 + L2 health
22+
$(PY) -m compose status
3223

33-
down: ## stop L2 native procs + remove L1 services
34-
-./scripts/stack-down.sh
35-
$(PY) -m boxlite_local down
24+
logs: ## tail a component log (COMPONENT=api|all)
25+
$(PY) -m compose logs $(COMPONENT)
3626

37-
wipe: ## stop L2 + remove L1 + wipe data dir
38-
-./scripts/stack-down.sh
39-
$(PY) -m boxlite_local down --wipe
27+
restart: ## restart L2 proc(s) / recreate L1 box(es) (COMPONENTS="runner dex")
28+
$(PY) -m compose restart $(COMPONENTS)
4029

41-
ps: ## list running boxlite-local-* boxes
42-
$(PY) -m boxlite_local ps
30+
reset: ## wipe L2 runtime state (ARGS=--hard also rebuilds the schema)
31+
$(PY) -m compose reset $(ARGS)
4332

44-
doctor: ## run preflight checks (SDK + runtime + port conflicts)
45-
$(PY) -m boxlite_local doctor
46-
47-
test: ## run unit tests (no BoxLite required)
48-
$(PY) -m pytest tests/unit -q
49-
50-
itest: ## run integration smoke test (requires BoxLite runtime, ~30s)
51-
BOXLITE_INTEGRATION=1 $(PY) -m pytest tests/integration/test_multi_service.py -v -s
52-
53-
e2e: ## run comprehensive E2E suite (~60s, ~90s with smoke)
54-
BOXLITE_INTEGRATION=1 $(PY) -m pytest tests/integration/test_e2e_full.py -v -s
55-
56-
itest-all: ## run BOTH integration suites (~90s)
57-
BOXLITE_INTEGRATION=1 $(PY) -m pytest tests/integration -v
58-
59-
migrate: ## build local pg schema by running all TypeORM migrations from scratch
60-
cd ../api && set -a && . ../.env && set +a && \
61-
npx ts-node -P ./tsconfig.json -r tsconfig-paths/register \
62-
../node_modules/typeorm/cli.js migration:run -d ./src/migrations/data-source.ts
63-
64-
seed-init-data: ## ensure dashboard-required base data (admin org, default region, wait runner)
65-
./scripts/seed-init-data.sh
66-
67-
# ── L2 stack wrappers (L1 + native API/Runner/Proxy/Dashboard) ───────────────
68-
69-
stack-build: ## build native runner + proxy binaries
70-
./scripts/stack-build.sh
71-
72-
stack-up: ## ensure L1 up + start all L2 native processes (idempotent)
73-
./scripts/stack-up.sh $(COMPONENTS)
74-
75-
stack-down: ## stop all L2 native processes (L1 stays up; add ARGS=--all to stop L1 too)
76-
./scripts/stack-down.sh $(ARGS) $(COMPONENTS)
77-
78-
stack-restart: ## restart one or more L2 components (runner = also rebuild)
79-
./scripts/stack-restart.sh $(COMPONENTS)
80-
81-
stack-status: ## one-screen health check across L1 + L2
82-
./scripts/stack-status.sh
83-
84-
stack-logs: ## tail logs (COMPONENT=api|runner|proxy|dashboard|all)
85-
./scripts/stack-logs.sh $(COMPONENT)
86-
87-
stack-reset: ## wipe L2 runtime state (PG user data + runner home; L1 + schema preserved)
88-
./scripts/stack-reset.sh
89-
90-
stack-reset-hard: ## like stack-reset, but also re-applies prod schema baseline
91-
./scripts/stack-reset.sh --hard
92-
93-
stack-nuke: ## ABSOLUTE NUKE: L1 boxes destroyed + data wiped + logs cleared
94-
./scripts/stack-reset.sh --nuke
95-
96-
stack-rebuild-l1-box: ## destroy + recreate one L1 box (BOX=dex|registry|...) — for stuck stateful services
97-
@if [ -z "$(BOX)" ]; then echo "usage: make stack-rebuild-l1-box BOX=<name>"; echo "e.g. BOX=dex when login returns stale tokens"; echo "e.g. BOX=registry when image pulls hang"; exit 1; fi
98-
$(BOXLITE_CLI) rm boxlite-local-$(BOX) --force
99-
python -m boxlite_local up $(BOX)
33+
nuke: ## tear down EVERYTHING (L1 boxes + data + logs)
34+
$(PY) -m compose nuke

0 commit comments

Comments
 (0)