feat(setup): add Apple container orchestrator for the storage stack#3347
feat(setup): add Apple container orchestrator for the storage stack#3347anthonykurtdavid wants to merge 5 commits into
Conversation
Add scripts/setup/apple-container.sh to run the full LightRAG storage stack (PostgreSQL, Neo4j, Milvus standalone + etcd + minio) plus the LightRAG API server on Apple's native `container` CLI (macOS 26, Apple Silicon) — a Docker-Compose-free path for arm64 dev boxes that need the production-like PG/Neo4j/Milvus backends without Docker Desktop. Apple `container` 1.0.0 has no Compose support, so the script reimplements: - start ordering + health-wait loops (no depends_on / healthcheck), reusing the repo's PORT_HEX /proc/net/tcp probe idiom; - service wiring by the container IP `container` assigns on a shared network (1.0.0 has no working container-to-container service DNS, apple/container#856), so no sudo and no DNS setup are required; - named volumes (host bind mounts of DB data dirs are broken on Apple container, apple/container#333). All images are arm64-verified. Postgres uses pgvector/pgvector:pg18 (the setup template's AGE image is amd64-only; AGE is not needed since graph=Neo4j and vector=Milvus). Milvus uses the CPU tag v2.6.11, not the -gpu tag. LLM and embeddings run over outbound HTTPS (e.g. OpenAI); no GPU and no vLLM. Convenience `make apple-up`/`apple-down`/`apple-status`/`apple-logs`/ `apple-restart`/`apple-pull` targets wrap the script, reusing the Makefile's existing bash-4 interpreter resolver. Also adds docs/AppleContainerSetup.md (linked from README.md) and offline tests in tests/setup/test_apple_container.py; the generated .apple-container.env is git-ignored. Shellcheck-, ruff- and pre-commit-clean; validated end-to-end on macOS 26 / Apple Silicon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d34e71a045
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Address PR review feedback: - Prefix all stack containers with `lightrag-` (configurable via LIGHTRAG_AC_PREFIX) so down/up never stop, remove, or reuse a same-named container from another project; logs/restart now validate the service name. - Make container_ip accept both the `ipv4Address` (container CLI 1.0.x) and a plain `address` field; document the namespacing in AppleContainerSetup.md. Re-validated end-to-end on macOS 26 / Apple M2: stack up with lightrag-* names, /health healthy, data preserved (106 entities); shellcheck + offline tests clean.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8886188ead
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Address PR review feedback: derive the storage volume names from the same prefix as the container names (VOLUME_PREFIX from LIGHTRAG_AC_PREFIX), so two stacks configured with different prefixes never share Postgres/Neo4j/Milvus data and a `down --purge` from one never deletes the other's volumes. Re-validated on macOS 26 / Apple M2: up creates namespaced lightrag_* volumes, stack /health healthy; shellcheck and 9 offline tests clean.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a20d7217da
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Address PR review feedback: - Read Postgres/Neo4j/MinIO credentials from the source .env (precedence: shell override > .env value > dev default) so a password set in .env is used to create the database and matches what the lightrag container connects with, instead of being silently overwritten with the `rag` default. - Write MILVUS_DB_NAME (default `lightrag`) into the generated env-file, since MilvusVectorDBStorage requires it in addition to MILVUS_URI; a minimal .env would otherwise fail LightRAG's startup validation and time out `up`. Re-validated on macOS 26 / Apple M2 with a fresh purge: Postgres created with the .env user, stack /health healthy; 11 offline tests and shellcheck clean.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca973bc70a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| grep -vE "^${key}=" "$file" > "${file}.tmp" 2>/dev/null || true | ||
| mv "${file}.tmp" "$file" |
There was a problem hiding this comment.
When generate_env_file copies a user's .env and calls chmod 600, the first _set_kv call immediately writes a new ${file}.tmp using the process umask and then mvs it over the private file. With the default macOS umask 022, the final .apple-container.env becomes mode 644, exposing copied API keys/passwords to other local users despite the script's promise to keep it private. Re-apply chmod 600 after the rewrite or create the temp file with restrictive permissions before replacing the original.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed — _set_kv now creates its temp file under umask 077, so every rewrite (and the final mv) keeps .apple-container.env at mode 600; the initial chmod 600 is no longer undone. Added a test asserting the generated file's permissions.
Address PR review feedback: _set_kv created the temp file under the process umask (022 -> 644) and mv'd it over the private .apple-container.env, undoing the initial chmod 600 and exposing the copied API keys/passwords to other local users. Create the temp under umask 077 so every rewrite keeps mode 600, and add a test asserting the generated file's permissions.
2 participants
Description
Adds a native Apple
containerorchestrator so Apple Silicon users can run LightRAG's full Postgres / Neo4j / Milvus storage stack (plus the LightRAG API server) without Docker Desktop. Apple'scontainerruntime (v1.0.0) has no Docker Compose support and a few gaps that makedocker-compose-full.ymlunusable as-is, so this ships a self-contained bash orchestrator that reimplements the missing pieces.Related Issues
None. Works around three upstream Apple-runtime limitations: no Compose, no container-to-container service DNS (apple/container#856), and broken bind mounts for DB data dirs (apple/container#333).
Changes Made
scripts/setup/apple-container.sh— orchestrator withup/down/status/logs/restart/pull/help. Creates a dedicated network, launches Postgres, Neo4j, Milvus (standalone + etcd + MinIO) and LightRAG in dependency order with health-wait loops, and wires dependents by the container IP captured atuptime.docs/AppleContainerSetup.md— deployment guide (prerequisites, image table, storage/volume/port mapping, Compose-feature comparison, troubleshooting, trust-boundary note). Linked fromREADME.md.tests/setup/test_apple_container.py— 8pytest.mark.offlineunit tests (help/arg parsing,generate_env_filestorage→IP mapping,_set_kvidempotency,bash -nsyntax, image-tag pins). All daemon-free, so they run in CI on Linux.Makefile—make apple-up/apple-down/apple-status/apple-logs/apple-restart/apple-pullconvenience targets (reuse the Makefile's bash-4 resolver)..gitignore— ignores the generated.apple-container.env(contains copied secrets).How it works:
uplaunches services in order, reads each dependency's vmnet IP viacontainer inspect, and bakes it into the next service's environment. LightRAG receives a generated.apple-container.env(a copy of your.envwith storage endpoints rewritten to in-network IPs), so your real.envis never mutated. Databases are not published to host ports (matchingdocker-compose-full.yml); LightRAG reaches them in-network. Image tags mirror the Compose stack; Postgres usespgvector/pgvector:pg18(the setup template's AGE image is amd64-only) and Milvus uses the CPU (non--gpu) tag.Checklist
Additional Notes
Scope / limitations (honest):
containerruntime and bash 4+. Not a replacement for the Docker Compose path.container1.0.0 has no service DNS) → rundown && upto re-wire after a dependency's IP changes.Testing: 8 offline unit tests pass;
pre-commit(ruff-format + ruff) andshellcheckare clean. Manually validated end-to-end on macOS 26 / Apple M2: full stack up oncontainer1.0.0, ingested a document → 106 entities / 134 relations, a real RAG query returned HTTP 200, and data persisted across adown/upcycle. CI cannot exercise the container daemon, so daemon-dependent paths were validated manually only.