Skip to content

Commit f2f60db

Browse files
wyattgill9claude
andauthored
symphony: move the elixir runtime into packages/symphony (#782)
## Summary Move the Symphony Elixir runtime into `packages/symphony`, absorbed from [indexable-inc/symphony](https://github.com/indexable-inc/symphony) at `c9e709208c3ae161e24f625b9f3808a288c859ed`. After symphony#268 moved the room stack (room-server, the Tauri/Svelte UI) into the ix monorepo, the dedicated repo was the Elixir runtime alone; this PR gives it its long-term home in index so the standalone repo can retire. Follows up on the reverted whole-repo subtree attempt (#767#779/#780): this time only the Elixir part moves, per the plan agreed in Slack (room → ix, elixir → index). ## What this adds - `packages/symphony/`: the runtime (`elixir/`), the engine wire fixtures (`contracts/fixtures`, kept beside `elixir/` because the contract tests resolve `../../contracts`), the bundled example pack (`workflows/example`), `bin/run-nix`, and docs. The 30 MB of `.github` demo media, the standalone CI workflows, and the PR template did not move; the `pr_body.check` mix task that validated that template is deleted as vestigial. - `packages.<sys>.symphony`: the launcher, parity with the standalone flake's `packages.default` (Nushell wrapper exec'ing `bin/run-nix` with Elixir 1.19/OTP 28, gh, git, openssh, cacert on PATH). Production deploys keep working the same way: stage source, `mix deps.get`, `mix run --no-halt`. - `nixosModules.symphony` via `modules/services/symphony/`: the service module, unchanged, auto-discovered under the same attr name ix imports from the symphony flake today. - `checks.x86_64-linux.symphony-elixir`: the standalone repo's required lane (`mix compile --warnings-as-errors`, `mix format --check-formatted`, `mix credo`, `mix test`; 384 tests) as a sandboxed derivation. Deps come from a `fetchMixDeps` fixed-output derivation; rebar is pinned; the lazy_html C++ NIF (test-only, LiveView HTML assertions) is satisfied by seeding elixir_make's artifact cache with the upstream release tarball, which elixir_make still verifies against the `checksum.exs` pinned in the dep. The advisory lane (dialyzer, sobelow, deps.audit, coveralls) stays a local `make quality` run; see `packages/symphony/docs/quality.md`. - `devShells.<sys>.symphony`: parity with the standalone devshell (elixir, erlang, codex, gh, git, openssh). - Eval assertions (`tests/default.nix`, `symphony` group) pinning the module's unit env contract (`SYMPHONY_WORKFLOW_PACK` default, primary-repo export, ExecStart shape, EnvironmentFile pass-through, hostRuntime gating) that ix's hil deployment and worker module read. ## The room-server seam (deliberately untouched) The `symphony` flake input stays exactly as #780 restored it: pinned to the last rev that still builds `room-server`, feeding `pkgs.symphony-room-server` into `images/dev/symphony-codex` and its eval tests. room-server's source now lives in the private ix monorepo, so the public image cannot build it once the symphony repo goes away. Resolving that seam (move the image to ix, or have ix layer room-server onto a public base image) is the remaining blocker for retiring the repo; the input comment in `flake.nix` now says so. Do not `nix flake update symphony` until then: symphony@main no longer exports `room-server`, so a bump breaks the image eval. ## ix follow-up (after this merges) - `inputs.symphony.packages.<sys>.default` → `inputs.index.packages.<sys>.symphony` in `nix/modules/services/host/symphony/module.nix` and `symphony-runtime/module.nix` - `inputs.symphony.nixosModules.symphony` → `inputs.index.nixosModules.symphony` - `inputs.symphony.packages.<sys>.codex` → `pkgs.codex` (the symphony flake's `codex` output was a plain re-export for pin visibility) - drop ix's `symphony` input and the `inputs.symphony.follows` line on its `index` input - then resolve the image seam, drop index's `symphony` input, and archive indexable-inc/symphony ## Validation - `nix build .#checks.x86_64-linux.symphony-elixir` (384 tests, 0 failures, sandboxed) - `nix build .#ciChecks.x86_64-linux.eval` (aggregate includes the new `ix-test-symphony` and the untouched `ix-test-symphony-codex`) - `nix build .#packages.x86_64-linux.symphony` and `.#devShells.x86_64-linux.symphony` - `nix eval .#nixosModules --apply builtins.attrNames` lists `symphony` - `nix run .#lint` (nixfmt, statix, deadnix, ast-grep, ast-grep-test all green) - `git diff --check` clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Move the Symphony Elixir runtime into packages/symphony > - Relocates the Symphony Elixir runtime into [packages/symphony/elixir](https://github.com/indexable-inc/index/pull/782/files#diff-c3bd38f0e22ac85e5dd9316c2d736a934c9f00aa9e8b68e2e956b30fbf2ddaeb), establishing it as a self-contained Mix project (`symphony_elixir` v0.2.0) with its own toolchain, Nix derivation, and CI quality gate. > - Adds a full OTP application with role-based supervision: control-plane nodes boot the web endpoint, workflow/skill catalogs, cron/Slack triggers, GitHub App token management, and the IR runtime; worker nodes boot a minimal Slipstream client. > - Introduces the IR execution layer: `RunGraph`, `Node`, `Attempt`, `Store` (JSON-backed persistence), `Materializer`, `Graph` (ready-node scheduling and failure propagation), `Recovery`, and a `DynamicSupervisor` that resumes pending runs after restart. > - Adds a DSL pipeline — `Lexer`, `Parser`, `Interpreter`, and `Schema` — that compiles `.sym` workflow files into IR nodes, with `WorkflowCatalog` hot-reloading them from disk. > - Exposes a Phoenix web layer with LiveView dashboards for runs, workflows, skills, and statistics, plus a JSON API at `/api/v1` with run controls and webhook receivers for GitHub, Linear, and Slack. > - Adds a NixOS module at [modules/services/symphony/default.nix](https://github.com/indexable-inc/index/pull/782/files#diff-de2dbd88d424fd159a6aa9ddcfef4d9505fb4d90f66eed66936038c967e6e7bf) and a `bin/run-nix` entrypoint for production deployment, with a new `symphony` developer shell in [lib/per-system.nix](https://github.com/indexable-inc/index/pull/782/files#diff-9879a1f03396eb758ff538e44a00fb409b99bc0bbfac4656755b05716d036683). > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 8b5445b.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 104bf2e commit f2f60db

157 files changed

Lines changed: 24454 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

flake.nix

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@
5757
inputs.nixpkgs.follows = "nixpkgs";
5858
};
5959

60+
# Provides only `room-server` for images/dev/symphony-codex (through
61+
# `pkgs.symphony-room-server` in lib/overlay.nix). The Elixir runtime
62+
# itself lives in packages/symphony now; room-server's source moved to
63+
# the ix monorepo, so this pin stays on the last symphony rev that still
64+
# builds it and retires once the image's room-server seam moves too.
6065
symphony = {
6166
url = "github:indexable-inc/symphony/main";
6267
inputs.nixpkgs.follows = "nixpkgs";

lib/per-system.nix

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -633,6 +633,11 @@ let
633633
printf '%s\n' '${forced}' > "$out"
634634
'';
635635
run-records-session = repoPackages.run.passthru.tests.recordsSession;
636+
# Symphony's required quality lane (compile -Werror, mix format,
637+
# credo, mix test) as a sandboxed derivation; see
638+
# packages/symphony/default.nix. The advisory lane (dialyzer,
639+
# sobelow, deps.audit) stays a local `mix quality` run.
640+
symphony-elixir = repoPackages.symphony.passthru.tests.elixir;
636641
# Deterministic alloc-count gate for indexbench: runs the counting-
637642
# allocator demo bench once through `indexbench assert` and fails if its
638643
# allocation count exceeds the declared budget. Reproducible, unlike
@@ -809,5 +814,19 @@ in
809814
pkgs.jemalloc
810815
];
811816
};
817+
818+
# Dev loop for packages/symphony: the Elixir/OTP pairing the runtime pins
819+
# (1.19 on 28) plus the host tools bin/run-nix expects. codex is the plain
820+
# nixpkgs CLI; authenticate it before `nix run .#symphony`.
821+
symphony = pkgs.mkShellNoCC {
822+
packages = [
823+
(ix.languages.elixir.toolchain pkgs { version = "1.19"; })
824+
(ix.languages.erlang.toolchain pkgs { version = "28"; })
825+
pkgs.codex
826+
pkgs.gh
827+
pkgs.git
828+
pkgs.openssh
829+
];
830+
};
812831
};
813832
}
Lines changed: 353 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,353 @@
1+
# NixOS service module for the Symphony runtime.
2+
#
3+
# Minimal opinionated systemd unit. Reads secrets from an EnvironmentFile
4+
# you control, so you can wire any secret manager (sops-nix, agenix,
5+
# Bitwarden Secrets Manager, AWS Secrets Manager, etc.) underneath. For
6+
# Bitwarden Secrets Manager specifically, set `secretsCommand` to a
7+
# `bws run -- ...` invocation; the unit will wrap ExecStart with it.
8+
{
9+
config,
10+
lib,
11+
pkgs,
12+
...
13+
}:
14+
let
15+
inherit (lib)
16+
mkEnableOption
17+
mkIf
18+
mkOption
19+
optionalString
20+
types
21+
;
22+
23+
cfg = config.services.symphony;
24+
in
25+
{
26+
options.services.symphony = {
27+
enable = mkEnableOption "Symphony runtime";
28+
29+
package = mkOption {
30+
type = types.package;
31+
description = "Symphony package (provides /bin/symphony from this flake's default output).";
32+
};
33+
34+
user = mkOption {
35+
type = types.str;
36+
default = "symphony";
37+
description = "Unix user the service runs as. Set to an existing user, or let DynamicUser handle it.";
38+
};
39+
40+
stateDir = mkOption {
41+
type = types.path;
42+
default = "/var/lib/symphony";
43+
description = "Directory for runs, workspaces, logs, and the staged runtime copy.";
44+
};
45+
46+
httpPort = mkOption {
47+
type = types.port;
48+
default = 4040;
49+
description = "Phoenix HTTP listener port.";
50+
};
51+
52+
primaryRepo = mkOption {
53+
type = types.nullOr types.path;
54+
default = null;
55+
description = "Absolute path to the primary repository checkout (SYMPHONY_PRIMARY_REPO).";
56+
};
57+
58+
repoRoot = mkOption {
59+
type = types.nullOr types.path;
60+
default = null;
61+
description = "Optional parent directory of sibling repository checkouts (SYMPHONY_REPO_ROOT). Defaults to the parent of primaryRepo.";
62+
};
63+
64+
workflowPack = mkOption {
65+
type = types.str;
66+
default = "example";
67+
description = "Built-in workflow pack name; ignored when packDir is set.";
68+
};
69+
70+
packDir = mkOption {
71+
type = types.nullOr types.path;
72+
default = null;
73+
description = "Absolute path to an external workflow pack (SYMPHONY_PACK_DIR). Takes precedence over workflowPack.";
74+
};
75+
76+
roomRegistryUrl = mkOption {
77+
type = types.nullOr types.str;
78+
default = null;
79+
description = ''
80+
Central room.ix.dev base URL each run's room-server registers its
81+
backend with (SYMPHONY_ROOM_REGISTRY_URL). Drives both the room UI's
82+
transcript view and the Slack "Run details" deep link. Unset disables
83+
registration and the Slack link. The matching write token is a secret;
84+
supply SYMPHONY_ROOM_REGISTRY_TOKEN via environmentFile.
85+
'';
86+
};
87+
88+
roomAdvertiseHost = mkOption {
89+
type = types.nullOr types.str;
90+
default = null;
91+
description = ''
92+
Address a provisioned per-run room-server binds and advertises so
93+
room.ix.dev can reach it to proxy the run's transcript
94+
(SYMPHONY_ROOM_ADVERTISE_HOST). Set to this host's tailnet address when
95+
room.ix.dev runs elsewhere; unset keeps the loopback default, reachable
96+
only when room.ix.dev shares the host.
97+
'';
98+
};
99+
100+
roomServerUrl = mkOption {
101+
type = types.nullOr types.str;
102+
default = null;
103+
description = ''
104+
Standing room-server URL for `:local` / `{:room, url}` placements that
105+
do not provision their own per-run server (SYMPHONY_ROOM_SERVER_URL).
106+
'';
107+
};
108+
109+
extraEnvironment = mkOption {
110+
type = types.attrsOf types.str;
111+
default = { };
112+
description = ''
113+
Additional environment variables exported to the service. Use for
114+
non-secret config: LINEAR_WORKSPACE_SLUG, SYMPHONY_BOT_USERNAME,
115+
SYMPHONY_BOT_EMAIL, SYMPHONY_GITHUB_APP_OWNER_REPO,
116+
SYMPHONY_GITHUB_STATS_QUERY, SYMPHONY_SLACK_NOTIFY_CHANNEL, etc.
117+
'';
118+
};
119+
120+
environmentFile = mkOption {
121+
type = types.nullOr types.path;
122+
default = null;
123+
description = ''
124+
Path to a systemd EnvironmentFile holding secrets:
125+
LINEAR_API_KEY, GITHUB_TOKEN, LINEAR_WEBHOOK_SECRET,
126+
GITHUB_WEBHOOK_SECRET, SLACK_SIGNING_SECRET, SLACK_BOT_OAUTH_TOKEN,
127+
SYMPHONY_GITHUB_APP_PRIVATE_KEY_BASE64, SYMPHONY_ROOM_REGISTRY_TOKEN,
128+
etc.
129+
Wire this to whichever secret manager you use (sops-nix, agenix, ...).
130+
Leave null if you use secretsCommand instead.
131+
'';
132+
};
133+
134+
secretsCommand = mkOption {
135+
type = types.nullOr (types.listOf types.str);
136+
default = null;
137+
example = [
138+
"bws"
139+
"run"
140+
"--project-id"
141+
"symphony-prod"
142+
"--"
143+
];
144+
description = ''
145+
Optional command that wraps ExecStart and injects secrets into the
146+
environment. Designed for Bitwarden Secrets Manager (`bws run --
147+
...`) or any compatible secret-injecting CLI. The wrapper command
148+
must exec its trailing arguments. Place the bws binary on the
149+
service's PATH via `path = [ pkgs.bws ];` or by adding it to
150+
runtimeInputs of the symphony package.
151+
152+
When set, the unit also expects BWS_ACCESS_TOKEN (or equivalent)
153+
to be exported via environmentFile or extraEnvironment.
154+
'';
155+
};
156+
157+
path = mkOption {
158+
type = types.listOf types.package;
159+
default = [ ];
160+
description = "Extra packages on the service PATH (e.g. pkgs.bws when using secretsCommand).";
161+
};
162+
163+
hostRuntime = mkOption {
164+
default = { };
165+
description = ''
166+
The host codex placement. When enabled, a workflow node that
167+
declares `location: host` (or the run's resolved fallback) runs
168+
codex directly on this machine as a real OS user, with no VM. The
169+
per-run room-server and the codex process it spawns run as
170+
`user` inside that user's home directory, launched as transient
171+
`systemd-run --uid` units. This option wires the polkit grant,
172+
PATH, and environment that path needs. It stays inert until
173+
`enable` is set.
174+
'';
175+
type = types.submodule {
176+
options = {
177+
enable = mkEnableOption "the host codex placement";
178+
179+
user = mkOption {
180+
type = types.str;
181+
default = "";
182+
description = "OS user codex runs as for host placement (SYMPHONY_HOST_USER). Must already exist with a home directory.";
183+
};
184+
185+
group = mkOption {
186+
type = types.nullOr types.str;
187+
default = null;
188+
description = "OS group for host runs (SYMPHONY_HOST_GROUP); omitted uses the user's primary group.";
189+
};
190+
191+
workspacesDir = mkOption {
192+
type = types.nullOr types.path;
193+
default = null;
194+
description = "Parent directory for run checkouts (SYMPHONY_HOST_WORKSPACES_DIR); defaults to <user home>/symphony-workspaces.";
195+
};
196+
197+
roomServerPackage = mkOption {
198+
type = types.nullOr types.package;
199+
default = null;
200+
description = "Package providing the codex-wrapped room-server launched as the host user (this flake's room-server output). Used by the per-run host placement.";
201+
};
202+
203+
keep = mkOption {
204+
type = types.bool;
205+
default = false;
206+
description = "Leave the unit and checkout in place after the turn for inspection (SYMPHONY_HOST_KEEP).";
207+
};
208+
};
209+
};
210+
};
211+
};
212+
213+
config = mkIf cfg.enable {
214+
assertions = [
215+
{
216+
assertion = !cfg.hostRuntime.enable || cfg.hostRuntime.user != "";
217+
message = "services.symphony.hostRuntime.user must be set when hostRuntime.enable is true.";
218+
}
219+
{
220+
assertion = !cfg.hostRuntime.enable || cfg.hostRuntime.roomServerPackage != null;
221+
message = "services.symphony.hostRuntime.roomServerPackage must be set when hostRuntime.enable is true.";
222+
}
223+
];
224+
225+
# The host runtime calls systemd's StartTransientUnit over D-Bus to run
226+
# codex as another user. A non-root service needs polkit authorization
227+
# for that. Scope the grant to the "symphony-host-" unit-name prefix so
228+
# the service cannot manage unrelated system units. See systemd-run(1)
229+
# and the polkit systemd1 actions documented at
230+
# https://www.freedesktop.org/software/systemd/man/latest/org.freedesktop.systemd1.html
231+
security.polkit = lib.mkIf cfg.hostRuntime.enable {
232+
enable = true;
233+
extraConfig = ''
234+
polkit.addRule(function(action, subject) {
235+
if (subject.user == "${cfg.user}" &&
236+
action.id == "org.freedesktop.systemd1.manage-units") {
237+
var unit = action.lookup("unit");
238+
if (unit && unit.indexOf("symphony-host-") == 0) {
239+
return polkit.Result.YES;
240+
}
241+
}
242+
});
243+
'';
244+
};
245+
246+
users.users = lib.mkIf (cfg.user == "symphony") {
247+
symphony = {
248+
isSystemUser = true;
249+
group = "symphony";
250+
home = cfg.stateDir;
251+
};
252+
};
253+
254+
users.groups = lib.mkIf (cfg.user == "symphony") {
255+
symphony = { };
256+
};
257+
258+
systemd.tmpfiles.rules = [
259+
"d ${cfg.stateDir} 0750 ${cfg.user} ${cfg.user} -"
260+
"d ${cfg.stateDir}/workspaces 0750 ${cfg.user} ${cfg.user} -"
261+
"d ${cfg.stateDir}/runs 0750 ${cfg.user} ${cfg.user} -"
262+
"d ${cfg.stateDir}/log 0750 ${cfg.user} ${cfg.user} -"
263+
];
264+
265+
systemd.services.symphony = {
266+
description = "Symphony runtime";
267+
wantedBy = [ "multi-user.target" ];
268+
after = [ "network-online.target" ];
269+
wants = [ "network-online.target" ];
270+
271+
path =
272+
cfg.path
273+
++ lib.optionals cfg.hostRuntime.enable [
274+
pkgs.systemd
275+
pkgs.getent
276+
cfg.hostRuntime.roomServerPackage
277+
];
278+
279+
environment = {
280+
SYMPHONY_STATE_DIR = cfg.stateDir;
281+
SYMPHONY_WORKSPACES_DIR = "${cfg.stateDir}/workspaces";
282+
SYMPHONY_RUNS_DIR = "${cfg.stateDir}/runs";
283+
SYMPHONY_LOGS_ROOT = "${cfg.stateDir}/log";
284+
SYMPHONY_HTTP_PORT = toString cfg.httpPort;
285+
SYMPHONY_WORKFLOW_PACK = cfg.workflowPack;
286+
}
287+
// (lib.optionalAttrs (cfg.primaryRepo != null) {
288+
SYMPHONY_PRIMARY_REPO = toString cfg.primaryRepo;
289+
})
290+
// (lib.optionalAttrs (cfg.repoRoot != null) {
291+
SYMPHONY_REPO_ROOT = toString cfg.repoRoot;
292+
})
293+
// (lib.optionalAttrs (cfg.packDir != null) {
294+
SYMPHONY_PACK_DIR = toString cfg.packDir;
295+
})
296+
// (lib.optionalAttrs (cfg.roomRegistryUrl != null) {
297+
SYMPHONY_ROOM_REGISTRY_URL = cfg.roomRegistryUrl;
298+
})
299+
// (lib.optionalAttrs (cfg.roomAdvertiseHost != null) {
300+
SYMPHONY_ROOM_ADVERTISE_HOST = cfg.roomAdvertiseHost;
301+
})
302+
// (lib.optionalAttrs (cfg.roomServerUrl != null) {
303+
SYMPHONY_ROOM_SERVER_URL = cfg.roomServerUrl;
304+
})
305+
// (lib.optionalAttrs cfg.hostRuntime.enable (
306+
{
307+
SYMPHONY_HOST_USER = cfg.hostRuntime.user;
308+
SYMPHONY_HOST_ROOM_SERVER_COMMAND = lib.getExe cfg.hostRuntime.roomServerPackage;
309+
}
310+
// (lib.optionalAttrs (cfg.hostRuntime.group != null) {
311+
SYMPHONY_HOST_GROUP = cfg.hostRuntime.group;
312+
})
313+
// (lib.optionalAttrs (cfg.hostRuntime.workspacesDir != null) {
314+
SYMPHONY_HOST_WORKSPACES_DIR = toString cfg.hostRuntime.workspacesDir;
315+
})
316+
// (lib.optionalAttrs cfg.hostRuntime.keep {
317+
SYMPHONY_HOST_KEEP = "true";
318+
})
319+
))
320+
// cfg.extraEnvironment;
321+
322+
serviceConfig = {
323+
Type = "simple";
324+
User = cfg.user;
325+
Group = cfg.user;
326+
ExecStart =
327+
let
328+
symphonyBin = "${cfg.package}/bin/symphony";
329+
wrapper = optionalString (cfg.secretsCommand != null) (
330+
lib.escapeShellArgs cfg.secretsCommand + " "
331+
);
332+
in
333+
"${wrapper}${symphonyBin}";
334+
Restart = "on-failure";
335+
RestartSec = "10s";
336+
StateDirectory = lib.mkIf (lib.hasPrefix "/var/lib/" cfg.stateDir) (
337+
lib.removePrefix "/var/lib/" cfg.stateDir
338+
);
339+
# Symphony spawns codex subprocesses and clones git repos, so
340+
# most sandboxing options need to stay permissive. Only enable
341+
# the cheap, safe ones.
342+
NoNewPrivileges = true;
343+
PrivateTmp = true;
344+
ProtectKernelTunables = true;
345+
ProtectKernelModules = true;
346+
ProtectControlGroups = true;
347+
}
348+
// (lib.optionalAttrs (cfg.environmentFile != null) {
349+
EnvironmentFile = cfg.environmentFile;
350+
});
351+
};
352+
};
353+
}

0 commit comments

Comments
 (0)