Skip to content

Commit 31c3b97

Browse files
committed
Discover running TensorFold engines for the device heartbeat
TensorFold Studio starts its own `tensorfold-native serve` on a new random loopback port every launch, so no registry entry can name it and IAK could not see it. The TensorFold status reader now also scans this user's processes for `tensorfold-native serve` / `tensorfold serve` (one `ps -U <uid> -o pid=,args=` on macOS, /proc on Linux with `comm` checked first, no sudo), takes the port from --port or, when there is none, from the listening socket via lsof, and the model and name from the command line. Engines are merged with the registry by port (the entry keeps its id and keyFile; an entry without an engine skips auto-detection) and published as source "discovered". GET-only rules unchanged. Keys come from files only: a registry keyFile, or the absolute --api-key-file a discovered engine was started with, read in the engine's own key-file format and refused when other users can read it. Process environments and --api-key values on a command line are never read or kept. TensorFold Studio makes a fresh key per launch, keeps it in memory and passes it as TENSORFOLD_API_KEY (app/electron/agent/engine/ supervisor.ts), so its engine reports metrics "auth_required", with its model taken from --name and labelled model_source "args". On by default when INTENT_TENSORFOLD_REGISTRY is set; INTENT_TENSORFOLD_DISCOVER=1 runs it without a registry, 0 (or `tensorfold-status --no-discover`) turns it off. New payload fields: name, app, model_source, memory.footprint_gb, top-level discovery (ok / off / unavailable). Tests: 15 new in test/tensorfold-discovery.test.mjs with ps, lsof and GET fixtures recorded from the three engines running on this Mac (Python 0.6.0, native 1.0.10, Studio), decoys, a fake /proc tree and a keyed engine.
1 parent 917efb0 commit 31c3b97

20 files changed

Lines changed: 1241 additions & 99 deletions

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
## Unreleased
44

55
### Added
6+
- **TensorFold engines are discovered from the process table**: the TensorFold status reader (heartbeat and `ide-agent-kit tensorfold-status`) now also finds this user's running `tensorfold-native serve` / `tensorfold serve` engines (one `ps` on macOS, `/proc` on Linux, no sudo; port from `--port`, else the listening socket via `lsof`), merges them with the registry by port and publishes them as `source: "discovered"`, so TensorFold Studio's engine on its random port gets a card. On by default with `INTENT_TENSORFOLD_REGISTRY`; `INTENT_TENSORFOLD_DISCOVER=1` runs it without a registry, `=0` or `--no-discover` turns it off. A discovered engine's `--api-key-file` is read in the engine's own format; process environments and command-line keys are never read, so the Studio engine (key in memory only) reports `metrics: "auth_required"` with its model from the command line. New fields: `name`, `app`, `model_source`, `memory.footprint_gb`, and a top-level `discovery` (`ok` / `off` / `unavailable`). Key files for TensorFold status reads, including a registry `keyFile`, are now refused when other users can read them, as the engine refuses them.
67
- **TensorFold status on the device heartbeat**: `uik-daemon` and `ide-agent-kit intent daemon` can publish the live status of each TensorFold server in the model registry under the device's `tensorfold` key, so CodeWatch can show a card per machine. Opt in with `INTENT_TENSORFOLD_REGISTRY=<path to models.json>`; `INTENT_TENSORFOLD_SCOPE=local` (default, loopback entries only) or `all`. Read with GET `/health`, `/metrics` and `/v1/slide/graph` only, never a completion. Publishes model, warming, version/backend/device when the server reports them, memory, running and waiting requests, token counters, tok/s from counter deltas between reads, draft acceptance, TTFT p50, KV cache use, Living Weights fact counts (never fact text) and the metrics auth state. An unreachable server is `reachable: false`; an unreadable registry is an `error`, never an empty list. New `ide-agent-kit tensorfold-status [--json]` prints the same object. The registry gains `engine` (`"tensorfold"` marks an entry, any other name skips it, absent means auto-detect). A key is sent only to an entry marked TensorFold that names its own `keyFile`, and only on the two gated routes; the fleet-wide `LLM_API_KEY` fallbacks are never used for a status read.
78
- **What is still waiting on you** (`bin/iak-pending.mjs`): asks every daemon the fleet knows about - this machine's plus every `watchdog-roster.json` entry with a `gate` - for its pending intents and prints them oldest first with the age in the leading column. Motivation: confirmation cards post to the room correctly and then scroll away; on 2026-09-20 four approval requests sat pending on one machine for up to 9 hours 12 minutes and the owner asked twice in one night what was waiting on him, because no surface anywhere could answer it. A host that could not be reached is reported under its own `COULD NOT ASK` heading and is never folded into "nothing pending"; a gate naming another machine by LAN IP is refused rather than probed and reported the same way. A daemon whose rows carry no `status` this version understands is a shape mismatch rather than an empty queue, and a fleet roster that exists but cannot be parsed or read exits `2` instead of silently shrinking the fleet to this machine (only a *missing* roster means "single-machine install"). An age that cannot be believed - missing, future-dated, or epoch-seconds - renders as `age unknown`, sorts to the top and survives `--older-than`, rather than being clamped to "fresh". Exit codes: `0` all clear, `10` pending, `11` some hosts unreachable, `12` no host reachable, `2` cannot run, precedence `12 > 11 > 10 > 0`; `--older-than` narrows what is printed and announced but can never produce a `0`. `--json` for machines. `--room` requires `--older-than` and announces each item at most once per escalation (threshold, then each doubling of its age), keeping a ledger at `~/.iak/pending-announced.json` that is written atomically before the post, carries forward entries for hosts it could not ask, rolls back when a post fails, and degrades to silence - never a storm - when it is missing, corrupt or unwritable. `examples/pending-launchd.plist` wires it to a timer. The gate token is resolved by the existing `resolveGateToken` helper, is sent only to hosts that helper already trusts, and never appears in output or argv.
89
- **Session degradation watch** (`bin/iak-degradation-watch.mjs`): a per-machine daemon that tails watched agents' Claude Code session transcripts and posts a room alert (pushed to the owner's phone via CodeWatch) the moment a session's `message.model` leaves the configured accepted set or its `permissionMode` differs from the expected one. Motivation: on 2026-07-18 an agent's session silently slipped to a small fallback model with manual permission prompts while the owner was away; the degraded agent kept posting — looping on stale tasks and reposting completed instructions — and cost an evening to diagnose. Alerts fire immediately on first detection (a wrong model in a transcript is a fact, not a flap) with a 30-minute per-signal cooldown and a recovery note. Config: `degradation_watch` block (`interval_sec`, `alert_room`, `agents[]` with `transcript_dir`, `accepted_models` prefix list, `expected_permission_mode`); API key reuses `poller.api_key`. `--once` and `--dry-run` flags for testing. Run one instance per machine under process supervision.

‎README.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -496,7 +496,9 @@ can show a card for it on each machine.
496496

497497
1. List the server in `config/models.json` with `"engine": "tensorfold"` (or
498498
leave `engine` out and let it be auto-detected). See
499-
`config/models.README.md`.
499+
`config/models.README.md`. Engines running on this machine, including the
500+
one TensorFold Studio starts on a random port, are found from the process
501+
table without an entry.
500502
2. Check what the card will say: `ide-agent-kit tensorfold-status` (add
501503
`--json` for the exact object).
502504
3. Turn it on for the publisher: set
@@ -505,7 +507,9 @@ can show a card for it on each machine.
505507

506508
It is read with GET requests only (`/health`, `/metrics`, `/v1/slide/graph`),
507509
never a completion, and an unreachable server shows as unreachable instead of
508-
stopping the heartbeat. The payload shape is in
510+
stopping the heartbeat. Keys are read only from files (`keyFile`, or an
511+
engine's `--api-key-file`); TensorFold Studio keeps its engine key in memory,
512+
so its card says `metrics: auth required`. The payload shape is in
509513
`packages/user-intent-kit/README.md`.
510514

511515
UIK v0.2.2 or later recommended for the `uik-daemon` bin. v0.2.0 introduced the daemon, v0.2.1 fixed a silent-exit in the example file, and v0.2.2 added agent-status republish on the heartbeat cadence so the agent slot does not expire while the daemon is running. Pre-0.2.2 daemons publish device state correctly but the agent slot still goes stale after its TTL.
@@ -1207,7 +1211,7 @@ ide-agent-kit gateway <health|agents|trigger|wake> [options]
12071211
ide-agent-kit memory <list|get|set|append|delete|search> [options]
12081212
ide-agent-kit rooms archive [--backfill] [--max-pages N] [--room R]
12091213
ide-agent-kit rooms search "words" [--from handle] [--since 2026-09-20] [--regex] [--limit N]
1210-
ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--timeout <ms>]
1214+
ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--no-discover] [--timeout <ms>]
12111215
ide-agent-kit init [--ide <claude-code|codex|cursor|vscode|gemini>] [--profile <balanced|low-friction>]
12121216
ide-agent-kit acp <spawn|list|status|send|close> [options]
12131217
ide-agent-kit keepalive <start|stop|status> [--pid-file <path>] [--heartbeat-sec <sec>]

‎bin/cli.mjs‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,10 @@ Usage:
102102
daemon: long-running publisher (desktop state + agent heartbeat) [--interval <sec>]
103103
Add INTENT_TENSORFOLD_REGISTRY=<models.json> to publish TensorFold status on each beat.
104104
105-
ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--timeout <ms>] [--config <path>]
106-
Live status of every TensorFold server in the model registry, read with GET only
107-
(/health, /metrics, /v1/slide/graph). Same object the device heartbeat publishes.
105+
ide-agent-kit tensorfold-status [--json] [--registry <path>] [--local] [--no-discover] [--timeout <ms>] [--config <path>]
106+
Live status of every TensorFold server in the model registry and every TensorFold engine
107+
running on this machine, read with GET only (/health, /metrics, /v1/slide/graph).
108+
Same object the device heartbeat publishes.
108109
109110
ide-agent-kit poll --rooms <room1,room2> --api-key <key> --handle <@handle> [--interval <sec>] [--config <path>]
110111
(Legacy) Poll GroupMind rooms with explicit CLI args. Prefer "rooms watch".
@@ -578,7 +579,12 @@ async function main() {
578579
console.error('Error: --timeout must be a positive number of milliseconds');
579580
process.exit(2);
580581
}
581-
const result = await runTensorFoldStatus({ registryPath, scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs });
582+
const envRegistry = String(process.env.INTENT_TENSORFOLD_REGISTRY || '').trim();
583+
const registryRequired = typeof opts.registry === 'string' || (envRegistry !== '' && !['0', 'off', 'false', 'no'].includes(envRegistry.toLowerCase()));
584+
const result = await runTensorFoldStatus({
585+
registryPath, registryRequired, discover: !opts['no-discover'],
586+
scope: opts.local ? 'local' : 'all', json: !!opts.json, timeoutMs,
587+
});
582588
(result.exitCode === 0 ? console.log : console.error)(result.text);
583589
process.exit(result.exitCode);
584590
}

‎config/models.README.md‎

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -82,13 +82,27 @@ Which entries count as TensorFold:
8282
both, so mark a keyed server `"engine": "tensorfold"` explicitly.
8383
* any other `engine`: never asked.
8484

85+
Engines running on this machine do not need an entry at all: the status
86+
reader also finds this user's `tensorfold-native serve` / `tensorfold serve`
87+
processes and reads them the same way (`source: "discovered"`). An entry on
88+
the same loopback port wins: its `id` and `keyFile` are used, and an entry
89+
with no `engine` needs no auto-detection. An entry with another `engine` on
90+
that port stays skipped. Turn discovery off with `INTENT_TENSORFOLD_DISCOVER=0`
91+
or `tensorfold-status --no-discover`.
92+
8593
**Keys.** A TensorFold server may gate `/metrics` and `/v1/slide/graph`
8694
behind a key (`/health` stays open, and says only `{"status": "ok"}` while
87-
keys are on). The status reader sends a key only to an entry marked
88-
`"engine": "tensorfold"` that names its own `keyFile`, and only on those two
89-
routes. It never uses `LLM_API_KEY` / `LLM_API_KEY_FILE`: those belong to some
90-
other server as far as a status read knows. Without a usable key it reports
91-
`metrics: "auth_required"` and the rest of the card reads unknown.
95+
keys are on). The status reader sends a key only on those two routes, and
96+
only from a file named for that server: the `keyFile` of an entry marked
97+
`"engine": "tensorfold"`, or the absolute `--api-key-file` a discovered
98+
engine was started with. The file is read the way the engine reads it (one
99+
key per line, `label: key` or a bare key, `#` comments) and refused, as the
100+
engine refuses it, when other users can read it (`chmod 600`). It never uses
101+
`LLM_API_KEY` / `LLM_API_KEY_FILE`, a key on a command line, or a process's
102+
environment. Without a usable key it reports `metrics: "auth_required"` and
103+
the rest of the card reads unknown. TensorFold Studio's engine is always in
104+
that state: the app makes a fresh key on every launch, keeps it in memory and
105+
passes it to the engine in the environment, never in a file.
92106

93107
```json
94108
{

‎packages/user-intent-kit/README.md‎

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -132,11 +132,21 @@ device card:
132132
```bash
133133
export INTENT_TENSORFOLD_REGISTRY=~/ide-agent-kit/config/models.json
134134
export INTENT_TENSORFOLD_SCOPE=local # default; `all` also reads remote entries
135+
export INTENT_TENSORFOLD_DISCOVER=1 # default with a registry; 0 turns it off; 1 alone runs without one
135136
```
136137

137138
- **GET only.** `/health`, `/metrics` and `/v1/slide/graph`, never a
138139
completion, so a status read cannot interfere with a Living Weights lesson.
139-
- **Off unless set.** Nothing is read or published without the registry path.
140+
- **Off unless set.** Nothing is read or published without the registry path
141+
or `INTENT_TENSORFOLD_DISCOVER=1`.
142+
- **Running engines are found too.** Each read also scans this user's
143+
processes for `tensorfold-native serve` and `tensorfold serve` (one `ps` on
144+
macOS, `/proc` on Linux, no sudo), takes the port from `--port` or, when
145+
there is none, from the listening socket (`lsof`), and the model and name
146+
from the command line. They are merged with the registry by port and
147+
published as `source: "discovered"`. This is how TensorFold Studio's engine,
148+
which gets a new random port every launch, gets a card. Process
149+
environments are never read.
140150
- **Not on the heartbeat's path.** It reads on its own timer at the heartbeat
141151
interval; the beat publishes the last reading and never waits on a socket.
142152
- **One key, replaced whole.** Everything goes under `tensorfold`, so a server
@@ -146,6 +156,12 @@ export INTENT_TENSORFOLD_SCOPE=local # default; `all` also reads remote entrie
146156
- **Which entries:** marked `"engine": "tensorfold"`, or auto-detected from
147157
their `/metrics`. See `config/models.README.md` in ide-agent-kit, including
148158
how a keyed server names its key.
159+
- **Keys come from files only.** A registry entry's `keyFile`, or the
160+
`--api-key-file` a discovered engine was started with, read in the engine's
161+
key-file format and refused when other users can read the file. TensorFold
162+
Studio keeps its engine key in memory and hands it to the engine through
163+
the environment, so its engine shows `metrics: "auth_required"`, the model
164+
from its command line, and the reason.
149165
- **TensorFold's default port is 8080, and so is `INTENT_MODEL_ENDPOINT`'s.**
150166
The served-model probe above may send one short generation request to a
151167
loopback endpoint to prove what is loaded. On a machine where TensorFold
@@ -160,12 +176,13 @@ The payload, per device:
160176
"checked_at": "2026-10-14T09:00:00.000Z",
161177
"scope": "local",
162178
"error": null,
179+
"discovery": "ok",
163180
"servers": [{
164181
"id": "mb-tensorfold", "host": "localhost", "port": 8091, "local": true,
165-
"source": "registry", "reachable": true, "reason": null,
166-
"model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "warming": false,
182+
"source": "registry", "name": null, "app": null, "reachable": true, "reason": null,
183+
"model": "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit", "model_source": "health", "warming": false,
167184
"version": null, "backend": null, "device": null,
168-
"memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2 },
185+
"memory": { "used_gb": 18.7, "peak_gb": 40.7, "budget_gb": 96.2, "footprint_gb": 19.2 },
169186
"requests": { "running": 0, "waiting": 0 },
170187
"tokens": { "prompt_total": 0, "generation_total": 0 },
171188
"tok_s": null, "tok_s_window_s": null, "decode_tok_s": null,
@@ -184,10 +201,19 @@ and "could not tell" never look the same.
184201

185202
What the numbers mean:
186203

187-
- `local`: the server runs on this machine (loopback entry). `source` is
188-
`registry` (marked `"engine": "tensorfold"`) or `detected`.
204+
- `local`: the server runs on this machine (a loopback entry or a discovered
205+
engine). `source` is `registry` (marked `"engine": "tensorfold"`),
206+
`detected` (auto-detected from its `/metrics` or `/health`) or `discovered`
207+
(found running in the process table). `name` is the engine's `--name`;
208+
`app` is `"TensorFold Studio"` for the engine the app bundles, else null.
209+
- `model_source`: `health` when `/health` named the model, `args` when it came
210+
from the engine's command line (a keyed `/health` says only "ok").
211+
- `discovery` (top level): `ok`, `off`, or `unavailable` when the process scan
212+
failed. An `unavailable` scan is not the same as "no engines running".
189213
- `memory`: GB, base 10, one decimal, like `mem_total_gb`. `used` is the
190-
engine's active memory, `budget` its configured budget.
214+
engine's active memory, `budget` its configured budget, `footprint` the
215+
process's physical footprint (macOS, including Metal buffers). The native
216+
MLX build reports only the footprint.
191217
- `tokens`: counters since the server started, for finished requests.
192218
- `tok_s`: generated tokens of requests that finished between two reads,
193219
divided by the seconds between them (`tok_s_window_s`). `null` on the first

‎packages/user-intent-kit/bin/uik-daemon.js‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,14 @@
7676
* Never a generation request. An unreachable server
7777
* publishes reachable:false; the heartbeat never
7878
* waits on it.
79+
* INTENT_TENSORFOLD_DISCOVER 1 | 0. Find TensorFold engines running on this
80+
* machine from the process table (this user's
81+
* `tensorfold-native serve` / `tensorfold serve`;
82+
* port from --port, else the listening socket) and
83+
* publish them as source "discovered", merged with
84+
* the registry by port. Default: on when
85+
* INTENT_TENSORFOLD_REGISTRY is set; set 1 to run it
86+
* with no registry at all, 0 to turn it off.
7987
* INTENT_TENSORFOLD_SCOPE local (default) | all. local = only entries on
8088
* loopback, i.e. servers on THIS machine, so each
8189
* machine's card carries its own servers. all = every

‎packages/user-intent-kit/src/index.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ export {
1717
STATUS_SCHEMA as TENSORFOLD_STATUS_SCHEMA,
1818
METRICS_STATES as TENSORFOLD_METRICS_STATES,
1919
} from './tensorfold-status.js';
20+
export { discoverTensorFoldEngines } from './tensorfold-discovery.js';
2021
export {
2122
ModelAvailabilityProbe,
2223
describeLocalModels,

0 commit comments

Comments
 (0)