You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 917efb0
Browse filesBrowse the repository at this point in the historyBrowse files
Device heartbeat reads TensorFold's /health and /metrics (#143)
IAK now reads the live status of each TensorFold server in the model
registry and puts it on the device card under one `tensorfold` key, so
CodeWatch can show a card per machine. Opt in with
INTENT_TENSORFOLD_REGISTRY=<path to models.json> on uik-daemon or
`ide-agent-kit intent daemon`; INTENT_TENSORFOLD_SCOPE=local (default)
or all.
Read with GET /health, GET /metrics and GET /v1/slide/graph only, never
a completion. Runs on its own timer at the heartbeat interval and the
beat reads the last snapshot, so a dead server costs a stale card and
never a late heartbeat. Unreachable is reachable:false with the reason;
an unreadable registry is an error, never an empty list. tok/s comes
from generation_tokens_total deltas between reads. Living Weights
publishes fact counts only, never fact text. version, backend and device
are read from /health when present and null otherwise.
Registry: `engine` ("tensorfold" marks an entry, any other name skips
it, absent means auto-detect from /metrics or the /health shape). A key
is sent only to an entry marked TensorFold that names its own keyFile,
only on the two gated routes; the fleet-wide LLM_API_KEY fallbacks are
not used for a status read (resolveEntryToken gains entryOnly).
New `ide-agent-kit tensorfold-status [--json]` prints the same object.
Requests carry Connection: close: a CLI run that exited with a kept-alive
socket left a ConnectionResetError traceback in the 0.6.0 server log.
Tests: 25 new, fixtures recorded from the live 0.6.0 server on 8091,
plus 401 /metrics, 404 slide graph, unreachable, hung socket, redirect
and heartbeat payload cases.
Co-authored-by: ThinkOffApp <233579950+ThinkOffApp@users.noreply.github.com>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,7 @@
3
3
## Unreleased
4
4
5
5
### Added
6
+
- **TensorFold status on the device heartbeat**: `uik-daemon` and `ide-agent-kit intent daemon` can publish the live status of each TensorFold server in the model registry under the device's `tensorfold` key, so CodeWatch can show a card per machine. Opt in with `INTENT_TENSORFOLD_REGISTRY=<path to models.json>`; `INTENT_TENSORFOLD_SCOPE=local` (default, loopback entries only) or `all`. Read with GET `/health`, `/metrics` and `/v1/slide/graph` only, never a completion. Publishes model, warming, version/backend/device when the server reports them, memory, running and waiting requests, token counters, tok/s from counter deltas between reads, draft acceptance, TTFT p50, KV cache use, Living Weights fact counts (never fact text) and the metrics auth state. An unreachable server is `reachable: false`; an unreadable registry is an `error`, never an empty list. New `ide-agent-kit tensorfold-status [--json]` prints the same object. The registry gains `engine` (`"tensorfold"` marks an entry, any other name skips it, absent means auto-detect). A key is sent only to an entry marked TensorFold that names its own `keyFile`, and only on the two gated routes; the fleet-wide `LLM_API_KEY` fallbacks are never used for a status read.
6
7
- **What is still waiting on you** (`bin/iak-pending.mjs`): asks every daemon the fleet knows about - this machine's plus every `watchdog-roster.json` entry with a `gate` - for its pending intents and prints them oldest first with the age in the leading column. Motivation: confirmation cards post to the room correctly and then scroll away; on 2026-09-20 four approval requests sat pending on one machine for up to 9 hours 12 minutes and the owner asked twice in one night what was waiting on him, because no surface anywhere could answer it. A host that could not be reached is reported under its own `COULD NOT ASK` heading and is never folded into "nothing pending"; a gate naming another machine by LAN IP is refused rather than probed and reported the same way. A daemon whose rows carry no `status` this version understands is a shape mismatch rather than an empty queue, and a fleet roster that exists but cannot be parsed or read exits `2` instead of silently shrinking the fleet to this machine (only a *missing* roster means "single-machine install"). An age that cannot be believed - missing, future-dated, or epoch-seconds - renders as `age unknown`, sorts to the top and survives `--older-than`, rather than being clamped to "fresh". Exit codes: `0` all clear, `10` pending, `11` some hosts unreachable, `12` no host reachable, `2` cannot run, precedence `12 > 11 > 10 > 0`; `--older-than` narrows what is printed and announced but can never produce a `0`. `--json` for machines. `--room` requires `--older-than` and announces each item at most once per escalation (threshold, then each doubling of its age), keeping a ledger at `~/.iak/pending-announced.json` that is written atomically before the post, carries forward entries for hosts it could not ask, rolls back when a post fails, and degrades to silence - never a storm - when it is missing, corrupt or unwritable. `examples/pending-launchd.plist` wires it to a timer. The gate token is resolved by the existing `resolveGateToken` helper, is sent only to hosts that helper already trusts, and never appears in output or argv.
7
8
- **Session degradation watch** (`bin/iak-degradation-watch.mjs`): a per-machine daemon that tails watched agents' Claude Code session transcripts and posts a room alert (pushed to the owner's phone via CodeWatch) the moment a session's `message.model` leaves the configured accepted set or its `permissionMode` differs from the expected one. Motivation: on 2026-07-18 an agent's session silently slipped to a small fallback model with manual permission prompts while the owner was away; the degraded agent kept posting — looping on stale tasks and reposting completed instructions — and cost an evening to diagnose. Alerts fire immediately on first detection (a wrong model in a transcript is a fact, not a flap) with a 30-minute per-signal cooldown and a recovery note. Config: `degradation_watch` block (`interval_sec`, `alert_room`, `agents[]` with `transcript_dir`, `accepted_models` prefix list, `expected_permission_mode`); API key reuses `poller.api_key`. `--once` and `--dry-run` flags for testing. Run one instance per machine under process supervision.
8
9
-**Send-to-session**: the :8788 daemon gains `POST /sessions/send {agent, text, from?}` — deliver text as user input into any named agent's live session — plus `GET /sessions/agents` for the target picker. Four delivery adapters, configured per agent under `mcp.confirmations.sessions.agents`: `wake` (spawn a wake/injection script), `gui` (same, chained behind a human-idle guard with `--wait` so it never types over the user), `tmux` (literal `send-keys` + Enter), and `forward` (relay to the owning machine's daemon, provenance passed through so the `[from …]` prefix is applied exactly once). Deliveries are accepted-async (202) and every attempt is receipted. This is the backend half of the CodeWatch send box; it drives any agent uniformly, including headless sessions the provider apps cannot reach.
The IDE Agent Kit and User Intent Kit are designed to be deployed together: IAK consumes the intent state for gating, UIK publishes your own heartbeats. Without the daemon your slot goes stale and other agents will treat you as offline.
488
488
489
+
#### TensorFold status on the device card
490
+
491
+
IAK, TensorFold, CodeWatch and GroupMind are meant to work as one system. If
492
+
you run a [TensorFold](https://github.com/ThinkOffApp/TensorFold) server, the
493
+
heartbeat can carry its live status (model, memory, requests, tok/s, draft
494
+
acceptance, time to first token, KV cache use, Living Weights) so CodeWatch
495
+
can show a card for it on each machine.
496
+
497
+
1. List the server in `config/models.json` with `"engine": "tensorfold"` (or
498
+
leave `engine` out and let it be auto-detected). See
499
+
`config/models.README.md`.
500
+
2. Check what the card will say: `ide-agent-kit tensorfold-status` (add
501
+
`--json` for the exact object).
502
+
3. Turn it on for the publisher: set
503
+
`INTENT_TENSORFOLD_REGISTRY=/path/to/config/models.json` in the
504
+
environment of `uik-daemon` or `ide-agent-kit intent daemon`, and restart it.
505
+
506
+
It is read with GET requests only (`/health`, `/metrics`, `/v1/slide/graph`),
507
+
never a completion, and an unreachable server shows as unreachable instead of
508
+
stopping the heartbeat. The payload shape is in
509
+
`packages/user-intent-kit/README.md`.
510
+
489
511
UIK v0.2.2 or later recommended for the `uik-daemon` bin. v0.2.0 introduced the daemon, v0.2.1 fixed a silent-exit in the example file, and v0.2.2 added agent-status republish on the heartbeat cadence so the agent slot does not expire while the daemon is running. Pre-0.2.2 daemons publish device state correctly but the agent slot still goes stale after its TTL.
@@ -18,6 +18,7 @@ A bare top-level array is also accepted.
18
18
|`host`| yes |**Tailnet MagicDNS name** (`asus1`, `mini`) or tailnet IP (`100.x.y.z`). See the rule below. |
19
19
|`port`| yes | Integer 1-65535. The OpenAI-style HTTP port, not the ssh port. |
20
20
|`kind`| yes |`openai`\|`vllm`\|`lmstudio`. Decides which list endpoint is called: `lmstudio` uses `/api/v1/models` and counts only loaded instances, the other two use `/v1/models`. |
21
+
|`engine`| no | Which server software answers on this port. `tensorfold` turns on the TensorFold status read-out for this entry (see below). Any other short name (`vllm`, `llama.cpp`, `ollama`) says "not TensorFold" and skips auto-detection. Omit it to let the status reader auto-detect. |
21
22
|`box`| no | Free text for humans: which machine, where it lives. Shown in `--json`, never probed. |
22
23
|`sharing`| no |`exclusive`\|`shared`, default `shared`. Advisory metadata for the switcher's ranking. The probe applies the same BUSY rules to both - it is documented in `docs/model-capacity.md` rather than silently inferred. |
23
24
|`owner`| no | Handle of whoever to ask before taking the box. |
@@ -64,6 +65,42 @@ two hosts behind the same router were in opposite path states to the same
64
65
Berlin box at the same moment. A value written into this shared file is wrong
65
66
for every caller except the one who wrote it.
66
67
68
+
## TensorFold entries
69
+
70
+
`ide-agent-kit tensorfold-status` and the device heartbeat (when
71
+
`INTENT_TENSORFOLD_REGISTRY` points at this file) read the live status of
72
+
every TensorFold server listed here: model, warming, memory, requests, token
73
+
counters, tok/s, draft acceptance, time to first token, KV cache use and
74
+
Living Weights. They send **GET /health, GET /metrics and GET /v1/slide/graph
75
+
only**, never a completion.
76
+
77
+
Which entries count as TensorFold:
78
+
79
+
*`"engine": "tensorfold"`: always, reachable or not.
80
+
* no `engine`: auto-detected when `/metrics` carries `tensorfold:` series or
81
+
`/health` has TensorFold's shape. A server started with an API key hides
82
+
both, so mark a keyed server `"engine": "tensorfold"` explicitly.
83
+
* any other `engine`: never asked.
84
+
85
+
**Keys.** A TensorFold server may gate `/metrics` and `/v1/slide/graph`
86
+
behind a key (`/health` stays open, and says only `{"status": "ok"}` while
87
+
keys are on). The status reader sends a key only to an entry marked
88
+
`"engine": "tensorfold"` that names its own `keyFile`, and only on those two
89
+
routes. It never uses `LLM_API_KEY` / `LLM_API_KEY_FILE`: those belong to some
90
+
other server as far as a status read knows. Without a usable key it reports
91
+
`metrics: "auth_required"` and the rest of the card reads unknown.
92
+
93
+
```json
94
+
{
95
+
"id": "gb10-tensorfold",
96
+
"host": "asus3",
97
+
"port": 8080,
98
+
"kind": "openai",
99
+
"engine": "tensorfold",
100
+
"keyFile": "~/.iak/tensorfold_gb10_api_key.txt"
101
+
}
102
+
```
103
+
67
104
## The rule: tailnet names, never LAN IPs
68
105
69
106
`host` must not be a private LAN address. The loader rejects `10.x.x.x`,
A value that could not be read is `null`, never `0`. `error` is set (and
182
+
`servers` empty) when the registry itself could not be read, so "no servers"
183
+
and "could not tell" never look the same.
184
+
185
+
What the numbers mean:
186
+
187
+
-`local`: the server runs on this machine (loopback entry). `source` is
188
+
`registry` (marked `"engine": "tensorfold"`) or `detected`.
189
+
-`memory`: GB, base 10, one decimal, like `mem_total_gb`. `used` is the
190
+
engine's active memory, `budget` its configured budget.
191
+
-`tokens`: counters since the server started, for finished requests.
192
+
-`tok_s`: generated tokens of requests that finished between two reads,
193
+
divided by the seconds between them (`tok_s_window_s`). `null` on the first
194
+
read and after a server restart. `decode_tok_s` is the server's own live
195
+
decode rate, where the server reports one (the native build does).
196
+
-`draft.acceptance`: accepted over drafted MTP tokens since start, `null`
197
+
until something was drafted.
198
+
-`ttft_p50_s`: median time to first token since start, estimated from the
199
+
histogram buckets.
200
+
-`kv_cache_usage`: the fullest stream's cache use, 0 to 1.
201
+
-`living_weights.available`: `/v1/slide/graph` answered (`false` on a 404,
202
+
`null` when it could not be read). Counts only: the fact text stays on the
203
+
machine. The native build answers the route even without `--slide`, so
204
+
there `facts` is the better signal.
205
+
-`metrics`: `ok`, `auth_required`, `auth_rejected` or `unavailable`.
206
+
-`version`, `backend`, `device`: read from `/health` when the server reports
207
+
them (newer builds), `null` otherwise.
208
+
126
209
> Note: The older `examples/iak-integration.js` is a one-shot demo and will exit once its setInterval timer is unref'd. Use `uik-daemon` for long-running deployments.
0 commit comments