feat(llamacpp): Hecate-managed local models — download + run via llama.cpp by chicoxyzzy · Pull Request #101 · hecatehq/hecate

chicoxyzzy · 2026-05-14T23:13:52Z

Summary

Adds first-party support for downloading and running local models on the bundled llama-server binary, similar to Ollama / LM Studio. The feature is engine-agnostic at the URL level (/hecate/v1/local-models/*) so future MLX or other runtimes share the namespace.

Lands behind a feature gate — when HECATE_LLAMA_SERVER_BIN is unset (or HECATE_LOCAL_MODELS not opted in), the surface stays dormant: handlers return a useful 503 with local_models_unavailable, no provider is auto-registered, no child processes spawn.

v1 — backend foundation, UI surface, Tauri sidecar bundling

Backend (internal/llamacpp/): typed Installer (sha-verified, atomic rename, cancellable), Runtime (single-child supervisor with crash listener), Proxy (gateway-internal reverse proxy with FlushInterval=-1 for streaming), Service facade composing them plus a curated Catalog.
HTTP API: GET /catalog, GET /installed, POST /install + SSE events, DELETE install/installed, GET /runtime, POST runtime/start|stop, plus the internal /hecate/internal/llamacpp/v1/{path...} proxy mount.
/v1/models integration: installed local models join the chat composer's model dropdown alongside provider-routed models.
Auto-registered provider: on boot, a managed llamacpp provider row is created pointing at the internal proxy. Operator overrides (matched by PresetID="llamacpp") are left alone — surfaces as ErrAutoProviderOperatorOwned for log-warning visibility.
UI (ui/src/features/providers/): LocalModelsCard summary in the Connections panel, LocalModelsSlideOver with Runtime / Installed / Catalog / Custom URL sections. SSE-driven install progress, single confirm-modal-gated Uninstall, per-row Start with hidden Start on the loaded row.
Tauri: llama-server joins claude-code and codex as a sidecar; resolved binary path is environment-wired into the gateway at startup.

v2 — lazy-download, LRU keep-warm, gated repos, HuggingFace browse

Headless lazy-download — HECATE_LOCAL_MODELS_LAZY_DOWNLOAD=on triggers a one-time download of llama-server from the pinned upstream llama.cpp release at boot. Atomic rename + chmod, sha-verified when the pin carries a digest, cached at <data_dir>/llamacpp/bin/. Tauri builds leave it OFF — the bundled binary is the source of truth there.
LRU keep-warm — HECATE_LOCAL_MODELS_MAX_RESIDENT=N keeps the N most-recently-used models loaded across requests; the (N+1)th EnsureLoaded evicts the coldest session. Default 1 preserves v1 single-child behavior. All sessions race-clean.
Gated HuggingFace repos — per-install token via InstallSpec.hf_token, env fallback via HUGGINGFACE_TOKEN. The installer attaches Authorization: Bearer <token> only when present. Token is not persisted at rest; v2 deliberately scopes to single-use.
HuggingFace browse + search — server-side proxy at GET /local-models/huggingface/search and .../repos/{owner}/{name}. Search pins filter=gguf, sort by downloads. File tree returns only .gguf files with LFS sha256 + size + canonical resolve URL. Browser never sees the HF token. UI surfaces a "Browse HuggingFace" section between Catalog and Custom URL with search → result rows (gated badge, download count) → per-repo file expander → Install drops into the existing install flow with sha256 attached.

Telemetry, observability, error mapping

New per-feature spans: local_model.install, local_model.runtime, local_model.proxy join the central span table with hecate.phase labels — TestAllTelemetryEventsHaveSpecificSpanAndPhase covers them.
Stable error codes: local_models_unavailable, local_model_not_installed, local_model_runtime_unavailable, local_model_install_already_running, local_model_install_not_found, huggingface_gated, huggingface_not_found, huggingface_upstream_error.
OTel install events: local_model.install.{started,progress,completed,failed,cancelled} with byte counts + sha256 attributes. Runtime crash events surface with reason + uptime.

Docs

docs/rfcs/local-models-llamacpp.md — full RFC: scope, architecture, error mapping, boot reconciliation contract, operator override path, v1/v2 boundary.
docs/local-models.md — operator guide: availability matrix, env knobs, HTTP API, error codes, troubleshooting recipes.
.env.example — HECATE_LLAMA_SERVER_BIN, HECATE_LOCAL_MODELS, HECATE_LOCAL_MODELS_LAZY_DOWNLOAD, HECATE_LOCAL_MODELS_MAX_RESIDENT, HUGGINGFACE_TOKEN.

Stats

52 files changed, +11,940 / −4. ~1880 backend tests green under -race; 734 UI tests green.

Test plan

Copilot

Pull request overview

Adds a first-party Hecate-managed local models feature backed by a bundled or lazily downloaded llama-server, with backend install/runtime/proxy support, UI entry points, Tauri sidecar wiring, telemetry, and operator documentation.

Changes:

Introduces internal/llamacpp service components for catalog installs, runtime supervision, proxying, HF browsing, binary resolution, and tests.
Wires local models into /hecate/v1/local-models/*, /v1/models, control-plane persistence, telemetry/profiler spans, and gateway startup.
Adds UI API/types/card surface, Tauri external binary staging/resolution, environment docs, and local-models documentation.

Reviewed changes

Copilot reviewed 51 out of 52 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
`ui/src/types/runtime.ts`	Adds local-models and HuggingFace API wire types.
`ui/src/lib/api.ts`	Adds local-models, runtime, install, SSE, and HF browser API helpers.
`ui/src/features/providers/ProvidersView.tsx`	Adds the local models card to the providers view.
`ui/src/features/providers/LocalModelsCard.tsx`	Implements runtime summary/dormant card UI.
`ui/src/features/providers/LocalModelsCard.test.tsx`	Tests local models card states and slide-over launch.
`tauri/src-tauri/tauri.conf.json`	Adds `llama-server` as an external binary.
`tauri/src-tauri/src/sidecar.rs`	Resolves `llama-server` and passes env vars to the gateway.
`scripts/fetch-llama-server.ts`	Adds a Bun script to download/stage llama.cpp sidecars.
`internal/telemetry/semconv.go`	Adds local-model telemetry attributes.
`internal/telemetry/contract.go`	Registers local-model events and span names.
`internal/profiler/tracer.go`	Maps local-model events to spans/phases.
`internal/llamacpp/types.go`	Defines local-model catalog/runtime/install public types.
`internal/llamacpp/telemetry.go`	Adds local-model OTel span/event helpers.
`internal/llamacpp/service.go`	Adds top-level local-model service facade and provider registration.
`internal/llamacpp/service_test.go`	Tests service availability, reconciliation, and auto-provider behavior.
`internal/llamacpp/runtime_test.go`	Tests runtime lifecycle and crash handling.
`internal/llamacpp/runtime_process.go`	Adds production `llama-server` process starter.
`internal/llamacpp/runtime_lru_test.go`	Tests multi-resident LRU runtime behavior.
`internal/llamacpp/proxy.go`	Adds gateway-internal reverse proxy to active llama.cpp runtime.
`internal/llamacpp/proxy_test.go`	Tests proxy routing, streaming, and error mapping.
`internal/llamacpp/installer_test.go`	Tests install, SHA mismatch, cancellation, and validation paths.
`internal/llamacpp/installer_gated_test.go`	Tests HF token handling for gated downloads.
`internal/llamacpp/huggingface.go`	Adds HF search and repo-file client.
`internal/llamacpp/huggingface_test.go`	Tests HF client query, filtering, and errors.
`internal/llamacpp/catalog_test.go`	Validates catalog entries, SHA pins, and paste URLs.
`internal/llamacpp/binary_resolver.go`	Adds cached/lazy `llama-server` binary resolver.
`internal/llamacpp/binary_resolver_test.go`	Tests resolver explicit/cache/download/error paths.
`internal/controlplane/store.go`	Adds installed-model state and shared mutation helpers.
`internal/controlplane/store_test.go`	Adds shared installed-model lifecycle tests.
`internal/controlplane/store_sqlite.go`	Persists installed models in SQLite-backed state.
`internal/controlplane/store_sqlite_test.go`	Tests installed-model SQLite lifecycle/round-trip.
`internal/controlplane/store_memory.go`	Persists installed models in memory store.
`internal/controlplane/store_memory_test.go`	Tests installed-model memory lifecycle.
`internal/api/server.go`	Mounts local-model public routes and internal proxy route.
`internal/api/response.go`	Adds stable local-model/HF error codes and guidance.
`internal/api/handler.go`	Wires local-model service and appends installed models to `/v1/models`.
`docs/telemetry.md`	Documents local-model telemetry spans and attributes.
`docs/rfcs/README.md`	Adds local-models RFC index entry.
`docs/README.md`	Links the local-models operator guide.
`docs/local-models.md`	Adds local-models operator guide.
`cmd/hecate/main.go`	Initializes binary resolver/service and auto-provider at gateway boot.
`.gitignore`	Ignores staged llama-server binaries.
`.env.example`	Documents local-model environment knobs.

Comments suppressed due to low confidence (1)

ui/src/lib/api.ts:353

This also places the HuggingFace token in the URL for repo-file lookups, which can leak through logs/history. Keep the token out of query parameters here as well (for example, switch to a POST body or Hecate-side Authorization header) so gated-repo browsing does not expose the operator's HF token.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

chicoxyzzy · 2026-05-15T07:15:09Z

All 13 Copilot review comments addressed across e91d4622, f7491294, and follow-ups. Resolution per comment:

#	Location	Fix
1	`internal/llamacpp/runtime.go:625` Stop hang on pre-watcher	`watcherDone` stays nil until the watcher goroutine is actually spawned; Stop's receive skips nil channels. Regression test `TestRuntime_StopWhilePreWatcher` reproduces the race (`e91d4622`).
2	`internal/llamacpp/proxy.go:82` GET /v1/models 400s	Added `methodHasJSONBody` + `isBodylessPath` predicates and a `forwardBodyless` path that routes through `ProxyRuntime.ActiveBaseURL`. Tests cover happy path, "nothing loaded" 503, and large body (`e91d4622`).
3	`ui/src/lib/api.ts:333` HF token in URL	Token now rides `Authorization: Bearer` end-to-end; handler reads from header with HUGGINGFACE_TOKEN env fallback; query-string token is explicitly ignored (regression test in `handler_local_models_test.go`).
4	`scripts/fetch-llama-server.ts:70` unpinned sha	Script fails closed when `sha256` is empty; `HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1` is the documented dev-only bypass (prints observed digest for recording). Backfilled `b4404` digest in `f7491294`.
5	`internal/llamacpp/binary_resolver.go:125` resolver no sha	`Resolve()` returns `ErrBinarySHARequired` when `AssetSHA256` is empty unless `AllowUnverifiedDownload` (test-only). `DefaultBinarySpec` now carries the pinned darwin/arm64 digest.
6	`tauri/src-tauri/src/sidecar.rs:164` debug resolution	Debug branch now checks `tauri/src-tauri/binaries/llama-server-<triple>` first (where the fetch script stages), with the historical repo-root fallback after.
7	`docs/local-models.md:341` stale out-of-scope	Section replaced with a Status block reflecting v1+v2 implemented; "Still out of scope" trimmed to Linux/Windows bundles, per-model fine config, auto-update, non-llama.cpp engines.
8	`docs/rfcs/README.md:28` "Not implemented"	Updated to mark v1+v2 implemented with the four v2 features called out.
9	`cmd/hecate/main.go:338` non-runtime handlers when binary missing	`localModelsService()` gate now also checks `FeatureAvailability().Available`; catalog/installed/install/HF endpoints return 503 when the binary is unresolved. `/runtime` keeps emitting the dormant introspection body via `localModelsServiceForIntrospection`.
10	`internal/llamacpp/proxy.go:143` 64 KiB body cap	Raised to 16 MiB matching the gateway's accepted chat payload size. Test `TestProxy_AcceptsLargeRequestBody` exercises a 1 MiB payload.
11	`internal/llamacpp/service.go:271` Provider.ID not set	Managed row now has explicit `ID="llamacpp"` (`managedProviderID` constant) so `owned_by` in `/v1/models` matches the registered provider.
12	`internal/llamacpp/service.go:236` stale URL on reboot	`EnsureAutoRegisteredProvider` distinguishes managed (ID match) from operator-override (different ID, same PresetID). Managed row gets re-upserted on each boot to refresh `BaseURL`. Regression test `TestService_EnsureAutoRegisteredProvider_RefreshesManagedRow`.
13	`internal/llamacpp/installer.go:510` stale gated msg	401/403 now points at the v2 recovery paths: missing token → set `hf_token` or `HUGGINGFACE_TOKEN`; rejected token → verify access + expiry.

Backend race suite (1889 tests) and UI suite (734 tests) both green. The Tauri matrix is also running for the first time on this PR — _tauri-shared.yml's publish-updater-website job declared contents: write, actions: write, which tauri-build.yml capped at contents: read, causing startup_failure for every run since 35b59cd4 (2026-05-01). Lifted the cap in e7a3643f with the security tradeoff documented inline.

Apply the reusable-workflow permission fix from #101 so tauri-build no longer dies during workflow startup. GitHub validates all permissions requested by the called workflow before release-only job conditions run, so the caller has to grant actions and contents write even for PR validation runs. Split the Tauri action into unsigned PR and signed release steps. Passing empty signing and notarization variables still makes the bundler try to sign, which is why #101 moved past startup and then failed on empty key/certificate decoding. PR validation now omits those variables entirely, while tagged releases keep the signing path. Also build Windows sidecars with the host executable suffix before staging them, matching the filename the Windows runner expects when copying hecate.exe and hecate-acp.exe into Tauri externalBin names. Refresh the generated Tauri capability schema so it matches the existing updater permission. Verified with: - ruby -e 'require "yaml"; %w[.github/workflows/tauri-build.yml .github/workflows/_tauri-shared.yml].each { |f| YAML.load_file(f) }' - just tauri-sidecar - cd tauri/src-tauri && cargo test

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

chicoxyzzy · 2026-05-15T09:37:14Z

All three findings addressed in bf048da4:

#	Finding	Fix
P1	`Stop()` lost the concurrently-starting child — promoted session no longer in pool, child leaked	internal/llamacpp/runtime.go checks `r.sessions[modelID] == session` at both lock reacquisitions in `EnsureLoaded` (after `starter.Start` and after health polling). If Stop dropped the session under us, kill the orphaned handle and return `ErrRuntimeNotRunning`. Releases the mutex during `handle.Stop` so `stopTimeout` doesn't pin the runtime; re-locks for the outer `defer r.mu.Unlock()`. Regression test `TestRuntime_StopDuringSpawnKillsOrphanedChild` uses a `blockingStarter` to keep the race window open and asserts `handle.stopCount == 1` plus empty session pool after the race.
P1	Headless auto-register stored relative `BaseURL` when `PublicURL` empty	cmd/hecate/main.go defers `EnsureAutoRegisteredProvider` until after `net.Listen` returns, so it can fall back to `"http://" + listener.Addr().String()` when `cfg.Server.PublicURL` is empty. Tauri still passes `PublicURL` explicitly; headless gateways enabled via `HECATE_LLAMA_SERVER_BIN` now get an absolute URL too.
P2	Placeholder `llama-server` stubs looked real to both sides	Justfile + CI workflow write `# hecate-llama-server-placeholder` sentinel into the stub. tauri/src-tauri/src/sidecar.rs's new `is_llama_server_placeholder` scans the first 512 bytes and rejects any match — `HECATE_LLAMA_SERVER_BIN` is never set to a stub. internal/llamacpp/service.go does the same check in `FeatureAvailability` for the headless path where an operator might point at the stub directly — returns `Reason: "binary_is_placeholder"`. Regression test in service_test.go.

Verified locally: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

Thanks for the careful review — the orphan-child case in particular was the kind of bug that would have been very confusing to debug live (runtime says idle, but ps shows a llama-server running).

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

Copilot

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 3 comments.

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

Captures the design for bundling llama-server as a third Tauri sidecar, storing GGUF files at <data_dir>/models/, and routing through a single auto-registered llamacpp provider whose BaseURL points at a gateway-internal proxy (/hecate/internal/llamacpp/v1/...). The proxy fans out by requested model id to the active llama-server child, started on demand and killed on switch. Captures the catalog story (compiled-in pinned HuggingFace GGUF URLs + paste-direct-URL escape hatch, no HF browse v1, no re-hosted registry), the storage tier mirror (memory + sqlite + postgres), the OTel event surface, four new stable error codes, and the feature flag default (on for Tauri, off for headless Go gateway). v1 platform is macOS arm64 only — Metal-enabled by default in upstream llama.cpp's prebuilt macOS arm64 build. Linux / Windows ride the wider Tauri matrix expansion (CPU + Vulkan, when that lands). Three decisions are still open and flagged inline: the exact 8 curated entries, whether paste-URL accepts repo URLs (currently no — clean error pointing at the direct .gguf URL), and gated-repo behaviour (currently out of v1 with a clean "not supported" error). Hand-off layered as backend → tauri → ui → devops per the RFC's final section. Closes a recurring gap on the "try a local model" path and matches the one-click UX operators expect from Ollama / LM Studio without us owning a CDN.

…rolplane storage Introduces the internal/llamacpp/ package with the public type surface (CatalogEntry, Capabilities, RuntimeState/Status, InstallSpec, ProgressEvent, FeatureAvailability), the compiled-in curated catalog, and the paste-URL parser that powers the eventual /hecate/v1/local-models/ install endpoint. The catalog ships 8 v1 entries — Llama 3.2 1B + 3B, Qwen 2.5 0.5B + 3B + 7B, Mistral 7B v0.3, Phi-3 mini, Gemma 2 2B, all Q4_K_M, all pinned at specific HuggingFace direct-download URLs sourced from bartowski's converter account (the same account LM Studio defaults to). SHA256 values are intentionally empty for the initial round so the rest of the surface can land before the backfill — CatalogSHA256Gaps() exposes the TODO list, and TestCatalogSHA256Gaps logs them informationally. The installer will be wired to warn on empty shas and reject mismatches when present. ParsePasteURL implements the v1 policy: direct GGUF URLs only, repo-page / tree URLs surface ErrPasteURLNotDirect with operator guidance to copy the file URL, non-.gguf paths surface ErrPasteURLNotGGUF. Slug derivation lowercases, replaces non-alphanum with hyphens, and strips the .gguf suffix. The persisted record lives in controlplane (InstalledModel) so it slots into the existing JSON-blob sqlite store without a separate table — adding a slice to State is the whole migration. The llamacpp package re-exports via type alias so handlers and UI stay on llamacpp.InstalledModel without a mapping layer. Store gains UpsertInstalledModel / DeleteInstalledModel; both go through shared applyUpsertInstalledModel / applyDeleteInstalledModel helpers that audit-log the right action verb (installed_model.created / .updated / .deleted) and preserve InstalledAt across updates. Memory and sqlite stores delegate to the helpers; a shared runStoreInstalledModelLifecycle test runs against both, plus a sqlite round-trip test verifies the JSON shape survives Snapshot(). Note: the RFC mentioned postgres as a third tier; the repo doesn't ship a postgres store today (only memory + sqlite). When postgres lands, the same applyUpsertInstalledModel / applyDeleteInstalledModel helpers wire in without changes. Verification: - go build ./... clean - go vet ./... clean - go test ./internal/controlplane/... ./internal/llamacpp/... → 62 pass - go test -race on the same set → clean

…rialized Adds internal/llamacpp/installer.go: the download path that POST /hecate/v1/local-models/install will sit on. Resolves InstallSpec (catalog id or paste-URL) to a concrete plan, streams the HTTP body into <data_dir>/models/<slug>.gguf.part while hashing, verifies sha256 when expected, and atomically renames to the final path before calling controlplane.Store.UpsertInstalledModel. Key shape decisions: - One-install-at-a-time. Install returns ErrInstallInProgress when another download is in flight (handler maps to 409 local_model_install_already_running). Cancel kills the in-flight download via context, removes the .part file, emits a cancelled event. - ProgressEvents go to a bounded channel. The download loop sample events on both a byte threshold (256 KiB default) and a wall-clock step (250 ms default) so fast LAN downloads don't drown the UI and slow downloads still update steadily. Slow consumers drop events rather than block the writer; terminal events (completed / failed / cancelled) are best-effort under the same policy. - Errors are classified to stable kinds (network / sha_mismatch / cancelled / disk / gated / invalid_url / unknown) so the UI can map to a recovery hint without parsing message strings. HTTP 401/403 maps to "gated" with the v1-not-supported note in the message. - HTTP client and Clock are injectable (HTTPDoer / Clock interfaces in InstallerOptions). Production wires http.DefaultClient + time.Now; tests inject an httptest TLS server and a frozen clock for deterministic EmittedAt. - InstallerStore is the slim Store interface the installer needs — Upsert/Delete only — so tests can stub without pulling controlplane. Tests cover happy path (sha verified, file at final path, registry upsert recorded), sha mismatch (hard fail with both expected/actual hashes, partial file removed, no upsert), 403 → ErrorKindGated, cancel during streaming (server emits a chunk, test waits for the "ready" signal, cancels, asserts cancelled event + cleanup), concurrent Install → ErrInstallInProgress, spec validation (empty, ambiguous, unknown catalog id), and Cancel-with-no-active. go test -race ./internal/llamacpp/... → 27 pass.

…ener Adds internal/llamacpp/runtime.go and runtime_process.go: the part that owns the llama-server child process. v1 holds at most one child at a time; switching models stops the active child before starting the new one. State machine: idle → starting → running → stopping → idle, with a side-state of failed when a start fails or a child crashes unexpectedly. EnsureLoaded(ctx, modelID) is the only entry point — it serializes through the runtime mutex, so concurrent callers queue. If the requested model is already running, it returns the live base URL immediately; otherwise it stops any existing child, spawns a new one, polls /health, and commits the transition. Mutex discipline: the mutex is RELEASED during the slow operations (spawn + health-poll) so Status reads don't stall, then re-acquired to commit. The crash listener takes the same mutex when classifying an unexpected exit, so the state machine never observes a torn transition. Concurrent stop while a crash arrives is handled by the watcher coalescing on the active-session pointer — if the session has been replaced under it, the watcher leaves the new state alone. ProcessStarter / ProcessHandle are interfaces so tests inject deterministic fakes. ExecProcessStarter is the production implementation: spawns llama-server with -m / --host / --port / -c flags, polls /health every 250 ms, stops with SIGTERM → SIGKILL escalation after the configured timeout, and reaps the child in a dedicated goroutine that always runs exactly once per Start. The exit info channel is buffered 1 so Stop can race the reaper without deadlocking. freeTCPPort picks a loopback port by opening :0 and reading what the kernel assigns — same pattern Tauri uses for the gateway port. Tests cover happy path, no-op re-EnsureLoaded for the same model, model switch (stops the previous child, starts a new one), idempotent Stop (twice in a row both succeed), crash → failed state with LastError populated, health-poll failure tears down the child, spawn failure surfaces as failed, ErrRuntimeUnavailable when the binary path is empty, and ActiveBaseURL's three return paths (idle / running-with-this-model / running-with-other-model). freeTCPPort sanity check rounds it out. go test -race ./internal/llamacpp/... → 37 pass.

… peek Adds internal/llamacpp/proxy.go: the HTTP handler that fronts the auto-registered llamacpp provider. Mounted at /hecate/internal/llamacpp/v1/* and reachable only from within the gateway — the BaseURL on the auto-managed provider row points here. Flow per request: 1. Read + bound the body (64 KiB peek limit). 2. Decode just the `model` field. Empty / malformed → 400. 3. Call ProxyRuntime.EnsureLoaded(model). Maps runtime errors to stable error codes: - ErrRuntimeUnavailable → 503 local_models_unavailable - ErrRuntimeNotRunning / ErrRuntimeWrongModel → 503 local_model_runtime_unavailable - store "not found" → 404 local_model_not_installed - anything else → 500 local_model_runtime_unavailable 4. httputil.ReverseProxy forwards to <runtime base>/v1/<rest>. FlushInterval=-1 so SSE/streamed responses pass through chunk-by-chunk. Authorization header is stripped before forwarding so an inbound gateway-side token never lands in llama-server's stderr log. Tests cover the happy path (POST /chat/completions ends up at the upstream's /v1/chat/completions with the model field intact and the inbound Authorization stripped), end-to-end streaming (upstream flushes three SSE chunks, proxy delivers them all to the client), each runtime-error → error-code mapping, bad JSON → 400, missing model field → 400, and the JSON error shape contract. InternalProxyPathPrefix() is exported so the upcoming service wiring can compose the auto-registered provider's BaseURL from the same constant the route mounts under. go test -race ./internal/llamacpp/... → 49 pass.

Adds internal/llamacpp/service.go: the single thing api/main.go reaches for to wire the local-models feature. Composes Catalog + Installer + Runtime + Proxy and owns: - The controlplane.Store → Runtime.ModelLookup adapter (controlplaneModelLookup). Linear scan over Snapshot — fine for v1. - FeatureAvailability(): three-state probe (available / binary_not_found / binary_not_executable) the UI calls before rendering the Connections card to skip per-endpoint probes. - ListInstalled(): boot-reconciles the registry against the filesystem. Rows whose .gguf file vanished are dropped from the store synchronously inside the list call — keeps the chat composer's model picker from offering files the operator already rm'd, without forcing them to clean up by hand. - EnsureAutoRegisteredProvider(): called once at gateway boot. Upserts one llamacpp provider with the gateway-internal proxy path as BaseURL. Matches on PresetID="llamacpp" so an operator-created row (also using the catalog preset) is detected and left alone — the service returns ErrAutoProviderOperatorOwned which the caller logs as a structured warning. Dormant feature (empty BinaryPath) skips registration entirely so a stale row doesn't point at a port we won't bind. Tests: - FeatureAvailability: three branches (missing / executable / non-executable). - ListInstalled: drops the ghost row + verifies the store reflects the deletion. - EnsureAutoRegisteredProvider: fresh state path creates the row with the expected BaseURL/Kind/Protocol; operator-override path returns the sentinel and doesn't add a duplicate; dormant service skips entirely. - Runtime-lookup adapter smoke check — guards the boring path that would silently break if the adapter regressed. go test -race ./internal/llamacpp/... ./internal/controlplane/... → 100 pass.

Mounts the public local-models surface at /hecate/v1/local-models/* and the gateway-internal proxy at /hecate/internal/llamacpp/v1/*. Public API (internal/api/handler_local_models.go): GET /hecate/v1/local-models/catalog — curated entries + per-entry installed flag GET /hecate/v1/local-models/installed — boot-reconciled list POST /hecate/v1/local-models/install — kicks off the download; returns {install_id} GET /hecate/v1/local-models/install/{id}/events — SSE: ProgressEvents (started → progress → completed/failed/cancelled) DELETE /hecate/v1/local-models/install/{id} — cancel the in-flight install DELETE /hecate/v1/local-models/installed/{model_id} — uninstall (stops runtime if loaded, removes file + row) GET /hecate/v1/local-models/runtime — merged availability + state snapshot POST /hecate/v1/local-models/runtime/start — EnsureLoaded(model_id) POST /hecate/v1/local-models/runtime/stop — idempotent stop Each error path maps to a stable code from the RFC: - local_models_unavailable → 503 when feature is dormant - local_model_not_installed → 404 when requested model isn't on disk - local_model_runtime_unavailable → 503 when start/proxy fails - local_model_install_already_running → 409 on concurrent install attempt - local_model_install_not_found → 404 on cancel/events for unknown id - invalid_request → 400 for empty/ambiguous specs, bad URLs - not_found → 404 for unknown catalog ids User messages + operator actions added alongside the existing default shapes in response.go. Runtime status handler is the odd one out — when localModels is nil it returns 200 with availability=false instead of 503, so the UI can render "not available in this build" without a second roundtrip. The RFC's preference for the dormant-feature card. Install SSE plumbing: the Installer's per-install ProgressEvent channel is consumed by AttachInstall() into a buffered fanout keyed by install_id. Subscribers (the SSE handler) replay buffered history on attach, then receive new events live. A 60-second grace period keeps the terminal events around so a tab reload during the final "completed" frame still sees the outcome. Service.Uninstall stops the runtime if the target is currently loaded, removes the .gguf file, deletes the registry row. Returns ErrInstalledModelNotFound for unknown ids so the handler can map to 404. /v1/models integration: handler.HandleModels appends installed local models as additional entries with provider="llamacpp", provider_kind="local", discovery_source="local_model_registry". display_name, size_bytes, and loaded (matches active model) flow through metadata so the chat composer's model picker can render "loaded" vs "not loaded" affordances without a second probe. Boot wiring (cmd/hecate/main.go): looks for HECATE_LLAMA_SERVER_BIN or HECATE_LOCAL_MODELS=on. When either is set, constructs the Service, calls SetLocalModelsService on the Handler, and runs EnsureAutoRegisteredProvider against the configured PublicURL. An ErrAutoProviderOperatorOwned response is logged at Info level — the operator's row wins. Init failures log Warn and leave the feature disabled (handlers return 503). Smoke tests cover happy-path + dormant path per endpoint: - dormant catalog → 503 local_models_unavailable - wired catalog → 200 + curated entries with installed=false - installed empty / reflects registry round-trip - runtime status dormant → availability=false - runtime status wired → available=true, state=idle - install spec validation (empty + unknown catalog id) - runtime stop idempotent on idle - uninstall: missing id → 400, unknown id → 404 - cancel-install: unknown id → 404 - runtime start: missing model_id → 400 Verification: - go test ./internal/api/... ./internal/llamacpp/... ./internal/controlplane/... → 514 pass - go test -race ./internal/llamacpp/... ./internal/controlplane/... → 100 pass - go build ./... clean

…models Adds the third externalBin entry (binaries/llama-server) so Tauri's bundler stages a per-platform llama.cpp llama-server binary alongside the hecate gateway and hecate-acp ACP bridge. Three pieces wire together: 1. scripts/fetch-llama-server.ts — bun script that downloads the pinned upstream llama.cpp release archive, verifies sha256 when pinned, extracts the llama-server binary, and stages it at tauri/src-tauri/binaries/llama-server-<triple>. v1 covers aarch64-apple-darwin only (Metal-enabled by default in upstream's macOS arm64 build); adding a target = adding a row to TARGETS. The pinned release lives in LLAMA_CPP_RELEASE — bumping it is a single deliberate edit. SHA256 left empty during v1 bring-up with a logged warning; backfill before stable release. 2. tauri/src-tauri/src/sidecar.rs — resolve_llama_server_binary() mirrors the existing hecate binary resolution: HECATE_LLAMA_SERVER_BIN override first, then debug-mode repo-root lookup, then release-mode triple-suffixed lookup next to the running executable. Returns Option so a missing binary surfaces as "feature dormant" instead of aborting Tauri startup. Wired into spawn_and_wait so the gateway child inherits HECATE_LLAMA_SERVER_BIN + HECATE_LOCAL_MODELS=on whenever the binary resolves — the gateway's boot path then mounts the local-models API and auto-registers the llamacpp provider. 3. tauri.conf.json — third externalBin entry. The bundler renames binaries/llama-server to llama-server-<triple> at build time. .gitignore entries for tauri/src-tauri/binaries/llama-server-* and a root-level /llama-server so a misplaced staged binary doesn't end up in commits. Verification: - cargo check clean (placeholder binaries staged) - cargo test --lib 7 pass The fetch script is the only sanctioned producer of the bundled binary path; the .app bundle ships it unmodified.

…odels Surfaces the new /hecate/v1/local-models/* API in the Connections workspace. LocalModelsCard.tsx — slot at the top of the Connections workspace, above the operator-configured providers. Two visible states: Dormant — the gateway has no bundled llama-server (HECATE_LLAMA_SERVER_BIN unset / not executable). Card renders a "Not bundled" badge with the operator-facing explanation and no actions. UI doesn't poll endpoints on a dormant build. Active — Hecate has the binary. Shows runtime state pill (Idle / Loading / Running / Error), installed-model count summary, and a "Manage models" primary CTA that opens the slide-over. Failure state surfaces the runtime's last_error inline. Background poll on an 8s interval keeps the card's state fresh without a long-lived SSE for the summary surface. A close handler re-polls immediately so a finished install reflects without waiting for the next tick. LocalModelsSlideOver.tsx — the manage surface. Four sections: Runtime — current state pill + active-model id + loopback port, Stop button when running / starting. Active install (conditional) — appears while an install is in flight or after a terminal event the operator hasn't dismissed. Live progress bar driven by the per-install SSE stream (subscribeLocalModelInstallEvents). Cancel button while running, Dismiss after terminal. Connection drop is treated as a terminal "failed" so the UI never sits spinning forever. Installed list — per-row Start (Primary), Uninstall (Danger), and a "loaded" pill for whichever model the runtime currently has resident. Uninstall confirms via shared ConfirmModal. Catalog — curated entries with Install buttons. Disabled while another install is running so a second click can't trip the installer's serialize-one-at-a-time guard. "installed" pill replaces the Install button once the registry row exists. Custom HF URL — paste-direct-GGUF input that drops into the same installer flow (just a different InstallSpec). Types in types/runtime.ts mirror the Go response envelopes: LocalModelCatalogEntry, LocalModelInstalled, LocalModelRuntimeStatus, LocalModelFeatureAvailability, LocalModelProgressEvent, etc. Subscribe helper joins the five SSE event names into one callback so the consuming component doesn't have to wire each kind separately. API helpers in lib/api.ts cover every public endpoint: catalog, installed, runtime, install, install events (SSE), cancel install, uninstall, start, stop. Same fetchJSON helper + HECATE_API base path as the rest of the surface. Verification: - bun run typecheck clean - bun run build clean (ProvidersView chunk: 48 KB → 58 KB) - bun run test 707 pass

Adds docs/local-models.md as the operator-facing guide for the Hecate-managed llama.cpp runtime — when the feature is available (HECATE_LLAMA_SERVER_BIN resolution), storage layout under GATEWAY_DATA_DIR/models/, how models flow into /v1/models without a gateway restart, the runtime state machine (idle/starting/running/ stopping/failed), the catalog policy (pinned HuggingFace GGUF URLs, no re-hosted registry, paste-URL escape hatch, no gated repos in v1), the full HTTP API table with stable error codes, env knobs, the operator-override-of-auto-provider behaviour, troubleshooting, and the explicit v1-out-of-scope list cross-referenced to the RFC. .env.example gains HECATE_LLAMA_SERVER_BIN + HECATE_LOCAL_MODELS documented above the existing logging section. Both are commented out by default — the desktop Tauri sidecar sets them automatically, and headless gateway operators opt in only when they have a binary they want the gateway to manage. docs/README.md gains a Local models row in the Operator Docs table so the doc is discoverable from the landing page. OTel span events for the install / runtime lifecycle are noted in the RFC but not emitted yet — documenting them in docs/events.md would describe surface that doesn't exist. Their emission lands in a follow-up alongside the rest of the docs/events.md additions for this feature.

… scope Resolves the three known gaps surfaced after the initial round. OTel emission ============= The RFC promised typed span events for the install / runtime / proxy lifecycle; the first round only had slog-level error logs. This pass wires real OTel spans: - 10 new event constants in internal/telemetry/contract.go (local_model.install.started/progress/completed/failed/cancelled, local_model.runtime.starting/started/stopped/crashed, local_model.proxy.routed) plus 15 hecate.local_model.* attribute constants in internal/telemetry/semconv.go covering install + runtime + per-request hop telemetry. - internal/llamacpp/telemetry.go: package-level OTel tracer + small startInstallSpan / startRuntimeSpan / recordInstallEvent / recordRuntimeStarted / recordRuntimeStopped / recordRuntimeCrashed / recordProxyRouted helpers. Mirrors the per-package tracer pattern internal/agentadapters/approvals.go uses for the approval coordinator. - Installer: install span begins inside Install() (detached from the request lifetime via context.WithoutCancel so it outlives the POST handler) and ends in the run goroutine's defer. emit() fans every ProgressEvent to the span as a span event; terminal failed sets span.SetStatus(Error, message). - Runtime: runtimeSession gains spawnedAt + span fields. EnsureLoaded starts the span on transition to RuntimeStarting, records the runtime.started event with ttfh_ms once /health passes, stopActiveLocked records runtime.stopped with reason=operator and uptime_ms, and watchChild records runtime.crashed with exit_code + signal on unexpected exit. transitionFailedLocked closes the span with Error status on spawn / health failures. - Proxy: every ServeHTTP records local_model.proxy.routed on the parent span (or a short-lived span if none is attached) so traces show inbound chat traffic against the runtime that served it. Tracer is the OTel SDK's NoopTracerProvider unless explicitly wired, so tests don't need a span exporter to pass. docs/telemetry.md gains a "Local Models Spans" section under Traces with the full attribute table. docs/local-models.md cross-references it. SHA256 backfill =============== All 8 curated catalog entries now ship with pinned sha256 + exact size_bytes pulled from HuggingFace's LFS metadata (api.huggingface.co/api/models/<repo>/tree/main → lfs.oid + lfs.size). The installer's existing mismatch hard-fail now actually triggers on a supply-chain anomaly instead of silently accepting any download. TestCatalogSHA256Gaps flipped from t.Logf to t.Fatalf — a future entry that ships without a pinned sha now fails the build. A new TestCatalogSHA256Format guards the hex format (64 lowercase chars). RFC + docs scope ================ The original RFC overcommitted on postgres. The repo only ships memory + sqlite controlplane stores today; the entire controlplane surface is in that shape. Amended the RFC to match reality: - "Mirrored across memory + sqlite, postgres when the repo grows it" instead of pretending postgres is in scope for this feature. - Removed the "Postgres mirror tested but never exercised in production" risk row. - Migration section now correctly notes the sqlite store persists State as a single JSON blob — adding the InstalledModels field *is* the migration, no new table. - ListInstalledModels removed from the Store interface signature in the RFC (the implementation uses Snapshot() instead). - Backend hand-off note clarified that postgres lands "when the repo grows it" rather than as a precondition. Verification ============ go build ./... clean go test ./internal/api/... ./internal/llamacpp/... ./internal/controlplane/... ./internal/telemetry/... → 603 pass go test -race ./internal/llamacpp/... ./internal/controlplane/... ./internal/telemetry/... → 189 pass

…→ chat The unit tests in internal/llamacpp/ exercise each component in isolation. This pass adds three integration tests in internal/api/ that drive the registered routes through real httptest.Servers and assert the full chain works end-to-end — the only verification I hadn't done before is "does the route registration + service composition actually route a chat completion through to the upstream when the operator hits Install → Start → /v1/chat/completions". Fakes (no real llama-server needed): - fakeLlamaServer: httptest.Server with /health (200 OK) and /v1/chat/completions (200 OK with a fixed assistant payload). Records call counts so tests can assert the proxy forwarded. - fakeGGUFSource: httptest.NewTLSServer that serves arbitrary bytes as the model file. TLS because ParsePasteURL requires https. - pinnedStarter / pinnedHandle: ProcessStarter that returns a handle whose Host/Port point at fakeLlamaServer. Runtime's /health probe and proxy forwarding both land there. Wraps the real HTTP probe rather than stubbing it, so the runtime's health-poll path is actually exercised. Tests: TestLocalModels_InstallStartChat_EndToEnd 1. POST /hecate/v1/local-models/install with a paste URL pointing at the GGUF fake. 2. Subscribe to GET /install/{id}/events SSE stream; parse the SSE wire shape (event: name + data: payload). 3. Assert the terminal event is completed with the expected sha256 and the file lands at <data_dir>/models/<slug>.gguf. 4. GET /installed surfaces the new row. 5. POST /runtime/start; assert state transitions to running. 6. POST /hecate/internal/llamacpp/v1/chat/completions with the model id. Confirm the proxy forwards to upstream, returns upstream's body, and the upstream chat call count incremented. TestLocalModels_UninstallRoundTrip Install → DELETE /installed/{id} → /installed is empty again. TestLocalModels_Catalog_DormantBuildReturns503 Wires a Handler without SetLocalModelsService, registers the real routes, asserts GET /catalog returns 503 with errCodeLocalModelsUnavailable. Sanity for the dormant build path through the actual mux. Uses the unexported registerLocalModelsRoutes function directly so route-registration regressions are caught by these tests, not shelved until production. Calls SetLocalModelsService through the same setter cmd/hecate uses at boot. The SSE reader is a small bufio.Scanner-based parser that handles the event: + data: wire shape. Bounded by a 10s context so a stuck install can't hang the suite. Verification: go test ./internal/api/... -run LocalModels → 16 pass go test -race the same → 16 pass go test ./internal/api/... ./internal/llamacpp/... ./internal/controlplane/... ./internal/telemetry/... → 606 pass

Adds vitest coverage for the two new UI components shipped in commit 9b628f3. The handler-level Go tests cover the API; these tests cover the rendering branches the operator actually sees. LocalModelsCard (9 tests): - Loading shell on first paint while /runtime hasn't resolved. - Dormant tile with "Not bundled" / "Binary unusable" / "Disabled" / "Unavailable" badges per FeatureAvailability.reason. - Idle state with installed count summary. - Running state surfaces the active model name in the pill. - Empty-installed copy when the registry is fresh. - Failed state surfaces runtime.active.last_error inline. - Click Manage → SlideOver opens (asserted via the SlideOver-only Done button + Runtime section header). - /installed transient error doesn't trigger the dormant path — the 503 from /runtime is what flags the build, not /installed hiccups. LocalModelsSlideOver (11 tests): - All four sections present on first load (Runtime / Installed / Catalog / Custom HF URL). - Runtime pill + Stop button surface only when a model is loaded, loopback port renders so operators can curl it directly. - Active model row hides Start, shows the "loaded" badge. - Install button hits installLocalModel with catalog_id, then subscribes to SSE; published progress events update the % bar. - Completion event flips the active install to terminal state + triggers /installed refresh. - Cancel button calls cancelLocalModelInstall with the right id. - Catalog Install buttons disable while an install is running (v1 serializes; second click can't trip the installer's in-flight guard). - Uninstall opens ConfirmModal before firing the API call. - Start button on a non-active installed row calls startLocalModel. - Paste-URL flow submits the typed URL via installLocalModel. - InlineError surfaces a /catalog fetch failure. Tests mock the lib/api helpers and drive the components directly — no actual gateway interaction. The SSE callback is captured via subscribeLocalModelInstallEvents' mock impl so tests can publish events synchronously. Verification: bun run test → 35 files / 727 pass

Closes the first v2 follow-up from the RFC: the headless gateway can now resolve a llama-server binary without requiring an explicit HECATE_LLAMA_SERVER_BIN. Tauri builds keep using the bundled sidecar; CLI / dev gateways opt into on-demand download via HECATE_LOCAL_MODELS_LAZY_DOWNLOAD=on. internal/llamacpp/binary_resolver.go: BinaryResolver — three-step resolution chain: 1. ExplicitPath (HECATE_LLAMA_SERVER_BIN) — short-circuits if the file is regular + executable. 2. Cache hit at <data_dir>/llamacpp/bin/llama-server — works even when AllowDownload is false, so a Tauri build that once populated the cache still serves it. 3. Lazy download — gated on AllowDownload. Fetches the pinned upstream archive, verifies sha256 when pinned, extracts the inner binary atomically (write to .part → fsync → chmod +x → rename). Errors are typed sentinels the caller maps to operator-facing states: - ErrBinaryUnavailable no cache, no download permission - ErrBinaryNoUpstream unsupported GOOS/GOARCH - ErrBinarySHAMismatch archive sha differs from the pin - ErrBinaryInnerMissing upstream renamed the binary DefaultBinarySpec() carries the production pin — release tag matches scripts/fetch-llama-server.ts so Tauri sidecar bundling and headless lazy-download converge on the same release. Resolve caches the resolved path on the resolver struct so repeated calls within a process don't hammer disk or network. cmd/hecate/main.go: shouldInitLocalModels() gates init on any of three env knobs: HECATE_LLAMA_SERVER_BIN (Tauri / operator override), HECATE_LOCAL_MODELS=on (force init), or the new HECATE_LOCAL_MODELS_LAZY_DOWNLOAD=on. Boot calls resolver.Resolve with a 60s deadline; resolution failures other than ErrBinaryUnavailable log a warning and the feature stays dormant. Tests cover every branch: - Explicit path wins + bypasses cache/download. - Non-executable explicit path errors. - Cache hit works with AllowDownload=false. - Lazy download happy path → extracted binary at the cache path, executable, body matches the zip's inner file. - Repeat Resolve doesn't re-hit upstream (in-memory cache). - sha256 mismatch hard-fails with ErrBinarySHAMismatch, no file persisted. - AllowDownload=false on a cold cache → ErrBinaryUnavailable. - Inner-path missing in archive → ErrBinaryInnerMissing. - HTTP error from upstream surfaces with status code in the error message. Docs: .env.example documents HECATE_LOCAL_MODELS_LAZY_DOWNLOAD with the Tauri vs CLI guidance. docs/local-models.md env table gains the new knob row. docs/rfcs strikes through the v1 out-of-scope item and notes where the v2 codepath lives. Verification: go build ./... clean go test -race ./internal/llamacpp/... 71 pass

Closes the second v2 follow-up from the RFC. With the default MaxResident=1 the runtime behaves identically to v1 (single child, restart-on-switch). With MaxResident>1, the runtime keeps the N most-recently-used models resident and evicts the coldest when the (N+1)th EnsureLoaded arrives. Operators opt in via HECATE_LOCAL_MODELS_MAX_RESIDENT=N. The trade is RAM for switch latency — N resident models can swap with near-zero overhead; the operator gates on memory pressure, Hecate doesn't auto-tune. internal/llamacpp/runtime.go: Runtime gains: - sessions map[string]*runtimeSession — pool keyed by model id. - lruOrder []string — head=oldest, tail=newest. - primaryID — the "primary" session (most recently touched). - r.active continues to point at the primary so existing v1 paths still resolve via r.active. With MaxResident=1, the pool has at most one entry; r.active == sessions[primaryID]. EnsureLoaded(modelID): - Hot path: model already resident → bump LRU, promote to primary, return URL without touching the child. - Cold path: evict LRU children until len(sessions) < cap, then spawn + health-poll + insert. Eviction is the same path as v1's stop-then-start when cap=1. ActiveBaseURL(modelID): - Empty modelID → v1-compat: return the primary's URL. - Non-empty → look up in sessions map. Bumps LRU + promotes to primary on hit. Returns ErrRuntimeWrongModel when the model isn't loaded (carries the resident list in the message). Stop() now shuts down every resident child, not just the primary. stopActiveLocked is retained as a thin wrapper over the new stopSessionLocked for the legacy method-name compat. watchChild promotes the LRU tail as the new primary when a non-primary session crashes; state stays "running" if other sessions are still resident, falls to "failed" only when the pool is empty. SessionsSnapshot() exposes the per-session view (LRU order, with Primary bool) for UI callers that want the multi-resident detail. MaxResident() returns the configured cap. Both are additive — v1 callers keep using Status() and see the primary unchanged. cmd/hecate/main.go wires HECATE_LOCAL_MODELS_MAX_RESIDENT via a small parseMaxResident helper that returns 0 (default) on missing or unparseable input — boot is the wrong place to be noisy about env typos when the safe default is the v1 behavior. Tests (runtime_lru_test.go) cover: - Both models resident after two consecutive EnsureLoads at cap 2. - Hot-path re-load returns the same URL without spawning. - LRU eviction picks the coldest, not the most-recently-touched. - ActiveBaseURL works for any resident model id. - Primary follows the most-recently-touched session, including via ActiveBaseURL touches. - Stop tears down every resident child. - Crash of a non-primary session removes it from the pool while the primary keeps serving (state stays "running"). - 8 concurrent goroutines hitting EnsureLoaded never exceed the cap. - MaxResident=0 falls through to default 1 (v1 behavior). All existing v1 tests in runtime_test.go pass unchanged — the default cap of 1 is the only path they ever exercised. Docs: .env.example documents the new knob with the RAM-vs-latency trade-off. docs/local-models.md env table gains a row. docs/rfcs strikes the v1 out-of-scope entry. Verification: go build ./... clean go test -race ./internal/llamacpp/... → 80 pass go test -race ./internal/llamacpp/... ./internal/api/... ./internal/controlplane/... → 539 pass

Closes the third v2 follow-up. Operators can now install gated HuggingFace models (Meta's official Llama checkpoints, Google's official Gemma, etc.) by supplying their HF access token. The token is **not persisted**. It rides the install request and the env var only — a future PR can introduce encrypted at-rest storage when the cipher plumbing is sorted out for a generic "named credential" surface. Surface: - InstallSpec.HFToken (Go) / hf_token (JSON) — per-install token, set in the POST /install body. - HUGGINGFACE_TOKEN env var — headless / CI fallback. Spec token wins when both are set. - UI: Custom HuggingFace URL section gains a second input (password-type, autoComplete=new-password + data-1p-ignore + data-lpignore so password managers don't try to autofill). Token clears after a successful install. Installer: resolveSpec resolves the token in priority order (spec → env) and stamps it onto the installPlan. download() attaches `Authorization: Bearer <token>` only when the plan carries one — public installs never carry an empty/spurious header, since some CDNs interpret presence-of-Authorization as a signal to enforce auth. Existing 401/403 → ErrorKindGated path is unchanged; with a valid token the upstream returns 2xx and the install completes normally. API: handler.HandleLocalModelsInstall forwards req.HFToken through to llamacpp.InstallSpec.HFToken. UI lib/api: installLocalModel's signature gains the optional hf_token field. Tests (installer_gated_test.go): - Gated repo with the right token → completes, Authorization header on the upstream call. - Gated repo without a token → ErrorKindGated (covered by existing test, reasserted with the auth-recording upstream). - Env-var fallback works when InstallSpec.HFToken is empty. - Spec token wins over env token. - Public install (no token, no env) sends no Authorization header at all — regression guard. Env-mutating tests run serially (t.Setenv is incompatible with t.Parallel); helper authRecorder is shared across them. Docs: docs/local-models.md gains a "Gated HuggingFace repos" section. .env.example documents HUGGINGFACE_TOKEN. docs/rfcs strikes the v1 out-of-scope entry; model-card preview remains TBD. Verification: go test ./internal/llamacpp/... 85 pass go test -race ./internal/llamacpp/... 85 pass bun run test -- LocalModels 20 pass bun run typecheck clean bun run build clean

Adds a server-side proxy to HF Hub's API so the operator can search GGUF-tagged repos and pick a specific quant from inside the Manage slide-over. The browser never sees the HF token; the gateway forwards it on the operator's behalf. Backend - internal/llamacpp/huggingface.go: HuggingFaceClient.SearchModels + ListRepoFiles. SearchModels pins filter=gguf and sort=downloads; ListRepoFiles returns only .gguf files with the LFS sha256 + size + canonical resolve URL. Per-call token via Bearer header. Typed errors for gated (401/403) and not-found (404). - internal/llamacpp/service.go: HuggingFaceOptions now plumbs through ServiceOptions so handler tests can swap baseURL to an httptest server. - internal/api: HandleLocalModelsHFSearch + HandleLocalModelsHFRepoFiles handlers; writeHFError maps client errors to the new stable codes huggingface_gated / huggingface_not_found / huggingface_upstream_error. Routes registered at GET /hecate/v1/local-models/huggingface/... - internal/telemetry + profiler: SpanLocalModelInstall / SpanLocalModelRuntime / SpanLocalModelProxy join the central span table; the local_model.install.*, local_model.runtime.*, and local_model.proxy.* event prefixes route there with hecate.phase labels — fixes TestAllTelemetryEventsHaveSpecificSpanAndPhase. UI - ui/src/types/runtime.ts: HuggingFaceModel + HuggingFaceFile wire types. - ui/src/lib/api.ts: searchHuggingFaceModels + listHuggingFaceRepoFiles helpers. - LocalModelsSlideOver: Browse HuggingFace section between Catalog and Custom URL — search input + result rows + per-row Show files expander. Picking a file Install drops into the existing install flow with the LFS sha256 attached. Already-installed files render the 'installed' badge instead of an Install button (matched by source_url). Tests - internal/llamacpp/huggingface_test.go: query construction, limit clamping (1000→100, 0→20), gated 403 mapping, 404 mapping, file filter + LFS sha extraction, repo-id validation, concurrent requests under -race. - internal/api/handler_local_models_test.go: 5 new HF handler tests covering happy path, gated 403, dormant 503, repo file listing, not-found, and token forwarding through the proxy. - ui/.../LocalModelsSlideOver.test.tsx: 7 new tests covering the browse section render, search invocation, result row contents (download count formatting, gated badge), file list expansion, Install path with sha256, the 'installed' badge for already- installed files, error surfacing, and token forwarding to both search + file-list calls. Docs - RFC out-of-scope: HuggingFace browse / search struck through with v2 reference. - docs/local-models.md: new HF browse section, two new HTTP API rows, three new error code rows.

Security - HF token now rides the `Authorization: Bearer` header end-to-end — UI helper, gateway handler, and the upstream HF call. Query-string `?token=` is removed from the API contract so the secret can't leak through dev-server logs, browser history, or proxy access logs. Handler reads from header → HUGGINGFACE_TOKEN env fallback; legacy ?token= is explicitly ignored (regression test guards it). Correctness — runtime - Stop no longer deadlocks on a session that's still in the start window: watcherDone is now created after the watcher goroutine is spawned. Stop's `<-watcher` receive sees nil-channel for sessions that never reached "running" and skips the wait. Regression test (`TestRuntime_StopWhilePreWatcher`) reproduces the race with a blockingStarter and verifies Stop completes within 500ms. Correctness — proxy - GET /v1/models now bypasses the model-body peek: body-less methods (GET/HEAD/OPTIONS) and the /models path forward straight to the active session via ProxyRuntime.ActiveBaseURL. v1 returned 400 because peekModel returned an empty model field. New tests cover the happy path, the "nothing loaded" 503, and a large-body request that previously tripped the 64 KiB cap. - Body cap raised from 64 KiB → 16 MiB to match the gateway's own request body cap. Real chat completions with conversation history, tool definitions, and multimodal blobs comfortably exceeded 64 KiB. Correctness — auto-provider - Managed row now has an explicit ID="llamacpp" matching the `owned_by` field the /v1/models integration advertises. Without this, the controlplane store derived ID="llama-cpp" from Name and routing lookups by provider ID failed silently. - EnsureAutoRegisteredProvider distinguishes "row we own" (ID=llamacpp) from "operator override" (different ID, PresetID=llamacpp). On reboot it refreshes the managed row's BaseURL so a desktop launch on a different port doesn't leave the row pointing at a stale internal-proxy URL. Operator overrides remain untouched — ErrAutoProviderOperatorOwned still fires for those. Correctness — dormancy - localModelsService gate now checks FeatureAvailability().Available in addition to the nil check. When HECATE_LOCAL_MODELS=on but the binary is unresolved, non-introspection handlers return 503 with local_models_unavailable instead of pretending the feature works. /runtime status still returns 200 + availability=false so the UI can render the dormant state with its reason. Supply chain — binary verification - BinaryResolver fails closed on the lazy-download path when Spec.AssetSHA256 is empty (ErrBinarySHARequired). Tests opt into the unverified path via the new AllowUnverifiedDownload escape hatch; production never sets it. - scripts/fetch-llama-server.ts now refuses to stage a sidecar binary without a pinned sha256. Dev workflow for bumping the release tag: set HECATE_ALLOW_UNVERIFIED_LLAMA_SERVER=1, observe the printed digest, record it in TARGETS[].sha256, run again. Tauri sidecar resolution - Debug-build resolver checks tauri/src-tauri/binaries/llama-server-<triple> in addition to the historical repo-root fallback. Operators who run the documented `bun scripts/fetch-llama-server.ts` step now see the staged binary picked up; previously the feature stayed dormant in dev because the resolver looked in the wrong place. UX - Installer's 401/403 message no longer says "v1 does not support gated repos" — gated support landed. New message points at the right recovery path: missing token → set hf_token or HUGGINGFACE_TOKEN; rejected token → verify access + expiry. Docs - docs/rfcs/README.md status updated: v1 + v2 implemented (only Linux/Windows bundles remain out of scope). - docs/local-models.md "Out of scope (v1)" replaced with a Status section that reflects what's actually implemented + a focused "Still out of scope" list.

Tauri's externalBin resolution runs during `cargo test` and `cargo build` on the host triple, so the binary must exist even on platforms the bundle doesn't actually ship — otherwise the build script errors with `resource path 'binaries/llama-server-<triple>' doesn't exist` before any test runs. Two paths: - `.github/workflows/test.yml` — the Linux x86_64 Rust test runner. Add `llama-server` to the existing placeholder-staging loop so the job has a triple-suffixed file at the expected path. - `justfile` — extract a new `tauri-llama-sidecar` recipe that the matrix Tauri build (`_tauri-shared.yml`) picks up automatically through the `tauri-sidecar` dep chain. The macOS arm64 branch tries the real fetch script when `bun` is available; every other triple drops an executable shell stub. The desktop bundle still only ships the macOS arm64 llama-server — Linux / Windows remain out of scope.

Both the headless lazy-download path and the Tauri fetch script now verify the upstream archive against the pinned digest: llama-b4404-bin-macos-arm64.zip sha256: 48bf9261b859386db34e23f6447638282e1144c63fdb8bf8ab8380d63d4ff485 size: 60.2 MB Computed by downloading the upstream release archive directly from the b4404 release page; smoke-tested locally by running `scripts/fetch-llama-server.ts --target aarch64-apple-darwin` and confirming the staged binary reports `version: 4404` on `--version`. Production effect: HECATE_LOCAL_MODELS_LAZY_DOWNLOAD=on now works end-to-end on darwin/arm64 — previously the resolver short-circuited with ErrBinarySHARequired because the pin was empty. Tauri builds were unaffected (they bundle the binary directly via externalBin). docs/local-models.md updated to reflect the digest-always-verifies behaviour and to point operators at DefaultBinarySpec when bumping the release tag.

Three lifecycle / readiness edge cases the second review surfaced. P1 — Stop() lost concurrently-starting child: EnsureLoaded inserted a session into r.sessions, released the mutex during starter.Start, then re-acquired the mutex and promoted the session at line 533 without checking whether Stop() had removed it under it. If Stop ran during the spawn, it saw session.handle == nil (we hadn't assigned it yet), skipped the handle-stop, and deleted the session. EnsureLoaded then promoted a session that was no longer in the pool, leaving the runtime reporting Running with a child that subsequent Stop() iterations of r.sessions never found — orphan child leak. Fix: at both lock-reacquisition points in EnsureLoaded (after starter.Start and after health polling), check that r.sessions[modelID] still points at our session. If not, Stop dropped it — kill the now-orphaned handle and return ErrRuntimeNotRunning. Release the mutex during the handle.Stop call so the long-tail stopTimeout doesn't pin the runtime, re-lock before returning so the existing deferred Unlock at the top of the function runs cleanly. Regression test (TestRuntime_StopDuringSpawnKillsOrphanedChild) uses a blockingStarter that pauses Start() until the test releases it. It runs Stop() in the gap, then unblocks the spawn, then asserts the returned handle's stopCount == 1 and the runtime ends in idle with an empty session pool. The existing TestRuntime_StopWhilePreWatcher (deadlock guard) is kept as a sibling. P1 — Headless auto-registration stored relative provider URL: cmd/hecate/main.go called EnsureAutoRegisteredProvider with cfg.Server.PublicURL. When that was empty (the common headless case), EnsureAutoRegisteredProvider built BaseURL as strings.TrimRight("", "/") + "/hecate/internal/llamacpp/v1" = "/hecate/internal/llamacpp/v1" and persisted it. Any downstream that resolved the URL — including the chat composer's routing — saw a relative path and failed. Tauri set PublicURL explicitly so it never tripped; headless / dev gateways enabled via HECATE_LLAMA_SERVER_BIN did. Fix: defer the EnsureAutoRegisteredProvider call until after listener.Addr() is known (it has to happen after net.Listen), and fall back to "http://" + listener.Addr().String() when PublicURL is empty. Tauri's explicit PublicURL still wins. P2 — Placeholder llama-server stubs looked like real binaries: `just tauri-llama-sidecar` writes a `#!/bin/sh\nexit 0\n` executable for non-arm64-darwin targets so Tauri's externalBin resolution succeeds at build time. The Rust sidecar resolver only checked `is_file()` and the Go service only checked `mode.IsRegular() && executable bit`. Both passed for the stub, so the dev/Linux Tauri build would set HECATE_LLAMA_SERVER_BIN=/path/to/exit-0-script, the gateway would auto-register the llamacpp provider with `Available: true`, the UI would advertise local-models, and Install / Start would fail only when the operator clicked them. Fix: write a sentinel comment "hecate-llama-server-placeholder" into the stub body and detect it in both the Tauri resolver (new is_llama_server_placeholder in sidecar.rs) and the Go service (FeatureAvailability returns Reason: "binary_is_placeholder"). Updated Justfile, .github/workflows/test.yml CI stub, and added a regression test (TestService_FeatureAvailability "binary is sentinel placeholder" subtest). Verified: 1891 Go tests pass under -race, 734 UI tests pass, 7 Rust tests pass, actionlint clean on all three workflows.

# Conflicts: # .github/workflows/test.yml # Justfile # cmd/hecate/main.go # docs/README.md # docs/rfcs/README.md # internal/api/handler.go # internal/api/response.go # internal/controlplane/store.go # internal/controlplane/store_memory.go # internal/controlplane/store_memory_test.go # internal/controlplane/store_sqlite.go # internal/controlplane/store_sqlite_test.go # internal/controlplane/store_test.go # internal/profiler/tracer.go # internal/telemetry/contract.go # internal/telemetry/semconv.go # tauri/src-tauri/src/sidecar.rs # tauri/src-tauri/tauri.conf.json # ui/src/lib/api.ts # ui/src/types/runtime.ts

Copilot AI review requested due to automatic review settings May 14, 2026 23:13

Copilot started reviewing on behalf of chicoxyzzy May 14, 2026 23:14 View session

Copilot AI reviewed May 14, 2026

View reviewed changes

chicoxyzzy force-pushed the feature/llamacpp branch from f77a044 to e7a3643 Compare May 15, 2026 07:12

chicoxyzzy force-pushed the feature/llamacpp branch 2 times, most recently from 1899e57 to f749129 Compare May 15, 2026 07:48

chicoxyzzy mentioned this pull request May 15, 2026

ci: gate desktop packaging behind tests #103

Merged

chicoxyzzy force-pushed the feature/llamacpp branch from f749129 to 6caaa71 Compare May 15, 2026 09:07

chicoxyzzy force-pushed the feature/llamacpp branch from bf048da to 5aef623 Compare May 15, 2026 09:48

chicoxyzzy force-pushed the feature/llamacpp branch from 5aef623 to 142121f Compare May 15, 2026 14:14

chicoxyzzy force-pushed the feature/llamacpp branch from 142121f to cc252d7 Compare May 15, 2026 23:26

chicoxyzzy requested a review from Copilot May 15, 2026 23:29

Copilot started reviewing on behalf of chicoxyzzy May 15, 2026 23:29 View session

Copilot AI reviewed May 15, 2026

View reviewed changes

Comment thread internal/api/handler.go

Comment thread internal/llamacpp/service.go

Comment thread internal/llamacpp/runtime.go

chicoxyzzy force-pushed the feature/llamacpp branch from cc252d7 to 9a05910 Compare May 16, 2026 03:45

chicoxyzzy added 21 commits May 16, 2026 09:17

chicoxyzzy force-pushed the feature/llamacpp branch from 9a05910 to 7355368 Compare May 16, 2026 07:36

Navigation Menu

Uh oh!

feat(llamacpp): Hecate-managed local models — download + run via llama.cpp - #101

feat(llamacpp): Hecate-managed local models — download + run via llama.cpp#101
chicoxyzzy wants to merge 22 commits into
masterfrom
feature/llamacpp

chicoxyzzy commented May 14, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chicoxyzzy commented May 15, 2026

Uh oh!

chicoxyzzy commented May 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

chicoxyzzy commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

v1 — backend foundation, UI surface, Tauri sidecar bundling

v2 — lazy-download, LRU keep-warm, gated repos, HuggingFace browse

Telemetry, observability, error mapping

Docs

Stats

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chicoxyzzy commented May 15, 2026

Uh oh!

chicoxyzzy commented May 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chicoxyzzy commented May 14, 2026 •

edited

Loading