bug: dashboard-api enable/disable HTTP endpoints don't fall back to EXTENSIONS_DIR — built-in extensions return 404

[yasinBursali]

Bug Report: dashboard-api enable/disable HTTP endpoints don't fall back to EXTENSIONS_DIR — built-in extensions return 404

Severity: Medium
Category: Infra / Lookup Bug
Platform: All (Linux, macOS, Windows/WSL2)
Confidence: Confirmed

Follow-up of: #299 (Host agent rejects all built-in extensions from API management) and #320 (apply_template silently skips built-in extensions).
PR Light-Heart-Labs#907 fixed the host-agent layer of #299 (verified working — see my comment on #299). PR Light-Heart-Labs#909 fixed the internal _activate_service helper for the template-apply path (see my comment on #320). But the public dashboard-api HTTP handlers enable_extension and disable_extension still only check USER_EXTENSIONS_DIR. The user-facing capability promised by these PRs doesn't ship at the public API surface.

Description

PR Light-Heart-Labs#907 (the dashboard-api side of #299) was supposed to "allow API management of built-in extensions". The _assert_not_core helper was correctly relaxed (only the 4 always-on services in ALWAYS_ON_SERVICES are now blocked at the gate), but the public HTTP handlers enable_extension and disable_extension still only look in USER_EXTENSIONS_DIR. Calling them for a built-in extension like n8n returns 404 Extension not installed: n8n, even though dream-n8n is healthy and the extension directory exists at dream-server/extensions/services/n8n/.

The internal _activate_service (used by template apply via Light-Heart-Labs#909) DID get the EXTENSIONS_DIR fallback. The public HTTP endpoints did not.

Affected File(s)

• dream-server/extensions/services/dashboard-api/routers/extensions.py:1037 — enable_extension (public HTTP handler POST /api/extensions/{service_id}/enable)
• dream-server/extensions/services/dashboard-api/routers/extensions.py:1147 — disable_extension (public HTTP handler POST /api/extensions/{service_id}/disable)

For comparison: _activate_service at L985 (the internal helper, fixed by Light-Heart-Labs#909) already does the right thing.

Root Cause

Function	Line	Checks built-in fallback (EXTENSIONS_DIR)?
_activate_service (internal)	985	✅ Yes — if in_user and user_dir.is_dir(): ext_dir = user_dir; elif in_builtin and builtin_dir.is_dir(): ext_dir = builtin_dir
enable_extension (public HTTP)	1037	❌ No — only (USER_EXTENSIONS_DIR / service_id).resolve()
disable_extension (public HTTP)	1147	❌ No — only (USER_EXTENSIONS_DIR / service_id).resolve()

The enable_extension body (current state on the integration branch):

@router.post("/api/extensions/{service_id}/enable")
def enable_extension(service_id: str, ...):
    _validate_service_id(service_id)
    _assert_not_core(service_id)               # ← #907 relaxed this; now only blocks always-on
    ext_dir = (USER_EXTENSIONS_DIR / service_id).resolve()    # ← still USER-only
    if not ext_dir.is_relative_to(USER_EXTENSIONS_DIR.resolve()):
        raise HTTPException(status_code=404, detail=f"Extension not found: {service_id}")
    if not ext_dir.is_dir():
        raise HTTPException(status_code=404, detail=f"Extension not installed: {service_id}")
    ...

The _assert_not_core change is correct but useless without the corresponding lookup change: the request gets past the _assert_not_core gate, then immediately 404s on the directory check because the built-in lives in EXTENSIONS_DIR, not USER_EXTENSIONS_DIR.

Evidence

On a fresh install with Light-Heart-Labs#907 (and Light-Heart-Labs#909) merged. n8n is part of the default --all install:

$ KEY=$(grep ^DASHBOARD_API_KEY= ~/dream-server/.env | cut -d= -f2)
$ H="Authorization: Bearer $KEY"

# Always-on services correctly blocked (this part works — _assert_not_core fix is fine):
$ curl -X POST -H "$H" http://127.0.0.1:3002/api/extensions/llama-server/disable
{"detail":"Cannot modify always-on service: llama-server"}                  # ← 403, correct

$ curl -X POST -H "$H" http://127.0.0.1:3002/api/extensions/dashboard-api/disable
{"detail":"Cannot modify always-on service: dashboard-api"}                 # ← 403, correct

# Built-in extensions can't be managed (this is the bug):
$ curl -X POST -H "$H" http://127.0.0.1:3002/api/extensions/n8n/disable
{"detail":"Extension not installed: n8n"}                                   # ← 404, wrong

# Confirm n8n is actually installed and running:
$ docker ps --filter name=dream-n8n --format '{{.Names}} {{.Status}}'
dream-n8n Up 16 minutes (healthy)

$ ls dream-server/extensions/services/n8n/
README.md  compose.yaml  manifest.yaml

For comparison, the host-agent layer (PR Light-Heart-Labs#907's other half, fixed in bin/dream-host-agent.py) DOES handle built-ins correctly. Direct test against 127.0.0.1:7710 from the host:

$ curl -X POST -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
    -d '{"service_id":"n8n"}' http://127.0.0.1:7710/v1/extension/stop
{"status": "ok", "service_id": "n8n", "action": "stop"}                     # ← 200, works

So the host-agent can handle built-in management. The dashboard-api just doesn't reach it because of the lookup bug at the higher layer.

Platform Analysis

• macOS: Affected — same Python code path runs in the dashboard-api container regardless of host OS.
• Linux: Affected, verified end-to-end on WSL2.
• Windows/WSL2: Affected, same.

Reproduction

1. Fresh install with the open PR stack fix(models): platform-aware activation + download cancel Light-Heart-Labs/DreamServer#893–909 merged. n8n is part of the default --all install.

2. KEY=$(grep ^DASHBOARD_API_KEY= ~/dream-server/.env | cut -d= -f2)
   curl -X POST -H "Authorization: Bearer $KEY" http://127.0.0.1:3002/api/extensions/n8n/disable
   

3. Expected: 200 with disable result (PR fix(host-agent,dashboard-api): allow API management of built-in extensions Light-Heart-Labs/DreamServer#907's title says "allow API management of built-in extensions").
4. Actual: 404 {"detail":"Extension not installed: n8n"}.

Impact

Medium. Always-on enforcement (the security side of Light-Heart-Labs#907) is correct — llama-server, open-webui, dashboard, dashboard-api all return the new 403 message. The user-facing capability promised by the PR title ("allow API management of built-in extensions") doesn't ship at the public API surface.

Internal callers that go through _activate_service (i.e. template apply via apply_template) would in theory benefit, but those are themselves blocked by a separate :ro mount conflict — see #331.

Suggested Approach

Mirror _activate_service's lookup pattern in both enable_extension (L1037) and disable_extension (L1147):

# replace the existing USER_EXTENSIONS_DIR-only resolve block with:
user_dir = (USER_EXTENSIONS_DIR / service_id).resolve()
builtin_dir = (EXTENSIONS_DIR / service_id).resolve()
in_user = user_dir.is_relative_to(USER_EXTENSIONS_DIR.resolve())
in_builtin = builtin_dir.is_relative_to(EXTENSIONS_DIR.resolve())
if not in_user and not in_builtin:
    raise HTTPException(status_code=404, detail=f"Extension not found: {service_id}")
if in_user and user_dir.is_dir():
    ext_dir = user_dir
elif in_builtin and builtin_dir.is_dir():
    ext_dir = builtin_dir
else:
    raise HTTPException(status_code=404, detail=f"Extension not installed: {service_id}")

This is the same pattern PR Light-Heart-Labs#909 already added to _activate_service — copy it. Better still: refactor so all three call sites share a single _resolve_extension_dir(service_id) helper.

Note that fixing the lookup is necessary but not sufficient: disable_extension then needs to perform a rename (compose.yaml → compose.yaml.disabled) on the extensions/ mount, which is :ro due to PR Light-Heart-Labs#908. So the lookup fix should land alongside the host-agent delegation fix proposed in #331.

Test gap

TestAssertNotCoreAllowsBuiltins (added by Light-Heart-Labs#907 in test_extensions.py:2008) only validates the _assert_not_core helper in isolation. It does NOT exercise the full enable/disable HTTP path against a built-in extension. A direct e2e test against POST /api/extensions/n8n/disable would have caught this immediately. Same gap pattern as #331 — the unit test mocks the dependency (the helper / the tmpdir) and never hits the layer where the bug actually lives.

Cross-references

• Host agent rejects all built-in extensions from API management — CORE_SERVICE_IDS misused as blocklist #299 (open) — parent issue at the host-agent layer. PR fix(host-agent,dashboard-api): allow API management of built-in extensions Light-Heart-Labs/DreamServer#907's host-agent fix is verified working; this is the dashboard-api side that didn't get the corresponding update.
• apply_template silently skips built-in (non-catalog) extensions — all 11 templates affected #320 (open) — parent issue at the template-apply layer. Same lookup bug class, fixed for _activate_service but not for the public HTTP handlers.
• bug: read-only extensions mount blocks built-in template activation — every catalog template apply fails 500 #331 (filed in this same test pass) — related: the rename in disable_extension will fail with EROFS due to the :ro mount even after the lookup is fixed. Both fixes need to land together.

---

Filed during full-stack integration test of open PR stack Light-Heart-Labs#893–909 on Light-Heart-Labs/DreamServer@c0600ca3. Environment: WSL2 / Ubuntu 24.04 / NVIDIA RTX 3070 Laptop / Tier 1.

[yasinBursali]

Confirmed reproducing on macOS during the same integration-test run (all 17 PRs Light-Heart-Labs#893–909 merged, Mac Apple Silicon, install at /Volumes/X/dream-server-test).

Identical error on identical endpoint:

$ curl -s -X POST -H "Authorization: Bearer $KEY" http://127.0.0.1:3002/api/extensions/llama-server/disable
{"detail":"Cannot modify always-on service: llama-server"}   ← 403 ✓

$ curl -s -X POST -H "Authorization: Bearer $KEY" http://127.0.0.1:3002/api/extensions/n8n/disable
{"detail":"Extension not installed: n8n"}                    ← 404 ✗

I independently traced it to the exact same cause while writing up my own findings — line 1047 in enable_extension:

ext_dir = (USER_EXTENSIONS_DIR / service_id).resolve()
if not ext_dir.is_relative_to(USER_EXTENSIONS_DIR.resolve()):
    raise HTTPException(status_code=404, detail=f"Extension not found: {service_id}")
if not ext_dir.is_dir():
    raise HTTPException(status_code=404, detail=f"Extension not installed: {service_id}")

— missing the EXTENSIONS_DIR branch that _activate_service:991-1008 has.

This is the same gap in disable_extension:1153 (also USER_EXTENSIONS_DIR-only). Not cross-platform-specific, just cross-platform reproducible.

One additional wrinkle on the way down: even after fixing this lookup, _scan_compose_content:204 would still block most built-ins because their compose files declare services: {<service-name>} that matches CORE_SERVICE_IDS. The only built-in that slips past both checks is comfyui because its compose.yaml has a services: {} stub (the real services live in compose.nvidia.yaml/compose.amd.yaml overlays). So the full "first-class manageable extensions" promise of the PR requires fixes in at least three places:

1. enable_extension + disable_extension lookup (this issue)
2. _scan_compose_content needs a built-in trust bypass (or the check should be on the manifest id, not the internal compose services: map)
3. \+ the :ro mount conflict from bug: read-only extensions mount blocks built-in template activation — every catalog template apply fails 500 #331 once the rename path is reachable

FYI: my Mac test also hit the :ro mount crash (#331) when I forced the rename path. Filing my matching report under that issue.

(Runtime testing on macOS, integration branch = upstream/main + all 17 open yasinBursali PRs.)

[yasinBursali]

Test Results — 12APRdevelopments branch (commit 2e238b3), WSL2 Ubuntu 24.04.4

Status: NOT REPRODUCED — behavior appears correct for the current API

Tested:

POST /api/extensions/whisper/enable       →  200 OK  (action: enabled, restart_required: false)
POST /api/extensions/llama-server/enable  →  403  "Cannot modify always-on service: llama-server"
POST /api/extensions/open-webui/enable    →  403  "Cannot modify always-on service: open-webui"

_resolve_extension_dir() (extensions.py line 187) checks USER_EXTENSIONS_DIR first, then falls back to EXTENSIONS_DIR:

user_dir = (USER_EXTENSIONS_DIR / service_id).resolve()
if user_dir.is_relative_to(USER_EXTENSIONS_DIR.resolve()) and user_dir.is_dir():
    return user_dir
builtin_dir = (EXTENSIONS_DIR / service_id).resolve()
if builtin_dir.is_relative_to(EXTENSIONS_DIR.resolve()) and builtin_dir.is_dir():
    return builtin_dir

Built-in core services (llama-server, open-webui) are correctly blocked by _assert_not_core() with a 403. Extension services in EXTENSIONS_DIR (whisper, embeddings, etc.) work fine.

If the original issue was about a different scenario (e.g., EXTENSIONS_DIR env var pointing to a non-existent path), that would need a specific test setup to reproduce.

[yasinBursali]

Verified Fixed — 12APRdevelopments branch (commit 2e238b3)

Live test on WSL2:

POST /api/extensions/whisper/enable       →  200 OK  {"action": "enabled", ...}
POST /api/extensions/llama-server/enable  →  403  "Cannot modify always-on service: llama-server"
POST /api/extensions/open-webui/enable    →  403  "Cannot modify always-on service: open-webui"

Built-in extensions (whisper, tts, embeddings, n8n, etc.) are now correctly manageable via the API. Only the 4 truly always-on base-compose services (llama-server, open-webui, dashboard, dashboard-api) return 403.

_resolve_extension_dir() correctly falls back from USER_EXTENSIONS_DIR to EXTENSIONS_DIR — no more 404 on built-in extension enable/disable.

Fixed by upstream PRs Light-Heart-Labs#907 + Light-Heart-Labs#909.

Closing.