apply_template silently skips built-in (non-catalog) extensions — all 11 templates affected

[yasinBursali]

Bug Report: apply_template silently skips built-in extensions — all 11 templates affected

Severity: High
Category: Infra
Platform: All
Confidence: Confirmed

Description

POST /api/templates/{id}/apply silently skips any built-in extension that is not currently healthy. The response reports results[svc_id] = "skipped: Extension not installed: X" and restart_required: false with enabled_count: 0, which looks like a successful apply to the frontend even though nothing was enabled.

Affected File(s)

• dream-server/extensions/services/dashboard-api/routers/templates.py — apply_template(), _BASE_COMPOSE_SERVICES
• dream-server/extensions/services/dashboard-api/routers/extensions.py — _activate_service()

Root Cause

_activate_service(service_id) resolves ext_dir = USER_EXTENSIONS_DIR / service_id. Built-in extensions (n8n, litellm, langfuse, tts, whisper, openclaw, perplexica, searxng, privacy-shield, token-spy, comfyui) live in EXTENSIONS_DIR, not USER_EXTENSIONS_DIR. The check ext_dir.is_dir() returns False, raising HTTPException(404, "Extension not installed: n8n").

apply_template wraps the call in try/except HTTPException and records the result as "skipped: Extension not installed: n8n" — no user-visible error.

The comment in templates.py says _activate_service checks both user-installed and built-in extension dirs, but this is incorrect. The code only checks USER_EXTENSIONS_DIR.

_BASE_COMPOSE_SERVICES = frozenset({"llama-server", "open-webui", "dashboard", "dashboard-api"}) — correctly excludes always-running services but does not include other built-in extensions (n8n, litellm, etc.).

Evidence

# extensions.py:977
def _activate_service(service_id: str) -> dict:
    ext_dir = (USER_EXTENSIONS_DIR / service_id).resolve()   # ← only USER_EXTENSIONS_DIR
    if not ext_dir.is_dir():
        raise HTTPException(status_code=404, detail=f"Extension not installed: {service_id}")

Compare with _is_dep_enabled() at line 936, which correctly checks both dirs:

if (EXTENSIONS_DIR / dep / "compose.yaml").exists():   # ← correctly checks EXTENSIONS_DIR
    return True
if (USER_EXTENSIONS_DIR / dep / "compose.yaml").exists():
    return True

Services silently skipped per template

Template	Built-in services silently skipped if not healthy
ai-coding-workspace	n8n, openclaw
chat-playground	tts, whisper
creative-studio	comfyui, tts, whisper
developer-homelab	n8n, token-spy
llm-platform	litellm, langfuse, token-spy, n8n
ml-data-workshop	litellm, langfuse, n8n
multi-agent-laboratory	openclaw, langfuse, n8n
no-code-ai-builder	n8n
personal-knowledge-base	perplexica, searxng, whisper
private-research-hub	searxng, perplexica, privacy-shield
voice-assistant	whisper, tts, n8n

All 11 templates are affected.

Platform Analysis

All platforms. The template system is a dashboard-api feature with no platform-specific branching.

Reproduction

1. Have a DreamServer install where n8n is not running (or disable it: rename extensions/services/n8n/compose.yaml → compose.yaml.disabled)
2. POST /api/templates/no-code-ai-builder/apply with valid API key
3. Response shows "n8n": "skipped: Extension not installed: n8n" and enabled_count: 0
4. UI shows "Template applied" but n8n is not running

Impact

The template system's primary purpose — setting up groups of services with one click — is broken for built-in services that are not currently running. All 11 templates fail silently for a majority of their listed services on systems where those services aren't already healthy.

Suggested Approach

Add a built-in extension activation path in apply_template that checks EXTENSIONS_DIR / svc_id / "compose.yaml.disabled" and renames it to compose.yaml (same mechanism _activate_service uses for USER_EXTENSIONS_DIR). Mirror the pattern from _is_dep_enabled() which already correctly checks both directories.

---

Filed by automated infra auditor after full-sweep of changes merged 2026-04-06 → 2026-04-11 on upstream/main @ c0600ca.

[yasinBursali]

Status check from integration test of PR stack Light-Heart-Labs#893–909

Environment: WSL2 / Ubuntu 24.04 / NVIDIA RTX 3070 Laptop / Tier 1
Branch tested: local/integration-test containing all 17 open yasinBursali PRs merged --no-ff onto Light-Heart-Labs/DreamServer@c0600ca
Install: bash install.sh --all --non-interactive --no-comfyui → phase 13, 17 healthy containers

Verdict

Structurally fixed by PR Light-Heart-Labs#909 — but the fix is unreachable at runtime due to a separate :ro mount conflict introduced by PR Light-Heart-Labs#908. The original "silent skip" symptom is gone (templates now fail loudly), but they fail with the wrong error.

Evidence — _activate_service now checks both directories

PR Light-Heart-Labs#909 modifies _activate_service in routers/extensions.py (L985-1034) to check both USER_EXTENSIONS_DIR and EXTENSIONS_DIR. From the deployed dashboard-api container:

# routers/extensions.py L985+
def _activate_service(service_id: str) -> dict:
    """ ... Checks both USER_EXTENSIONS_DIR (user-installed) and EXTENSIONS_DIR
    (built-in) so templates can enable built-in extensions like n8n, tts, etc. ... """
    user_dir = (USER_EXTENSIONS_DIR / service_id).resolve()
    builtin_dir = (EXTENSIONS_DIR / service_id).resolve()

    in_user = user_dir.is_relative_to(USER_EXTENSIONS_DIR.resolve())
    in_builtin = builtin_dir.is_relative_to(EXTENSIONS_DIR.resolve())
    if not in_user and not in_builtin:
        raise HTTPException(status_code=404, detail=f"Extension not found: {service_id}")

    if in_user and user_dir.is_dir():
        ext_dir = user_dir
    elif in_builtin and builtin_dir.is_dir():
        ext_dir = builtin_dir
    else:
        raise HTTPException(status_code=404, detail=f"Extension not installed: {service_id}")
    ...
    os.rename(str(disabled_compose), str(enabled_compose))   # ← problem

This is exactly what this issue suggested. The lookup precedence is correct (user dir wins, built-in is fallback). The _BASE_COMPOSE_SERVICES frozenset has been replaced by ALWAYS_ON_SERVICES (this issue mentions the original was incorrectly used as a generic blocklist; PR Light-Heart-Labs#907 split the concepts — see #299 for the host-agent layer of that fix).

But it's unreachable at runtime — :ro mount blocks the rename

PR Light-Heart-Labs#908 (security fix for the .env :ro→writable regression, #317) restored :ro on the dashboard-api container's mounts, including extensions/:

$ docker inspect dream-dashboard-api | grep extensions
/home/rosenrot/dream-server/extensions -> /dream-server/extensions(ro)

When _activate_service runs the os.rename at L1032, the destination is on a read-only filesystem and the call fails:

$ curl -X POST -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
    -d '{}' http://127.0.0.1:3002/api/templates/creative-studio/apply
Internal Server Error

File "/app/routers/templates.py", line 188, in apply_template
    deps_enabled, result = await asyncio.to_thread( ... )
File "/app/routers/templates.py", line 136, in _activate_with_lock
    main_result = _activate_service(sid)
File "/app/routers/extensions.py", line 1032, in _activate_service
    os.rename(str(disabled_compose), str(enabled_compose))
OSError: [Errno 30] Read-only file system:
    '/dream-server/extensions/services/comfyui/compose.yaml.disabled'
    -> '/dream-server/extensions/services/comfyui/compose.yaml'

So the symptom in this issue's body — "silently skips ... restart_required: false ... enabled_count: 0 ... looks like a successful apply" — is now replaced by a hard 500. Better in the sense that the failure is no longer silent. Worse in the sense that every catalog template fails (verified across all 11 listed templates: every one includes at least one built-in extension).

Filed as #331. The suggested fix there is to mirror PR Light-Heart-Labs#908's pattern: add a host-agent endpoint like POST /v1/extension/activate and have _activate_service delegate the rename to it, the same way PR Light-Heart-Labs#908 delegates .env writes. That preserves both #317's security boundary and this issue's built-in resolution capability.

Adjacent gap — public enable/disable HTTP endpoints still don't recognize built-ins

Beyond _activate_service (the internal helper used by template apply), the public enable_extension (L1037) and disable_extension (L1147) handlers in the same file STILL only look in USER_EXTENSIONS_DIR. So POST /api/extensions/n8n/disable returns 404 Extension not installed: n8n even though dream-n8n is healthy and the directory exists. Filed as #333. PR Light-Heart-Labs#907 took care of the host-agent layer (verified working — see comment on #299) but the dashboard-api HTTP layer was only partially updated.

Test gap that allowed this to land

The unit test test_activate_service_resolves_builtin_with_disabled_compose (added by PR Light-Heart-Labs#909) uses monkeypatch to set EXTENSIONS_DIR and USER_EXTENSIONS_DIR to tmp_path-based dirs, which are always writable. The test correctly validates the rename logic in isolation but cannot detect the :ro mount conflict that Light-Heart-Labs#908 introduces in production. An integration test that mounts the real container (or at least asserts mount mode against the compose file) would have caught both #320's failure mode and the new #331 failure mode at the same time.

Summary

• ✅ PR fix(dashboard-api): non-blocking apply_template + resolve built-in extensions Light-Heart-Labs/DreamServer#909 adds the EXTENSIONS_DIR fallback as this issue requested
• ✅ The "silent skip" no longer happens
• ❌ The fix is unreachable due to the :ro mount that PR fix(security): route .env writes through host agent, restore :ro mount Light-Heart-Labs/DreamServer#908 (separately) restored
• ❌ Public HTTP enable/disable endpoints have the same lookup bug — see bug: dashboard-api enable/disable HTTP endpoints don't fall back to EXTENSIONS_DIR — built-in extensions return 404 #333

This issue should probably stay open until the #331 fix lands, since the user-facing capability isn't actually delivered yet.

---

Filed during full-stack integration test of open PR stack Light-Heart-Labs#893–909 on Light-Heart-Labs/DreamServer@c0600ca3. Cross-referenced from #331, #333.

[yasinBursali]

Addressed by PR Light-Heart-Labs#909 (currently draft, pending upstream review).