Dashboard API enable/activate doesn't invalidate .compose-flags cache — CLI sees stale stack

[yasinBursali]

Problem

When the dashboard API enables an extension (_activate_service in routers/extensions.py), it renames compose.yaml.disabled → compose.yaml but does not invalidate .compose-flags.

The CLI's `get_compose_flags` reads .compose-flags from cache (fast path in dream-cli:295) — which was written at install time and only regenerated when the CLI itself toggles extensions (cmd_enable/cmd_disable via _regenerate_compose_flags).

Result: after API-driven enable, any subsequent CLI command (`dream start`, `dream restart`, `dream stop`) operates on a stale stack that doesn't include the newly enabled services.

Reproduction

1. Install DreamServer, some extensions disabled (e.g., tts, whisper)
2. From the dashboard UI, click Enable on tts or apply a template containing tts
3. Dashboard API calls _activate_service("tts") which renames extensions/services/tts/compose.yaml.disabled → compose.yaml
4. Run `dream start` from the terminal
5. Bug: tts is not started. Docker reports dream-tts as an orphan container. The resolved compose stack doesn't include the tts compose file.
6. Root cause: .compose-flags still contains the old flag list from install time, without extensions/services/tts/compose.yaml

Impact

• PR 2 (service templates) directly triggers this: template apply renames compose files but doesn't update the cache, so the required follow-up `dream start` misses the newly enabled services.
• Any API-driven extension toggle silently breaks CLI operations until the user manually removes `.compose-flags` or `dream disable && dream enable` the same service.

Fix Options

Option A: Invalidate cache from API

Have _activate_service (and any other API path that toggles compose files) remove or regenerate .compose-flags after the rename. Simplest fix:
```python
import os
compose_flags = INSTALL_DIR / ".compose-flags"
if compose_flags.exists():
compose_flags.unlink()
```
This falls through to dynamic resolution on next CLI invocation.

Option B: Drop the cache entirely

.compose-flags was added as an install-time fast path. The resolver is fast enough at runtime that the cache may not be worth the cache-invalidation complexity. Remove get_compose_flags fast path, always resolve dynamically.

Option C: Use mtime-based cache invalidation

Compare the cache file mtime against all extensions/services/*/compose.yaml* mtimes. Regenerate if any extension compose file is newer.

Recommended

Option A — it's minimal and correct. The dashboard API already imports INSTALL_DIR from config. Add cache invalidation to both _activate_service (enable path) and disable_extension (disable path).

Files Affected

• `dream-server/extensions/services/dashboard-api/routers/extensions.py` — `_activate_service`, `disable_extension`
• `dream-server/extensions/services/dashboard-api/routers/templates.py` — indirectly via `_activate_service` calls
• `dream-server/dream-cli` — already correct (`_regenerate_compose_flags` is called from CLI enable/disable paths)

[yasinBursali]

Addressed by upstream PR Light-Heart-Labs#890 — fix(dashboard-api): invalidate .compose-flags via host agent on mutations.

[yasinBursali]

Reopening — PR Light-Heart-Labs#890 only partially fixed this

I closed this issue earlier today after matching it to upstream PR Light-Heart-Labs#890 (fix(dashboard-api): invalidate .compose-flags via host agent on mutations). After running a fresh QA pass against a local QA branch built from upstream/main + the merged PRs, the bug still reproduces on one specific path of enable_extension.

Root cause

enable_extension in dream-server/extensions/services/dashboard-api/routers/extensions.py has two execution branches:

1. Stopped-start branch (lines 1043–1064) — fires when the extension's compose.yaml is already on disk (the case for library extensions like jupyter that ship with an active compose.yaml, not compose.yaml.disabled). This branch calls _call_agent("start", service_id) and _write_initial_progress(...) and then returns at line 1056 — without ever calling _call_agent_invalidate_compose_cache().

2. Activate branch (lines ~1083–1101) — fires when compose.yaml.disabled has to be renamed to compose.yaml. This branch correctly calls _call_agent_invalidate_compose_cache() at line 1101, gated by if enabled_services:.

PR Light-Heart-Labs#890 added the invalidate call to branch 2 but missed branch 1. Both branches return the same {"action": "enabled", ...} response, so the API caller cannot distinguish them.

Reproducer

On a freshly installed stack (commit cd11612 + Light-Heart-Labs#890/Light-Heart-Labs#891/Light-Heart-Labs#892 merged):

KEY=$(grep '^DASHBOARD_API_KEY=' ~/dream-server/.env | cut -d= -f2)
API=http://127.0.0.1:3002

# 1. Install jupyter (library extension that ships with compose.yaml, not .disabled)
curl -sf -H "Authorization: Bearer $KEY" -X POST $API/api/extensions/jupyter/install
test ! -e ~/dream-server/.compose-flags && echo "install invalidated cache OK"

# 2. Plant a known cache file so we can detect whether enable touches it
bash ~/dream-server/scripts/resolve-compose-stack.sh \
  --script-dir ~/dream-server --tier 1 --gpu-backend nvidia \
  > ~/dream-server/.compose-flags
md5sum ~/dream-server/.compose-flags
# -> e.g. 982bae4fa6b7a9222d64cb91a17b8e21

# 3. Enable jupyter
curl -sf -H "Authorization: Bearer $KEY" -X POST $API/api/extensions/jupyter/enable
# -> {"id":"jupyter","action":"enabled","restart_required":false,"message":"Extension started."}

# 4. The cache file should be GONE per PR #890's contract. But it isn't.
md5sum ~/dream-server/.compose-flags
# -> 982bae4fa6b7a9222d64cb91a17b8e21  (byte-identical to step 2)

By contrast, disable, uninstall, and install all correctly invalidate the cache after this same PR — those handlers call _call_agent_invalidate_compose_cache() at lines 1189, 1252, and 886 respectively.

Why the original test missed it

PR Light-Heart-Labs#890's added unit tests cover the path where compose.yaml.disabled exists. The library extensions that ship with an active compose.yaml go through a different codepath that wasn't exercised. Any user who installs a library extension and then immediately calls enable (e.g., via the Dashboard "Install → Enable" flow on a freshly catalog-installed extension) hits the gap.

Suggested fix

Add _call_agent_invalidate_compose_cache() to the stopped-start branch inside _extensions_lock(), before _call_agent("start", service_id):

# Stopped case: compose.yaml exists but container is not running — just start it
if enabled_compose.exists():
    with _extensions_lock():
        st = os.lstat(enabled_compose)
        if stat.S_ISLNK(st.st_mode):
            raise HTTPException(...)
        _scan_compose_content(enabled_compose)
    _write_initial_progress(service_id)
    _call_agent_invalidate_compose_cache()   # <-- ADD
    agent_ok = _call_agent("start", service_id)
    ...

The lock release between _scan_compose_content and the cache invalidation is fine because the existing call sites also invalidate outside the per-mutation lock.

Severity

Medium. Same severity profile as the original #298 — the CLI sees a stale compose set after an enable, until something else regenerates the cache. PR Light-Heart-Labs#890 reduced the surface but didn't close it for the library-extension install→enable path, which is the most common end-user flow.

Independently validated by both static source review (4 invalidate call sites at lines 886/1101/1189/1252; the L1101 site is in a branch that this enable path bypasses) and runtime reproduction with the planted-cache test above.

[yasinBursali]

Verification Result — CONFIRMED (deployed image) / SUPERSEDED (source)

Verified against: upstream/main @ c0600ca + live install at /Volumes/X/dream-server-test
Platform scope: All (macOS + Linux — same Python router on both)

---

Evidence

Static analysis — running Docker container (dream-dashboard-api)

disable_extension in running container (extracted via docker exec):

# lines 1139–1159 of /app/routers/extensions.py (running image)
    with _extensions_lock():
        st = os.lstat(enabled_compose)
        if stat.S_ISLNK(st.st_mode):
            raise HTTPException(...)
        os.rename(str(enabled_compose), str(disabled_compose))   # ← rename happens
                                                                   # ← NO cache invalidation
    logger.info("Disabled extension: %s", service_id)
    return {"id": service_id, "action": "disabled", ...}

enable_extension in running container (lines 1050–1070):

    with _extensions_lock():
        for dep in missing_deps:
            result = _activate_service(dep)          # renames compose.yaml.disabled → compose.yaml
            ...
        result = _activate_service(service_id)       # same
        if result.get("action") == "enabled":
            enabled_services.append(service_id)
    # ← NO _call_agent_invalidate_compose_cache() — jumps directly to agent start loop
    for svc_id in enabled_services:
        if not _call_agent("start", svc_id): ...

Confirmed: grep -n '_call_agent_invalidate_compose_cache' /app/routers/extensions.py exits 1 (zero matches).

---

Live reproduction

Setup: baserow installed at data/user-extensions/baserow/compose.yaml (enabled).

.compose-flags before API call:

mtime: Apr 11 04:10:21 2026  size: 889
...  -f data/user-extensions/baserow/compose.yaml   ← listed

API call: POST /api/extensions/baserow/disable

{"id": "baserow", "action": "disabled", "restart_required": true}

Filesystem after:

• data/user-extensions/baserow/compose.yaml.disabled — rename succeeded ✓
• .compose-flags mtime: Apr 11 04:10:21 2026 (unchanged)
• .compose-flags still contains: -f data/user-extensions/baserow/compose.yaml — stale entry pointing to a file that no longer exists ✗

Same result for enable path: re-enabled baserow, .compose-flags mtime still unchanged.

Impact: CLI's get_compose_flags() fast path (dream-cli line 371–376) returns the stale cache. Any subsequent dream start / restart / stop passes a -f data/user-extensions/baserow/compose.yaml flag for a file that doesn't exist, causing docker compose to error.

---

Source at c0600ca — fix IS present

The fix was already merged into upstream/main. Both handlers call _call_agent_invalidate_compose_cache():

# extensions.py — enable_extension (lines 1099–1101 in fresh-install source)
        # Invalidate .compose-flags cache so dream-cli picks up the new enabled set
        if enabled_services:
            _call_agent_invalidate_compose_cache()

# extensions.py — disable_extension (line 1189 in fresh-install source)
        os.rename(str(enabled_compose), str(disabled_compose))
        _call_agent_invalidate_compose_cache()          # ← fix

# templates.py — apply_template (lines 150, 178)
        _call_agent_invalidate_compose_cache()

Host agent endpoint (dream-host-agent.py lines 114–116 + 606–617):

def invalidate_compose_cache() -> None:
    (INSTALL_DIR / ".compose-flags").unlink(missing_ok=True)   # simple delete

---

Inaccuracy in issue description

The reproduction steps reference extensions/services/tts/compose.yaml.disabled. This is the built-in extension path. _activate_service and enable_extension operate exclusively on USER_EXTENSIONS_DIR (~/.dream-server/user-extensions/), not on extensions/services/. The bug affects user-installed extensions (baserow, and anything installed from the library). Built-in extensions in extensions/services/ are toggled by a separate path that was not traced here.

---

Status

The code fix is in source at c0600ca. The deployed Docker image (dream-dashboard-api) was built from an older commit and does not contain the fix. Rebuilding and redeploying the image from c0600ca will resolve this — no new code changes needed.

---

Verified via static analysis + live runtime test on 2026-04-11. Automated verifier.

[yasinBursali]

Addressed by upstream draft PR Light-Heart-Labs#904.

[yasinBursali]

Verified at runtime — PR Light-Heart-Labs#904's cache invalidation holds on the integration branch

Environment: WSL2 / Ubuntu 24.04 / NVIDIA RTX 3070 Laptop / Tier 1
Branch: local/integration-test (PR stack Light-Heart-Labs#893–909 merged onto Light-Heart-Labs/DreamServer@c0600ca)
Install: bash install.sh --all --non-interactive --no-comfyui → phase 13, 17 healthy containers

What I tested

Source-level inspection of the deployed dream-dashboard-api container's routers/extensions.py. Looked for _call_agent_invalidate_compose_cache() call sites.

Result

$ docker exec dream-dashboard-api grep -n "_call_agent_invalidate_compose_cache\|invalidate_compose_cache" /app/routers/extensions.py
425:def _call_agent_invalidate_compose_cache() -> None:
427:    url = f"{AGENT_URL}/v1/compose/invalidate-cache"
891:        _call_agent_invalidate_compose_cache()
1073:        _call_agent_invalidate_compose_cache()
1121:        _call_agent_invalidate_compose_cache()
1209:        _call_agent_invalidate_compose_cache()
1272:        _call_agent_invalidate_compose_cache()

The cache-invalidate helper is defined at L425 and called from 5 sites covering the enable/disable/install/uninstall mutation paths. The L1073 site is exactly what PR Light-Heart-Labs#904 added: invalidate the compose-flags cache before the host-agent starts a stopped extension, so dream-cli picks up the new compose state on the next read.

The host-agent's /v1/compose/invalidate-cache endpoint is also reachable end-to-end (verified during the Light-Heart-Labs#908 PUT test in another part of this test pass — host-agent journal shows successful invalidate-cache calls).

Status

PR Light-Heart-Labs#904's fix holds. This issue can stay closed.

What was NOT tested

End-to-end "disable, modify state, re-enable, observe stale-cache absence" against an actually-installed library extension. Would need to install something like baserow from the extensions library first. I'll do that as part of a separate runtime probe and add a follow-up here if I find anything unexpected.

---

Verified during full-stack integration test of open PR stack Light-Heart-Labs#893–909.

[yasinBursali]

Follow-up: end-to-end disable→enable cycle on a real library extension

In my earlier comment I noted I had source-level evidence (5 call sites for _call_agent_invalidate_compose_cache()) but had not exercised the disable→re-enable path end-to-end against a real installed extension. Just did that.

Test

Installed baserow from the extensions library (smallest test target), then disabled and re-enabled it via the dashboard-api, and counted cache-invalidate events in the host-agent journal.

$ KEY=$(grep ^DASHBOARD_API_KEY= ~/dream-server/.env | cut -d= -f2)
$ H="Authorization: Bearer $KEY"

# 1. Install
$ curl -X POST -H "$H" -H "Content-Type: application/json" -d '{}' \
    http://127.0.0.1:3002/api/extensions/baserow/install
{"id":"baserow","action":"installed","restart_required":false,"progress_endpoint":"/api/extensions/baserow/progress","message":"Extension installed and starting."}

# 2. Wait for image pull + container start (~90s)
$ docker ps --filter name=baserow
dream-baserow  Up About a minute (health: starting)

# 3. Disable
$ curl -X POST -H "$H" http://127.0.0.1:3002/api/extensions/baserow/disable
# (response varies — see "UX observation" note below)
$ ls ~/dream-server/data/user-extensions/baserow/compose.yaml*
.../compose.yaml.disabled                     # ← rename happened
$ docker ps -a --filter name=baserow
dream-baserow  Exited (137) ...               # ← container stopped

# 4. Re-enable (this drives the L1073 stopped-restart path that #904 added)
$ curl -X POST -H "$H" http://127.0.0.1:3002/api/extensions/baserow/enable
{"id":"baserow","action":"enabled","enabled_services":["baserow"],"restart_required":false,"message":"Extension enabled and started."}
$ docker ps --filter name=baserow
dream-baserow  Up Less than a second (health: starting)

Cache-invalidate count

$ journalctl --user -u dream-host-agent.service --since "5 minutes ago" \
    | grep -c "compose-flags cache invalidated"
4

Four cache-invalidate events fired across the install + disable + re-enable cycle:

1. Install — compose-flags cache invalidated (the install path PR fix(restart,extensions): use compose up -d and invalidate cache on enable Light-Heart-Labs/DreamServer#904 doesn't touch directly, but the host-agent invalidates anyway)
2. First disable — compose-flags cache invalidated immediately after the rename compose.yaml → compose.yaml.disabled completes
3. Re-enable — compose-flags cache invalidated BEFORE the host-agent's start extension: baserow call, exactly as PR fix(restart,extensions): use compose up -d and invalidate cache on enable Light-Heart-Labs/DreamServer#904 specifies for the stopped-restart path at L1073
4. Re-enable's start hook — compose-flags cache invalidated again as part of the start handshake

The "Extension enabled and started" response message and the restart_required: false flag confirm the re-enable took the L1073 stopped-restart path that PR Light-Heart-Labs#904 added — the path that says "compose.yaml exists, container is stopped, just start it without doing a full enable rename, but invalidate cache first." This is exactly the symptom this issue (#298) describes — and the new path correctly invalidates cache before the start, so dream-cli (which reads .compose-flags) sees the updated state on the next read.

UX observation worth noting (not blocking, not the core bug)

The first disable curl I issued timed out the client at 10s while baserow was still in "health: starting" state. The host-agent log shows the disable handler took ~11s to complete the agent-stop call (the container was newborn and not ready to gracefully shut down). Once the second disable hit (after the container was stopped), it returned Extension already disabled: baserow instantly. No data corruption — the disable did complete server-side after the curl client gave up — but the response time is long enough that a real user might re-click the disable button and create a confusing sequence.

This isn't a #298 / Light-Heart-Labs#904 issue per se; it's a "graceful stop on a not-yet-healthy container is slow" timing observation. Probably worth a separate follow-up if you care about the dashboard UX. Mentioning here so it's discoverable.

Status

PR Light-Heart-Labs#904's fix is solid end-to-end. The L1073 stopped-restart cache invalidate fires at the right place, dream-cli will see fresh state immediately after enable. This issue can definitely stay closed.

Cleaning up baserow now (uninstall) so I don't leave test state behind.

---

Verified during full-stack integration test of open PR stack Light-Heart-Labs#893–909.