Skip to content

Prompts: rework get_prompt() to read from EngineConfig (mirror MCP registry pattern) #623

Description

@ahmedennaifer

Problem

idun_agent_engine.prompts.helpers.get_prompt() resolves through YAML file → Manager API only. The DB-backed EngineConfig.prompts assembled by idun_agent_standalone.services.engine_config.assemble_engine_config() on every boot and reload is never consulted by user agent code that calls get_prompt("system_prompt").

Symptoms:

  • Editing a prompt via /admin/api/v1/prompts mutates the DB, but user code keeps reading YAML values.
  • User code that calls get_prompt(...) at module import time snapshots the YAML value forever — even /reload doesn't update it.
  • The constitution says "DB is steady-state truth in standalone. YAML seeds at first boot only." Today's helper violates that for prompts.

Direction

Follow the MCP registry pattern that already exists in the codebase. See libs/idun_agent_engine/src/idun_agent_engine/mcp/registry.py:

```python
_active_registry: MCPClientRegistry | None = None

def set_active_registry(registry: MCPClientRegistry | None) -> None: ...
def get_active_registry() -> MCPClientRegistry | None: ...
```

Mirror for prompts:

```python

libs/idun_agent_engine/src/idun_agent_engine/prompts/registry.py

_active_prompts: list[PromptConfig] | None = None

def set_active_prompts(prompts: list[PromptConfig] | None) -> None: ...
def get_active_prompts() -> list[PromptConfig] | None: ...
```

Wire-up:

  • Bootlibs/idun_agent_standalone/src/idun_agent_standalone/app.py: after assemble_engine_config(session) returns, call set_active_prompts(engine_config.prompts).
  • Reloadlibs/idun_agent_standalone/src/idun_agent_standalone/services/reload.py:commit_with_reload: in the success branch after reload_callable(assembled) returns, call set_active_prompts(assembled.prompts). Never in the failure branch (rollback preserves the prior prompts).

Resolution order in get_prompt() (and get_prompts()):

  1. Explicit config_path= arg (tests, scripts).
  2. get_active_prompts() if not None — standalone path (new).
  3. IDUN_CONFIG_PATH YAML (bare engine + YAML, unchanged).
  4. Manager API (SaaS / Manager path, unchanged).

This keeps all three deployment shapes working, adds the in-process snapshot for standalone, and matches the MCP precedent so the next contributor reads it as a familiar pattern rather than a new abstraction.

Acceptance criteria

  • prompts/registry.py ships with set_active_prompts + get_active_prompts, mirroring mcp/registry.py shape.
  • get_prompt() resolution order updated; YAML/Manager paths untouched for non-standalone deployments.
  • Standalone boot writes set_active_prompts(engine_config.prompts) after assembly.
  • Standalone commit_with_reload writes set_active_prompts(assembled.prompts) in the success branch only.
  • Unit test: standalone test boots, edits a prompt via admin REST, calls get_prompt(...) and observes the new value (no `/reload` needed beyond the normal admin pipeline).
  • Unit test: bare-engine YAML mode still resolves through IDUN_CONFIG_PATH.
  • Unit test: SaaS Manager mode still resolves through get_prompts_from_api().
  • Autouse test fixture clears _active_prompts between tests so state doesn't bleed.

Out of scope

  • Prompt versioning / labels (Langfuse-style `label='production'` semantics). Track separately if wanted.
  • HTTP self-call to /admin/api/v1/prompts — explicitly rejected, see design discussion.
  • DB-direct access from a sync sessionmaker in standalone — rejected in favor of in-memory snapshot.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions