Scaling an ADK Assistant to 100+ Programs

[suryasinghmv]

We currently have an ADK-based learning assistant implemented for a single program. The assistant works well, but the current setup assumes a one-to-one relationship between a program and the server/runtime (program-specific prompts and learning data are coupled to the deployment).

We are now looking to scale this learning assistant to support 100+ mastery programs, where:

1. The agent logic and sub-agents remain the same
2. Only the prompts, tone/persona, and learning data differ per program
3. We want to avoid deploying 100 separate services or containers

The key challenge is designing a clean, maintainable way for a single ADK runtime to dynamically load:

program-specific configuration (prompts, persona, rules)

program-specific learning data (lessons, transcripts, precomputed artifacts)

…at runtime, based on a program_id, without introducing brittle branching logic, duplicated code, or performance regressions.

What We’re Considering

Our current thinking is to:

1. Treat each program as configuration + data, not code
2. Keep a single ADK learning assistant runtime
3. Load program manifests and data dynamically at request time (with caching)
4. Use shared prompt templates with program-specific variables

Before locking this in, we’d love to sanity-check this approach with the community.

[llalitkumarrr]

This seems to be a good strategy. However, for the best user experience, please keep a close watch on latency. It could be helpful to design a root agent that oversees multiple sub-agents, each sub-agent could then handle specific program data and tools. I suggest looking into RAG+CAG for this problem and refining the instructions will be extremely important.