Skip to content

Allow agent to have multiple backend instances #2562

Open
@achimnol

Description

@fregataa's contribution of dummy agent backends (#1283) has allowed scalability tests for large-scale clusters.

Though, it is still cumbersome to actually configure and spawn hundreds of agents because we have to copy-and-edit agent.toml and execute ag start-server for hunderds times. Of course, we could automate this process by scripting, but it poses a psychological hurdle to start.

As an extension of #496, let's allow a single agent to host multiple agent backend instances from a single configuration file.

  • Allow Jinja templating of agent.toml to easily populate multiple backend configurations (including dummies).
    • Expand the structure of agent's local-config to handle multiple backend instances.
    • Store the rendered config file to /tmp (while parsing it in memory), so that it could be inspected by engineers and admins.
    • Let's keep backward-compatibility to existing agent.toml files.
  • Pass the agent ID when invoking the agent RPC calls and route the call to the designated backend instance.
    • From the manager's perspective, there are multiple agents sharing the same physical RPC endpoint.

For "real" backends, we need to resolve #496 first to get it actually working, but for "dummy" backends we can start right now.

  • In production, there won't be more than 2-3 backend instances per agent and only single one in most cases.
  • In scalability testing, there may be hundreds and thousands of backend instances multiplexed via asyncio in a single process and single thread.

Metadata

Assignees

No one assigned

    Labels

    comp:agentRelated to Agent componenturgency:1If no other duties are available, volunteer to help.

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions