Skip to content

Allow agent to have multiple backend instances #2562

Open
@achimnol

Description

@achimnol

@fregataa's contribution of dummy agent backends (#1283) has allowed scalability tests for large-scale clusters.

Though, it is still cumbersome to actually configure and spawn hundreds of agents because we have to copy-and-edit agent.toml and execute ag start-server for hunderds times. Of course, we could automate this process by scripting, but it poses a psychological hurdle to start.

As an extension of #496, let's allow a single agent to host multiple agent backend instances from a single configuration file.

  • Allow Jinja templating of agent.toml to easily populate multiple backend configurations (including dummies).* Expand the structure of agent's local-config to handle multiple backend instances.

  • Store the rendered config file to /tmp (while parsing it in memory), so that it could be inspected by engineers and admins.

  • Let's keep backward-compatibility to existing agent.toml files.

  • Pass the agent ID when invoking the agent RPC calls and route the call to the designated backend instance.* From the manager's perspective, there are multiple agents sharing the same physical RPC endpoint.

For "real" backends, we need to resolve #496 first to get it actually working, but for "dummy" backends we can start right now.

  • In production, there won't be more than 2-3 backend instances per agent and only single one in most cases.
  • In scalability testing, there may be hundreds and thousands of backend instances multiplexed via asyncio in a single process and single thread.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions