Open
Description
@fregataa's contribution of dummy agent backends (#1283) has allowed scalability tests for large-scale clusters.
Though, it is still cumbersome to actually configure and spawn hundreds of agents because we have to copy-and-edit agent.toml
and execute ag start-server
for hunderds times. Of course, we could automate this process by scripting, but it poses a psychological hurdle to start.
As an extension of #496, let's allow a single agent to host multiple agent backend instances from a single configuration file.
- Allow Jinja templating of
agent.toml
to easily populate multiple backend configurations (including dummies).- Expand the structure of agent's local-config to handle multiple backend instances.
- Store the rendered config file to
/tmp
(while parsing it in memory), so that it could be inspected by engineers and admins. - Let's keep backward-compatibility to existing
agent.toml
files.
- Pass the agent ID when invoking the agent RPC calls and route the call to the designated backend instance.
- From the manager's perspective, there are multiple agents sharing the same physical RPC endpoint.
For "real" backends, we need to resolve #496 first to get it actually working, but for "dummy" backends we can start right now.
- In production, there won't be more than 2-3 backend instances per agent and only single one in most cases.
- In scalability testing, there may be hundreds and thousands of backend instances multiplexed via asyncio in a single process and single thread.