Multiple agent instances on a single host

[cedricdv]

It should be possible to run multiple instance of elastic-agent, each with their own configuration, on a single host.

The use case is ELKs own popularity: with the growing adoption of ELK for monitoring, more and more users will be faced with the situation where there are 2 or more independent parties that want to monitor the same machine.

Definition of done: when multiple agents, both with elevated privileges, with their own config and fleet connection, can coexist on the same host.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[cmacknz]

CC @nimarezainia for a product opinion.

At a technical level, the groundwork for this feature already exists: https://github.com/elastic/elastic-agent?tab=readme-ov-file#development-installations

[nimarezainia]

@cedricdv could you elaborate a bit more on what you mean by "their own configuration"?

if the goal here is to provide multi-tenancy of sorts from a management perspective, once we complete Space awareness for Fleet, we will move to providing the same in Integrations app also, then each team would won their own set of integrations. If you need "their own configuration" to be sent to a different pipeline, you can achieve that by routing in Logstash or using Kafka.

Based on your description, sounds like that collection should happen once, but the data collected to be curated for different teams.

[cedricdv]

With "Their own config" I mean a separate fleet agent instance with its own fleet configuration and set of integrations. For full context: the business case is one where we support our software running on machines managed by our customer. They use elastic agent for system and security purposes, we use it for application insight. But since they already have an agent running for their own system management, we are currently unable to deploy our own agent for monitoring the applications.

[nimarezainia]

@cedricdv so you can't be a tenant on their instance of the agent? as in, this is not a multi-tenancy use case from a management perspective.

If you could install multiple agents on the same host, The reality of it is that the Agent Policy would most probably be exactly the same for those policies. Obviously the integrations and their settings will vary. So if we provided space awareness for integrations, your team could carve out a space for the integrations you own and the agent policy would be owned by the customer who manages the setup. This will all depend on the answer to the question above. Sounds like they are not willing (or it's not part f the business model) to carve out a space for your team.

[cedricdv]

We have defined custom log ingests in our fleet policies to deal with our applications logs. So, asking our customer to support us with logs is not really doable because we'd have to ask them to fix us an update each time we change something to the logging in our applications.

The monitoring and reporting needs from our customer are related to system and security management, they want to make sure that the machines run smooth and securely. Our needs are focused around our applications and their health: how many items are in specific queues? how many errors of this type do we get across all customers on this version? So yes, in reality there will be some overlap, for example in the system metrics and logging, but for the rest the policies will be very different from each other depending on the reporting needs of each party.

Sharing an instance of Agent supporting multiple fleet configurations would also not be a perfect match since supported integrations are tied to the Agent version and this in turn is to the fleet server and Kibana version. Each party will want to keep versions of it's ELK stack under it's own control. So it would be preferable that each organization can run it's own instance of elastic agent, possibly even running different versions of the agent executable.