Support context variables in the Elastic Agent policy output section

[blakerouse]

Describe the enhancement:

At the moment variables from composable providers are supported in the inputs.* only. Support for this variables outside of the inputs.* is required to allow other areas, like outputs or fleet to use values.

Issue with this change is at the moment go-ucfg provides the variable replacement outside of inputs.* and that supports the a very similar syntax that will conflict. To solve this issue we can add a fallback where if no prefix of the provider is provided for a variable then it will default to the env provider which is all the go-ucfg supports at the moment. That allows the replacement of go-ucfg outside of inputs.*. Removing the need to support any variable replacement from go-ucfg.

Describe a specific use case for the enhancement or feature:

This is needed to allow provides to provide variables outside of inputs, explicitly because provider variables can change through out the live of the Elastic Agent.

What is the definition of done?

When all context providers work outside of inputs. and go-ucfg variable replacement can be removed.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[blakerouse]

I have been looking into this more and this will need to be scoped to certain sections in the policy. Like variables should not be usable in providers as that would be using variables to affect variables.

I think it would be best to just say it is allowed in these areas. outputs, inputs, fleet, but not agent.

[nimarezainia]

@blakerouse can you please help me understand how this would apply to an outputs definition? (as an example)
and could you elaborate on when you say that its allowed in "fleet but not agent" - I'm getting wrapped around the axle on that comment. All config from fleet ends up on the agent.

[blakerouse]

@nimarezainia I am speaking of the keys in the configuration, sorry that was not clear. The following top-level keys would allow the ability for variables to be present.

• outputs.*
• inputs.*
• fleet.*

These following keys would not allow variables to be present:

• agent.*
• providers.*

[blakerouse]

I updated this issue for to just be for the output section of the policy as the fleet.* section is a little different and I filed specific issues for that.