Injected proxy variables into endpoint elasticsearch does not include NO_PROXY variable

[janniten]

Hello,
We did an upgrade from 8.14.2 to 8.16.1.
In our environment the agent has the HTTP_PROXY, HTTPS-PROXY and NO_PROXY configured.
NO_PROXY variable is not being injected into endpoint .
As a result, the elastic endpoint is trying to connect the ES cluster Via proxy, which is not correct.

Thank you

• Version: 8.16.1
• Operating System: Windows / linux
• Discuss Forum URL: https://elasticstack.slack.com/archives/C02J2JBS0FP/p1733218957317399
• Steps to Reproduce: Upgrade from 8.14.2 to 8.16.1

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[michel-laterman]

HTTP_PROXY/HTTPS_PROXY injection was added with: #5044
We'll probably need to alter the injectProxyURL func to add NO_PROXY.

@intxgo, would the expected value in the config be no_proxy: string, or no_proxy: []string?

[intxgo]

Since the env vars are injected as a regular output proxy settings, there's no place for NO_PROXY (endpoint won't recognize it, ignore it). However since the output URL is known, I think the NO_PROXY can be matched against the output URL at Agent side before deciding to inject HTTP_PROXY/HTTPS_PROXY variable to the output config.

[intxgo]

Alternative solution would be to pass all three HTTP_PROXY, HTTPS_PROXY, NO_PROXY to Endpoint as Agent env variables, a new structure in the config message instead of injecting them to the output configuration. We will need to handle this on Endpoint side too, but in general Endpoint proxy table already has place for storing Agent env variables.

[ManojPraveeen]

Hi Team,

The issue is related to case 01791476. From what I understand from the @intxgo statement, you’re suggesting including these proxy configurations directly in the agent's environment settings, right?

It would look something like the configuration below:

proxy_settings:
  http_proxy: 
  https_proxy:
  no_proxy: 

Adding the above parameters to the configuration should fix the issue, correct me if I’m wrong.

[intxgo]

@ManojPraveeen it's more complicated. Since quite some time we're not forwarding the config between components anymore, instead we have a GRPC base communication, see demo client https://github.com/elastic/elastic-agent-client

The GRPC is roughly equivalent to the config, however currently the protobuf definition doesn't have a place to forward the Agent env proxy settings, so instead Agent's env variables (only the first 2) are evaluated at Agent side and injected as regular proxy setting as if it came from Fleet. If we follow the path to revert the injection, introducing a proper means to forward all the variables via protobuf, we will have to update Agent and Endpoint to support the change. BTW, since these are env variables, they won't be ever stored in Endpoint vault as they are volatile.

The other option, a quicker patch to existing solution, was to evaluate all three env variables at Agent side to see if NO_PROXY nullifies HTTP_PROXY setting for the Elasticsearch address.

[ManojPraveeen]

@intxgo
From what I understand the Elastic Agent is currently only uses the HTTP_PROXY and HTTPS_PROXY settings. You want to implement a logic from agent end, So when the agent tries to connect directly to Elasticsearch using NO_PROXY, it should bypass the proxy settings. Is that correct?

If that's what you are saying, could you explain how we can achieve this workaround?

[intxgo]

@ManojPraveeen there is already a workaround in place #5044 which unfortunately didn't account for NO_PROXY so either it has to be further improved or reverted to be replaced with a new mechanism

[Archerymaister]

Hey everybody,
what is the current state of this problem? From my understanding, this is affecting every system where the http_proxy/https_proxy variable is set, correct? Is a fix planned in the next release?

[cmacknz]

From my understanding, this is affecting every system where the http_proxy/https_proxy variable is set, correct?

It affects every system that wants to use NO_PROXY with Elastic Defend. It should not affect other integrations. Elastic Defend runs as a separate service from the Elastic Agent service and doesn't get environment variables set on the primary agent service unless we forward them. It also appears that Elastic Defend itself doesn't support the NO_PROXY filter yet so manually setting the environment variable on the Elastic Endpoint service isn't a workaround.

There is a PR attempting to address this open now #6675. We'll know which release it will target once it merges.

[Archerymaister]

Good to hear, that there is a solution in sight! Thank you!