Prevent connection to localhost:4317 when no port is exposed

[ale-ima]

Hi,
I am running jmx_exporter as a Java Agent exporter in OpenTelemetry mode with Tomcat10.1 and with the following arguments to the JVM (notice that no port is used): -javaagent:/usr/lib/jmxexporter/jmx_prometheus_javaagent-1.5.0.jar=/etc/jmxexporter/jmx_exporter.yml -Dotel.java.global-autoconfigure.enabled=true

In /etc/jmxexporter/jmx_exporter.yml I have the following config:

openTelemetry:
  endpoint: https://my-otel-endpoint.example
  protocol: http/protobuf
  interval: 10
rules:
  ...

In tomcat logs I see the following log every minute:

04-Dec-2025 00:00:00.867 SEVERE [OkHttp http://localhost:4317/...] e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.sdk.internal.ThrottlingLogger.doLog Failed to export metrics. The request could not be executed. Error message: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317
        java.net.ConnectException: Failed to connect to localhost/[0:0:0:0:0:0:0:1]:4317
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:297)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:207)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.exporter.sender.okhttp.internal.RetryInterceptor.intercept(RetryInterceptor.java:96)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
                at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
                at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
                at java.base/java.lang.Thread.run(Thread.java:1583)
                Suppressed: java.net.ConnectException: Failed to connect to localhost/127.0.0.1:4317
                        ... 21 more
                Caused by: java.net.ConnectException: Connection refused
                        at java.base/sun.nio.ch.Net.pollConnect(Native Method)
                        at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
                        at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
                        at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
                        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
                        at java.base/java.net.Socket.connect(Socket.java:751)
                        at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.platform.Platform.connectSocket(Platform.kt:128)
                        at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:295)
                        ... 20 more
        Caused by: java.net.ConnectException: Connection refused
                at java.base/sun.nio.ch.Net.pollConnect(Native Method)
                at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
                at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
                at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
                at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
                at java.base/java.net.Socket.connect(Socket.java:751)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.platform.Platform.connectSocket(Platform.kt:128)
                at e1723a08afd7bca35570fd31a7656f59.io.prometheus.metrics.shaded.io_opentelemetry_2_16_0_alpha.okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:295)

Is there a way to prevent this self connection when OpenTelemetry mode is used and no ports is exposed?
Thanks!

[dhoard]

@ale-ima a few questions...

1. does it work if using endpoint: https://my-otel-endpoint.example:<port>/api/v1/otlp
2. does it work if using endpoint: https://my-otel-endpoint.example:<port>
3. does it work if using endpoint: https://my-otel-endpoint.example/api/v1/otlp

We have a integration test for 1, but not for 2 and 3. I'll need to add them.

[ale-ima]

Thanks for your answer.
I should have been more clear: the exporter works fine, I do get the data into my otel endpoint.
And I am using https://my-otel-endpoint.example/api/v1/otlp, I mistakenly deleted the path when replacing the actual URL.

But even though I get the data, the logs gets spammed by failing connections to a non exposed port.
I presume it is because I did not specify a port in -javaagent:/usr/lib/jmxexporter/jmx_prometheus_javaagent-1.5.0.jar=/etc/jmxexporter/jmx_exporter.yml but I don't want to because multiple JVMs are monitored on a single host and ports will conflict.
I am guessing the exporter expects the port to be exposed even though I have no use for it because I am using OTEL only.

[dhoard]

The listen port is to configure the HTTP endpoint for Prometheus pull requests. If using OpenTelemetry mode only, this doesn't come into play.

Which version of the exporter are you using?
Are there any OpenTelemetry environment variables in use?

[ale-ima]

I use the latest version (1.5.0)
Yes, the following environment variables are set up

OTEL_SERVICE_NAME=alt-metrics
OTEL_RESOURCE_ATTRIBUTES=service.instance.id=alt-main-app2-d13,app=t8080

[dhoard]

Can you test setting these via the exporter.yaml and removing the environment variables?

openTelemetry:
  endpoint: https://my-otel-endpoint.example
  protocol: http/protobuf
  interval: 10
  serviceName: alt-metrics
  resourceAttributes: service.instance.id=alt-main-app2-d13,app=t8080

[ale-ima]

With these in the exporter.yaml the JVM (Tomcat) crashes on startup.
The only logs I could get are the following
catalina.2025-12-30.log :

30-Dec-2025 17:00:59.227 INFO [main] e1723a08afd7bca35570fd31a7656f59.io.prometheus.jmx.logger.Logger.log Starting ...
30-Dec-2025 17:00:59.321 INFO [main] e1723a08afd7bca35570fd31a7656f59.io.prometheus.jmx.logger.Logger.log HTTP enabled [false]
30-Dec-2025 17:00:59.322 INFO [main] e1723a08afd7bca35570fd31a7656f59.io.prometheus.jmx.logger.Logger.log OpenTelemetry enabled [true]
30-Dec-2025 17:00:59.322 INFO [main] e1723a08afd7bca35570fd31a7656f59.io.prometheus.jmx.logger.Logger.log Starting OpenTelemetry ...

If that can help, I am running Tomcat 10.1.44 on Java 21.0.8 (Oracle JDK)

[dhoard]

@ale-ima This has been difficult to track down. This appears to be related to core OpenTelemetry code bundled in the JMX Exporter.

Are you setting any OpenTelemetry system (-Dotel.X) Java properties?

[dhoard]

@ale-ima any updates?