stroom-proxy forwarding failure if destination name resolution fails at startup

[stroomdev00]

stroom-proxy fails to forward any logs if it's initially unable to resolve the destination address, even when the name resolution starts working correctly. Presumably the lookup or its failure is cached at proxy startup and the cache never renewed.

Meanwhile, it is happy to talk to the feed checking URL on the same host when resolution is fixed so I guess this uses an different resolution method?

A simple systemctl restart stroom-proxy seems to resolve the issue once name resolution is working.

7.8 and 7.9

[stroomdev00]

Also stops forwarding and fails to restart on timeout:

INFO   [2025-04-03T12:10:10.410Z] [pool-3-thread-39] stroom.proxy.app.handler.RemoteFeedStatusService -  Got feed status 'Receive' for 'XXX' using url 'https://xxx/api/feedStatus/v1'
INFO   [2025-04-03T12:10:17.315Z] [pool-3-thread-39] stroom.proxy.app.handler.RemoteFeedStatusService -  Got feed status 'Receive' for 'XXX' using url 'https://xxx/api/feedStatus/v1'
ERROR  [2025-04-03T12:10:43.952Z] [Forwarding queue transfer 1] stroom.proxy.repo.ParallelExecutor -  Read timed out
stroom.proxy.app.handler.ForwardException: Read timed out
        at stroom.proxy.app.handler.ForwardException.recoverable(ForwardException.java:33)
        at stroom.proxy.app.handler.HttpSender.post(HttpSender.java:240)
        at stroom.proxy.app.handler.HttpSender.send(HttpSender.java:116)
        at stroom.proxy.app.handler.ForwardHttpPostDestination.add(ForwardHttpPostDestination.java:53)
        at stroom.proxy.app.handler.Forwarder.add(Forwarder.java:92)
        at stroom.proxy.app.handler.DirQueueTransfer.run(DirQueueTransfer.java:21)
        at stroom.proxy.repo.ParallelExecutor.run(ParallelExecutor.java:162)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
        at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
        at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
        at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
        at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
        at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
        at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
        at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
        at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1461)
        at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1066)
        at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
        at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
        at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:250)
        at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:56)
        at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:331)
        at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:196)
        at com.codahale.metrics.httpclient5.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:62)
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.lambda$execute$0(InternalExecRuntime.java:236)
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:791)
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:233)
        at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:121)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:199)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:150)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:174)
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:162)
        at stroom.proxy.app.handler.HttpSender.post(HttpSender.java:216)
        ... 8 common frames omitted

[stroomdev00]

Moved the stack trace that was in this comment to a new issue as it is probably unrelated.

#4851