Transient "java.net.SocketTimeoutException: timeout" Exception

[kostasxmeta]

Hello, we've been seeing this socket timeout issue for the past several months. It is transient i.e. it's not easily reproducible, and we have not been able to link it to a specific type of query, but we've been seeing it consistently a few times per day.

This seems to happen here:

presto/presto-client/src/main/java/com/facebook/presto/client/JsonResponse.java

Line 131 in 6e88468

try (Response response = client.newCall(request).execute()) {

The full stacktrace:

...
Caused by: java.io.UncheckedIOException: java.net.SocketTimeoutException: timeout
	at com.facebook.presto.jdbc.internal.client.JsonResponse.execute(JsonResponse.java:154)
	at com.facebook.presto.jdbc.internal.client.StatementClientV1.<init>(StatementClientV1.java:135)
	at com.facebook.presto.jdbc.internal.client.StatementClientFactory.newStatementClient(StatementClientFactory.java:24)
	at com.facebook.presto.jdbc.QueryExecutor.startQuery(QueryExecutor.java:46)
	at com.facebook.presto.jdbc.PrestoConnection.startQuery(PrestoConnection.java:759)
	at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:263)
	... 9 more
Caused by: java.net.SocketTimeoutException: timeout
	at com.facebook.presto.jdbc.internal.okio.Okio$4.newTimeoutException(Okio.java:230)
	at com.facebook.presto.jdbc.internal.okio.AsyncTimeout.exit(AsyncTimeout.java:285)
	at com.facebook.presto.jdbc.internal.okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
	at com.facebook.presto.jdbc.internal.okio.RealBufferedSource.indexOf(RealBufferedSource.java:345)
	at com.facebook.presto.jdbc.internal.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:217)
	at com.facebook.presto.jdbc.internal.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:211)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:187)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at com.facebook.presto.jdbc.internal.client.OkHttpUtil.lambda$userAgent$0(OkHttpUtil.java:77)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at com.facebook.presto.jdbc.internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at com.facebook.presto.jdbc.internal.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
	at com.facebook.presto.jdbc.internal.okhttp3.RealCall.execute(RealCall.java:77)
	at com.facebook.presto.jdbc.internal.client.JsonResponse.execute(JsonResponse.java:131)
	... 14 more
Caused by: java.net.SocketTimeoutException
	at java.base/sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:214)
	at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
	at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
	at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1096)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:816)
	at com.facebook.presto.jdbc.internal.okio.Okio$2.read(Okio.java:139)
	at com.facebook.presto.jdbc.internal.okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
	... 37 more

It seems related to the okhttp library and I found this "guide to timeouts" which seems relevant: https://www.baeldung.com/okhttp-timeouts

Does anyone have any ideas as to what might be going on here and what the best way to fix it would be?

Thanks in advance!

[sunriseLe]

Do you have any idea? We also have the same problem