Client socket changes between aws-sdk v2 and v3

[camhart]

Describe the issue

We recently updated from aws-sdk v2 to v3. We did normal testing, and everything was good. But once we pushed to production, with a production load, aws-sdk v3 started to give the error below:

WARN @smithy/node-http-handler:WARN - socket usage at capacity=50 and 766 additional requests are enqueued.
See https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-maxsockets.html
or increase socketAcquisitionWarningTimeout=(millis) in the NodeHttpHandler config.

This should really be a documented change. I'm not sure what the difference is between v2 and v3, but I'd assume others are going to hit this problem.

I had to work with an AWS support engineer to resolve it, since the change isn't documented anywhere that I could find. See ticket #174544667100759.

In case other's come across this, I had to update my s3 and dynamodbdocumentclient to allow for more sockets. Default is 50. Keep alive should be enabled by default already

const s3 = new S3Client({
    requestHandler: {
        httpsAgent: { maxSockets: 400 }, //this is what fixed it
    },
});

const client = new DynamoDBClient({
    requestHandler: {
        httpsAgent: { maxSockets: 400 }, //this is what fixed it
    },
});

Links

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/dynamodb/
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/

[aBurmeseDev]

Hi @camhart - thanks for reaching out and your feedback on the documentation.

I'm glad you were able to resolve the issue and I couldn't find the internal ticket reference to the case number you shared, however, I can share this doc that explains requestHander and socket exhaustion. Another docs/developer guide you shared also explains the behavior around configuring maxSockets. Hope that helps!

[camhart]

I see the documentation. Perhaps I should have read more, however in my entire time going from v2 -> v3 I never happened across any of those documents. I'd recommend a big warning in the api documentation for each client construction indicating a breaking change for those migrating from v2 -> v3. And consider increasing the default max sockets to a value larger than 50. What was it in v2?

[marianomerlo]

Hi! I've just opened #7416 which seems to be duplicated of this.

Were you able to find the root cause of this? I mean, I understand that increasing sockets "is a solution" but why do I need more sockets now? Because according to: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-configuring-maxsockets.html, in v2 the default is also 50 so I don't understand why v2 works fine with 50 sockets and in v3 we need to increment it.

[aBurmeseDev]

Closing as the original issue had been resolved. Please refer to this docs "A note on socket exhaustion".

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.