aws-sdk: Increase in TimeoutErrors in Amplify hotswap builds

[ShadowCat567]

Describe the bug

Starting from 11/14, we have started seeing TimeoutError: Resource is not in the expected state due to waiter status: TIMEOUT, with a frequency we have never seen before. This error was added here in version 2.167.0.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.166.0

Expected Behavior

Very few to none TimeoutError: Resource is not in the expected state due to waiter status: TIMEOUT appearing to our customers.

Current Behavior

TimeoutError: Resource is not in the expected state due to waiter status: TIMEOUT has become one of the most common error messages our customers are receiving.

Reproduction Steps

1. Setup amplify (Gen2) https://docs.amplify.aws/react/start/manual-installation/
2. Setup a function https://docs.amplify.aws/react/build-a-backend/functions/set-up-function/
3. Make a change in lambda code

Possible Solution

Roll back this PR: #31702

Additional Information/Context

No response

CDK CLI Version

2.167.0

Framework Version

No response

Node.js Version

=Node18

OS

Linux/Mac/Windows

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

Can't reproduce this using 2.168.0. The lambda func hot swapping looks good to me. Please provide more reproducible details as well as any screenshots if possible. Thank you.

[pahud]

Also, our latest version is 2.168.0 now. Can you verify with this version?

[ShadowCat567]

I have edited the description to provide some additional info about repro instructions, but this issue is very fickle and is hard to reproduce consistently. It does appear in the most recent cdk version.

[pahud]

I can't reproduce it but this could be the potential root cause:

The timeout occurs because AWS Lambda requires the function to be in an Active state and have LastUpdateStatus=Successful before it can be updated again. CDK waits for this state before proceeding, but has different wait times depending on the Lambda configuration:

aws-cdk/packages/aws-cdk/lib/api/hotswap/lambda-functions.ts

Lines 333 to 336 in 27babe6

	await lambda.waitUntilFunctionUpdated(delaySeconds, {
	FunctionName: functionName,
	});
	}

AWS SDK v3 has a different waiter implementation, timeout handling and delay strategy that handle requests differently from v2. This could affect how timeouts are processed.

I guess we should either increase the timeout somehow from here or add status checking like

const response = await lambda.getFunctionConfiguration({
  FunctionName: functionName
});
if (response.State === 'Active' && response.LastUpdateStatus === 'Successful') {
  // Proceed with hotswap
}

I'll bring this up to the core team for further investigation.