fix(lambda): translate self-managed Kafka Endpoints key to the SDK enum value by go-to-k · Pull Request #1398 · go-to-k/cdkd

go-to-k · 2026-08-09T03:57:36Z

Summary

LambdaEventSourceMappingProvider cast the CFn SelfManagedEventSource blob
straight to the SDK type. CFn spells the bootstrap-server map key
Endpoints.KafkaBootstrapServers, while the SDK models Endpoints as
Partial<Record<EndPointType, string[]>> keyed by the enum value
KAFKA_BOOTSTRAP_SERVERS.

Because Endpoints is a map rather than a modeled structure, the AWS SDK v3
serializer does not drop the unknown key — it forwards it and the service
rejects the request. So CreateEventSourceMapping failed outright for every CDK
SelfManagedKafkaEventSource user, not silently.

What changed

create() renames KafkaBootstrapServers -> KAFKA_BOOTSTRAP_SERVERS.
readCurrentState() applies the inverse, so cdkd state (which holds the CFn
spelling) and the AWS-current snapshot compare on the same key instead of
firing guaranteed drift on every clean run of a self-managed-Kafka ESM.
Every other member of the blob is copied through untouched — both siblings of
Endpoints and unrecognised keys inside it. Anything not re-shapeable (a
non-object blob, a missing / non-object / array Endpoints) is forwarded
verbatim, exactly as the raw cast this replaces did: turning a
silent-drop fix into a new silent drop would be the same bug class in the
other direction, and AWS must stay the layer that rejects a malformed
template. Unit tests pin each of those arms, plus non-mutation of the
caller's properties object (cdkd state is saved from it, so an in-place
rename would bake the SDK spelling into state).
Note the pass-through is not a general solution: a FUTURE EndPointType
whose CFn spelling also diverges reproduces AWS::Lambda::EventSourceMapping: self-managed Kafka Endpoints.KafkaBootstrapServers not translated to SDK enum key — CreateEventSourceMapping fails #1384 exactly and needs its own
entry. That is called out in the helper's JSDoc.
Two existing create tests used the SDK spelling in CFn-property position
(they encoded the bug's blind spot); corrected to the CFn spelling.

Test plan

vp run check / typecheck:test / build / test — 515 files, 8694 tests, 0 type errors.
Revert probe: restoring the raw casts in the real provider makes both new
tests fail (create ships KafkaBootstrapServers, readCurrentState emits
KAFKA_BOOTSTRAP_SERVERS); restoring turns them green.
Real AWS: new tests/integration/lambda-esm-self-managed-kafka fixture.
The ESM points at non-existent brokers and is created enabled: false — the
API validates the request shape, not connectivity — so no Kafka cluster (or
its cost) is needed while the exact wire path the bug broke is exercised. A
successful Phase 1 deploy IS the proof; verify.sh additionally reads the
enum key back from AWS, asserts an in-place update keeps the UUID, and
asserts a clean destroy. Run 2026-08-09: all assertions green, destroy 5
deleted / 0 errors, 0 orphans. Recorded in
docs/_generated/integ-last-run.tsv.

verify.sh also runs a cdkd drift phase between deploy and update: the
read-side inverse is otherwise unreachable from an integ, because
readCurrentState feeds observedProperties / drift and NOT the deploy diff —
deleting it would have left every other phase green.

Notes

Two live-run traps, both fixed and both recorded as memory rules:

AWS returned the two brokers in the opposite order from the one submitted, so
the first readback compare failed while the fix was working correctly. The
assertion now sorts both sides.
DeleteSecret is eventually consistent: the new Secret gone-assert fired on
an otherwise-clean destroy and the secret was gone a minute later. It now
polls, like the ESM delete probe.

Deferred

AWS::Lambda::EventSourceMapping is still absent from NESTED_KEY_TARGETS
(scripts/gen-nested-key-coverage.ts), so its seven other forwarded config
blobs stay unaudited by the critic. Adding a target requires a schema-fixture
re-capture and may surface unrelated divergences, so it belongs with the
critic-target expansion tracked in (#1393) rather than in this bug fix.

Closes #1384

…um value CFn spells the bootstrap-server map key `SelfManagedEventSource.Endpoints.KafkaBootstrapServers`, while the SDK models `Endpoints` as `Partial<Record<EndPointType, string[]>>` keyed by the enum VALUE `KAFKA_BOOTSTRAP_SERVERS`. Because `Endpoints` is a MAP rather than a modeled structure, the AWS SDK v3 serializer forwards the unknown CFn key verbatim instead of dropping it and the service rejects the request - so CreateEventSourceMapping failed outright for every CDK SelfManagedKafkaEventSource user. - create: rename KafkaBootstrapServers -> KAFKA_BOOTSTRAP_SERVERS - readCurrentState: the inverse, so state (which holds the CFn spelling) and the AWS-current snapshot compare on the same key instead of firing guaranteed drift on every clean run - every other member of the blob is copied through untouched, so a future SDK addition needs no change here - unit tests for both directions, verified to fail without the fix - new tests/integration/lambda-esm-self-managed-kafka fixture: a disabled ESM against non-existent brokers (the API validates request shape, not connectivity) so a successful deploy IS the proof; verify.sh also reads the enum key back from AWS, asserts an in-place update keeps the UUID, and asserts a clean destroy Closes #1384

…cord the run AWS does not preserve the submitted order of Endpoints.KAFKA_BOOTSTRAP_SERVERS (the first live run came back b-2 before b-1), so the readback compare now sorts both sides.

…ead of dropping it Review follow-up on the #1384 fix. - renameEndpointsKey returned undefined for any non-object input, so a truthy-but-malformed blob was SILENTLY DROPPED where the raw cast it replaced forwarded it to AWS. Turning a silent-drop fix into a new silent drop is the exact regression class this repo tracks; the helper now returns non-reshapeable input verbatim and AWS stays the layer that rejects a bad template. - new unit tests: sibling members beside Endpoints and beside the renamed key survive; the caller's properties object is not mutated (state is saved from it, so an in-place rename would bake the SDK spelling into state); malformed and array-valued input pass through; the two directions COMPOSE back to the exact template blob. - corrected two more pre-existing roundtrip fixtures that spelled the SDK enum key in CFn-property position. - verify.sh: added a cdkd drift phase (the read-side inverse was otherwise unreachable from the integ - observedProperties does not feed the deploy diff), a Creating/Updating settle wait before the update phase (UpdateEventSourceMapping rejects a still-creating mapping with a non-retryable ResourceInUseException), a poll loop around the async ESM delete probe, and a gone-assert for the Secret.

DeleteSecret is eventually consistent the same way DeleteEventSourceMapping is: the first run of the new assertion failed while describe-secret still answered 200 on an otherwise-clean destroy, and the secret was gone a minute later. Poll instead of asserting once.

Review follow-up. The cdkd drift phase added in the previous commit proved nothing: drift's baseline is observedProperties, captured at deploy time from the SAME readCurrentState call, so both comparison sides carry whatever spelling toCfnSelfManagedEventSource emits and the rename cancels - deleting the inverse left the phase green. Worse, its comment told the next maintainer the path was covered. Assert the PERSISTED spelling instead: state must hold the template's KafkaBootstrapServers and must NOT hold KAFKA_BOOTSTRAP_SERVERS. That side is asymmetric and does fail when the inverse is dropped. The drift run is kept as an end-to-end agreement check, now branching on its exit code so a command error (2) is not reported as a drift finding (1). Also gate the pre-update settle loop on the TERMINAL state set (Enabled/Disabled) rather than 'not Creating' - Enabling/Disabling would otherwise break out early. Verified against real AWS: state assertion green, ESM settled at Disabled, destroy 5 deleted / 0 errors / 0 orphans.

## [0.278.5](v0.278.4...v0.278.5) (2026-08-09) ### Bug Fixes * **lambda:** translate self-managed Kafka Endpoints key to the SDK enum value ([#1398](#1398)) ([8c947d4](8c947d4))

github-actions · 2026-08-09T05:07:25Z

🎉 This PR is included in version 0.278.5 🎉

The release is available on:

Your semantic-release bot 📦🚀

go-to-k added 5 commits August 9, 2026 12:46

test(integ): make the Kafka broker assertion order-insensitive and re…

f1e3208

…cord the run AWS does not preserve the submitted order of Endpoints.KAFKA_BOOTSTRAP_SERVERS (the first live run came back b-2 before b-1), so the readback compare now sorts both sides.

go-to-k mentioned this pull request Aug 9, 2026

audit: nested-key critic blind spots found by the 0809 sweep — mixed-case SDK models, file-global literal heuristic, missing targets, selective sub-key forwards #1393

Open

go-to-k merged commit 8c947d4 into main Aug 9, 2026
5 checks passed

go-to-k deleted the fix/1384-lambda-esm-kafka-endpoints branch August 9, 2026 05:06

github-actions Bot added the released label Aug 9, 2026

Navigation Menu

Uh oh!

fix(lambda): translate self-managed Kafka Endpoints key to the SDK enum value - #1398

fix(lambda): translate self-managed Kafka Endpoints key to the SDK enum value#1398
go-to-k merged 5 commits into
mainfrom
fix/1384-lambda-esm-kafka-endpoints

go-to-k commented Aug 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Aug 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

go-to-k commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Test plan

Notes

Deferred

Uh oh!

Uh oh!

github-actions Bot commented Aug 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

go-to-k commented Aug 9, 2026 •

edited

Loading