Skip to content

fix(dynamodb): reset the table-level GlobalTable on-demand write ceiling when the template drops it - #1439

Merged
go-to-k merged 7 commits into
mainfrom
fix/1434-globaltable-ondemand-reset
Aug 9, 2026
Merged

fix(dynamodb): reset the table-level GlobalTable on-demand write ceiling when the template drops it#1439
go-to-k merged 7 commits into
mainfrom
fix/1434-globaltable-ondemand-reset

Conversation

@go-to-k

@go-to-k go-to-k commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

Removing WriteOnDemandThroughputSettings from a AWS::DynamoDB::GlobalTable
template never set flatChanged, so no UpdateTable went out and the old
ceiling stayed live in AWS while cdkd reported success — the absent-field-reset
silent drop (#1160), the table-level sibling of the per-GSI case #1423 closed.

The reset merges per member rather than branching on "the whole block
disappeared" (the shape the #1433 review caught: a block can survive with its
member dropped, which is the likelier user edit), and only fires when the table
is PAY_PER_REQUEST on both sides of the deploy — the ceiling is not a live,
resettable setting otherwise, and that condition also subsumes suppressing the
reset during a billing-mode flip (see the Review section for why "not flipping"
alone was wrong). oldBilling / newBilling are now resolved once above step 3
and reused by step 4, so the two sites share one binding.

The issue asked for three fields; only ONE is implementable

Live-probed against real AWS (us-east-1 source + us-west-2 replica, one global
table per probe, both torn down). Full transcripts are on the issue.

field reset payload result
table-level WriteOnDemandThroughputSettings -1 works — accepted, reads back as ABSENCE, partial removal preserves the sibling
replica OnDemandThroughputOverride -1 accepted but stored literally as -1
replica OnDemandThroughputOverride {} accepted, then wedged the table in UPDATING for over an hour, value unchanged
replica OnDemandThroughputOverride omit member ValidationException: There are no actions specified…
replica ProvisionedThroughputOverride -1 / 0 rejected, must have value greater than or equal to 1
replica ProvisionedThroughputOverride {} InternalServerError

Table-level -1 was verified independently rather than inherited from
#1423, since reset semantics are field-specific.

The two replica overrides are deliberately not implemented: there is no
payload that resets them, and shipping any of the above would trade a silent
no-op for a live defect (a nonsense stored value, or a wedged table). The
registry schema agrees by construction — it declares those members
"minimum": 1, so no in-band sentinel exists. Suggest re-titling #1434 to the
table-level field and either closing the replica halves as
not-currently-possible or splitting them out with a CloudFormation-side A/B as
their acceptance criterion. #1434 is intentionally left open (Refs, not
Closes) so the record does not claim all three were fixed.

Self-review catch

The first cut swapped Number() for toFiniteNumber(), which routed a
present-but-unresolvable value (an unresolved intrinsic) into the reset
branch — silently CLEARING the ceiling the template was trying to set, i.e. the
same silent-wrong-action class this PR exists to remove. The gate now tests raw
key presence and the coercion is left exactly as it was, so such a value still
reaches AWS and fails loudly. Pinned by a test.

Test plan

  • 8 units: full removal, PARTIAL removal, value-changed (never the
    sentinel), stringly-typed coercion, present-but-unresolvable,
    PROVISIONED -> PROVISIONED drop, billing-flip suppression, no-change
    redeploy. Revert-proofed — neutering the reset fails exactly the first two
    while the rest stay green, since they pin behavior the change does not alter.
  • Integ (dynamodb-globaltable, real AWS, PASS in 253s, 3 deleted / 0
    errors / 0 orphans): a new drop-table-ondemand-limit UPDATE mode alongside
    the existing drop-gsi-ondemand-limits (AWS::DynamoDB::GlobalTable: removing a per-GSI on-demand limit silently no-ops (absent-field reset, #1160 class) #1423) one. verify.sh asserts
    BEFORE and AFTER, not just after — step 13b drops only the per-GSI limit,
    so the table-level ceiling must still read 200 going in. Without that
    precondition the post-drop absence check would also pass if the ceiling had
    never reached AWS at all.
  • The table-level READ ceiling is deliberately NOT asserted: the canonical
    Billing.onDemand({ maxReadRequestUnits }) is never wired by cdkd, so pinning
    it would encode that gap as expected behavior.

Related finding filed, not fixed here

The probes surfaced a separate, more severe gap: the table-level on-demand READ
ceiling (Replicas[local].ReadOnDemandThroughputSettings) is never wired AT ALL
— dropped on the way in rather than merely un-reset. The repo's own
verbatim-cdk synth unit fixture already carried the value with no assertion
over it. Filed as #1436.

Review

/review-pr tiered this 1-reviewer (215 LOC / 6 files would be inline,
bumped one step by the src/provisioning/providers/** up-bias plus >1 fix-back
commit). A pr-code-reviewer pass ran against the branch and found one real
defect introduced by this PR, now fixed:

  • !billingModeFlipping was the wrong predicate. PROVISIONED -> PROVISIONED
    is also "not flipping", so a provisioned template that carried
    WriteOnDemandThroughputSettings (legal in the CFn schema, reachable from
    hand-authored L1 and from pre-fix state) and then dropped it would have sent
    OnDemandThroughput on a provisioned table and earned a ValidationException —
    a NEW failure where the pre-fix behavior was a correct no-op. The gate now
    requires PAY_PER_REQUEST on BOTH sides, which subsumes the flip suppression
    (a flip makes the two sides differ) and adds the missing "the ceiling is
    actually live" condition. Pinned by a test.

Two further findings were filed rather than fixed here, to keep this PR's blast
radius on the table-level path:

Verified clean by the reviewer: the raw-presence gate, the oldBilling /
newBilling hoist (identical values at all 8 downstream consumers; nothing
mutates the property bags in between), leaving create() unchanged, and the
readCurrentState / drift interaction (the always-emit placeholder means the
post-reset baseline and the next read agree, so no phantom drift and no reset
loop).

Refs #1434

…ing when the template drops it

Removing WriteOnDemandThroughputSettings never set flatChanged, so no
UpdateTable went out and the old ceiling stayed live in AWS while cdkd
reported success - the absent-field-reset silent drop (#1160), the
table-level sibling of the per-GSI case #1423 closed.

The reset merges per member rather than branching on "the whole block
disappeared" (the shape the #1433 review caught: a block can survive with
its member dropped, which is the likelier user edit), and is suppressed
while the billing mode is flipping, since dropping the on-demand block on
the way to PROVISIONED is the natural template edit rather than a clear
request. oldBilling / newBilling are now resolved once above step 3 and
reused by step 4 so the two sites cannot disagree about what flipping
means.

The issue asked for three fields; live probes proved only ONE is
implementable. Table-level -1 was verified independently rather than
inherited from #1423 (reset semantics are field-specific): it was
accepted and DescribeTable then returned the untouched sibling alone, so
the reset reads back as ABSENCE, never as -1.

The two replica overrides have no reset mechanism and are deliberately
left alone. -1 is accepted but STORED LITERALLY, which would be strictly
worse than the current no-op; the documented empty-override form hangs
the table in UPDATING for over an hour with the value unchanged; an entry
carrying only IndexName is rejected. ProvisionedThroughputOverride is the
same by construction - the registry schema declares the members
"minimum": 1, and live -1 / 0 fail validation while {} returns
InternalServerError. Full transcripts are on the issue.

Tests: 6 units (full removal, partial removal, value-changed, stringly
typed coercion, billing-flip suppression, no-change redeploy).
Revert-proofed: neutering the reset fails exactly the first two.

Refs #1434
…ltable fixture

Adds a `drop-table-ondemand-limit` UPDATE mode alongside the existing
`drop-gsi-ondemand-limits` (#1423) one, so the table-level reset is
exercised against real AWS rather than only against a mocked client.

verify.sh asserts BEFORE and AFTER, not just after: step 13b drops only
the per-GSI limit, so the table-level ceiling must still read 200 going
in. Without that precondition the post-drop absence check would also pass
if the ceiling had never reached AWS at all.

The table-level READ ceiling is deliberately NOT asserted: the canonical
Billing.onDemand({ maxReadRequestUnits }) is never wired by cdkd (issue
#1436), so pinning it would encode that gap as expected behavior.

Refs #1434
…hing AWS

Self-review catch on the #1434 reset. Gating the reset branch on "did the
value coerce?" instead of "is the key present?" routed an unresolved
intrinsic into the reset, silently CLEARING the ceiling the template was
trying to set. The pre-existing behavior - forward it and let AWS reject
the request loudly - is correct, so the gate now tests raw presence and
the coercion is left exactly as it was.

Also drops the redundant oldBillingMode/newBillingMode aliases: the pair
is now hoisted under its original oldBilling/newBilling names, so there is
one binding rather than two names for the same value.

Refs #1434
…demand on BOTH sides

Reviewer catch on PR #1439. The reset was gated on "the billing mode is
not flipping", but PROVISIONED -> PROVISIONED is also not flipping — so a
provisioned template that carried WriteOnDemandThroughputSettings (legal
in the CFn schema, reachable from hand-authored L1 and from state written
before this fix) and then dropped it would send OnDemandThroughput on a
PROVISIONED table and earn a ValidationException, where the pre-fix
behavior was a correct no-op.

Requiring PAY_PER_REQUEST on both sides subsumes the flip suppression (a
flip makes the two sides differ) and adds the missing "the ceiling is
actually live" condition.

Refs #1434
@go-to-k
go-to-k merged commit 654391b into main Aug 9, 2026
7 checks passed
@go-to-k
go-to-k deleted the fix/1434-globaltable-ondemand-reset branch August 9, 2026 16:51
github-actions Bot pushed a commit that referenced this pull request Aug 9, 2026
## [0.278.17](v0.278.16...v0.278.17) (2026-08-09)

### Bug Fixes

* **dynamodb:** reset the table-level GlobalTable on-demand write ceiling when the template drops it ([#1439](#1439)) ([654391b](654391b))
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 0.278.17 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant