Skip to content

Fix the LocalStack testing steps in the prismic-snapshots README - #443

Open
rcantin-w wants to merge 5 commits into
mainfrom
prismic-snapshots-readme-localstack
Open

rcantin-w wants to merge 5 commits into
mainfrom
prismic-snapshots-readme-localstack

Conversation

@rcantin-w

@rcantin-w rcantin-w commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

What does this change?

Fixes the LocalStack steps in the prismic-snapshots README, which don't work as written.

Three things:

  • localstack start exits without a LOCALSTACK_AUTH_TOKEN — LocalStack 2026.03.0 moved to a single image that needs an account. Pinning localstack/localstack:4.14 runs without one, which is all these S3-only tests need.
  • Step 3 told you to create wellcomecollection-prismic-backups, but test-prismic-backup-trigger.js has always used wellcomecollection-prismic-assets, so you'd point the script at a bucket that isn't there. The download bucket name was already right.
  • Added a note that yarn test:trigger can report success when its S3 calls have failed, so don't trust the exit code.

The bucket step also now says to verify the buckets exist, and the steps after docker run say to use a second terminal — the first one is held by LocalStack, which makes the bucket step easy to skip.

How to test

  1. Start LocalStack. It holds the terminal, so run everything below in a second one.

    docker run --rm -p 4566:4566 localstack/localstack:4.14
  2. Create the buckets and confirm they're there.

    aws --endpoint-url=http://localhost:4566 s3 mb s3://wellcomecollection-prismic-assets
    aws --endpoint-url=http://localhost:4566 s3 mb s3://wellcomecollection-prismic-downloads
    aws --endpoint-url=http://localhost:4566 s3 ls

    If the CLI can't find credentials, prefix those with AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test — LocalStack doesn't check them.

  3. Uncomment S3_ENDPOINT in testing/*.js, and put PRISMIC_BEARER_TOKEN in .env.

  4. Run yarn test:trigger from infrastructure/prismic-snapshots. It pages through ~12k assets in under ten seconds and ends with a real bucket and key rather than undefined.

  5. Confirm it wrote something.

    aws --endpoint-url=http://localhost:4566 s3 ls s3://wellcomecollection-prismic-assets/media-library/

    You should get latest-batches.json and a timestamped prismic-assets-*.json.

Ignore the "will fail on S3 operations without LocalStack" line in the output — it prints whenever AWS_ACCESS_KEY_ID is unset, whether or not LocalStack is working. A successful run says it too.

Skip step 2 and you get the failure this PR is about: NoSuchBucket out of getPreviousFetchTime, then Bucket: undefined / Key: undefined and exit 0. Prismic is never reached, so a missing bucket and a bad token look identical.

On main, step 1 exits with an auth token error and the README's bucket step creates a bucket the trigger script never touches, so the run writes nothing and still prints its success block.

Have we considered potential risks?

Docs only — no Lambda, Terraform or dependency changes, so nothing deployed moves.

The pin to localstack/localstack:4.14 will go stale. It's the last image that starts without an account and LocalStack aren't patching it, so anyone who needs more than S3 here will want a token and a current image instead.

The note about yarn test:trigger describes how it behaves today. If someone picks up #442 and makes the script exit non-zero, the note should go.

For AI / future debugging

#442 says both test scripts "print Done even when an S3 call fails". Neither half holds, and the real shape is worth knowing. There's no "Done" string in the test scripts or the Lambdas at all — grep -ri done only hits scripts/deploy.sh. And the two scripts fail in opposite directions.

Written by Claude Code.

Three corrections found while actually running the test scripts against
LocalStack:

- `localstack start` exits without a LOCALSTACK_AUTH_TOKEN, since LocalStack
  2026.03.0 consolidated to a single image that requires an account. Pinning
  localstack/localstack:4.14 runs without one and is enough for these S3-only
  tests.

- The trigger bucket to create is wellcomecollection-prismic-assets, not
  -backups. test-prismic-backup-trigger.js has set BUCKET_NAME to -assets all
  along, so following the README gave the script a bucket that did not exist.
  The download bucket name was already correct.

- A failed trigger run can look like a pass. prepareAssetsForDownload catches
  its own errors and returns a statusCode/body object rather than throwing, so
  the test script takes its .then() branch, prints "Batches written to" with an
  undefined bucket and key, and exits 0. The download script does not behave
  this way: it throws on partial failure and exits 1, so the README note is
  scoped to the trigger.

Refs #442, which recorded the first two points as "being fixed in #426". #426
is superseded by #438, which already took @aws-sdk/client-s3 to 3.1161.0 on
main, so the README fixes land here instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rcantin-w rcantin-w mentioned this pull request Nov 9, 2026
rcantin-w and others added 4 commits November 13, 2026 16:35
Hit this running the steps on this branch: LocalStack was up, the test
still failed with NoSuchBucket. Step 1's docker run is foreground and
never returns, so the terminal is held and the bucket step two items
later gets skipped — you go to a new terminal and straight to the test.

So: say the first terminal is occupied, add an s3 ls to prove the
buckets exist before anything depends on them, and note that --rm
loses them on every restart.

The rest is recognising the failure when it happens. The trigger
Lambda reads latest-asset-snapshot-metadata.json before it calls
Prismic, so a missing bucket throws there, gets swallowed by the
catch in prepareAssetsForDownload, and prints undefined/undefined with
exit 0 — identical to what a bad PRISMIC_BEARER_TOKEN would look like
from the exit code. Worth saying out loud, since the obvious reading
of that output is that the token is wrong.

Also flagged the "will fail on S3 operations without LocalStack" line,
which is gated on AWS_ACCESS_KEY_ID being unset rather than on whether
LocalStack is reachable, so a fully successful run prints it too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rcantin-w
rcantin-w requested a lite review from Copilot November 17, 2026 12:32
@rcantin-w rcantin-w added the Lower priority Prioritise other PRs over this one unless it's getting old label Nov 17, 2026
@rcantin-w
rcantin-w marked this pull request as ready for review November 17, 2026 12:33
@rcantin-w
rcantin-w requested a review from a team as a code owner November 17, 2026 12:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The README’s verification depends on external LocalStack behavior and availability of the 4.14 image, which must be confirmed before the documented AWS CLI commands can be relied upon.

No open issues 🎉

What changed in this PR

Updates the prismic-snapshots README with corrected, clearer LocalStack testing instructions for the manual S3 workflow.

Changes:

  • Uses the token-free localstack/localstack:4.14 image and explains the two-terminal setup.
  • Corrects the trigger bucket name and adds bucket-existence verification.
  • Documents credential guidance, output verification, and misleading successful exit statuses from yarn test:trigger.
File Description
infrastructure/​prismic-snapshots/​README.md Corrects and expands the LocalStack setup, bucket creation, verification, credential, and test-result guidance.

Review effort: Lite

@rcantin-w rcantin-w moved this from Backlog to Ready for review in Digital experience Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Lower priority Prioritise other PRs over this one unless it's getting old

Projects

Status: Ready for review

Development

Successfully merging this pull request may close these issues.

2 participants