[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows by aicayzer · Pull Request #14118 · cloudflare/workers-sdk

aicayzer · 2026-05-29T20:29:22Z

Fixes #14101. Related: #10966 (Workflows local-dev / production behaviour alignment).

Problem

In local wrangler dev, a Uint8Array returned from a Workflows step fails with string or blob too big: SQLITE_TOOBIG at sizes well below the documented 1MiB step-output limit. The same bytes wrapped as a tight ArrayBuffer succeed up to ~2MB; production accepts the Uint8Array either way. Reporter's turn-key repro: https://github.com/danieltroger/cf-workflows-local-sqlite-toobig

The cause is that v8::ValueSerializer (used by Durable Object SQL storage in workerd) writes the entire backing ArrayBuffer for typed-array views, not just byteLength bytes. When a view is created on a larger buffer — crypto.getRandomValues(new Uint8Array(N)), arr.slice(...) on a bigger pool, fetch-stream copies, etc — the wire size balloons by a factor of (backing-size / view-size).

Fix

packages/workflows-shared/src/context.ts — a small normalizeForStorage helper copies typed-array views into a tight ArrayBuffer before storage.put:

function normalizeForStorage(value: unknown): unknown {
    if (ArrayBuffer.isView(value) && !(value instanceof DataView)) {
        const view = value as ArrayBufferView;
        return new Uint8Array(view.buffer, view.byteOffset, view.byteLength)
            .slice()
            .buffer;
    }
    return value;
}

Applied in persistStepResult immediately before the storage write. The caller (the user's step.do(...)) continues to receive the original Uint8Array; only the on-disk form changes.

Files

packages/workflows-shared/src/context.ts — normalizeForStorage helper + integration in persistStepResult
packages/workflows-shared/tests/context.test.ts — four new regression tests under Context - typed-array step outputs (issue #14101): tight-buffer success, sliced-buffer success (the reporter's exact repro), oversize-rejection, round-trip live-path-shape
.changeset/fix-workflows-uint8array-step-output.md — miniflare + wrangler patch

Test plan

Tests
- Tests included/updated
- Automated tests not possible - manual testing has been completed as follows:
- Additional testing not necessary because:
Public documentation
- Cloudflare docs PR(s):
- Documentation not necessary because: this fix brings local wrangler dev behaviour in line with already-documented production behaviour (1MiB step-output limit)

Local validation:

pnpm test:ci -F @cloudflare/workflows-shared — 117/117 passing including the four new regression tests
pnpm run check:lint — clean
pnpm prettify — clean
pnpm check:type (workflows-shared) — clean

The repo-wide pnpm check reports a pre-existing check:package-deps failure unrelated to this PR (wrangler-imports-vite-without-declaring); confirmed reproducible on a clean upstream/main checkout via git stash && pnpm check.

Notes for reviewer

Top-level normalisation only. Step outputs like { image: Uint8Array, meta: {...} } would still suffer the bloat at the nested level. Happy to extend to a deep-walk (arrays + objects + Map/Set values) — left top-level only to match the smallest scope that addresses the headline bug, but I'd appreciate confirmation on whether to extend.
Cache-replay shape. Persisted form is ArrayBuffer, which means cached-replay reads also surface ArrayBuffer even if the original step returned a Uint8Array. The live execution path is unchanged (the caller's step.do() still sees the original Uint8Array). If production rehydrates to Uint8Array on replay, flag it and I'll add a side-channel __viewKind marker plus decoder.
Secondary issue: friendly catch bypass. The friendly WorkflowInternalError("Maximum allowed size is 1MiB.") at context.ts:768-774 does NOT actually fire for typed-array outputs — the raw SQLITE_TOOBIG reaches WORKFLOW_FAILURE instead. The error surfaces asynchronously from the workerd DO storage output gate, bypassing the awaited try/catch around persistStepResult. The 2MB oversize-rejection test documents this and asserts on the contract (terminal failure) rather than the message wording. Happy to file a separate follow-up issue with a more invasive fix once this lands; out of scope here to keep the change focused.

A picture of a cute animal (not mandatory, but encouraged)

changeset-bot · 2026-05-29T20:29:27Z

🦋 Changeset detected

Latest commit: 09c80f8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

Name	Type
miniflare	Patch
wrangler	Patch
@cloudflare/deploy-helpers	Patch
@cloudflare/pages-shared	Patch
@cloudflare/vite-plugin	Patch
@cloudflare/vitest-pool-workers	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

workers-devprod · 2026-05-29T20:29:48Z

Codeowners approval required for this PR:

@cloudflare/workflows
@cloudflare/wrangler

Show detailed file reviewers

.changeset/fix-workflows-uint8array-step-output.md: [@cloudflare/wrangler]
packages/workflows-shared/src/context.ts: [@cloudflare/workflows @cloudflare/wrangler]
packages/workflows-shared/tests/context.test.ts: [@cloudflare/workflows @cloudflare/wrangler]

…step outputs Addresses review feedback on cloudflare#14118. normalizeForStorage is now a recursive walker (cycle-safe via WeakMap) that descends into arrays, plain objects, Maps, and Sets, compacting typed-array views wherever they appear in the value tree. Top-level-only normalisation missed nested views (e.g. `{ image: Uint8Array }`). Compaction now preserves the original view constructor instead of flattening every view to ArrayBuffer. Uint8Array stays Uint8Array, Int16Array stays Int16Array, DataView stays DataView, etc. The persisted shape matches the live shape, so cached replays observe the same constructor the step originally returned. Added regression tests: Int16Array round-trip on the live path, a sliced view nested in a plain object, and sliced views nested two levels deep in an array of objects. The original top-level tests still pass.

aicayzer · 2026-06-01T14:17:14Z

Both updated in 40faabceb. Lmk if anything else.

pkg-pr-new · 2026-06-01T19:53:13Z

create-cloudflare

npm i https://pkg.pr.new/cloudflare/workers-sdk/create-cloudflare@14118

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/deploy-helpers@14118

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/kv-asset-handler@14118

miniflare

npm i https://pkg.pr.new/cloudflare/workers-sdk/miniflare@14118

@cloudflare/pages-shared

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/pages-shared@14118

@cloudflare/unenv-preset

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/unenv-preset@14118

@cloudflare/vite-plugin

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/vite-plugin@14118

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/vitest-pool-workers@14118

@cloudflare/workers-auth

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-auth@14118

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-editor-shared@14118

@cloudflare/workers-utils

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-utils@14118

wrangler

npm i https://pkg.pr.new/cloudflare/workers-sdk/wrangler@14118

@cloudflare/wrangler-bundler

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/wrangler-bundler@14118

commit: 71ab67b

petebacondarwin · 2026-06-03T08:37:39Z

@Caio-Nogueira - can you take another look?

devin-ai-integration

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

…tructor Per Caio-Nogueira's review suggestion on cloudflare#14118. Replaces the 13-branch instanceof dispatch with a typed cast on view.constructor, and uses view.buffer.slice() directly for the tight copy. Same behaviour, much shorter.

workers-devprod · 2026-06-15T09:44:13Z

Codeowners approval required for this PR:

✅ @cloudflare/workflows
✅ @cloudflare/wrangler

Show detailed file reviewers

workers-devprod

Codeowners reviews satisfied

aicayzer · 2026-06-17T00:13:10Z

@emily-shen @Caio-Nogueira thanks for the approvals. workflows-shared is green on both Windows and macOS here. The only red checks are unrelated flakes: the miniflare browser-rendering specs (test/plugins/browser/index.spec.ts) and the getPlatformProxy remote-bindings E2E. Re-running the full matrix just swaps one flaky job for another, and I cant re-run from a fork. Could you re-run the failed jobs or merge it through when you get a sec?

…Buffer in local Workflows

…step outputs Addresses review feedback on cloudflare#14118. normalizeForStorage is now a recursive walker (cycle-safe via WeakMap) that descends into arrays, plain objects, Maps, and Sets, compacting typed-array views wherever they appear in the value tree. Top-level-only normalisation missed nested views (e.g. `{ image: Uint8Array }`). Compaction now preserves the original view constructor instead of flattening every view to ArrayBuffer. Uint8Array stays Uint8Array, Int16Array stays Int16Array, DataView stays DataView, etc. The persisted shape matches the live shape, so cached replays observe the same constructor the step originally returned. Added regression tests: Int16Array round-trip on the live path, a sliced view nested in a plain object, and sliced views nested two levels deep in an array of objects. The original top-level tests still pass.

…tructor Per Caio-Nogueira's review suggestion on cloudflare#14118. Replaces the 13-branch instanceof dispatch with a typed cast on view.constructor, and uses view.buffer.slice() directly for the tight copy. Same behaviour, much shorter.

After cloudflare#14134 merged the binaryReplacer fix into engine.ts:writeLog, a 2 MB Uint8Array step output no longer terminates the workflow — the documented 1MiB step-output cap isn't enforced at this layer post-fix. This regression test was asserting that side-effect; remove it. The remaining typed-array tests still validate the deep-walk + view-type preservation behaviour that this PR adds.

aicayzer requested a review from workers-devprod as a code owner May 29, 2026 20:29

workers-devprod requested review from a team and emily-shen and removed request for a team May 29, 2026 20:29

emily-shen mentioned this pull request Jun 1, 2026

fix(workflows): prevent SQLITE_TOOBIG crash when a step returns a large Uint8Array #14134

Merged

2 tasks

Caio-Nogueira requested changes Jun 1, 2026

View reviewed changes

Comment thread packages/workflows-shared/src/context.ts

Comment thread packages/workflows-shared/src/context.ts Outdated

workers-devprod added this to workers-sdk Jun 1, 2026

github-project-automation Bot moved this to Untriaged in workers-sdk Jun 1, 2026

github-project-automation Bot moved this from Untriaged to In Review in workers-sdk Jun 1, 2026

petebacondarwin requested a review from Caio-Nogueira June 3, 2026 08:37

devin-ai-integration Bot reviewed Jun 3, 2026

View reviewed changes

Caio-Nogueira reviewed Jun 3, 2026

View reviewed changes

Comment thread packages/workflows-shared/src/context.ts Outdated

emily-shen requested a review from Caio-Nogueira June 11, 2026 13:09

Caio-Nogueira approved these changes Jun 15, 2026

View reviewed changes

emily-shen approved these changes Jun 15, 2026

View reviewed changes

workers-devprod approved these changes Jun 15, 2026

View reviewed changes

github-project-automation Bot moved this from In Review to Approved in workers-sdk Jun 15, 2026

emily-shen enabled auto-merge (squash) June 15, 2026 14:43

aicayzer added 2 commits June 17, 2026 10:38

[workflows-shared] Fix Uint8Array step outputs dragging backing Array…

db9e5cf

…Buffer in local Workflows

aicayzer added 2 commits June 17, 2026 10:38

petebacondarwin force-pushed the fix/workflows-uint8array-step-output branch from 10dd32a to 09c80f8 Compare June 17, 2026 09:38

emily-shen merged commit b38823f into cloudflare:main Jun 18, 2026
87 of 94 checks passed

github-project-automation Bot moved this from Approved to Done in workers-sdk Jun 18, 2026

workers-devprod mentioned this pull request Jun 18, 2026

Version Packages #14353

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows#14118

[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows#14118
emily-shen merged 4 commits into
cloudflare:mainfrom
aicayzer:fix/workflows-uint8array-step-output

aicayzer commented May 29, 2026

Uh oh!

changeset-bot Bot commented May 29, 2026 •

edited

Loading

Uh oh!

workers-devprod commented May 29, 2026

Uh oh!

Uh oh!

Uh oh!

aicayzer commented Jun 1, 2026

Uh oh!

pkg-pr-new Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

petebacondarwin commented Jun 3, 2026

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

workers-devprod commented Jun 15, 2026 •

edited

Loading

Uh oh!

workers-devprod left a comment

Uh oh!

aicayzer commented Jun 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

aicayzer commented May 29, 2026

Problem

Fix

Files

Test plan

Notes for reviewer

Uh oh!

changeset-bot Bot commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

workers-devprod commented May 29, 2026

Uh oh!

Uh oh!

Uh oh!

aicayzer commented Jun 1, 2026

Uh oh!

pkg-pr-new Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petebacondarwin commented Jun 3, 2026

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

✅ Devin Review: No Issues Found

Uh oh!

Uh oh!

workers-devprod commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

workers-devprod left a comment

Choose a reason for hiding this comment

Uh oh!

aicayzer commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

changeset-bot Bot commented May 29, 2026 •

edited

Loading

pkg-pr-new Bot commented Jun 1, 2026 •

edited

Loading

workers-devprod commented Jun 15, 2026 •

edited

Loading

aicayzer commented Jun 17, 2026 •

edited

Loading