[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows by aicayzer · Pull Request #14118 · cloudflare/workers-sdk

aicayzer · 2026-05-29T20:29:22Z

Fixes #14101. Related: #10966 (Workflows local-dev / production behaviour alignment).

Problem

In local wrangler dev, a Uint8Array returned from a Workflows step fails with string or blob too big: SQLITE_TOOBIG at sizes well below the documented 1MiB step-output limit. The same bytes wrapped as a tight ArrayBuffer succeed up to ~2MB; production accepts the Uint8Array either way. Reporter's turn-key repro: https://github.com/danieltroger/cf-workflows-local-sqlite-toobig

The cause is that v8::ValueSerializer (used by Durable Object SQL storage in workerd) writes the entire backing ArrayBuffer for typed-array views, not just byteLength bytes. When a view is created on a larger buffer — crypto.getRandomValues(new Uint8Array(N)), arr.slice(...) on a bigger pool, fetch-stream copies, etc — the wire size balloons by a factor of (backing-size / view-size).

Fix

packages/workflows-shared/src/context.ts — a small normalizeForStorage helper copies typed-array views into a tight ArrayBuffer before storage.put:

function normalizeForStorage(value: unknown): unknown {
    if (ArrayBuffer.isView(value) && !(value instanceof DataView)) {
        const view = value as ArrayBufferView;
        return new Uint8Array(view.buffer, view.byteOffset, view.byteLength)
            .slice()
            .buffer;
    }
    return value;
}

Applied in persistStepResult immediately before the storage write. The caller (the user's step.do(...)) continues to receive the original Uint8Array; only the on-disk form changes.

Files

packages/workflows-shared/src/context.ts — normalizeForStorage helper + integration in persistStepResult
packages/workflows-shared/tests/context.test.ts — four new regression tests under Context - typed-array step outputs (issue #14101): tight-buffer success, sliced-buffer success (the reporter's exact repro), oversize-rejection, round-trip live-path-shape
.changeset/fix-workflows-uint8array-step-output.md — miniflare + wrangler patch

Test plan

Tests
- Tests included/updated
- Automated tests not possible - manual testing has been completed as follows:
- Additional testing not necessary because:
Public documentation
- Cloudflare docs PR(s):
- Documentation not necessary because: this fix brings local wrangler dev behaviour in line with already-documented production behaviour (1MiB step-output limit)

Local validation:

pnpm test:ci -F @cloudflare/workflows-shared — 117/117 passing including the four new regression tests
pnpm run check:lint — clean
pnpm prettify — clean
pnpm check:type (workflows-shared) — clean

The repo-wide pnpm check reports a pre-existing check:package-deps failure unrelated to this PR (wrangler-imports-vite-without-declaring); confirmed reproducible on a clean upstream/main checkout via git stash && pnpm check.

Notes for reviewer

Top-level normalisation only. Step outputs like { image: Uint8Array, meta: {...} } would still suffer the bloat at the nested level. Happy to extend to a deep-walk (arrays + objects + Map/Set values) — left top-level only to match the smallest scope that addresses the headline bug, but I'd appreciate confirmation on whether to extend.
Cache-replay shape. Persisted form is ArrayBuffer, which means cached-replay reads also surface ArrayBuffer even if the original step returned a Uint8Array. The live execution path is unchanged (the caller's step.do() still sees the original Uint8Array). If production rehydrates to Uint8Array on replay, flag it and I'll add a side-channel __viewKind marker plus decoder.
Secondary issue: friendly catch bypass. The friendly WorkflowInternalError("Maximum allowed size is 1MiB.") at context.ts:768-774 does NOT actually fire for typed-array outputs — the raw SQLITE_TOOBIG reaches WORKFLOW_FAILURE instead. The error surfaces asynchronously from the workerd DO storage output gate, bypassing the awaited try/catch around persistStepResult. The 2MB oversize-rejection test documents this and asserts on the contract (terminal failure) rather than the message wording. Happy to file a separate follow-up issue with a more invasive fix once this lands; out of scope here to keep the change focused.

A picture of a cute animal (not mandatory, but encouraged)

…Buffer in local Workflows

changeset-bot · 2026-05-29T20:29:27Z

🦋 Changeset detected

Latest commit: 71ab67b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

Name	Type
miniflare	Patch
wrangler	Patch
@cloudflare/pages-shared	Patch
@cloudflare/vite-plugin	Patch
@cloudflare/vitest-pool-workers	Patch
@cloudflare/wrangler-bundler	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

workers-devprod · 2026-05-29T20:29:48Z

Codeowners approval required for this PR:

@cloudflare/workflows
@cloudflare/wrangler

Show detailed file reviewers

.changeset/fix-workflows-uint8array-step-output.md: [@cloudflare/wrangler]
packages/workflows-shared/src/context.ts: [@cloudflare/workflows @cloudflare/wrangler]
packages/workflows-shared/tests/context.test.ts: [@cloudflare/workflows @cloudflare/wrangler]

…step outputs Addresses review feedback on cloudflare#14118. normalizeForStorage is now a recursive walker (cycle-safe via WeakMap) that descends into arrays, plain objects, Maps, and Sets, compacting typed-array views wherever they appear in the value tree. Top-level-only normalisation missed nested views (e.g. `{ image: Uint8Array }`). Compaction now preserves the original view constructor instead of flattening every view to ArrayBuffer. Uint8Array stays Uint8Array, Int16Array stays Int16Array, DataView stays DataView, etc. The persisted shape matches the live shape, so cached replays observe the same constructor the step originally returned. Added regression tests: Int16Array round-trip on the live path, a sliced view nested in a plain object, and sliced views nested two levels deep in an array of objects. The original top-level tests still pass.

aicayzer · 2026-06-01T14:17:14Z

Both updated in 40faabceb. Lmk if anything else.

pkg-pr-new · 2026-06-01T19:53:13Z

create-cloudflare

npm i https://pkg.pr.new/cloudflare/workers-sdk/create-cloudflare@14118

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/deploy-helpers@14118

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/kv-asset-handler@14118

miniflare

npm i https://pkg.pr.new/cloudflare/workers-sdk/miniflare@14118

@cloudflare/pages-shared

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/pages-shared@14118

@cloudflare/unenv-preset

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/unenv-preset@14118

@cloudflare/vite-plugin

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/vite-plugin@14118

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/vitest-pool-workers@14118

@cloudflare/workers-auth

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-auth@14118

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-editor-shared@14118

@cloudflare/workers-utils

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/workers-utils@14118

wrangler

npm i https://pkg.pr.new/cloudflare/workers-sdk/wrangler@14118

@cloudflare/wrangler-bundler

npm i https://pkg.pr.new/cloudflare/workers-sdk/@cloudflare/wrangler-bundler@14118

commit: 71ab67b

petebacondarwin · 2026-06-03T08:37:39Z

@Caio-Nogueira - can you take another look?

devin-ai-integration

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

…tructor Per Caio-Nogueira's review suggestion on cloudflare#14118. Replaces the 13-branch instanceof dispatch with a typed cast on view.constructor, and uses view.buffer.slice() directly for the tight copy. Same behaviour, much shorter.

After cloudflare#14134 merged the binaryReplacer fix into engine.ts:writeLog, a 2 MB Uint8Array step output no longer terminates the workflow — the documented 1MiB step-output cap isn't enforced at this layer post-fix. This regression test was asserting that side-effect; remove it. The remaining typed-array tests still validate the deep-walk + view-type preservation behaviour that this PR adds.

[workflows-shared] Fix Uint8Array step outputs dragging backing Array…

eda217f

…Buffer in local Workflows

aicayzer requested a review from workers-devprod as a code owner May 29, 2026 20:29

workers-devprod requested review from a team and emily-shen and removed request for a team May 29, 2026 20:29

emily-shen mentioned this pull request Jun 1, 2026

fix(workflows): prevent SQLITE_TOOBIG crash when a step returns a large Uint8Array #14134

Merged

2 tasks

Caio-Nogueira requested changes Jun 1, 2026

View reviewed changes

Comment thread packages/workflows-shared/src/context.ts

Comment thread packages/workflows-shared/src/context.ts Outdated

Merge branch 'main' into fix/workflows-uint8array-step-output

ccfeeac

workers-devprod added this to workers-sdk Jun 1, 2026

github-project-automation Bot moved this to Untriaged in workers-sdk Jun 1, 2026

github-project-automation Bot moved this from Untriaged to In Review in workers-sdk Jun 1, 2026

petebacondarwin requested a review from Caio-Nogueira June 3, 2026 08:37

devin-ai-integration Bot reviewed Jun 3, 2026

View reviewed changes

Caio-Nogueira reviewed Jun 3, 2026

View reviewed changes

Comment thread packages/workflows-shared/src/context.ts Outdated

aicayzer added 2 commits June 3, 2026 15:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows#14118

[workflows-shared] Fix Uint8Array step outputs dragging backing ArrayBuffer in local Workflows#14118
aicayzer wants to merge 5 commits into
cloudflare:mainfrom
aicayzer:fix/workflows-uint8array-step-output

aicayzer commented May 29, 2026

Uh oh!

changeset-bot Bot commented May 29, 2026 •

edited

Loading

Uh oh!

workers-devprod commented May 29, 2026

Uh oh!

Uh oh!

Uh oh!

aicayzer commented Jun 1, 2026

Uh oh!

pkg-pr-new Bot commented Jun 1, 2026 •

edited

Loading

Uh oh!

petebacondarwin commented Jun 3, 2026

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

aicayzer commented May 29, 2026

Problem

Fix

Files

Test plan

Notes for reviewer

Uh oh!

changeset-bot Bot commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

workers-devprod commented May 29, 2026

Uh oh!

Uh oh!

Uh oh!

aicayzer commented Jun 1, 2026

Uh oh!

pkg-pr-new Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petebacondarwin commented Jun 3, 2026

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

✅ Devin Review: No Issues Found

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

changeset-bot Bot commented May 29, 2026 •

edited

Loading

pkg-pr-new Bot commented Jun 1, 2026 •

edited

Loading