@smithy/core@3.24.0: "Buffer is not defined" in browser — toUint8Array relative import bypasses browser field after serde consolidation

[gogakoreli]

Bug Report

Summary

@smithy/core@3.24.0 causes ReferenceError: Buffer is not defined in browser environments during AWS SDK request signing. The core/serde package consolidation (PR #1985, commit 8963b91) moved @smithy/util-utf8 into @smithy/core as a submodule but lost the per-file browser field mappings that previously redirected Node.js-specific code to browser-safe alternatives.

This is a sibling regression to #2022 (crypto not defined on Node 18) — both stem from the same consolidation effort losing environment-specific code paths.

Stack Trace

ReferenceError: Buffer is not defined
    at fromString (buffer-from.js:12:1)
    at fromUtf8 (fromUtf8.js:3:1)
    at toUint8Array (toUint8Array.js:4:1)
    at SignatureV4.createStringToSign (SignatureV4Base.js:33:1)
    at SignatureV4.getSignature (SignatureV4.js:132:1)
    at SignatureV4.signRequest (SignatureV4.js:123:1)
    at async AwsSdkSigV4Signer.sign (AwsSdkSigV4Signer.js:42:1)
    at async httpSigningMiddleware.js:19:1

Affected Versions

• Broken: @smithy/core@3.24.0 (published May 8, 2026) and @smithy/core@3.24.1
• Last working: @smithy/core@3.23.17
• Bundler: webpack 5 (likely affects all bundlers that rely on the browser field)

Root Cause Analysis

The call chain

toUint8Array.ts
  └─ import { fromUtf8 } from "./fromUtf8"       ← relative import
       └─ fromUtf8.ts
            └─ import { fromString } from "../util-buffer-from/buffer-from"
                 └─ Buffer.from(input, encoding)   ← 💥 doesn't exist in browser

What was lost during consolidation

Before — standalone @smithy/util-utf8@4.2.0 had per-file browser mappings in its package.json:

"browser": {
  "./dist-es/fromUtf8": "./dist-es/fromUtf8.browser",
  "./dist-es/toUtf8": "./dist-es/toUtf8.browser"
}

These told webpack: "when resolving ./fromUtf8 inside this package, use fromUtf8.browser.js (which uses TextEncoder) instead of fromUtf8.js (which uses Buffer.from())."

After — @smithy/core@3.24.0 only has entry-point level mappings:

"browser": {
  "./dist-es/submodules/serde/index.js": "./dist-es/submodules/serde/index.browser.js"
}

This correctly routes the top-level import (@smithy/core/serde → index.browser.js), and index.browser.js correctly exports from fromUtf8.browser.ts. However, toUint8Array.ts is the same file in both entry points, and it has a relative import:

// packages/core/src/submodules/serde/util-utf8/toUint8Array.ts
import { fromUtf8 } from "./fromUtf8";  // always resolves to Node.js version

Once webpack is inside the package following relative imports, the entry-point browser mapping has no effect. The relative ./fromUtf8 resolves to fromUtf8.js (Node.js, Buffer.from()) — never fromUtf8.browser.js (browser, TextEncoder).

Why PR #2009 didn't fix this

PR #2009 (commit 7ec62a0, "fix browser bundler metadata") restructured the approach to use variant index files for browser/native. The index.browser.ts correctly imports from ./util-utf8/fromUtf8.browser. But it didn't address the internal cross-reference — toUint8Array.ts still imports from ./fromUtf8 (the Node version), and this file is shared by both index.ts and index.browser.ts.

Regression Cross-References

Date	Event	Reference
Apr 29	Consolidation tracking issue opened	#1980
May 1	core/serde consolidation PR merged	#1985 (commit 8963b91)
May 6	Browser metadata fix PR merged (entry-point only)	#2009 (commit 7ec62a0)
May 8	@smithy/core@3.24.0 published — regression goes live	npm
May 8	Sibling bug reported: crypto is not defined on Node 18	#2022
May 11	@smithy/core@3.24.1 published — uuid fix only, Buffer bug persists	npm

Suggested Fixes

Option A — Add per-file browser mappings to @smithy/core's package.json:

"browser": {
  "./dist-es/submodules/serde/util-utf8/fromUtf8.js": "./dist-es/submodules/serde/util-utf8/fromUtf8.browser.js",
  "./dist-es/submodules/serde/util-utf8/toUtf8.js": "./dist-es/submodules/serde/util-utf8/toUtf8.browser.js",
  "./dist-es/submodules/serde/util-buffer-from/buffer-from.js": false
}

Option B — Create a browser-specific toUint8Array.browser.ts that imports from ./fromUtf8.browser and export it from index.browser.ts.

Option C — Refactor toUint8Array.ts to use TextEncoder directly (no dependency on fromUtf8 at all), since TextEncoder is available in all modern browsers and Node.js 16+:

export const toUint8Array = (data: string | ArrayBuffer | ArrayBufferView): Uint8Array => {
  if (typeof data === "string") {
    return new TextEncoder().encode(data);
  }
  // ...
};

Current Workaround

For webpack users, inject Buffer globally via ProvidePlugin:

const webpack = require('webpack');

module.exports = {
  plugins: [
    new webpack.ProvidePlugin({
      Buffer: ['buffer', 'Buffer'],
    }),
  ],
  resolve: {
    fallback: {
      buffer: require.resolve('buffer/'),
    },
  },
};

Environment

• Bundler: webpack 5
• Target: browser (ES2020)
• Framework: React 18
• AWS SDK: @aws-sdk/client-* (latest, depends on @smithy/core@^3.24.0)

[gogakoreli]

Additional Findings: 5 More Identical Bugs in the Same Consolidation

After auditing all internal imports within @smithy/core's serde submodule, the toUint8Array → fromUtf8 issue reported above is just the first to surface (because SigV4 signing fires on every request). There are at least 5 more browser-designated files that import Node.js-specific code via relative paths — all reachable from index.browser.ts.

Authoritative Evidence: Pre-Consolidation Browser Fields

These standalone packages had per-file browser field mappings that were lost during consolidation:

Package (pre-consolidation)	browser field (from npm registry)
@smithy/util-utf8@4.2.0	"./dist-es/fromUtf8": "./dist-es/fromUtf8.browser", "./dist-es/toUtf8": "./dist-es/toUtf8.browser"
@smithy/util-base64@4.2.0	"./dist-es/fromBase64": "./dist-es/fromBase64.browser", "./dist-es/toBase64": "./dist-es/toBase64.browser"
@smithy/util-stream@4.4.0	7 per-file mappings for headStream, splitStream, sdk-stream-mixin, createBufferedReadable, ChecksumStream, getAwsChunkedEncodingStream, createChecksumStream

@smithy/core@3.24.0's browser field only maps entry-point indexes — none of these per-file redirects survived.

Affected Files (all within packages/core/src/submodules/serde/)

1. util-base64/toBase64.browser.ts → Node fromUtf8

import { fromUtf8 } from "../util-utf8/fromUtf8";  // ← Node version (Buffer.from)

Exported from index.browser.ts line 57. Triggers on any toBase64("string") call in browser.

2. util-stream/sdk-stream-mixin.browser.ts → Node toBase64 + Node toUtf8

import { toBase64 } from "../util-base64/toBase64";  // ← Node version (Buffer)
import { toUtf8 } from "../util-utf8/toUtf8";        // ← Node version (Buffer)

Exported from index.browser.ts line 93. Triggers on .transformToString("base64") or .transformToString("utf-8") on any SDK response body.

3. util-stream/stream-collector.browser.ts → Node fromBase64

import { fromBase64 } from "../util-base64/fromBase64";  // ← Node version (Buffer)

Used by sdk-stream-mixin.browser.ts. Triggers when collecting a Blob response body (FileReader → base64 → decode path).

4. util-stream/blob/Uint8ArrayBlobAdapter.ts → ALL Node versions

import { fromBase64 } from "../../util-base64/fromBase64";  // ← Node
import { toBase64 } from "../../util-base64/toBase64";      // ← Node
import { fromUtf8 } from "../../util-utf8/fromUtf8";        // ← Node
import { toUtf8 } from "../../util-utf8/toUtf8";            // ← Node

Exported from index.browser.ts line 87. Triggers on any Uint8ArrayBlobAdapter usage (response body conversion).

5. util-stream/checksum/createChecksumStream.browser.ts → Node toBase64

import { toBase64 } from "../../util-base64/toBase64";  // ← Node version (Buffer)

Exported from index.browser.ts line 89. Triggers on streaming responses with checksum validation (e.g., S3 downloads).

Why Only toUint8Array Surfaces First

Execution Order	Code Path	When It Fires
1st 💥	toUint8Array → fromUtf8 → Buffer	Every request (SigV4 signing)
2nd	toBase64.browser → fromUtf8 → Buffer	Request body encoding
3rd	sdk-stream-mixin.browser → toBase64/toUtf8 → Buffer	Response body consumption
4th	Uint8ArrayBlobAdapter → all → Buffer	Response body conversion
5th	stream-collector.browser → fromBase64 → Buffer	Blob response collection
6th	createChecksumStream.browser → toBase64 → Buffer	Streaming checksum validation

SigV4 signing fires before the request leaves the browser, so toUint8Array crashes first. The others would surface progressively as response bodies are consumed — but a Buffer polyfill workaround masks them all.

Root Pattern

The .browser.ts files were written assuming the old per-file browser field would redirect their relative imports at bundle time. After consolidation, those redirects no longer exist, so ../util-utf8/fromUtf8 resolves to the Node.js version everywhere.

Suggested Comprehensive Fix

Rather than adding dozens of per-file browser mappings, consider making the browser files import explicitly from the .browser variants:

- import { fromUtf8 } from "../util-utf8/fromUtf8";
+ import { fromUtf8 } from "../util-utf8/fromUtf8.browser";

Or use the dependency injection pattern (already used successfully in retry and endpoints submodules) to pass environment-specific implementations from the index file.

[kuhe]

Hi, a fix has been released in https://www.npmjs.com/package/@smithy/core/v/3.24.2.

[gogakoreli]

that was quick! thanks