Skip to content

feat: Use Uint8Array instead of Buffer to adopt widely available API#801

Open
rogaldh wants to merge 39 commits intosolana-foundation:masterfrom
hoodieshq:development-replace-buffer-with-uint8array
Open

feat: Use Uint8Array instead of Buffer to adopt widely available API#801
rogaldh wants to merge 39 commits intosolana-foundation:masterfrom
hoodieshq:development-replace-buffer-with-uint8array

Conversation

@rogaldh
Copy link
Copy Markdown
Contributor

@rogaldh rogaldh commented Dec 30, 2025

Description

Replace Buffer usage with Uint8Array across the codebase for browser compatibility. Introduces a comprehensive byte-helper library (app/shared/lib/bytes.ts) that provides Uint8Array-native equivalents for all common Buffer operations: base64, hex, UTF-8 encoding/decoding, integer reads/writes (u8, u16LE, u32LE), concat, equals, alloc, and a polymorphic bytes() constructor. toBuffer() is retained solely as an adapter for external libraries (@solana/web3.js, @coral-xyz/anchor, @solana/spl-account-compression) that still require Buffer.

Type of change

  • New feature
  • Other (please describe): Internal refactor - Buffer to Uint8Array migration

Screenshots

n/a

Testing

Regression testing (preview deployment)

Address pages

SAS attestation

Transaction pages

Inspector and search

Quick checklist (from regression)

Base64

  • IDL Download: Jupiter DCA -> Download IDL -> verify valid JSON
  • Security.txt: Marinade shows security.txt info correctly
  • Transaction Logs: Serum tx, verify logs display correctly
  • Transaction Inspector: inspector, load a transaction, verify simulation works
  • Verified Builds: Orca Whirlpool, verified build badge appears

Hex

  • Hex Data Display: Serum tx, open "Raw" tab, verify hex data displays correctly
  • Verified Build Hash: Orca Whirlpool, program hash matches expected value
  • Serum/OpenBook Titles: Serum tx, instruction titles decode correctly

UTF-8

  • IDL Download: Jupiter DCA, verify JSON file contains valid UTF-8
  • Security.txt Export: Marinade, base64 export works correctly
  • Verified Builds: Orca Whirlpool, OtterSec verification check works

Buffer.alloc

  • Transaction Inspector: inspector, load ATA creation transaction, verify parsing
  • Non-existent Account: non-existent address, page loads without errors

Related Issues

n/a

Checklist

  • My code follows the project's style guidelines
  • I have added tests that prove my fix/feature works
  • All tests pass locally and in CI
  • I have updated documentation as needed
  • I have run build:info script to update build information
  • CI/CD checks pass

@vercel
Copy link
Copy Markdown

vercel bot commented Dec 30, 2025

@rogaldh is attempting to deploy a commit to the Solana Foundation Team on Vercel.

A member of the Team first needs to authorize it.

@mikhd mikhd force-pushed the development-replace-buffer-with-uint8array branch from 1f8afe4 to 396b413 Compare March 27, 2026 16:04
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
explorer Ready Ready Preview, Comment Mar 31, 2026 5:40pm

Request Review

@rogaldh rogaldh marked this pull request as ready for review March 31, 2026 14:15
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 31, 2026

Greptile Summary

This PR replaces Buffer usage across 42 files with Uint8Array-native equivalents, centralised in a new app/shared/lib/bytes.ts helper library, to improve browser compatibility. A toBuffer() adapter is retained only for the three external libraries (@solana/web3.js, @coral-xyz/anchor, @solana/spl-account-compression) that still require Buffer.

Key observations:

  • bytes.ts implementation is solid: toBase64 uses the safe per-element Array.from loop (avoiding the V8 argument-count overflow fixed in a prior round), fromHex normalises odd-length hex before calling the native path (also fixed), and toBuffer correctly preserves byteOffset/byteLength when wrapping subarrays.
  • isValidBase64 is now genuinely correct: The old SearchBar.tsx implementation called Buffer.from(str, 'base64') which never throws, so it always returned true. The migrated version uses atob() which does throw on invalid input — this is a net bug fix.
  • readUint32LE DataView usage is correct: The new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength) pattern correctly accounts for the array's byte offset, so sliced arrays (e.g. instruction.data.slice(1, 5)) are read accurately.
  • Minor API design concern: bytes.ts exports both readU8 (throws RangeError on out-of-bounds) and readUint8 (silently returns 0 via ?? 0) with nearly identical names but different error semantics, alongside the duplicate concat / concatBytes pair. These are not runtime bugs in the current PR but create a confusing surface for future callers.

Confidence Score: 5/5

PR is safe to merge; all previous round concerns are addressed and no P0/P1 issues were found.

All remaining findings are P2 style/quality suggestions (duplicate function pairs in bytes.ts). The migration logic is correct throughout, the toBase64 overflow and fromHex odd-length inconsistency from prior review rounds are both fixed, and the isValidBase64 change is a genuine bug fix. No correctness or data-integrity issues found.

app/shared/lib/bytes.ts — minor API surface cleanup opportunity (readU8 vs readUint8, concat vs concatBytes)

Important Files Changed

Filename Overview
app/shared/lib/bytes.ts New central byte-helper library replacing Buffer; generally well-implemented but exports two near-identical read-byte functions with different error semantics and two equivalent concat functions
app/shared/lib/tests/bytes.test.ts Thorough test coverage for all byte-helper functions including native API, fallback paths, and Buffer parity checks
app/components/instruction/ed25519/Ed25519DetailsCard.tsx Migrated from Buffer.readUInt8/readUInt16LE to readUint8/readUint16LE helpers; behavioral change on malformed data (silent 0 vs throw) is acceptable in this rendering context
app/components/SearchBar.tsx Replaced local isValidBase64 with the shared implementation; the new version actually validates base64 (atob throws on invalid input) unlike the old Buffer.from which never threw — this is a correct bug fix
app/components/inspector/into-parsed-data.ts Replaced Buffer.alloc/equals/discriminatorToBuffer with Uint8Array equivalents; logic preserved correctly
app/features/idl/interactive-idl/model/anchor/anchor-interpreter.ts Replaced Buffer.from(string) with fromUtf8(string) for bytes/defined-type arguments; semantically identical since Buffer.from without encoding defaults to UTF-8
app/features/idl/interactive-idl/model/pda-generator/seed-builder.ts Migrated to Uint8Array-native bytes helpers; BN LE serialisation and hex/UTF-8 seed conversion preserved correctly
app/features/security-txt/lib/fromProgramData.ts Replaced Buffer-based base64 decode and indexOf with Uint8Array equivalents; logic unchanged
app/components/instruction/pyth/program.ts Removed uint8ArrayToBuffer dependency; replaced buffer slice + toString('utf-8') with toUtf8(b.slice(...))
app/utils/verified-builds.tsx Migrated fromBase64/toHex/fromUtf8 to shared byte helpers; no logic changes
app/components/instruction/serum/types.ts Replaced Buffer.readUInt32LE with readUint32LE helper; DataView correctly uses byteOffset so slice+read semantics are preserved

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Callers across 42 files] -->|string / ArrayLike / ArrayBuffer| B["bytes() constructor"]
    A -->|hex string| C[fromHex]
    A -->|base64 string| D[fromBase64]
    A -->|UTF-8 string| E[fromUtf8]

    B --> F[Uint8Array]
    C --> F
    D --> F
    E --> F

    F -->|encode output| G[toHex]
    F -->|encode output| H[toBase64]
    F -->|decode output| I[toUtf8]
    F -->|external lib boundary| J[toBuffer → Buffer]

    J --> K["@solana/web3.js\n@coral-xyz/anchor\n@solana/spl-account-compression"]

    style J fill:#f9e4b7,stroke:#c9a227
    style K fill:#fce8e8,stroke:#c0392b
Loading

Reviews (2): Last reviewed commit: "fix: improve implementations for fromHex..." | Re-trigger Greptile

@rogaldh
Copy link
Copy Markdown
Contributor Author

rogaldh commented Mar 31, 2026

@greptile-apps check again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants