Skip to content

perf: avoid per-token mapper slice allocation - #483

Merged
alecthomas merged 1 commit into
alecthomas:masterfrom
srinathshrestha:perf/mapper-allocation
Sep 16, 2026
Merged

alecthomas merged 1 commit into
alecthomas:masterfrom
srinathshrestha:perf/mapper-allocation

Conversation

@srinathshrestha

Copy link
Copy Markdown
Contributor

Follow-up to #482.

The mapping lexer currently creates a temporary combined []Mapper for each token, appending the global & token-specific mapper slices. This PR changes it to iterate over those existing slices directly. The mapper order, error short-circuiting, original-token-type selection, and existing EOF behavior are covered by tests.

Isolated mapping-wrapper benchmark on my machine, using an allocation-free lexer with one global and one token-specific identity mapper:

before: ~86.4 ns/op    16 B/op    1 alloc/op
after:  ~64.6 ns/op     0 B/op    0 allocs/op

These are median results from 10 runs. This benchmark isolates mapper dispatch and is not intended to represent whole-parser performance.

All existing tests, the new focused tests, race tests, example tests, and the configured linter pass.

@alecthomas
alecthomas merged commit 59aab48 into alecthomas:master Sep 16, 2026
2 checks passed
@alecthomas

Copy link
Copy Markdown
Owner

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants