Skip to content

Optimize Effect Schema and add experimental JIT and AOT compilers - #7908

Draft
gcanti wants to merge 29 commits into
mainfrom
schema-compiler
Draft

gcanti wants to merge 29 commits into
mainfrom
schema-compiler

Conversation

@gcanti

@gcanti gcanti commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Optimizes the Effect Schema interpreter and adds experimental JIT and AOT compilers. All three modes use the existing SchemaParser APIs and remain composable.

Usage

Enable lazy JIT compilation for the application:

import "effect/unstable/schema/SchemaJITCompiler/enable"

Or enable one AST and its dependencies:

import * as SchemaJITCompiler from "effect/unstable/schema/SchemaJITCompiler"

SchemaJITCompiler.enable(User.ast)

Generate and install AOT decoders:

import * as SchemaAOTCompilerBuild from "effect/unstable/schema/SchemaAOTCompiler/Build"

SchemaAOTCompilerBuild.build({
  modules: import.meta.glob("./schemas/*.ts"),
  baseUrl: import.meta.url,
  outFile: "./generated/schema-aot.js"
})

Import the generated module at application startup. Decoding is included by default; encode, is, and make roots are opt-in. SchemaAOTCompiler.compile(targets) provides the lower-level source-generation API.

Design

The interpreter, JIT, AOT, and manually installed decoders share one registry keyed by AST identity. JIT compiles lazily and falls back to the interpreter when dynamic function construction is unavailable or compilation fails. AOT uses the same code generator without new Function, emits only the requested operations and their transitive dependencies, and shares decoder factories that generate identical code. Transformations and middleware run once.

Performance

Median time per operation on Node 24.12.0, V8 13.6, Apple M3. Measurements use five isolated rounds, 100 ms warmup, 300 ms measurement, and shared batch sizes. Main interpreter values come from paired base/head runs and are normalized to the branch median from the cross-library run to avoid process-level CPU-frequency drift. Effect uses public SchemaParser APIs. Valibot 1.5.0 uses parse and is. Zod 4.6.2 uses parse and validate with { jitless: true }, or z.compile(schema, { strict: true }). Lower is better. Bold values are within 5% of the fastest result in each row.

Interpreted

Case Main interpreter Branch interpreter Valibot Zod jitless
Moltar parseSafe, valid 325.8 ns 323.7 ns 352.7 ns 400.2 ns
Moltar parseSafe, invalid 3.11 µs 3.09 µs 3.50 µs 4.82 µs
Moltar assertLoose, valid 326.8 ns 326.5 ns 421.9 ns 462.7 ns
Moltar assertLoose, invalid 131.2 ns 120.7 ns 58.3 ns 96.6 ns
Array of 32 Structs 3.17 µs 3.20 µs 2.95 µs 3.73 µs
Record of 32 Structs 3.87 µs 3.94 µs 3.86 µs 5.77 µs
Discriminated Union, 8 members 97.5 ns 95.5 ns 398.4 ns 1.29 µs
Struct with 32 transformations 1.88 µs 1.53 µs 1.54 µs 1.76 µs
Construct 32 defaulted fields 2.22 µs 865.3 ns 1.05 µs 1.50 µs
Construct Array of 32 Structs 3.13 µs 3.13 µs 2.95 µs 3.77 µs

Compiled

Case JIT AOT Zod compile
Moltar parseSafe, valid 5.42 ns 5.47 ns 5.26 ns
Moltar parseSafe, invalid 3.34 µs 3.38 µs 4.34 µs
Moltar assertLoose, valid 2.78 ns 2.78 ns 3.47 ns
Moltar assertLoose, invalid 1.07 ns 1.09 ns 1.35 ns
Array of 32 Structs 128.0 ns 126.1 ns 130.6 ns
Record of 32 Structs 594.1 ns 594.6 ns 1.52 µs
Discriminated Union, 8 members 15.7 ns 15.8 ns 30.2 ns
Struct with 32 transformations 141.5 ns 137.4 ns 207.0 ns
Construct 32 defaulted fields 362.8 ns 357.0 ns 482.1 ns
Construct Array of 32 Structs 134.4 ns 144.7 ns 126.2 ns

Paired runs against main found no interpreter regressions. The 32-field transformation improved by 18.5%, and the invalid Moltar guard improved by 7.7%.

The construction rows are reference comparisons. Effect uses SchemaParser.make, while Valibot and Zod parse equivalent schemas because they have no separate construction API.

Zod compilation has no comparable result for trailing-rest tuples, strict oneOf, middleware, or recursive schemas.

Resource costs

Incremental costs are median per-schema slopes between 100 and 500 distinct schemas across five fresh-process rounds. Lower is better. Retained heap includes schemas, public parsers, and their runtime state. Fixed module imports and fixture inputs are excluded. Importing the JIT compiler retains about 118 KiB of additional fixed heap in this source-tree setup.

Preparation CPU includes adapter creation and the first call. AOT also includes importing and installing the generated module, but excludes build-time generation. Generated AOT source is operation-specific: 2.07 KiB/schema for Struct decode, 0.62 KiB/schema for guards, 2.90 KiB/schema for an Array of 32 Structs, 14.70 KiB/schema for 32 transformations, and 11.67 KiB/schema for 32-default construction. Sharing identical decoder factories and omitting unused dependency fast paths reduce an eight-member discriminated Union from 17.72 to 5.46 KiB/schema. Sharing cold property handling and failure construction reduces retained V8 code for 32 transformations from 27.32/27.46 KiB/schema (JIT/AOT) to 13.15/12.53 KiB/schema without a measurable hot-path regression. On diagnostic paths, reusing one field parser per shared AST lowers retained heap by about 0.1 KiB at two fields, 0.3 KiB at four, 0.76 KiB at eight, and 3.4 KiB at 32; hot runtime is unchanged.

Retained JavaScript heap

Case Effect JIT Effect AOT Zod compile
Struct 5.33 KiB/schema 7.90 KiB/schema 9.37 KiB/schema
Struct invalid 10.66 KiB/schema 11.63 KiB/schema 16.25 KiB/schema
Struct guard 5.00 KiB/schema 5.59 KiB/schema 9.13 KiB/schema
Array of 32 Structs 9.73 KiB/schema 14.47 KiB/schema 12.01 KiB/schema
Discriminated Union 27.90 KiB/schema 35.90 KiB/schema 47.97 KiB/schema
32 transformations 45.67 KiB/schema 43.50 KiB/schema 101.37 KiB/schema
Construct 32 defaults 72.00 KiB/schema 73.95 KiB/schema 123.30 KiB/schema

Runtime preparation CPU

Case Effect JIT Effect AOT Zod compile
Struct 30.7 µs/schema 37.7 µs/schema 55.8 µs/schema
Struct invalid 103.1 µs/schema 85.4 µs/schema 157.1 µs/schema
Struct guard 28.4 µs/schema 24.3 µs/schema 54.0 µs/schema
Array of 32 Structs 64.6 µs/schema 82.6 µs/schema 88.3 µs/schema
Discriminated Union 98.5 µs/schema 125.4 µs/schema 236.2 µs/schema
32 transformations 537.1 µs/schema 432.1 µs/schema 372.5 µs/schema
Construct 32 defaults 406.9 µs/schema 371.9 µs/schema 651.4 µs/schema

Bundle size

Consumer Minified + gzip Delta
Main, no compiler 17.55 KB baseline
Branch, no compiler 17.88 KB +0.33 KB
Branch with JIT 24.12 KB +6.24 KB
Branch with AOT 19.92 KB +2.04 KB

Across 15 stable Schema fixtures, the branch adds 0.25 to 0.44 KB gzip, averaging 0.33 KB.

Constraints

  • Install JIT or AOT before the first parser execution that should use it.
  • Rebuild AOT output after changing Schema definitions or the Effect version.
  • AOT generation loads the selected application modules at build time.

Detailed architecture, compatibility, memory, and benchmark notes are in packages/effect/SCHEMA.md.

Verification

  • Full Schema suite
  • JIT and AOT integration tests
  • AOT execution without dynamic function construction
  • AOT builder output through public SchemaParser APIs
  • Typecheck, compiler type tests, lint, and circular-dependency checks

@changeset-bot

changeset-bot Bot commented Sep 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1882532

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-slopcop effect-slopcop Bot added the 4.0 label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 35.00 KB 34.58 KB +0.42 KB (+1.21%)
basic.ts 6.87 KB 6.87 KB 0.00 KB (0.00%)
batching.ts 9.95 KB 9.95 KB 0.00 KB (0.00%)
brand.ts 6.45 KB 6.45 KB 0.00 KB (0.00%)
cache.ts 10.77 KB 10.77 KB 0.00 KB (0.00%)
config.ts 21.83 KB 21.51 KB +0.32 KB (+1.50%)
differ.ts 20.67 KB 20.32 KB +0.35 KB (+1.72%)
http-client.ts 21.94 KB 21.94 KB 0.00 KB (0.00%)
http-router.ts 36.92 KB 36.92 KB 0.00 KB (0.00%)
logger.ts 10.88 KB 10.88 KB 0.00 KB (0.00%)
metric.ts 9.02 KB 9.02 KB 0.00 KB (0.00%)
optic.ts 6.70 KB 6.70 KB 0.00 KB (0.00%)
pubsub.ts 15.10 KB 15.10 KB 0.00 KB (0.00%)
queue.ts 11.85 KB 11.85 KB 0.00 KB (0.00%)
schedule.ts 10.96 KB 10.96 KB 0.00 KB (0.00%)
schema-binary.ts 39.82 KB 39.51 KB +0.31 KB (+0.78%)
schema-class.ts 20.38 KB 20.06 KB +0.32 KB (+1.61%)
schema-fromJsonSchemaDocument.ts 31.18 KB 30.93 KB +0.25 KB (+0.80%)
schema-representation-roundtrip.ts 26.60 KB 26.34 KB +0.26 KB (+0.99%)
schema-string-transformation.ts 13.96 KB 13.63 KB +0.33 KB (+2.39%)
schema-string.ts 11.55 KB 11.12 KB +0.43 KB (+3.89%)
schema-template-literal.ts 15.70 KB 15.41 KB +0.29 KB (+1.88%)
schema-toArbitrary.ts 34.55 KB 34.10 KB +0.44 KB (+1.30%)
schema-toCodeDocument.ts 24.84 KB 24.56 KB +0.28 KB (+1.14%)
schema-toCodecJson.ts 19.61 KB 19.27 KB +0.34 KB (+1.76%)
schema-toEquivalence.ts 19.75 KB 19.38 KB +0.37 KB (+1.92%)
schema-toFormatter.ts 19.85 KB 19.49 KB +0.36 KB (+1.86%)
schema-toJsonSchemaDocument.ts 24.15 KB 23.77 KB +0.38 KB (+1.59%)
schema-toRepresentation.ts 19.89 KB 19.54 KB +0.35 KB (+1.80%)
schema.ts 19.59 KB 19.27 KB +0.32 KB (+1.66%)
stm.ts 12.80 KB 12.80 KB 0.00 KB (0.00%)
stream.ts 9.83 KB 9.83 KB 0.00 KB (0.00%)

@gcanti
gcanti force-pushed the schema-compiler branch 2 times, most recently from 035d2b3 to 3ab7d8a Compare September 7, 2026 06:51
@effect-janitor effect-janitor Bot added the enhancement New feature or request label Sep 7, 2026
@gcanti
gcanti force-pushed the schema-compiler branch 3 times, most recently from b1330e5 to d303cc6 Compare September 11, 2026 17:33
@gcanti gcanti changed the title Schema compiler Add experimental Schema JIT and AOT compilers Sep 11, 2026
@gcanti gcanti changed the title Add experimental Schema JIT and AOT compilers Optimize Effect Schema and add experimental JIT and AOT compilers Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant