Skip to content

Optimize Effect Schema and add experimental JIT and AOT compilers - #7908

Draft
gcanti wants to merge 23 commits into
mainfrom
schema-compiler
Draft

gcanti wants to merge 23 commits into
mainfrom
schema-compiler

Conversation

@gcanti

@gcanti gcanti commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Optimizes the Effect Schema interpreter and adds experimental JIT and AOT compilers. All three modes use the existing SchemaParser APIs and remain composable.

Usage

Enable lazy JIT compilation for the application:

import "effect/unstable/schema/SchemaJITCompiler/enable"

Or enable one AST and its dependencies:

import * as SchemaJITCompiler from "effect/unstable/schema/SchemaJITCompiler"

SchemaJITCompiler.enable(User.ast)

Generate and install AOT decoders:

import * as SchemaAOTCompilerBuild from "effect/unstable/schema/SchemaAOTCompiler/Build"

SchemaAOTCompilerBuild.build({
  modules: import.meta.glob("./schemas/*.ts"),
  baseUrl: import.meta.url,
  outFile: "./generated/schema-aot.js"
})

Import the generated module at application startup. Decoding is included by default; encode, is, and make roots are opt-in. SchemaAOTCompiler.compile(targets) provides the lower-level source-generation API.

Design

The interpreter, JIT, AOT, and manually installed decoders share one registry keyed by AST identity. JIT compiles lazily and falls back to the interpreter when dynamic function construction is unavailable or compilation fails. AOT uses the same code generator without new Function, emits only the requested operations and their transitive dependencies, and shares decoder factories that generate identical code. Transformations and middleware run once.

Performance

Median time per operation on Node 24.12.0, V8 13.6, Apple M3. Measurements use five isolated rounds, 100 ms warmup, 300 ms measurement, and shared batch sizes. Effect uses public SchemaParser APIs. Valibot 1.5.0 uses parse and is. Zod 4.6.2 uses parse and validate with { jitless: true }, or z.compile(schema, { strict: true }). Lower is better. Bold values are within 5% of the fastest result in each row.

Interpreted

Case Main interpreter Branch interpreter Valibot Zod jitless
Moltar parseSafe, valid 363.9 ns 326.3 ns 353.7 ns 400.3 ns
Moltar parseSafe, invalid 3.05 µs 3.06 µs 3.51 µs 4.79 µs
Moltar assertLoose, valid 332.3 ns 326.8 ns 427.3 ns 469.9 ns
Moltar assertLoose, invalid 129.2 ns 118.4 ns 57.7 ns 96.3 ns
Array of 32 Structs 3.15 µs 3.11 µs 3.16 µs 3.79 µs
Record of 32 Structs 3.81 µs 3.81 µs 3.81 µs 5.83 µs
Discriminated Union, 8 members 91.7 ns 93.8 ns 397.4 ns 1.29 µs
Struct with 32 transformations 2.00 µs 1.52 µs 1.52 µs 1.77 µs
Construct 32 defaulted fields 2.17 µs 868.1 ns 1.06 µs 1.48 µs
Construct Array of 32 Structs 3.15 µs 3.11 µs 2.99 µs 3.73 µs

Compiled

Case JIT AOT Zod compile
Moltar parseSafe, valid 5.55 ns 5.51 ns 5.33 ns
Moltar parseSafe, invalid 3.39 µs 3.44 µs 4.42 µs
Moltar assertLoose, valid 2.95 ns 2.93 ns 3.46 ns
Moltar assertLoose, invalid 1.04 ns 1.04 ns 1.34 ns
Array of 32 Structs 132.6 ns 129.9 ns 127.6 ns
Record of 32 Structs 594.2 ns 594.3 ns 1.51 µs
Discriminated Union, 8 members 15.5 ns 15.7 ns 29.6 ns
Struct with 32 transformations 154.2 ns 155.4 ns 204.3 ns
Construct 32 defaulted fields 356.8 ns 358.1 ns 481.8 ns
Construct Array of 32 Structs 133.9 ns 145.0 ns 128.6 ns

Paired runs against main found no interpreter regressions. The 32-field transformation improved by 19.5%, and the invalid Moltar guard improved by 8%.

The construction rows are reference comparisons. Effect uses SchemaParser.make, while Valibot and Zod parse equivalent schemas because they have no separate construction API.

Zod compilation has no comparable result for trailing-rest tuples, strict oneOf, middleware, or recursive schemas.

Resource costs

Incremental costs are median per-schema slopes between 100 and 500 distinct schemas across five fresh-process rounds. Lower is better. Retained heap includes schemas, public parsers, and their runtime state. Fixed module imports and fixture inputs are excluded. Importing the JIT compiler retains about 118 KiB of additional fixed heap in this source-tree setup.

Preparation CPU includes adapter creation and the first call. AOT also includes importing and installing the generated module, but excludes build-time generation. Generated AOT source is operation-specific: 2.07 KiB/schema for Struct decode, 0.62 KiB/schema for guards, 2.90 KiB/schema for an Array of 32 Structs, 28.24 KiB/schema for 32 transformations, and 11.67 KiB/schema for 32-default construction. Sharing identical decoder factories and omitting unused dependency fast paths reduce an eight-member discriminated Union from 17.72 to 5.46 KiB/schema.

Retained JavaScript heap

Case Effect JIT Effect AOT Zod compile
Struct 5.33 KiB/schema 7.90 KiB/schema 9.37 KiB/schema
Struct invalid 10.66 KiB/schema 11.63 KiB/schema 16.25 KiB/schema
Struct guard 5.00 KiB/schema 5.59 KiB/schema 9.13 KiB/schema
Array of 32 Structs 9.82 KiB/schema 14.47 KiB/schema 12.01 KiB/schema
Discriminated Union 27.90 KiB/schema 38.54 KiB/schema 47.97 KiB/schema
32 transformations 81.56 KiB/schema 81.11 KiB/schema 101.37 KiB/schema
Construct 32 defaults 72.00 KiB/schema 73.96 KiB/schema 123.30 KiB/schema

Runtime preparation CPU

Case Effect JIT Effect AOT Zod compile
Struct 31.7 µs/schema 39.2 µs/schema 55.6 µs/schema
Struct invalid 103.4 µs/schema 83.4 µs/schema 156.9 µs/schema
Struct guard 28.0 µs/schema 24.1 µs/schema 52.2 µs/schema
Array of 32 Structs 66.2 µs/schema 82.9 µs/schema 88.6 µs/schema
Discriminated Union 96.1 µs/schema 126.2 µs/schema 234.1 µs/schema
32 transformations 940.0 µs/schema 760.8 µs/schema 371.0 µs/schema
Construct 32 defaults 404.3 µs/schema 373.0 µs/schema 654.9 µs/schema

Bundle size

Consumer Minified + gzip Delta
Main, no compiler 17.55 KB baseline
Branch, no compiler 17.84 KB +0.29 KB
Branch with JIT 23.85 KB +6.01 KB
Branch with AOT 19.77 KB +1.93 KB

Across 15 stable Schema fixtures, the branch adds 0.25 to 0.44 KB gzip, averaging 0.33 KB.

Constraints

  • Install JIT or AOT before the first parser execution that should use it.
  • Rebuild AOT output after changing Schema definitions or the Effect version.
  • AOT generation loads the selected application modules at build time.

Detailed architecture, compatibility, memory, and benchmark notes are in packages/effect/SCHEMA.md.

Verification

  • Full Schema suite
  • JIT and AOT integration tests
  • AOT execution without dynamic function construction
  • AOT builder output through public SchemaParser APIs
  • Typecheck, compiler type tests, lint, and circular-dependency checks

@changeset-bot

changeset-bot Bot commented Sep 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3cb8c6c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-slopcop effect-slopcop Bot added the 4.0 label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 35.00 KB 34.58 KB +0.42 KB (+1.21%)
basic.ts 6.87 KB 6.87 KB 0.00 KB (0.00%)
batching.ts 9.95 KB 9.95 KB 0.00 KB (0.00%)
brand.ts 6.45 KB 6.45 KB 0.00 KB (0.00%)
cache.ts 10.77 KB 10.77 KB 0.00 KB (0.00%)
config.ts 21.83 KB 21.51 KB +0.32 KB (+1.50%)
differ.ts 20.67 KB 20.32 KB +0.35 KB (+1.72%)
http-client.ts 21.93 KB 21.93 KB 0.00 KB (0.00%)
http-router.ts 36.92 KB 36.92 KB 0.00 KB (0.00%)
logger.ts 10.88 KB 10.88 KB 0.00 KB (0.00%)
metric.ts 9.02 KB 9.02 KB 0.00 KB (0.00%)
optic.ts 6.70 KB 6.70 KB 0.00 KB (0.00%)
pubsub.ts 15.10 KB 15.10 KB 0.00 KB (0.00%)
queue.ts 11.85 KB 11.85 KB 0.00 KB (0.00%)
schedule.ts 10.96 KB 10.96 KB 0.00 KB (0.00%)
schema-binary.ts 39.82 KB 39.51 KB +0.31 KB (+0.78%)
schema-class.ts 20.38 KB 20.06 KB +0.32 KB (+1.61%)
schema-fromJsonSchemaDocument.ts 31.18 KB 30.93 KB +0.25 KB (+0.80%)
schema-representation-roundtrip.ts 26.60 KB 26.34 KB +0.26 KB (+0.98%)
schema-string-transformation.ts 13.94 KB 13.63 KB +0.31 KB (+2.26%)
schema-string.ts 11.55 KB 11.12 KB +0.43 KB (+3.89%)
schema-template-literal.ts 15.69 KB 15.41 KB +0.27 KB (+1.78%)
schema-toArbitrary.ts 34.55 KB 34.10 KB +0.44 KB (+1.30%)
schema-toCodeDocument.ts 24.84 KB 24.56 KB +0.28 KB (+1.14%)
schema-toCodecJson.ts 19.61 KB 19.27 KB +0.34 KB (+1.76%)
schema-toEquivalence.ts 19.75 KB 19.38 KB +0.37 KB (+1.92%)
schema-toFormatter.ts 19.85 KB 19.49 KB +0.36 KB (+1.85%)
schema-toJsonSchemaDocument.ts 24.15 KB 23.77 KB +0.38 KB (+1.59%)
schema-toRepresentation.ts 19.89 KB 19.54 KB +0.35 KB (+1.80%)
schema.ts 19.59 KB 19.27 KB +0.32 KB (+1.66%)
stm.ts 12.80 KB 12.80 KB 0.00 KB (0.00%)
stream.ts 9.83 KB 9.83 KB 0.00 KB (0.00%)

@gcanti
gcanti force-pushed the schema-compiler branch 2 times, most recently from 035d2b3 to 3ab7d8a Compare September 7, 2026 06:51
@effect-janitor effect-janitor Bot added the enhancement New feature or request label Sep 7, 2026
@gcanti
gcanti force-pushed the schema-compiler branch 3 times, most recently from b1330e5 to d303cc6 Compare September 11, 2026 17:33
@gcanti gcanti changed the title Schema compiler Add experimental Schema JIT and AOT compilers Sep 11, 2026
@gcanti gcanti changed the title Add experimental Schema JIT and AOT compilers Optimize Effect Schema and add experimental JIT and AOT compilers Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant