Skip to content

De-generic executor and refactor ORM stack#349

Open
KKould wants to merge 17 commits into
mainfrom
refactor-runtime-orm-stack
Open

De-generic executor and refactor ORM stack#349
KKould wants to merge 17 commits into
mainfrom
refactor-runtime-orm-stack

Conversation

@KKould

@KKould KKould commented Jun 12, 2026

Copy link
Copy Markdown
Member

Summary

This branch reduces compile-time pressure in the execution layer by de-genericizing executor nodes. The executor no longer carries the transaction/storage type parameter through every node; instead, shared execution state is routed through a runtime trait, so executor node code can be compiled once instead of being monomorphized per storage backend.

Key changes:

  • Remove the transaction generic from executor nodes and move table, view, index, statistics metadata, cursor, tuple-buffer, and DDL-apply access behind the execution runtime.
  • Normalize catalog and planner metadata arenas, including reusable table/column metadata slots and plan arena materialization for execution state.
  • Replace bincode/serde-based tuple/catalog serialization paths with project-local serdes implementations and table codec buffer reuse.
  • Rework parser, binder, and ORM integration so parser APIs are isolated, binder entry points consume owned statements, and ORM queries bind through normal binder-generated plans.
  • Reduce optional runtime dependencies by feature-gating parser, macros, rocksdb, copy, decimal, time, pyo3/wasm-related paths, and trimming shell/server dependency footprint.
  • Add execution and optimizer cleanups around static evaluator dispatch, column pruning bookkeeping, parameterized index probes, DESCRIBE column refs, set membership/subquery handling, and allocation reduction.
  • Update ORM/macros examples, README/docs, sqllogictest coverage, unit tests, and wasm examples for the new runtime and binding paths.

Binary Size Comparison

Measured as stripped release binaries on x86_64-unknown-linux-gnu with rustc 1.88.0. The KiteSQL binary is a small ORM/typed API smoke example that creates a table, inserts two rows, fetches them, and drops the table.

Important: the PR measurements intentionally do not enable the parser feature. main still has parser/sqlparser as a non-optional dependency, so main cannot be measured with the same no-parser feature set.

Target Kind Features / scope Size
KiteSQL main c0e63a0 memory embedded database orm,macros; parser not feature-gated on main 7.25 MiB
KiteSQL PR 4ee5c66 memory embedded database orm; parser disabled 2.25 MiB
KiteSQL main c0e63a0 RocksDB embedded database orm,macros,rocksdb; parser not feature-gated on main 15.74 MiB
KiteSQL PR 4ee5c66 RocksDB embedded database orm,rocksdb; parser disabled 10.08 MiB
KiteSQL main c0e63a0 LMDB embedded database orm,macros,lmdb; parser not feature-gated on main 8.27 MiB
KiteSQL PR 4ee5c66 LMDB embedded database orm,lmdb; parser disabled 2.60 MiB
rusqlite 0.34.0 + bundled SQLite SQLite binding / embedded SQLite minimal in-memory SQL smoke app 1.31 MiB
SeaORM 2.0.0-rc.38 ORM framework, not a database minimal SQLite query-builder/model app, no DB engine linked 0.39 MiB
Diesel 2.3.10 + bundled SQLite ORM/query builder, not a database minimal SQLite ORM app with bundled libsqlite3 1.36 MiB
Turso embedded database 0.7.0-pre.7 and pinned 0.5.0 did not compile on this rustc due transitive/proc-macro API requirements n/a

KiteSQL PR reduction vs main in this measurement:

  • memory: 7.25 MiB -> 2.25 MiB, down 68.9%
  • RocksDB: 15.74 MiB -> 10.08 MiB, down 36.0%
  • LMDB: 8.27 MiB -> 2.60 MiB, down 68.5%

TPCC Throughput Comparison

Measured on the current PR branch at 192c5c9 with ./scripts/run_tpcc_matrix.sh after removing old TPCC database state. The runner uses 1 warehouse, TPCC default 720s measurement windows, and removes each backend database path before running the variant. Raw logs were generated locally during the run and are not committed. Note: this is a single local long-window run; follow-up same-machine 120s checks did not reproduce a large regression at 65a57bf, so these TPCC numbers should be treated as preliminary until rerun under a controlled benchmark state.

Backend Profile TpmC Measure Time Status
KiteSQL LMDB embedded database / LMDB 56877 720s ok
KiteSQL RocksDB embedded database / RocksDB 27697 720s ok
SQLite balanced profile 39815 720s ok
SQLite practical profile 42407 720s ok

Testing

  • cargo test --workspace
  • make wasm-build && make wasm-examples
  • Size smoke builds for KiteSQL memory/RocksDB/LMDB with parser disabled on this PR branch

@KKould KKould self-assigned this Jun 12, 2026
@KKould KKould added the enhancement New feature or request label Jun 12, 2026
KKould added 2 commits June 13, 2026 03:50
Remove the transaction type parameter from executor nodes and route execution state through a runtime trait so executor code is compiled once instead of monomorphized per storage backend.
@KKould KKould changed the title Refactor runtime and ORM execution stack De-generic executor and refactor ORM stack Jun 12, 2026
@KKould KKould force-pushed the refactor-runtime-orm-stack branch from 44e2d3f to 2d22d96 Compare June 12, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant