HyperDB

The reactive database for local-first apps. Define schemas, queries, selectors, and actions once, then use that data layer in the browser and on the server.

📖 Full documentation → hyperdb.will-be-done.app

What it solves

HyperDB brings the developer experience of a backend like Convex to a database that can run in the browser and on the server. You describe data with typed schemas, read it through reactive selectors, and change it through transactional actions. The same slice of schema, selectors, and actions can be shared by the client and backend; only the storage driver differs.

It is designed for the parts of local-first apps where plain client state starts to strain:

• Efficient inserts into sorted data. Every table index is backed by a real B-tree, so inserting into a sorted collection stays O(log n) instead of rebuilding or shifting a whole array. This fits fractional indexing in local-first apps.
• Explicit query execution. SQL is powerful, but the query text does not usually tell you whether the database will use an index or scan a whole table. In HyperDB, selectors name the table index they read and build explicit bounds over it, so the code shows the access path it will take.
• Fine-grained reactivity. Selectors record exactly which index ranges they scanned, so a mutation only re-runs the selectors that overlap it, without proxies or observer().
• Lazy persistent reads when you need them. HybridDB combines a persistent primary store with an in-memory B-tree cache. Reads return cached ranges when possible and load missing ranges from the primary store on demand. Writes update the cache first so the UI can respond immediately, then flush to the primary store in order.
• Preloaded indexes without preloaded rows. PreloadedHybridDB loads every declared index into memory as key-to-id entries at startup, then batch-loads only the entity rows selected by a scan and caches them by id. Transactions publish memory changes before background persistence. asyncDispatch runs cached actions before returning its promise, suspending when an operation needs database I/O, a busy transaction lock, or an unresolved promise yielded by the action. Await execAsync(db.waitForPersistence()) when you need committed changes saved to the primary store. Both hybrid runtimes reject transactions before publishing memory changes when 100 persistence batches are unsettled; await persistence and retry in a new transaction. Rejected commits finish the transaction, while rollback remains safe for cleanup. On any runtime, a failed commit() did not commit: roll back and retry in a new transaction. Ordinary PreloadedHybridDB inserts preserve duplicate-ID checks during background persistence.
• Run the same logic on the backend. Because a table index is just a B-tree, the same schema, selectors, and actions run against a persistent store on the server (SQLite today, pg/mongodb in future). The runtime reads only the rows a selector touches instead of hydrating the whole dataset into memory.
• JavaScript selectors and actions. Selectors and actions are ordinary JS: loops, conditionals, function calls. You get fast indexed lookups underneath, not a query language to learn.

The devtool records selector runs and mutations in a call tree:

Who needs this

Reach for HyperDB when you want structured, queryable, reactive data shared across your whole stack:

• Local-first apps that work offline and sync to a server in the background, plus a server that runs the very same schema and sync logic.
• Apps with rich data models (tasks, documents, boards) that need indexed lookups and ordering on both client and server.
• Large sorted collections you reorder or insert into with fractional indexing, where array-based state becomes costly.
• Anywhere you'd otherwise duplicate models and queries between frontend and backend.

Installation

npm install @will-be-done/hyperdb

The React devtool ships separately. It traces every selector run and mutation into a browsable call tree, so you can see which index a slow view scanned. For HybridDB reads, select nodes are labeled in-mem or persist to show whether the returned rows came from the memory cache or the primary persistent store, and trace rows get an in-mem badge when no select fell through to a persistent scan — every read was served from the memory cache (mutations, which flush separately, don't affect it). You can sort traces by creation time, duration, or rows fetched, and when you switch traces, the active detail tab stays selected so comparison stays focused:

npm install @will-be-done/hyperdb-devtool

Quick start

// 1. Define a typed table (id + query indexes)
import {
  defineTable,
  v,
  type ExtractSchema,
  type ExtractSchemaInput,
} from "@will-be-done/hyperdb";

export const tasksTable = defineTable("tasks", {
  id: v.string(),
  projectId: v.string(),
  title: v.string(),
  slug: v.string(),
  orderToken: v.string(),
})
  .index("byProjectOrder", ["projectId", "orderToken"])
  .index("bySlug", ["slug"], { type: "uniqhash" })
  .index("byIds", ["id"]);

export type Task = ExtractSchema<typeof tasksTable>; // what reads return
export type NewTask = ExtractSchemaInput<typeof tasksTable>; // what writes accept

ExtractSchema and ExtractSchemaInput differ only where a validator accepts more than it stores: a v.arrayBuffer() column accepts any typed array or DataView on write and always reads back as an ArrayBuffer.

// 2. Create shared builders
import { createSelector, createAction } from "@will-be-done/hyperdb";

export const selector = createSelector({
  validateArgs: process.env.NODE_ENV === "development",
});
export const action = createAction({
  validateArgs: process.env.NODE_ENV === "development",
});

// 3. Write a selector and an action as plain generators
import { selectFrom, insert, v } from "@will-be-done/hyperdb";
import { selector, action } from "./builders";
import { tasksTable } from "./schema";

export const projectTasks = selector({
  name: "projectTasks",
  args: { projectId: v.string() },
  handler: function* ({ projectId }) {
    return yield* selectFrom(tasksTable, "byProjectOrder")
      .where((q) => q.eq("projectId", projectId))
      .order("asc");
  },
});

export const createTask = action({
  name: "createTask",
  args: { id: v.string(), projectId: v.string(), title: v.string() },
  handler: function* ({ id, projectId, title }) {
    yield* insert(tasksTable, [
      { id, projectId, title, slug: id, orderToken: id },
    ]);
  },
});

Queries can also return OR branches with or(...) or arrays from where. When combined with .order(...), those branches are merged into the index order before rows are returned.

// 4. Create a HybridDB (IndexedDB primary + in-memory cache)
import { DB, HybridDB, SubscribableDB, execAsync } from "@will-be-done/hyperdb";
import { openIndexedDBDriver } from "@will-be-done/hyperdb/drivers/idb";
import { BptreeInmemDriver } from "@will-be-done/hyperdb/drivers/inmemory";
import { tasksTable } from "./schema";

export async function createAppDB() {
  const primary = new DB(await openIndexedDBDriver("task-app"), {
    runtimeRowsValidation: process.env.NODE_ENV === "development",
    freezeArgs: process.env.NODE_ENV === "development",
    freezeRows: process.env.NODE_ENV === "development",
  });
  const cache = new DB(new BptreeInmemDriver());
  const db = new SubscribableDB(new HybridDB(primary, cache));

  await execAsync(db.loadTables([tasksTable]));

  // Optional: warm the cache with the whole table so future reads can stay in memory.
  // await execAsync(
  //   db.preloadTables([{ table: tasksTable, scanIndex: "byIds" }]),
  // );

  return db;
}

AsyncSqlDriver is also exercised against Turso Database's browser WASM engine in the shared driver conformance suite.

If all index keys fit in memory but all entity rows do not, use new SubscribableDB(new PreloadedHybridDB(primary)) instead. Its loadTables call automatically preloads every index on every loaded table; scans resolve ordered IDs in memory, including non-ID uniqhash value-to-ID pointers, and batch-fetch only unresolved rows through the built-in byId entity index. No explicit preloadTables call or extra B-tree byIds index is needed. Repeated loadTables calls are incremental: previously loaded tables remain available while supplied table definitions are added or refreshed. Supplied tables preload concurrently by default. Pass { preloadConcurrency: n } to bound concurrent whole-table reads, or { preloadConcurrency: "whole" } to state the default explicitly. Drivers may serialize internally; higher concurrency can temporarily retain several decoded tables before their rows are released. Exact byId misses reconcile rows added by another connected runtime. Apply changesets already persisted by another runtime sharing the primary with externalStorageMergeTrait. Their normal merge operations update the preloaded snapshot and invalidate subscribers, while external inserts persist idempotently instead of failing as duplicates.

If your whole app state can be loaded into memory at startup, you may not need HybridDB. A plain new SubscribableDB(new DB(new BptreeInmemDriver())) keeps reads and writes synchronous, so you can use useSyncSelector, useSyncDispatch, selectSync, and syncDispatch without promise or cache-miss tradeoffs. For one-off reads that should reuse the selector cache, use selectCachedSync, selectCachedAsync, or selectCachedMaybeAsync. For subscriptions outside React, use createSelectorStoreSync / createCachedSelectorStoreSync when selector results are available synchronously. Use createAsyncSelectorStore / createCachedSelectorStoreAsync when a HybridDB/async driver run may need to resolve a promise and expose query state.

SubscribableDB also exposes lifecycle hooks: mutation hooks such as afterInsert, afterUpsert, afterDelete, and afterChange, plus afterScan for successful index scans. HybridDB keeps the primary store persistent while serving cached index ranges from memory and loading missing range portions from the primary store.

import {
  DBProvider,
  useAsyncSelector,
  useAsyncDispatch,
} from "@will-be-done/hyperdb/react";
import type { SubscribableDB } from "@will-be-done/hyperdb";
import { HyperDBDevtools } from "@will-be-done/hyperdb-devtool/react";
import { createTask, projectTasks } from "./tasks";

function Tasks({ projectId }: { projectId: string }) {
  const { data: tasks = [], isFetching } = useAsyncSelector({
    selector: projectTasks,
    args: { projectId },
    defaultValue: [],
  });
  const dispatch = useAsyncDispatch();

  return (
    <>
      <button
        disabled={isFetching}
        onClick={() =>
          void dispatch(
            createTask({
              id: crypto.randomUUID(),
              projectId,
              title: "New task",
            }),
          )
        }
      >
        Add task
      </button>
      <ul>
        {tasks.map((task) => (
          <li key={task.id}>{task.title}</li>
        ))}
      </ul>
    </>
  );
}

export function App({ db }: { db: SubscribableDB }) {
  return (
    <DBProvider value={db}>
      <Tasks projectId="p1" />
      {/* Drop in the devtool to trace selectors and mutations */}
      <HyperDBDevtools db={db} initialIsOpen={false} />
    </DBProvider>
  );
}

useAsyncSelector attempts the selector once for its initial snapshot. If the run finishes synchronously, that value is visible immediately; if it returns a promise, the hook shows defaultValue, initialData, or placeholderData until that same promise resolves. The hook uses createCachedSelectorStoreAsync internally, so the same async subscription behavior is available without React. defaultValue may be a value or a zero-argument function that returns the value.

Undo and redo

Attach transaction history to a SubscribableDB, selecting the domain tables that users can undo:

import { execSync, execAsync } from "@will-be-done/hyperdb";
import { undoMiddleware, withoutUndo } from "@will-be-done/hyperdb/undo";

const history = undoMiddleware(db, {
  tables: [tasksTable],
  maxUndoLevels: 100,
});

execSync(history.undo()); // after a recorded edit; throws if no undo is available
execSync(history.redo());
// Use await execAsync(history.undo()) with asynchronous runtimes.

function* edit() {
  yield* history.withoutUndo(function* () {
    yield* upsert(tasksTable, [task]); // skip this manager only
  });
  yield* withoutUndo(function* () {
    yield* upsert(tasksTable, [otherTask]); // skip every manager
  });
}

The manager exposes queues, canUndo / canRedo, limits, subscriptions, withGroup, createGroup, and dropUndo / dropRedo. Replay runs atomically and restores only the fields each update wrote; it reports a conflict rather than overwriting fields changed outside history. Columns annotated with .with(undoIgnore()) are never undone, and skip(traits) leaves writes out of history by trait. Track source tables and let mutation hooks rebuild derived data. History is in memory. Groups with no net changes preserve redo. With HybridDB, a background persistence failure can occur after the cache commit and history queues have moved. See Undo & Redo for callback exclusions outside actions, grouping rules, hooks, and conflict handling.

Typed selectors and actions

Selector handlers are typed to yield only reads, so writing from a selector is a compile error, and the runtime rejects it too. Annotate reusable helpers with SelectorGenerator<T> for reads or ActionGenerator<T> for helpers that may write.

Errors and shutdown

Runtime errors extend HyperDBError and carry a stable code such as DUPLICATE_ID, VALIDATION, WRITES_DISALLOWED, HYBRID_DB_CRASHED, or DB_CLOSED. Match on the code with isHyperDBError(error, code). Mistakes in how tables, actions, or queries are defined (such as a blank action name or conflicting query conditions) throw a plain Error, for which isHyperDBError returns false. HyperDB does not log the errors it throws.

Every driver, SQLite included, reports a write that reuses an existing id (or repeats one inside a batch) as DuplicateIdError with code DUPLICATE_ID and the colliding id. A uniqhash violation is a different error: the storage engine's own error passes through unchanged, for SQLite a UNIQUE constraint failed error.

Use dispatchMaybeAsync(db, action(args)) when you want a synchronous result whenever no step had to wait. Close a runtime with await execAsync(db.close()): it flushes hybrid persistence, releases driver-owned resources, clears subscribers, and unregisters the DB from devtools. Concurrent close() calls share one close. HybridDB also waits for direct writes already updating the primary and cache, without waiting for open, uncommitted transactions. SQLite inserts remain all-or-nothing when a handler catches a trigger failure. Hybrid shutdown continues if its caller abandons the command. A failed SubscribableDB.close() still rejects new operations while allowing cleanup retries. Subscriber callbacks receive readonly Op[]; leave shared operations and rows unchanged, and copy the values you need to edit.

Entry points

Import path	Contents
@will-be-done/hyperdb	Core: defineTable, v, selectFrom, builders, DB, HybridDB, PreloadedHybridDB, SubscribableDB
@will-be-done/hyperdb/react	React hooks and DBProvider
@will-be-done/hyperdb/undo	Undo/redo managers, scoped exclusions, grouping, and conflict errors
@will-be-done/hyperdb/tracing	Tracing store and tracer configuration
@will-be-done/hyperdb/drivers/inmemory	BptreeInmemDriver
@will-be-done/hyperdb/drivers/sqlite	SqlDriver, AsyncSqlDriver
@will-be-done/hyperdb/drivers/idb	openIndexedDBDriver, IdbDriver
@will-be-done/hyperdb-devtool/react	HyperDBDevtools, HyperDBDevtoolsPanel (separate package)

Development

pnpm install
pnpm run test           # hyperdb and devtool suites (browser mode, Chromium)
pnpm run test:coverage  # same, with a coverage report for @will-be-done/hyperdb

The coverage report is written to packages/hyperdb/coverage/; open index.html there for the HTML view.

Learn more

• Introduction: what HyperDB is and when to use it
• Why HyperDB?: the data-modeling and reactivity problems it is built to solve
• How HyperDB Works: the mental model
• Quickstart: define a table, run a query, wire it into React
• LLM Cheat Sheet: compact context to paste into another project
• Storage Drivers: in-memory, IndexedDB, SQLite
• Devtools & Tracing: inspect selector runs and mutations
• Building a Sync Engine: share change-tracking code across client and server

On the server the persistent store is SQLite today (MongoDB and PostgreSQL are not supported yet). HyperDB gives you the storage, query, and reactivity primitives, and you build synchronization on top with the built-in primitives.