From 3a1c2524ab41b1e271557db793374e3d6f408725 Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 14:37:42 -0700 Subject: [PATCH 1/7] chore(docs): exclude generated and point-in-time Markdown from markdownlint Two path exclusions, dropping 33 of the 126 findings without touching a document. Both configs get the entry because markdownlint-cli2 reads only .markdownlint-cli2.jsonc and the editor extension reads only .markdownlintignore; letting them drift means the editor flags what CI-adjacent tooling ignores, or worse, the reverse. Root CHANGELOG.md (7 findings) is release-please output -- it is the sole changelog-path in release-please-config.json, so AGENTS.md forbids hand-editing it and the next release would clobber any fix anyway. Its findings are exactly that generator's shape: four MD012 double-blank-lines between release sections and three MD013 over-long generated commit lines. The exclusion is deliberately the bare path, not **/CHANGELOG.md, because the nine per-crate changelogs are hand-maintained and must stay linted. In .markdownlintignore the leading slash is what buys that, since it reads gitignore patterns where an unanchored CHANGELOG.md matches at every level; the cli2 list is globs, so it needs none. Verified with the `ignore` package the extension uses: root ignored, all nine per-crate changelogs still linted. docs/superpowers/** (26 findings) are point-in-time planning artifacts. AGENTS.md describes them as a spec and plan written before a feature and reviewed alongside the change that implements it; nothing maintains them after. The findings are dominated by prose shape rather than defects -- 13 MD013 lines of 500-1549 characters and 8 MD046 indented blocks inside a 1100-line plan -- so fixing them means a large reflow diff through settled records, which is also where the risk of mangling a code block is highest and the payoff lowest. Excluded specs alongside plans: same artifact class, and it keeps the question from being relitigated the first time a spec trips a rule. Not excluded for lack of a reader: excluded for lack of a maintainer. --- .markdownlint-cli2.jsonc | 12 ++++++++++++ .markdownlintignore | 14 ++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc index 3b48ce3..30b1116 100644 --- a/.markdownlint-cli2.jsonc +++ b/.markdownlint-cli2.jsonc @@ -24,6 +24,18 @@ // reference, so hand-edits are undone by the next refresh. "hyperdb-mcp/scripts/dc_sql_reference.md", + // Root changelog only. release-please generates it -- it is the sole + // "changelog-path" in release-please-config.json -- so any hand-fix is + // clobbered on the next release. Deliberately NOT "**/CHANGELOG.md": the + // nine per-crate changelogs are hand-maintained and stay in scope. + "CHANGELOG.md", + + // Point-in-time planning artifacts (see AGENTS.md "Documentation + // Conventions"): a spec and plan are written before a feature, reviewed + // alongside the change that implements it, and then not maintained. + // Reflowing prose in a settled record is churn no reader benefits from. + "docs/superpowers/**", + // npm platform sub-packages: packaging boilerplate, largely generated "hyperdb-api-node/npm/**", "hyperdb-mcp/npm/**" diff --git a/.markdownlintignore b/.markdownlintignore index f5b42c6..ff803d9 100644 --- a/.markdownlintignore +++ b/.markdownlintignore @@ -13,6 +13,20 @@ test_results/ # so hand-editing it is undone by the next refresh. hyperdb-mcp/scripts/dc_sql_reference.md +# Root changelog only -- release-please generates it (the sole changelog-path +# in release-please-config.json), so hand-fixes are clobbered on the next +# release. The leading slash is load-bearing: an unanchored "CHANGELOG.md" +# would also hide the nine hand-maintained per-crate changelogs, which are in +# scope. (markdownlint-cli2 matches its own list as globs, not gitignore +# patterns, so the equivalent entry there needs no slash.) +/CHANGELOG.md + +# Point-in-time planning artifacts (see AGENTS.md "Documentation +# Conventions"): a spec and plan are written before a feature, reviewed +# alongside the change that implements it, and then not maintained. +# Reflowing prose in a settled record is churn no reader benefits from. +docs/superpowers/ + # npm platform sub-packages: packaging boilerplate, largely generated hyperdb-api-node/npm/ hyperdb-mcp/npm/ From 345057216ee53d4b763d8f20384940ecc6cbc176 Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 14:41:26 -0700 Subject: [PATCH 2/7] docs: tag all 63 untagged code fences with a language (MD040) Clears MD040 to zero across 20 files. 61 blocks take `text`, per the repo convention for command output, ASCII diagrams, error messages and templates: architecture and data-flow diagrams, source-tree listings, hyperd/CLI --help output, stress-test result summaries, rustc error text, commit-message templates, and the MCP tool-call transcripts in the hyperdb-mcp README, SMOKE_TESTS and ROADMAP. Those transcripts read like a function call but match no real grammar -- `kv_set store=smoke key=k value=v -> {...}` is neither Python nor JS -- so `text` is the honest tag rather than borrowing a highlighter that would mis-colour it. Two blocks got a real language: a `cargo run` invocation in ROW_MAPPING (bash) and a table schema with an SQL comment in BENCHMARK_GUIDE (sql). Fixed by line number, never by pattern. A naive bulk-fixer corrupted 176 fences here once, because tagging an opener stops it matching a bare-fence test, so the CLOSER matches instead and becomes a spurious opener. The tagger is instead handed opener line numbers from a CommonMark state machine and asserts, per fence, that the line is both a bare fence and classified as an opener; anything else aborts. The state machine independently reproduced markdownlint exactly -- 68/68 MD040 line numbers and 9/9 MD046 -- before it was trusted. Structural proof, all 69 in-scope files: block count and a SHA of every block's content are unchanged; only info strings differ. Independently, the diff is 63 insertions and 63 deletions with every changed line a fence line, so no content line entered or left a block. The one 3-space-indented fence kept its indent. Nothing was added or removed, which also leaves hyperdb-mcp's README-coupled tests intact: doctor_readme_contract keys off a line window around exact `hyperdb-mcp doctor` lines (still at 286/287) and doctor_tests splits on `## CLI Reference` then does substring checks, none of which a fence info string can perturb. --- AGENTS.md | 4 +-- CONTRIBUTING.md | 4 +-- DEVELOPMENT.md | 4 +-- docs/BENCHMARK_GUIDE.md | 2 +- docs/GITHUB_OPERATIONS.md | 8 ++--- docs/ROW_MAPPING.md | 2 +- hyperdb-api-core/docs/DEVELOPMENT-client.md | 4 +-- hyperdb-api-core/docs/DEVELOPMENT-protocol.md | 2 +- hyperdb-api-core/docs/DEVELOPMENT-types.md | 2 +- hyperdb-api-derive/README.md | 4 +-- hyperdb-api-node/AGENTS.md | 2 +- hyperdb-api-node/DEVELOPMENT.md | 2 +- hyperdb-api-node/README.md | 2 +- .../examples/hyper-explorer/README.md | 2 +- hyperdb-api-salesforce/DEVELOPMENT.md | 2 +- hyperdb-api/DEVELOPMENT.md | 2 +- hyperdb-api/tests/stress_test/README.md | 8 ++--- hyperdb-mcp/README.md | 34 +++++++++---------- hyperdb-mcp/ROADMAP.md | 4 +-- hyperdb-mcp/SMOKE_TESTS.md | 32 ++++++++--------- 20 files changed, 63 insertions(+), 63 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 2d816a2..64cc485 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,7 +43,7 @@ This is a **pure-Rust implementation** of the Hyper database API, using the Post The codebase uses a **layered architecture**. The flagship user-facing crate is `hyperdb-api`; its implementation details live in `hyperdb-api-core`, which preserves three internal submodules (`types`, `protocol`, `client`) that contributors navigate independently. Two optional companion crates extend the public surface. -``` +```text ┌─────────────────────────────────────────────────────┐ │ hyperdb-api (High-level API, public) │ │ - Connection, AsyncConnection, HyperProcess │ @@ -251,7 +251,7 @@ Three *other* crates do define features. `hyperdb-api` is flag-free; the workspa Tests are organized by crate: -``` +```text hyperdb-api/tests/ # Integration tests (high-level API) hyperdb-api/tests/common/ # Shared test utilities hyperdb-api-core/tests/ # Client-level integration tests diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ef03d45..8c86a40 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -155,7 +155,7 @@ This project uses [Conventional Commits](https://www.conventionalcommits.org/) t ## Commit Message Structure -``` +```text (): @@ -184,7 +184,7 @@ This project uses [Conventional Commits](https://www.conventionalcommits.org/) t ## Examples -``` +```text feat: add support for batch query execution fix(hyperdb-api-core): resolve type mismatch diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index f1c5938..898c7c5 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -103,7 +103,7 @@ The API uses **lifetime annotations** to provide compile-time guarantees that re are used correctly. All dependent types (`Inserter`, `Catalog`, `Rowset`) carry a `'conn` lifetime tying them to their parent `Connection`: -``` +```text Connection (owns data) ├── Inserter<'conn> │ └── CopyInWriter<'conn> @@ -565,7 +565,7 @@ cargo test -p hyperdb-api --test integration_test ### Test Structure -``` +```text hyperdb-api/tests/ # Integration tests (high-level API) hyperdb-api/tests/common/ # Shared test utilities hyperdb-api-core/tests/ # Client-level integration tests diff --git a/docs/BENCHMARK_GUIDE.md b/docs/BENCHMARK_GUIDE.md index e1d0b5b..e8331b5 100644 --- a/docs/BENCHMARK_GUIDE.md +++ b/docs/BENCHMARK_GUIDE.md @@ -17,7 +17,7 @@ specific questions. All benchmarks share the same schema so numbers compare directly: -``` +```sql measurements(id INT NOT NULL, sensor_id INT, value DOUBLE, timestamp BIGINT) -- 24 bytes / row ``` diff --git a/docs/GITHUB_OPERATIONS.md b/docs/GITHUB_OPERATIONS.md index 87ad463..a79b0ad 100644 --- a/docs/GITHUB_OPERATIONS.md +++ b/docs/GITHUB_OPERATIONS.md @@ -51,7 +51,7 @@ the GitHub Release after merging the release PR) or via manual `workflow_dispatch` with an explicit tag input (for re-runs or emergency releases). Structure: -``` +```text verify ← full test suite + hyperd URL check, single-platform │ └─► publish ← crates.io publish in dependency order @@ -104,7 +104,7 @@ without needing Rust toolchains or manual hyperd setup. **Structure:** -``` +```text verify-ci ← checks that CI passed for this commit (gh api commit status) │ └─► build-npm (matrix × 4 platforms) @@ -340,7 +340,7 @@ semver: `feat!:` will bump `1.2.3` → `2.0.0` as expected. To stabilize the API and cut `1.0.0`, add a `Release-As: 1.0.0` footer to a conventional-commit on `main`: -``` +```text feat: stabilize public API Release-As: 1.0.0 @@ -351,7 +351,7 @@ Release-As: 1.0.0 For an `-rc.N` / `-alpha.N` / `-beta.N` release, add a footer to a commit on `main`: -``` +```text Release-As: 0.2.0-rc.1 ``` diff --git a/docs/ROW_MAPPING.md b/docs/ROW_MAPPING.md index 88289c5..88c0d7a 100644 --- a/docs/ROW_MAPPING.md +++ b/docs/ROW_MAPPING.md @@ -8,7 +8,7 @@ struct mapping. Start with the simplest that fits your situation. All six forms are demonstrated end-to-end in one runnable example: -``` +```bash cargo run -p hyperdb-api --example row_mapping_forms ``` diff --git a/hyperdb-api-core/docs/DEVELOPMENT-client.md b/hyperdb-api-core/docs/DEVELOPMENT-client.md index 1974fef..747eb52 100644 --- a/hyperdb-api-core/docs/DEVELOPMENT-client.md +++ b/hyperdb-api-core/docs/DEVELOPMENT-client.md @@ -18,7 +18,7 @@ For user-facing documentation, see [README.md](README.md). ### Module Map -``` +```text src/ lib.rs # Crate root, re-exports config.rs # Config builder for TCP connections @@ -129,7 +129,7 @@ in-flight streaming queries. machine that handles all three transfer modes. The state transitions and per-mode paths are documented in that module's rustdoc. Key states: -``` +```text ReadInitialResults -> RequestStatus -> ReadStatus -> RequestResults -> ReadResults -> Finished ``` diff --git a/hyperdb-api-core/docs/DEVELOPMENT-protocol.md b/hyperdb-api-core/docs/DEVELOPMENT-protocol.md index 7de9a90..197722c 100644 --- a/hyperdb-api-core/docs/DEVELOPMENT-protocol.md +++ b/hyperdb-api-core/docs/DEVELOPMENT-protocol.md @@ -29,7 +29,7 @@ encoding, use LittleEndian for the value bytes. The inline doc comments in ### Module Layout -``` +```text src/ lib.rs # Crate root, re-exports message/ diff --git a/hyperdb-api-core/docs/DEVELOPMENT-types.md b/hyperdb-api-core/docs/DEVELOPMENT-types.md index fd0e921..b234abf 100644 --- a/hyperdb-api-core/docs/DEVELOPMENT-types.md +++ b/hyperdb-api-core/docs/DEVELOPMENT-types.md @@ -23,7 +23,7 @@ For user-facing documentation (type mapping, serialization traits, API surface), ### Data Flow -``` +```text User Rust value │ ▼ diff --git a/hyperdb-api-derive/README.md b/hyperdb-api-derive/README.md index b4cbd85..c623e5c 100644 --- a/hyperdb-api-derive/README.md +++ b/hyperdb-api-derive/README.md @@ -151,12 +151,12 @@ With `features = ["compile-time"]` and `HYPERD_PATH` set, `query_as!` validates Bad SQL produces a `compile_error!` pointing at the SQL string literal: -``` +```text error: column "emai1" does not exist on any table in the query; check for a typo or a renamed/dropped column ``` -``` +```text error: `User` requires column "email" but the query does not project it; add it to the SELECT list or remove the field from `User` ``` diff --git a/hyperdb-api-node/AGENTS.md b/hyperdb-api-node/AGENTS.md index 864b82e..bcd041b 100644 --- a/hyperdb-api-node/AGENTS.md +++ b/hyperdb-api-node/AGENTS.md @@ -10,7 +10,7 @@ For documentation conventions specific to JavaScript/TypeScript, see the [Docume `hyperdb-api-node` is a **napi-rs** native addon that exposes the Rust `hyperdb-api` crate to JavaScript and TypeScript. The Rust side (`src/*.rs`) compiles to a `.node` shared library; the JS side (`index.js`, `index.d.ts`, `pool.mjs`, `arrow.mjs`) adds ergonomic wrappers on top. -``` +```text ┌──────────────────────────────────────────────────┐ │ JS/TS application code │ ├──────────────────────────────────────────────────┤ diff --git a/hyperdb-api-node/DEVELOPMENT.md b/hyperdb-api-node/DEVELOPMENT.md index 183d722..1bc370e 100644 --- a/hyperdb-api-node/DEVELOPMENT.md +++ b/hyperdb-api-node/DEVELOPMENT.md @@ -9,7 +9,7 @@ see [README.md](README.md). `hyperdb-api-node` is a three-layer stack: pure-JS extensions on top of napi-rs bindings on top of the Rust `hyperdb-api` crate. -``` +```text ┌───────────────────────────────────────────────────────┐ │ JS extensions │ │ index.js — native loader + tagged templates, │ diff --git a/hyperdb-api-node/README.md b/hyperdb-api-node/README.md index 2de0bd4..a4a1075 100644 --- a/hyperdb-api-node/README.md +++ b/hyperdb-api-node/README.md @@ -19,7 +19,7 @@ This package provides a native Node.js addon that gives JavaScript and TypeScrip ## Class Overview -``` +```text HyperProcess ──creates──▶ Connection ──returns──▶ RowData ConnectionBuilder ─builds─▶ Connection ──returns──▶ QueryStream ──yields──▶ RowData ConnectionPool ──pools──▶ Connection ──returns──▶ ColumnarStream ──yields──▶ ColumnarChunk diff --git a/hyperdb-api-node/examples/hyper-explorer/README.md b/hyperdb-api-node/examples/hyper-explorer/README.md index b03c1db..bb442b7 100644 --- a/hyperdb-api-node/examples/hyper-explorer/README.md +++ b/hyperdb-api-node/examples/hyper-explorer/README.md @@ -264,7 +264,7 @@ sequenceDiagram ## Project structure -``` +```text hyper-explorer/ ├── server/ # Express backend (TypeScript, tsx) │ ├── index.ts # Express, CORS, JSON, HyperProcess, pool, PORT, shutdown diff --git a/hyperdb-api-salesforce/DEVELOPMENT.md b/hyperdb-api-salesforce/DEVELOPMENT.md index 4f33a89..dbcf723 100644 --- a/hyperdb-api-salesforce/DEVELOPMENT.md +++ b/hyperdb-api-salesforce/DEVELOPMENT.md @@ -8,7 +8,7 @@ Internal architecture and contributor notes for the `hyperdb-api-salesforce` cra Authentication with Salesforce Data Cloud uses a two-stage token exchange: -``` +```text Stage 1: App --> POST {login_url}/services/oauth2/token --> OAuth Access Token Stage 2: App --> POST {instance_url}/services/a360/token --> DC JWT ``` diff --git a/hyperdb-api/DEVELOPMENT.md b/hyperdb-api/DEVELOPMENT.md index 14d0c79..87bb973 100644 --- a/hyperdb-api/DEVELOPMENT.md +++ b/hyperdb-api/DEVELOPMENT.md @@ -198,7 +198,7 @@ This is a cross-crate change. Start at the bottom of the stack: ### Test Structure -``` +```text hyperdb-api/tests/ # Integration tests (require live hyperd) hyperdb-api/tests/common/ # Shared test utilities (TestConnection) hyperdb-api/src/result.rs # Unit tests (arrow_path_tests, no hyperd needed) diff --git a/hyperdb-api/tests/stress_test/README.md b/hyperdb-api/tests/stress_test/README.md index e6a6b8c..26510af 100644 --- a/hyperdb-api/tests/stress_test/README.md +++ b/hyperdb-api/tests/stress_test/README.md @@ -178,7 +178,7 @@ Results from actual runs on a Mac Studio (M2 Ultra, 192 GB RAM): ### Moderate Load (30s, 12 threads, seed=7777) -``` +```text Duration: 30.5s Operations: 25,185 (0 failures) Throughput: 827 ops/sec, 297,808 insert-rows/sec @@ -190,7 +190,7 @@ Disk used: 856 MB ### High Load (2 min, 18 threads, seed=9999) -``` +```text Config: 5 DBs, 8 inserters, 6 query, 4 mixed think_time=0–5ms, batch=1k–50k rows Duration: 121.0s @@ -204,7 +204,7 @@ Disk used: 2,012 MB ### Replay of High Load Run -``` +```text Duration: 120.8s Operations: 13,720 (0 failures) Throughput: 113.5 ops/sec, 191,867 insert-rows/sec @@ -229,7 +229,7 @@ Disk used: 1,979 MB ## File Structure -``` +```text hyperdb-api/tests/ ├── stress_test_main.rs # Test entry points (#[ignore]) └── stress_test/ diff --git a/hyperdb-mcp/README.md b/hyperdb-mcp/README.md index 535086c..189ee9d 100644 --- a/hyperdb-mcp/README.md +++ b/hyperdb-mcp/README.md @@ -332,7 +332,7 @@ If hyperd repeatedly fails to start (3 attempts within 60 seconds — e.g., misc Ingest inline data and run a SQL query in a single call. -``` +```text query_data(data: '[{"region":"West","revenue":1200},...]', sql: 'SELECT region, SUM(revenue) FROM data GROUP BY region') ``` @@ -348,7 +348,7 @@ query_data(data: '[{"region":"West","revenue":1200},...]', sql: 'SELECT region, Ingest a file and run a SQL query in a single call. Streams from disk — handles files of any size. -``` +```text query_file(path: '/tmp/sales.parquet', sql: 'SELECT TOP 10 * FROM sales ORDER BY amount DESC') ``` @@ -365,7 +365,7 @@ query_file(path: '/tmp/sales.parquet', sql: 'SELECT TOP 10 * FROM sales ORDER BY Load inline data into a named local, persistent, or attached-database table. -``` +```text load_data(table: 'customers', data: '[{"id":1,"name":"Alice"},...]') ``` @@ -381,7 +381,7 @@ load_data(table: 'customers', data: '[{"id":1,"name":"Alice"},...]') Load a file into a named local, persistent, or attached-database table. -``` +```text load_file(table: 'orders', path: '/tmp/orders.csv') ``` @@ -404,7 +404,7 @@ local table. Pass the absolute path to the Iceberg table root (the directory containing `metadata/` and `data/`); hyperd's native Iceberg reader derives the schema and resolves the snapshot. -``` +```text load_iceberg(table: 'sales', path: '/lake/warehouse/db/sales') ``` @@ -423,7 +423,7 @@ Iceberg table metadata. Run a **read-only** SQL query against local (default), persistent, or an attached database. Accepts `SELECT`, `WITH`, `EXPLAIN`, `SHOW`, `VALUES`. For DDL/DML use `execute`. -``` +```text query(sql: 'SELECT c.name, SUM(o.amount) FROM orders o JOIN customers c ON o.customer_id = c.id GROUP BY c.name') ``` @@ -431,7 +431,7 @@ query(sql: 'SELECT c.name, SUM(o.amount) FROM orders o JOIN customers c ON o.cus Execute one or more **mutating** SQL statements as an atomic batch: `CREATE TABLE`, `INSERT`, `UPDATE`, `DELETE`, `DROP TABLE`, `ALTER`, `COPY`, etc. `sql` is an array of statements; multi-element batches run inside a transaction (all commit or all roll back). Single-element batches auto-commit, same as a one-off statement. Returns the per-statement affected row counts plus a total. Disabled in read-only mode. -``` +```text // Single statement (auto-commit) execute(sql: ['CREATE TABLE archived_orders AS SELECT * FROM orders WHERE year < 2024']) @@ -459,7 +459,7 @@ List all tables in the selected database with their schemas, column types, and r Return the schema, total row count, and first N rows of a table in a single call. -``` +```text sample(table: 'orders', n: 10) ``` @@ -483,7 +483,7 @@ Use it **before** `load_file` whenever you are unsure about types, or **after** reported `type` + `min` / `max` directly into a partial `schema` override on the subsequent `load_file` call. -``` +```text inspect_file(path: '/tmp/owid-population.csv') ``` @@ -531,7 +531,7 @@ only for the lifetime of the server process. #### `save_query` -``` +```text save_query(name: 'top_5_customers', sql: 'SELECT customer, SUM(amount) AS total FROM orders GROUP BY customer ORDER BY total DESC LIMIT 5', description: 'Biggest spenders this year') ``` @@ -547,7 +547,7 @@ first if you intend to overwrite. Non-read-only SQL is rejected with #### `delete_query` -``` +```text delete_query(name: 'top_5_customers') ``` @@ -584,7 +584,7 @@ Nine tools cover the surface: | `kv_pop` | Destructively read-and-remove the lowest-keyed entry (atomic) | `store`, `database`, `persist` | | `kv_clear` | Delete all keys in a store (returns count removed) | `store`, `database`, `persist` | -``` +```text kv_set(store: 'session', key: 'last_report', value: '{"rows": 4210}', database: 'persistent') kv_get(store: 'session', key: 'last_report') ``` @@ -607,7 +607,7 @@ Key properties: Write query results or a table to a file. -``` +```text export(table: 'orders', path: '~/Desktop/orders.parquet', format: 'parquet') export(sql: 'SELECT ...', path: '~/Desktop/analysis.hyper', format: 'hyper') ``` @@ -630,7 +630,7 @@ destination and materializes every user table from the selected source into it. Render a bounded quick diagnostic from a SQL query. This convenience tool is for inspecting or sharing one chart, not for dashboard/layout composition. -``` +```text chart(sql: 'SELECT product, SUM(revenue) as total FROM sales GROUP BY product', chart_type: 'bar', x: 'product', y: 'total', title: 'Revenue by Product') ``` @@ -684,7 +684,7 @@ bound, never zero. Monitor a directory for data files and auto-append them to a target table. -``` +```text watch_directory(path: '/tmp/inbox', table: 'events') unwatch_directory(path: '/tmp/inbox') ``` @@ -899,7 +899,7 @@ Hyper uses the Salesforce Data Cloud SQL dialect (PostgreSQL-compatible with ext Hyper does **not** support `ON CONFLICT` or `INSERT ... ON DUPLICATE KEY`. Use the `execute` tool's atomic batch shape instead: -``` +```text execute(sql: [ "UPDATE settings SET value = 'dark' WHERE key = 'theme'", "INSERT INTO settings (key, value) SELECT 'theme', 'dark' \ @@ -927,7 +927,7 @@ Full reference: [Data Cloud SQL Reference](https://developer.salesforce.com/docs ## CLI Reference -``` +```text hyperdb-mcp [OPTIONS] [COMMAND] Commands: diff --git a/hyperdb-mcp/ROADMAP.md b/hyperdb-mcp/ROADMAP.md index 18df1b8..96227a4 100644 --- a/hyperdb-mcp/ROADMAP.md +++ b/hyperdb-mcp/ROADMAP.md @@ -38,7 +38,7 @@ survive hyperd crashes transparently. Example — JOIN across a scratch `.hyper` file and the primary workspace, then land the result: -``` +```text attach_database(alias="src", kind="local_file", path="/tmp/scratch.hyper") copy_query( sql="SELECT s.id, s.name, p.amount FROM src.public.customers s JOIN orders p ON s.id = p.customer_id", @@ -68,7 +68,7 @@ keep these workarounds in mind: 1. **`.hyper` → `.hyper` export then load.** Single-table transfer. Fastest because `.hyper` is Hyper's native format. - ``` + ```text # in HyperDB (sandbox) export(table="scratch_data", path="/tmp/scratch.hyper", format="hyper") # then in HyperDB-persistent diff --git a/hyperdb-mcp/SMOKE_TESTS.md b/hyperdb-mcp/SMOKE_TESTS.md index 5ebbd91..4c0b817 100644 --- a/hyperdb-mcp/SMOKE_TESTS.md +++ b/hyperdb-mcp/SMOKE_TESTS.md @@ -66,7 +66,7 @@ it started in. The final section is a verification checklist for that. - The `hyperdb` MCP tools connected and responding. - Confirm the server is up and note its mode before you start: -``` +```text status ``` @@ -122,7 +122,7 @@ resource. ## 2. Create / read / overwrite (upsert) -``` +```text kv_set store=smoke key=greeting value="hello world" → {"stored": true, "created": true, "value_bytes": 11, "store": "smoke", "key": "greeting", "resolved_database": "local"} kv_get store=smoke key=greeting → {"found": true, "value": "hello world", "resolved_database": "local"} kv_get store=smoke key=does_not_exist → {"found": false, "value": null, "resolved_database": "local"} @@ -132,7 +132,7 @@ A miss is **not** an error — `found: false` with a `null` value. Batch writes are atomic and validate every key before writing: -``` +```text kv_set_many store=smoke_batch entries=[{"key":"batch_a","value":"A"},{"key":"batch_b","value":"B"}] → {"stored": 2, "created": 2, "overwritten": 0, "total_bytes": 2, "resolved_database": "local"} kv_list store=smoke_batch @@ -144,7 +144,7 @@ kv_clear store=smoke_batch **Overwrite must not create a duplicate row** (the backing table is indexless; `kv_set` is an app-side upsert): -``` +```text kv_size store=smoke → {"store": "smoke", "size": 1, "bytes": 11, "resolved_database": "local"} kv_set store=smoke key=greeting value="HELLO AGAIN" → {"stored": true, "resolved_database": "local", ...} kv_size store=smoke → {"store": "smoke", "size": 1, "bytes": 11, "resolved_database": "local"} # still 1, not 2 @@ -157,7 +157,7 @@ kv_get store=smoke key=greeting → {"found": true, " Seed a few keys, then list: -``` +```text kv_set store=smoke key=alpha value=1 kv_set store=smoke key=bravo value=2 kv_set store=smoke key=charlie value=3 @@ -175,7 +175,7 @@ emptied store disappears from the list; see §5). ## 4. Value fidelity — JSON, empty, large -``` +```text kv_set store=smoke key=config value='{"retries": 3, "nested": {"flag": true}}' kv_get store=smoke key=config → {"found": true, "value": "{\"retries\": 3, \"nested\": {\"flag\": true}}", "resolved_database": "local"} # byte-for-byte @@ -195,7 +195,7 @@ must stay distinct from a miss `{"found": false, "value": null}`. **Delete is idempotent and reports whether the key existed:** -``` +```text kv_delete store=smoke key=greeting → {"deleted": true, "resolved_database": "local", ...} # existed kv_delete store=smoke key=greeting → {"deleted": false, "resolved_database": "local", ...} # already gone — no error kv_delete store=smoke key=never_existed → {"deleted": false, "resolved_database": "local", ...} @@ -204,7 +204,7 @@ kv_delete store=smoke key=never_existed → {"deleted": false, "resolved_datab **`kv_pop` destructively removes the lowest-keyed entry** (a work-queue drain in ascending key order): -``` +```text # with keys [alpha, bravo, charlie, config, empty_val, big_blob] present kv_pop store=smoke → {"found": true, "key": "alpha", "value": "1", "resolved_database": "local"} kv_pop store=smoke → {"found": true, "key": "big_blob", "value": "...", "resolved_database": "local"} # 'b' < 'c' @@ -213,7 +213,7 @@ kv_pop store=smoke → {"found": true, "key": "bravo", "value": "2", "res **`kv_clear` empties the store and returns the count removed:** -``` +```text kv_size store=smoke → {"store": "smoke", "size": N, "bytes": B, "resolved_database": "local"} kv_clear store=smoke → {"store": "smoke", "removed": N, "resolved_database": "local"} kv_size store=smoke → {"store": "smoke", "size": 0, "bytes": 0, "resolved_database": "local"} @@ -224,7 +224,7 @@ of the remaining values' UTF-8 byte lengths at that point. **Empty-store edge cases:** -``` +```text kv_pop store=smoke → {"found": false, "resolved_database": "local"} # nothing to pop kv_clear store=smoke → {"store": "smoke", "removed": 0, "resolved_database": "local"} # idempotent kv_list_stores → {"count": 0, "stores": [], "resolved_database": "local"} # emptied store drops out @@ -238,7 +238,7 @@ kv_list_stores → {"count": 0, "stores": [], "resolved_database": "loca Violations are rejected as **`INVALID_ARGUMENT`** (not `INTERNAL_ERROR`) with a message that names the offending byte or the actual length: -``` +```text kv_set store=smoke key="has a space" value=x → error INVALID_ARGUMENT: "invalid name: KV key contains an invalid byte 0x20; allowed: A-Z a-z 0-9 _ . -" @@ -260,7 +260,7 @@ Only relevant when the server runs with `--read-only` (`status` shows `"read_only": true`). Start such a server yourself for this check — do not assume the shared daemon is read-only. -``` +```text # readers work: kv_get store=smoke key=k → {"found": false, "value": null, "resolved_database": "local"} kv_list store=smoke → {"store": "smoke", "count": 0, "keys": [], "resolved_database": "local"} @@ -299,7 +299,7 @@ an explicit `database` wins over `persist: true` (for example, `database=PeRsIsTeNt` resolves to `persistent`; mixed-case attached aliases resolve to the registry's lowercase alias. -``` +```text kv_set store=smoke_routing key=where value="local" # → local (default) kv_set store=smoke_routing key=where value="persistent" database=persistent # → persistent kv_set store=smoke_routing key=where2 value="via-flag" persist=true # → persistent (same DB) @@ -329,7 +329,7 @@ The backing table `_hyperdb_kv_store(store_name, key, value)` is hidden from store: annotate analytical rows with scratchpad metadata via a plain SQL join. **Run this in the local DB** (create a `smoke_`-prefixed table): -``` +```text kv_set store=product_notes key=P1 value="flagship - review pricing Q3" kv_set store=product_notes key=P3 value="discontinue candidate" @@ -351,7 +351,7 @@ Expected: P1 and P3 carry their notes; **P2 survives with `note: null`** ## 10. Table is hidden but accessible -``` +```text describe → table list does NOT include _hyperdb_kv_store query SELECT COUNT(*) FROM _hyperdb_kv_store → succeeds (directly queryable) ``` @@ -388,7 +388,7 @@ DB constraint — that limitation is documented, not a smoke-test failure.) Purge every scratch store and table, then confirm the databases are back to their starting state: -``` +```text kv_clear store=smoke kv_clear store=smoke_routing kv_clear store=smoke_routing database=persistent From 0e0382898a6b3d81e83d485fb4987efc175ce767 Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 14:46:47 -0700 Subject: [PATCH 3/7] docs: fix heading structure and code-block style (MD025/001/041/036/024/046) Clears every remaining non-MD013 rule: 11 MD025, 2 MD036, 2 MD024, and one each of MD001, MD041 and MD046. 93 findings down to 12. CONTRIBUTING.md had 12 H1s, one per top-level section, so every section claimed to be the document title. Demoted the 11 that are not the title to H2 and their H2 children to H3. The three H3s under "Issues, requests & ideas" deliberately stayed H3: they were the MD001 finding, an H1 -> H3 skip, and bringing their parent down to H2 makes them correctly incremented where they already are. Anchors are unaffected, which is worth stating because re-levelling is usually where anchors break: a GitHub slug derives from heading text only -- the hash count picks the h1..h6 tag, not the id. Verified rather than assumed, by running github-slugger over the file before and after: 22 anchors, identical sequence, and all six inbound links still resolve (#release-process, #commit-message-format, #contribution-checklist from AGENTS.md, hyperdb-api-node/DEVELOPMENT.md and docs/GITHUB_OPERATIONS.md; #what-contributors-do; plus intra-document #creating-a-pull-request and #commit-types-and-version-impact). No link needed updating. hyperdb-mcp/CHANGELOG.md carried "### Changed" and "### Fixed" twice under one "## [Unreleased]", the classic case of appending a second section instead of merging. Merged them and restored Keep a Changelog order (Added, Changed, Fixed), keeping each source block's internal order. That order is load-bearing here because two bullets cross-reference their own position: "supersedes the shorthand in the Added notes above" stays true since Added now sorts first, and "supersedes the older Unreleased note below" is intra-section -- both it and the note it supersedes are in the same Fixed block, untouched. Proved lossless by comparing the multiset of non-heading lines in the section: identical, with the two duplicate headings the only lines removed. BENCHMARK_GUIDE.md used a bold "**Hardware / software**" label as a heading in all four platform sections. Two were flagged; the other two escaped only because trailing italic placeholder text meant the line was not pure emphasis. Converted all four to the "#### Hardware / software" they were imitating, so the platform sections render alike. The placeholder notes became italic sentences ending in a period, matching the file's existing placeholder style -- a bare "*(placeholder)*" on its own line would have been a fresh MD036, since that rule exempts emphasis ending in punctuation and ")" does not qualify. SECURITY.md opened at H2 with no H1. Promoted it; same text, so the slug is unchanged, and nothing links to the file anyway. README.md's RHEL install snippet was indented rather than fenced -- now a bash fence. Structural proof over all 69 in-scope files: block count and per-block content SHAs unchanged. The README conversion shows as "1 indented->fenced, content identical", since the auditor hashes indented and fenced blocks alike, so the dedent is provably lossless. hyperdb-mcp/CHANGELOG.md holds no code blocks, so the section merge could not disturb one. --- CONTRIBUTING.md | 36 +++++++++++++++--------------- README.md | 6 +++-- SECURITY.md | 2 +- docs/BENCHMARK_GUIDE.md | 12 ++++++---- hyperdb-mcp/CHANGELOG.md | 48 ++++++++++++++++++---------------------- 5 files changed, 52 insertions(+), 52 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8c86a40..6bd481a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,13 +2,13 @@ This page lists the operational governance model of this project, as well as the recommendations and requirements for how to best contribute to Tableau Hyper Rust API. We strive to obey these as best as possible. As always, thanks for contributing – we hope these guidelines make it easier and shed some light on our approach and processes. -# Governance Model +## Governance Model -## Community Based +### Community Based The intent and goal of open sourcing this project is to increase the contributor and user base. The governance model is one where new project leads (`admins`) will be added to the project based on their contributions and efforts, a so-called "do-acracy" or "meritocracy" similar to that used by all Apache Software Foundation projects. -# Issues, requests & ideas +## Issues, requests & ideas Use GitHub Issues page to submit issues, enhancement requests and discuss ideas. @@ -40,7 +40,7 @@ Use GitHub Issues page to submit issues, enhancement requests and discuss ideas. If you're new to our project and looking for some way to make your first contribution, look for Issues labelled `good first contribution`. -# Code Style & Guidelines +## Code Style & Guidelines This project follows the **[Microsoft Pragmatic Rust Guidelines](https://microsoft.github.io/rust-guidelines/)**. The repo-specific adaptation — what is machine-enforced, what is reviewer-enforced, and the list of documented exceptions — is in [docs/RUST_GUIDELINES.md](docs/RUST_GUIDELINES.md). @@ -71,7 +71,7 @@ path-filtered, so a docs-only PR does not run it. When a lint genuinely cannot be satisfied for a given site, suppress it with `#[expect(lint_name, reason = "")]` rather than bare `#[allow(...)]` — the `reason` is mandatory and `#[expect]` auto-removes itself when the lint would no longer fire. See the [Exceptions](docs/RUST_GUIDELINES.md#exceptions) section of the guidelines page for the current workspace-level waivers. -# Contribution Checklist +## Contribution Checklist - [ ] Clean, simple, well styled code — conforms to [docs/RUST_GUIDELINES.md](docs/RUST_GUIDELINES.md) - [ ] Commits should be atomic and messages must be descriptive. Related issues should be mentioned by Issue number. @@ -92,7 +92,7 @@ When a lint genuinely cannot be satisfied for a given site, suppress it with `#[ - [ ] Reviews - Changes must be approved via peer code review -# Signed Commits +## Signed Commits This repo requires signed commits on `main`. Any PR whose commits are unsigned will be blocked at merge time — the GitHub Actions CI runs fine on unsigned commits, but the merge button won't enable. @@ -129,7 +129,7 @@ Two gotchas to avoid: GPG signing is also supported — see [GitHub's signing-commits guide](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) for the GPG and S/MIME paths. SSH is the recommended default for this repo. -# Creating a Pull Request +## Creating a Pull Request 1. **Ensure the bug/feature was not already reported** by searching on GitHub under Issues. If none exists, create a new issue so that other contributors can keep track of what you are trying to add/fix and offer suggestions (or let you know if there is already an effort in progress). 2. **Fork** the repository on GitHub. @@ -142,18 +142,18 @@ GPG signing is also supported — see [GitHub's signing-commits guide](https://d > **NOTE**: Be sure to [sync your fork](https://help.github.com/articles/syncing-a-fork/) before making a pull request. -# Contributor License Agreement ("CLA") +## Contributor License Agreement ("CLA") In order to accept your pull request, we need you to submit a CLA. You only need to do this once to work on any of Salesforce's open source projects. Complete your CLA here: -# Commit Message Format +## Commit Message Format This project uses [Conventional Commits](https://www.conventionalcommits.org/) to automate versioning and release management. Please format your commit messages accordingly. -## Commit Message Structure +### Commit Message Structure ```text (): @@ -169,7 +169,7 @@ This project uses [Conventional Commits](https://www.conventionalcommits.org/) t - **Body** (optional): Detailed explanation of the change - **Footer** (optional): Issue references -## Commit Types and Version Impact +### Commit Types and Version Impact | Commit Type | Version Bump | Example | |------------|--------------|---------| @@ -182,7 +182,7 @@ This project uses [Conventional Commits](https://www.conventionalcommits.org/) t > `fix:` for changes that end-users of the crate or npm package would notice. > A `fix(ci):` commit will trigger an unintended patch release. -## Examples +### Examples ```text feat: add support for batch query execution @@ -196,13 +196,13 @@ ci: fix chmod step in npm-build-publish workflow chore: update arrow dependency to 56 ``` -# Release Process +## Release Process This repo uses [release-please](https://github.com/googleapis/release-please) to fully automate version bumps, changelog generation, tagging, and the crates.io / npm publish dance. -## What contributors do +### What contributors do **Use [Conventional Commits](https://www.conventionalcommits.org/) for every PR title.** That's it. release-please reads the merged commits to figure out @@ -219,7 +219,7 @@ The **per-crate** `CHANGELOG.md` files are different: each carries a expected to append to them for user-visible API changes. See [AGENTS.md](AGENTS.md) reminder 8 for the policy and the full file list. -## What maintainers do +### What maintainers do The end-to-end flow lives in [`docs/GITHUB_OPERATIONS.md` → Cutting a release](docs/GITHUB_OPERATIONS.md#cutting-a-release). @@ -250,7 +250,7 @@ For pre-releases (`-rc.N`, `-alpha.N`, `-beta.N`), include a `Release-As:` footer in a commit on `main` — see [`docs/GITHUB_OPERATIONS.md`](docs/GITHUB_OPERATIONS.md#pre-releases). -## Published Crates +### Published Crates | Package | Registry | Notes | |---------|----------|-------| @@ -264,10 +264,10 @@ footer in a commit on `main` — see | `hyperdb-compile-check` | crates.io | Compile-time SQL validation backend. Not a workspace member (it declares its own `[workspace]` to break the dependency cycle), but release-please manages its version and it must be published for `hyperdb-api-derive`'s off-by-default `compile-time` feature to resolve. | | `hyperdb-api-node` | npm | Node.js/TypeScript bindings. `publish = false` for crates.io — the only crate in the tree that is not a Cargo publish target. | -# Code of Conduct +## Code of Conduct Please follow our [Code of Conduct](CODE_OF_CONDUCT.md). -# License +## License By contributing your code, you agree to license your contribution under the terms of our project [MIT](LICENSE-MIT) and [Apache-2.0](LICENSE-APACHE) dual license, and to sign the [Salesforce CLA](https://cla.salesforce.com/sign-cla). diff --git a/README.md b/README.md index 097324a..65d714a 100644 --- a/README.md +++ b/README.md @@ -364,8 +364,10 @@ This workspace builds with Red Hat's **system-native Rust toolchain and no `rustup`**, which is how enterprise environments typically consume it. RHEL provides `rust-toolset` in AppStream as a rolling Application Stream: - dnf install -y rust-toolset gcc gcc-c++ fontconfig-devel unzip - cargo build --release +```bash +dnf install -y rust-toolset gcc gcc-c++ fontconfig-devel unzip +cargo build --release +``` Notes for system-toolchain builds, all verified against `ubi9/ubi`: diff --git a/SECURITY.md b/SECURITY.md index b69c021..ca9d103 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,4 +1,4 @@ -## Security +# Security Please report any security issue to [https://www.sfdc.co/SubmitVuln](https://www.sfdc.co/SubmitVuln) as soon as it is discovered. This library limits its runtime dependencies in diff --git a/docs/BENCHMARK_GUIDE.md b/docs/BENCHMARK_GUIDE.md index e8331b5..27193f0 100644 --- a/docs/BENCHMARK_GUIDE.md +++ b/docs/BENCHMARK_GUIDE.md @@ -145,7 +145,7 @@ block from the suite's stdout. ### Platform: macOS (Apple Silicon) -**Hardware / software** +#### Hardware / software - **OS:** Darwin 26.6.2 (aarch64) - **CPU:** Apple M3 Max (14 physical / 14 logical cores) @@ -301,7 +301,9 @@ unchanged — opt into `ArrowInserter` and `executeQueryColumnar` / ### Platform: Linux (x86_64) -**Hardware / software** *(placeholder — replace with `host` block from your suite run)* +#### Hardware / software + +*Placeholder — replace with the `host` block from your suite run.* - **OS:** (e.g. Ubuntu 24.04) - **CPU:** @@ -326,7 +328,7 @@ unchanged — opt into `ArrowInserter` and `executeQueryColumnar` / ### Platform: Windows (x86_64, native) -**Hardware / software** +#### Hardware / software - **OS:** Windows 11 (build 26100) (x86_64) - **CPU:** Intel(R) Core(TM) i9-10980XE @ 3.00 GHz (18 physical / 36 logical cores) @@ -417,7 +419,9 @@ same process. ### Platform: Windows (x86_64 / WSL2) -**Hardware / software** *(placeholder)* +#### Hardware / software + +*Placeholder — fill in from the `host` block of your suite run.* - **OS:** (e.g. Ubuntu 22.04 under WSL2) - **CPU:** diff --git a/hyperdb-mcp/CHANGELOG.md b/hyperdb-mcp/CHANGELOG.md index 1886b5a..dde8e9b 100644 --- a/hyperdb-mcp/CHANGELOG.md +++ b/hyperdb-mcp/CHANGELOG.md @@ -7,33 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] -### Changed - -- The `arrow` and `parquet` dependencies moved from **58** to **59**. Not a - library-API change (this crate ships a binary), but it removes the `thrift` - dependency and the Apache Thrift excessive-size-allocation advisory with it: - `parquet` 58.x pinned `thrift ^0.17`, and `parquet` 59 dropped thrift - entirely. - -- `Engine::execute_in_transaction` now calls `hyperdb-api`'s `*_unguarded` - transaction methods instead of the deprecated `begin_transaction` / `commit` - / `rollback`, which 1.0.0 removed. No behavior change: the helper still takes - `&self`, so the RAII guard remains unavailable to it, and it still rolls back - before resuming an unwind. The `#[allow(deprecated)]` it needed is gone. - Moving to the guard still waits on - [issue #72](https://github.com/tableau/hyper-api-rust/issues/72). -- **BREAKING:** the minimum supported Rust version is now **1.88**, up from - 1.81, and the crate is compiled with **edition 2024**. 1.88 is the version - Red Hat Enterprise Linux 9.7 ships as `rust-toolset`. - -### Fixed - -- Public documentation on `PersistentAttachOutcome`, `ensure_exists_in`, - `list_in`, `upsert_stub_in`, `set_metadata_in` and `reconcile_in` no longer - links to private items, which made `cargo doc` fail under - `RUSTDOCFLAGS="-D warnings"`. The prose still names the internal helpers; it - just no longer tries to hyperlink to items a reader cannot navigate to. - ### Added - **`kv_set_many` tool** — atomic batch write accepting an array of `{key, value}` entries. Validates all keys before opening the transaction; an invalid key aborts the whole batch without writing anything. Default behavior (`overwrite` absent or `true`) reports `{stored, created, overwritten, total_bytes}`; with `overwrite: false`, existing keys are skipped (not errors) and the response reports `{stored, created, skipped, total_bytes}` where `created` is the number of keys newly inserted. `total_bytes` sums all submitted values, so it is an upper bound on bytes actually persisted when keys are skipped or duplicated. Each oversized entry (> 1 MiB) adds a keyed `warning` to a `warnings` array. @@ -74,6 +47,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Changed +- The `arrow` and `parquet` dependencies moved from **58** to **59**. Not a + library-API change (this crate ships a binary), but it removes the `thrift` + dependency and the Apache Thrift excessive-size-allocation advisory with it: + `parquet` 58.x pinned `thrift ^0.17`, and `parquet` 59 dropped thrift + entirely. + +- `Engine::execute_in_transaction` now calls `hyperdb-api`'s `*_unguarded` + transaction methods instead of the deprecated `begin_transaction` / `commit` + / `rollback`, which 1.0.0 removed. No behavior change: the helper still takes + `&self`, so the RAII guard remains unavailable to it, and it still rolls back + before resuming an unwind. The `#[allow(deprecated)]` it needed is gone. + Moving to the guard still waits on + [issue #72](https://github.com/tableau/hyper-api-rust/issues/72). +- **BREAKING:** the minimum supported Rust version is now **1.88**, up from + 1.81, and the crate is compiled with **edition 2024**. 1.88 is the version + Red Hat Enterprise Linux 9.7 ships as `rust-toolset`. - **KV attachment/read-only clarification (supersedes the shorthand in the Added notes above).** The global `--read-only` guard leaves the four KV readers available, but every `kv_*` call targeting a user attachment still @@ -91,6 +80,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Fixed +- Public documentation on `PersistentAttachOutcome`, `ensure_exists_in`, + `list_in`, `upsert_stub_in`, `set_metadata_in` and `reconcile_in` no longer + links to private items, which made `cargo doc` fail under + `RUSTDOCFLAGS="-D warnings"`. The prose still names the internal helpers; it + just no longer tries to hyperlink to items a reader cannot navigate to. - **Hyper-format export side-effect correction (supersedes the older Unreleased note below).** Export does not mutate its source database, but it creates or replaces the requested destination `.hyper` file and materializes From 5826f538690d5b97a00888c173c565f107045496 Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 14:49:03 -0700 Subject: [PATCH 4/7] docs: wrap the 12 over-long prose lines (MD013) Takes the count to zero. All 12 were single unwrapped paragraphs or changelog bullets of 529-1104 characters, in files whose median prose line is 68-79, so each was an outlier against its own file rather than something inherently unwrappable. Wrapped at 79. No inline disable was needed anywhere, and MD013 stays enabled globally. The rule's own configuration is why the set is this small and this clean: line_length is 500 with tables and code_blocks exempt, so long table rows and code never appear, and none of the 12 lines contains a link or a bare URL -- checked before wrapping, since a break inside a link destination is exactly the corruption that MD060 was disabled over. Wrapped by a tokenizer that treats code spans as atomic, so no break can land inside one, and that indents list continuations to the bullet's content column so the text stays in its list item. Every line is asserted to round trip: stripping the continuation indent and rejoining with single spaces has to reproduce the original line exactly, or the wrap aborts. Verified again independently afterwards, whole-file this time -- whitespace-normalized text is identical to the previous commit for all five files, so no word, code span or punctuation mark moved. hyperdb-mcp/README.md is the one file whose tests key off line positions, and the wrap shifted them by seven. Re-checked by re-implementing doctor_readme_contract's window logic against the edited file: the exact `hyperdb-mcp doctor` and `doctor --json` lines are still found, and the -12/+21 window around them still contains the "side-effect-free" and "does not start" phrases the test requires. The doctor_tests `## CLI Reference` extraction and its four substring checks also still pass, as does the `daemon ` foreground line. Static simulation, not an executed cargo run -- this is a docs-only change and no test was run. --- docs/TRANSACTIONS.md | 8 +++- hyperdb-api/CHANGELOG.md | 11 ++++- hyperdb-mcp/CHANGELOG.md | 20 +++++++++- hyperdb-mcp/DEVELOPMENT.md | 82 +++++++++++++++++++++++++++++++++----- hyperdb-mcp/README.md | 9 ++++- 5 files changed, 116 insertions(+), 14 deletions(-) diff --git a/docs/TRANSACTIONS.md b/docs/TRANSACTIONS.md index 57bf2b3..f108a12 100644 --- a/docs/TRANSACTIONS.md +++ b/docs/TRANSACTIONS.md @@ -28,7 +28,13 @@ txn.commit()?; #### 1. Exclusive Borrowing -`Connection::transaction(&mut self)` takes a mutable (exclusive) borrow of the connection, and `Transaction<'conn>` holds `&'conn mut Connection`. While the `Transaction` exists, the Rust borrow checker prevents any other code from accessing the raw connection — not even for read-only operations. This eliminates an entire class of bugs where application code accidentally issues SQL statements outside the transaction scope, causing data races or logic errors. The protection is enforced at compile time with zero runtime cost. +`Connection::transaction(&mut self)` takes a mutable (exclusive) borrow of the +connection, and `Transaction<'conn>` holds `&'conn mut Connection`. While the +`Transaction` exists, the Rust borrow checker prevents any other code from +accessing the raw connection — not even for read-only operations. This +eliminates an entire class of bugs where application code accidentally issues +SQL statements outside the transaction scope, causing data races or logic +errors. The protection is enforced at compile time with zero runtime cost. ```rust let mut conn = Connection::connect(endpoint, "db.hyper", CreateMode::DoNotCreate)?; diff --git a/hyperdb-api/CHANGELOG.md b/hyperdb-api/CHANGELOG.md index 841e023..42b030f 100644 --- a/hyperdb-api/CHANGELOG.md +++ b/hyperdb-api/CHANGELOG.md @@ -111,7 +111,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/). unification applied it workspace-wide. See the `hyperdb-bootstrap` and `hyperdb-api-salesforce` entries for detail. -- **BREAKING:** `KvStore::set`, `KvStore::set_as`, and `KvStore::set_batch` (plus their `AsyncKvStore` twins) now return `SetOutcome` or `BatchSetOutcome` instead of `Result<()>`, reporting whether each write created a new key or overwrote an existing one. The `created` signal eliminates silent data loss when an LLM accidentally clobbers existing KV data. Callers that ignored the `Result` (statement-position `set("k","v")?;`) — including `let _ = set(...)?;` — still compile unchanged. The genuinely breaking cases are callers that named the unit return (`let x: () = set(...)?;`) or that returned `set(...)` where a `Result<()>` was expected; these now see `SetOutcome`/`BatchSetOutcome` and must adapt. +- **BREAKING:** `KvStore::set`, `KvStore::set_as`, and `KvStore::set_batch` + (plus their `AsyncKvStore` twins) now return `SetOutcome` or + `BatchSetOutcome` instead of `Result<()>`, reporting whether each write + created a new key or overwrote an existing one. The `created` signal + eliminates silent data loss when an LLM accidentally clobbers existing KV + data. Callers that ignored the `Result` (statement-position `set("k","v")?;`) + — including `let _ = set(...)?;` — still compile unchanged. The genuinely + breaking cases are callers that named the unit return (`let x: () = + set(...)?;`) or that returned `set(...)` where a `Result<()>` was expected; + these now see `SetOutcome`/`BatchSetOutcome` and must adapt. ### Added diff --git a/hyperdb-mcp/CHANGELOG.md b/hyperdb-mcp/CHANGELOG.md index dde8e9b..4e122eb 100644 --- a/hyperdb-mcp/CHANGELOG.md +++ b/hyperdb-mcp/CHANGELOG.md @@ -9,8 +9,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Added -- **`kv_set_many` tool** — atomic batch write accepting an array of `{key, value}` entries. Validates all keys before opening the transaction; an invalid key aborts the whole batch without writing anything. Default behavior (`overwrite` absent or `true`) reports `{stored, created, overwritten, total_bytes}`; with `overwrite: false`, existing keys are skipped (not errors) and the response reports `{stored, created, skipped, total_bytes}` where `created` is the number of keys newly inserted. `total_bytes` sums all submitted values, so it is an upper bound on bytes actually persisted when keys are skipped or duplicated. Each oversized entry (> 1 MiB) adds a keyed `warning` to a `warnings` array. -- **`kv_set` — `value_path` parameter** — absolute path to a file whose contents become the value (read server-side). Provide exactly one of `value` or `value_path`; neither or both is `INVALID_ARGUMENT`. Reads any path the server process can read (same posture as `load_file` — no sandbox), with I/O errors preserved (`PermissionDenied` → `ErrorCode::PermissionDenied`, not collapsed to `FileNotFound`). A hard 64 MiB size cap is enforced against the file's metadata *before* reading, so a stray path to a huge file is rejected with `INVALID_ARGUMENT` instead of being slurped into memory. +- **`kv_set_many` tool** — atomic batch write accepting an array of + `{key, value}` entries. Validates all keys before opening the transaction; an + invalid key aborts the whole batch without writing anything. Default behavior + (`overwrite` absent or `true`) reports + `{stored, created, overwritten, total_bytes}`; with `overwrite: false`, + existing keys are skipped (not errors) and the response reports + `{stored, created, skipped, total_bytes}` where `created` is the number of + keys newly inserted. `total_bytes` sums all submitted values, so it is an + upper bound on bytes actually persisted when keys are skipped or duplicated. + Each oversized entry (> 1 MiB) adds a keyed `warning` to a `warnings` array. +- **`kv_set` — `value_path` parameter** — absolute path to a file whose + contents become the value (read server-side). Provide exactly one of `value` + or `value_path`; neither or both is `INVALID_ARGUMENT`. Reads any path the + server process can read (same posture as `load_file` — no sandbox), with I/O + errors preserved (`PermissionDenied` → `ErrorCode::PermissionDenied`, not + collapsed to `FileNotFound`). A hard 64 MiB size cap is enforced against the + file's metadata *before* reading, so a stray path to a huge file is rejected + with `INVALID_ARGUMENT` instead of being slurped into memory. - **`kv_set` — `overwrite` parameter** (default `true`) — when `false`, skips the write if the key already exists (calls `set_if_absent` instead of `set`), returning `{stored: false, created: false, existed: true}` with the original value unchanged. Eliminates silent data-loss from accidental overwrites. - **`kv_set` — `created` and `value_bytes` in response** — `created: true` means the key was newly inserted, `false` means an existing value was overwritten. `value_bytes` reports the UTF-8 byte length of the written value. - **`kv_set` — soft size warning** — values exceeding 1 MiB trigger a non-fatal `warning` field in the response steering the LLM toward `load_data` or a real table for large payloads. The write always succeeds. diff --git a/hyperdb-mcp/DEVELOPMENT.md b/hyperdb-mcp/DEVELOPMENT.md index 99a2c17..c4f79bb 100644 --- a/hyperdb-mcp/DEVELOPMENT.md +++ b/hyperdb-mcp/DEVELOPMENT.md @@ -101,7 +101,16 @@ Every ingest function (`ingest_json`, `ingest_csv`, `ingest_parquet_file`, `inge Three edges to this guarantee, all documented in `src/engine.rs`: 1. **DDL auto-commits.** Hyper commits `CREATE TABLE` / `DROP TABLE` immediately, regardless of the surrounding transaction. In `replace` mode the original table is already gone by the time INSERTs start, so a failed replace-mode ingest leaves an empty table rather than restoring the original. Append mode is fully atomic because it issues DDL only when the target doesn't exist and, when it does, no data is lost on failure. -2. **Panic safety.** `execute_in_transaction` wraps the closure in `catch_unwind(AssertUnwindSafe(...))`, issues a best-effort ROLLBACK on panic, and `resume_unwind`s the original payload. Without this, a panic inside the closure (unwrap on None, indexing OOB, arithmetic overflow) would leave an open transaction and every subsequent tool call would hit "transaction already in progress" — classified as `InternalError`, not `ConnectionLost`, so the reconnect path at `with_engine` would not rescue it and the engine would stay wedged until restart. Tested via `execute_in_transaction_rolls_back_on_panic` in `tests/transaction_tests.rs`. +2. **Panic safety.** `execute_in_transaction` wraps the closure in + `catch_unwind(AssertUnwindSafe(...))`, issues a best-effort ROLLBACK on + panic, and `resume_unwind`s the original payload. Without this, a panic + inside the closure (unwrap on None, indexing OOB, arithmetic overflow) would + leave an open transaction and every subsequent tool call would hit + "transaction already in progress" — classified as `InternalError`, not + `ConnectionLost`, so the reconnect path at `with_engine` would not rescue it + and the engine would stay wedged until restart. Tested via + `execute_in_transaction_rolls_back_on_panic` in + `tests/transaction_tests.rs`. 3. **Post-error wire-protocol quirk.** After a mid-transaction Hyper-level error (e.g. a NOT NULL violation on INSERT), the first SELECT after rollback may return an empty result set due to residual bytes on the connection. Retrying the query once restores normal behavior; the rollback itself is always correct. The `query_resilient` helper in `tests/transaction_tests.rs` is the robust pattern. --- @@ -217,15 +226,49 @@ Logs land next to the persistent file when one is supplied (so users find them i ## Daemon Mode Internals -`Engine::new` defaults to *daemon mode* — it tries `daemon::spawn::ensure_daemon(resolve_port_scan())` first, which discovers an existing daemon via `~/.hyperdb/daemon.json` (overridable via `HYPERDB_STATE_DIR`), else scans the port range for a running daemon, else auto-spawns one on the first free port as a detached background process. The Engine then connects via TCP (`Connection::connect(endpoint, …)`) without owning any `HyperProcess`, and records the daemon's `health_port` so the server's debounced `HEARTBEAT` targets the actual discovered port rather than re-resolving. +`Engine::new` defaults to *daemon mode* — it tries +`daemon::spawn::ensure_daemon(resolve_port_scan())` first, which discovers an +existing daemon via `~/.hyperdb/daemon.json` (overridable via +`HYPERDB_STATE_DIR`), else scans the port range for a running daemon, else +auto-spawns one on the first free port as a detached background process. The +Engine then connects via TCP (`Connection::connect(endpoint, …)`) without +owning any `HyperProcess`, and records the daemon's `health_port` so the +server's debounced `HEARTBEAT` targets the actual discovered port rather than +re-resolving. Falls back to local mode (per-session `hyperd` via `HyperProcess::new`) when the daemon can't be reached (including `AllOccupied` — the whole scan range is held by foreign processes), or always when `--no-daemon` is passed. -**Port resolution + identity.** `resolve_port_scan()` returns a `PortScan { base, span }`: when `HYPERDB_DAEMON_PORT` is set it pins that exact port (`span = 1`); otherwise it scans `span = DAEMON_PORT_SCAN_SPAN` (16) ports up from `DEFAULT_DAEMON_BASE_PORT` (7485 — deliberately *not* 7484, which is hyperd's conventional gRPC port). `probe_port` classifies each port as `OurDaemon` / `Camped` / `Refused`: liveness is no longer a bare TCP connect but an identity handshake — `health::ping_identified` sends `PING` and requires the reply's first two tokens to be exactly `PONG` and `hyperdb-mcp` (the third token is the daemon version). A foreign process that merely accepts TCP is `Camped` and skipped; only a `Refused` (connection-refused) port is treated as free to spawn on. `discover()` applies the same identity check before trusting `daemon.json`, so a stale or foreign-owned file is detected and removed. - -**Version takeover.** When discovery finds a running daemon, `maybe_take_over` compares the client's `version::MCP_VERSION` against the daemon's reported version via the pure `client_should_take_over` helper (`semver`). If the client is *strictly newer* it sends `STOP` (which drops the daemon's `HyperProcess`, stopping `hyperd`), waits for the health port to stop answering the identity ping, then respawns a fresh daemon on the same port. Equal/older/unparseable versions reuse the daemon — never a downgrade-kill. This makes upgrades take effect immediately instead of waiting for the old daemon to disappear. - -**Idle shutdown is opt-in.** `DaemonConfig.idle_timeout` is `Option`, set only when `--idle-timeout` or `HYPERDB_DAEMON_IDLE_TIMEOUT` is provided (flag wins over env). With neither set the idle-monitor branch of the `run_daemon` `tokio::select!` is replaced by `std::future::pending()` and never fires — the daemon (and `hyperd`) stay resident indefinitely so clients never pay the cold-start "restarting, please retry" round-trip. `DaemonState::last_activity` and the debounced `HEARTBEAT` plumbing still exist and only matter when the timeout is enabled. The hyperd restart-limit shutdown (below) is independent and always active. +**Port resolution + identity.** `resolve_port_scan()` returns a +`PortScan { base, span }`: when `HYPERDB_DAEMON_PORT` is set it pins that exact +port (`span = 1`); otherwise it scans `span = DAEMON_PORT_SCAN_SPAN` (16) ports +up from `DEFAULT_DAEMON_BASE_PORT` (7485 — deliberately *not* 7484, which is +hyperd's conventional gRPC port). `probe_port` classifies each port as +`OurDaemon` / `Camped` / `Refused`: liveness is no longer a bare TCP connect +but an identity handshake — `health::ping_identified` sends `PING` and requires +the reply's first two tokens to be exactly `PONG` and `hyperdb-mcp` (the third +token is the daemon version). A foreign process that merely accepts TCP is +`Camped` and skipped; only a `Refused` (connection-refused) port is treated as +free to spawn on. `discover()` applies the same identity check before trusting +`daemon.json`, so a stale or foreign-owned file is detected and removed. + +**Version takeover.** When discovery finds a running daemon, `maybe_take_over` +compares the client's `version::MCP_VERSION` against the daemon's reported +version via the pure `client_should_take_over` helper (`semver`). If the client +is *strictly newer* it sends `STOP` (which drops the daemon's `HyperProcess`, +stopping `hyperd`), waits for the health port to stop answering the identity +ping, then respawns a fresh daemon on the same port. Equal/older/unparseable +versions reuse the daemon — never a downgrade-kill. This makes upgrades take +effect immediately instead of waiting for the old daemon to disappear. + +**Idle shutdown is opt-in.** `DaemonConfig.idle_timeout` is `Option`, +set only when `--idle-timeout` or `HYPERDB_DAEMON_IDLE_TIMEOUT` is provided +(flag wins over env). With neither set the idle-monitor branch of the +`run_daemon` `tokio::select!` is replaced by `std::future::pending()` and never +fires — the daemon (and `hyperd`) stay resident indefinitely so clients never +pay the cold-start "restarting, please retry" round-trip. +`DaemonState::last_activity` and the debounced `HEARTBEAT` plumbing still exist +and only matter when the timeout is enabled. The hyperd restart-limit shutdown +(below) is independent and always active. ### hyperd liveness monitoring and restart @@ -252,7 +295,21 @@ Two new code paths fire `report_hyperd_error_to_daemon` (best-effort, 200ms time ### Known limitations -- **Hung-but-alive `hyperd`** (TCP listening, but unresponsive to queries) is NOT detected. The monitor's `try_wait()` returns `None` for a hung process; client tool calls hang on the read side without producing a `ConnectionLost` error. Operator recovery is `hyperdb-mcp daemon stop` followed by reconnect. Note the tradeoff introduced by resident-by-default: the idle timeout used to be an implicit backstop that reaped a wedged daemon after 30 min, after which the next client respawned a fresh one. With idle shutdown now opt-in (off by default), a hung-but-alive `hyperd` stays wedged until a client reports an error (fast-path `REPORT_HYPERD_ERROR`, which fires on a client-side `ConnectionLost`) or an operator runs `daemon stop`. This is an accepted tradeoff — keeping `hyperd` warm avoids the cold-start "restarting, please retry" round-trip on every active session, and genuine hyperd hangs are rare. A future enhancement could add a daemon-side liveness probe (a periodic trivial query with a timeout) to close the "all clients idle + hyperd hung" gap without reintroducing cold-start latency. +- **Hung-but-alive `hyperd`** (TCP listening, but unresponsive to queries) is + NOT detected. The monitor's `try_wait()` returns `None` for a hung process; + client tool calls hang on the read side without producing a `ConnectionLost` + error. Operator recovery is `hyperdb-mcp daemon stop` followed by reconnect. + Note the tradeoff introduced by resident-by-default: the idle timeout used to + be an implicit backstop that reaped a wedged daemon after 30 min, after which + the next client respawned a fresh one. With idle shutdown now opt-in (off by + default), a hung-but-alive `hyperd` stays wedged until a client reports an + error (fast-path `REPORT_HYPERD_ERROR`, which fires on a client-side + `ConnectionLost`) or an operator runs `daemon stop`. This is an accepted + tradeoff — keeping `hyperd` warm avoids the cold-start "restarting, please + retry" round-trip on every active session, and genuine hyperd hangs are rare. + A future enhancement could add a daemon-side liveness probe (a periodic + trivial query with a timeout) to close the "all clients idle + hyperd hung" + gap without reintroducing cold-start latency. - **Watchers** auto-recover from hyperd restarts: when an ingest fails with a connection-lost error, the watcher rebuilds its connection pool against the engine's current endpoint and retries the file once. Persistent failures (the second attempt also fails) fall through to the standard `failed/` move so a single broken file can't keep the watcher pinned in retry loops. See `src/daemon/{mod,discovery,health,run,spawn}.rs` for the full implementation. @@ -268,7 +325,14 @@ See `src/daemon/{mod,discovery,health,run,spawn}.rs` for the full implementation - **No `spawn_blocking`**: Engine calls are synchronous in an async runtime. Works today because `rmcp` handles dispatch, but explicit `spawn_blocking` would be more robust. - **Watcher thread per directory**: Each `watch_directory` call spawns a dedicated OS thread. For many concurrent watchers, a shared thread pool would be more efficient. - **Single shared `Connection`**: `Engine` owns exactly one `Connection` to `hyperd`, and every tool call serializes on an `Arc>>`. `hyperd` itself can handle many concurrent connections, so a `Connection` pool (or a dedicated watcher-thread `Connection`) would stop a long-running ingest from blocking unrelated tool calls. -- **`list_changed` is not fired on append-mode table creation**: Both `load_*` append mode and `watch_directory` auto-create the target table via `CREATE TABLE IF NOT EXISTS` when it doesn't exist, but neither path plumbs a "created vs existed" signal back from the ingest layer, so clients subscribed to `hyper://tables` miss the new entry until they rescan. The fix is to have `create_table` report whether it actually created vs. no-oped, bubble that up through the ingest return, and have both callers fire `notify_list_changed` only when a new table appears. +- **`list_changed` is not fired on append-mode table creation**: Both `load_*` + append mode and `watch_directory` auto-create the target table via + `CREATE TABLE IF NOT EXISTS` when it doesn't exist, but neither path plumbs a + "created vs existed" signal back from the ingest layer, so clients subscribed + to `hyper://tables` miss the new entry until they rescan. The fix is to have + `create_table` report whether it actually created vs. no-oped, bubble that up + through the ingest return, and have both callers fire `notify_list_changed` + only when a new table appears. --- diff --git a/hyperdb-mcp/README.md b/hyperdb-mcp/README.md index 189ee9d..fb394de 100644 --- a/hyperdb-mcp/README.md +++ b/hyperdb-mcp/README.md @@ -237,7 +237,14 @@ describe({ database: "persistent" }) sample({ table: "customers", database: "persistent" }) ``` -The `database` parameter is available on `query`, `execute`, `load_data`, `load_file`, `load_files`, `watch_directory`, `describe`, `sample`, `chart`, `export`, and `set_table_metadata`. The shorthand `persist: true` (sugar for `database: "persistent"`) is available on `load_data`, `load_file`, `load_files`, and `watch_directory`. Read tools generally accept a read-only user attachment; write tools require a writable one. The exception is the KV family: every `kv_*` call to a user attachment requires it to be writable because the backing table may need initialization. +The `database` parameter is available on `query`, `execute`, `load_data`, +`load_file`, `load_files`, `watch_directory`, `describe`, `sample`, `chart`, +`export`, and `set_table_metadata`. The shorthand `persist: true` (sugar for +`database: "persistent"`) is available on `load_data`, `load_file`, +`load_files`, and `watch_directory`. Read tools generally accept a read-only +user attachment; write tools require a writable one. The exception is the KV +family: every `kv_*` call to a user attachment requires it to be writable +because the backing table may need initialization. Every successful database-routed response includes the canonical `resolved_database`: `"local"`, `"persistent"`, or the lowercase attached From ed1dba3846e0e0c048d88bf87ad7b575e42406fc Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 15:01:12 -0700 Subject: [PATCH 5/7] chore(docs): lint docs/superpowers, relaxing only MD013 and MD046 Replaces the blanket docs/superpowers/** exclusion from 0b442c5 with a directory-scoped relaxation of the two rules that would force a reflow. The argument against excluding won: that tree is not a static archive. AGENTS.md mandates a spec and plan for every non-trivial feature, so new files land there regularly, and an exclusion leaves the editor silent exactly while one is being authored. The rules worth having then -- untagged fences, heading structure -- are precisely the ones an exclusion would suppress. A relaxation keeps them and drops only MD013 (plan prose runs past 1500 characters per line) and MD046 (plans mix indented and fenced blocks), which are the two that would demand rewriting a settled record. The root CHANGELOG.md exclusion is untouched; that one is generated, which is a different argument entirely. The mechanism is a nested docs/superpowers/.markdownlint.json, so one file governs both tools: markdownlint-cli2 resolves configuration per directory, and the extension's documented precedence is a ".markdownlint.{jsonc,json,...} file in the same or parent directory", so it walks up from whatever file is open. Confirmed both rather than assumed. The CLI now lints 68 files instead of 56, reports 0 issues, and the 1_88_uplift subdirectory two levels below the config went from 8 baseline findings to 0, which shows the config governs recursively and not just its own directory. The extension side is documented behaviour plus its own guidance that a CLI-first setup needs no further change to behave the same in the editor. .markdownlintignore loses its docs/superpowers/ entry, and the `ignore` package the extension uses agrees: the tree is linted at every depth, root CHANGELOG.md still ignored, all nine per-crate changelogs still in scope. The "extends" line is load-bearing and nearly went in as decoration. A nested config REPLACES ancestor configuration rather than merging with it -- measured, after a first attempt to measure it drew the opposite conclusion off a bad probe (a no-spaces long line, which MD013 never flags in its default non-strict mode). Dropping just the extends from this file takes the repo from 5 findings to 92: 70 MD060 and 17 MD010. MD060 is the rule this project disabled because a formatter satisfying it stripped the README's badge links, so a naive nested config would have quietly reintroduced it across the plan files. Extending the root config inherits all of it and overrides only the two rules named above. --- .markdownlint-cli2.jsonc | 6 ------ .markdownlintignore | 6 ------ docs/superpowers/.markdownlint.json | 33 +++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 12 deletions(-) create mode 100644 docs/superpowers/.markdownlint.json diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc index 30b1116..8b44128 100644 --- a/.markdownlint-cli2.jsonc +++ b/.markdownlint-cli2.jsonc @@ -30,12 +30,6 @@ // nine per-crate changelogs are hand-maintained and stay in scope. "CHANGELOG.md", - // Point-in-time planning artifacts (see AGENTS.md "Documentation - // Conventions"): a spec and plan are written before a feature, reviewed - // alongside the change that implements it, and then not maintained. - // Reflowing prose in a settled record is churn no reader benefits from. - "docs/superpowers/**", - // npm platform sub-packages: packaging boilerplate, largely generated "hyperdb-api-node/npm/**", "hyperdb-mcp/npm/**" diff --git a/.markdownlintignore b/.markdownlintignore index ff803d9..78aaf14 100644 --- a/.markdownlintignore +++ b/.markdownlintignore @@ -21,12 +21,6 @@ hyperdb-mcp/scripts/dc_sql_reference.md # patterns, so the equivalent entry there needs no slash.) /CHANGELOG.md -# Point-in-time planning artifacts (see AGENTS.md "Documentation -# Conventions"): a spec and plan are written before a feature, reviewed -# alongside the change that implements it, and then not maintained. -# Reflowing prose in a settled record is churn no reader benefits from. -docs/superpowers/ - # npm platform sub-packages: packaging boilerplate, largely generated hyperdb-api-node/npm/ hyperdb-mcp/npm/ diff --git a/docs/superpowers/.markdownlint.json b/docs/superpowers/.markdownlint.json new file mode 100644 index 0000000..a4fdbe4 --- /dev/null +++ b/docs/superpowers/.markdownlint.json @@ -0,0 +1,33 @@ +// Directory-scoped rules for docs/superpowers/ (specs and plans). +// +// These are point-in-time planning artifacts: a spec and plan are written +// before a feature, reviewed alongside the change that implements it, and then +// left alone (AGENTS.md, "Documentation Conventions"). Reflowing prose or +// re-indenting code blocks in a settled record is churn no reader benefits +// from, and it is the riskiest place to do it -- these are the longest files in +// the repo. So the two rules that would force a rewrite are off here: +// +// MD013 line-length -- plan prose runs to 1500+ characters per line +// MD046 code-block-style -- plans mix indented and fenced blocks +// +// Every other rule still applies, deliberately. New specs and plans are +// written all the time, and MD040 (untagged fences) plus the heading rules are +// the feedback worth having while authoring one. This is a relaxation, not an +// exclusion: an exclusion would leave the editor silent on a directory that is +// actively written to. +// +// "extends" is load-bearing, not decoration. A nested config REPLACES ancestor +// configuration rather than merging with it -- verified empirically: without +// this line, the root's `MD024.siblings_only` reverts to the default and fires +// on plan files, and `MD060: false` would likewise be lost, re-enabling the +// rule that was disabled for mangling links. Extending the root config keeps +// all of it and overrides only the two rules below. +// +// Governs both tools from one file: markdownlint-cli2 resolves config per +// directory, and the VS Code extension walks up from the file being edited +// (".markdownlint.{jsonc,json,...} file in the same or parent directory"). +{ + "extends": "../../.markdownlint.json", + "MD013": false, + "MD046": false +} From 585c81a0c038291c12b4927f1628ccfde48f2f9f Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 15:01:24 -0700 Subject: [PATCH 6/7] docs: tag the 5 untagged fences in docs/superpowers (MD040) Takes the repo to zero findings with docs/superpowers now linted. These are the 5 that 5185ad7 left behind because the tree was excluded at the time. Two blocks in the kv-store-m2 plan take `markdown`: both are Markdown intended to be pasted elsewhere -- a "### Key-value store (scratchpad)" section destined for readme.rs, and two changelog bullets for hyperdb-mcp's Unreleased section. The three in the LLM-ergonomics plan take `text`: each is a prose fragment to be appended to a #[tool(description = ...)] string, and two of them open with a deliberate leading space for that concatenation, which is preserved untouched since only the fence line is rewritten. Same tagger and same proof as 5185ad7, which matters more here than anywhere else: these are the longest files in the repo and the ones nobody re-reads, so a cascade would sit undetected. Checked first that no target block contains a fence line of its own -- the markdown block was the real risk, since a snippet documenting Markdown is how nesting shows up -- and the tagger still asserts per fence that the line is both a bare fence and classified as an opener by the state machine. Structural proof over all 69 files, against the branch base: 554 code blocks before and 554 after, every block's content SHA identical, no unclosed fence. Untagged fenced blocks are now 0 repo-wide, down from 68 at baseline. Independently, the diff is 5 insertions and 5 deletions with every changed line a fence line, so no content line entered or left a block. --- docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md | 4 ++-- docs/superpowers/plans/2026-07-11-kv-mcp-llm-ergonomics.md | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md b/docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md index bbbd874..de5091b 100644 --- a/docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md +++ b/docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md @@ -782,7 +782,7 @@ Expected: FAIL — `README missing mention of kv_get` (README not yet updated). - [ ] **Step 3: Add the `### Key-value store` subsection** under `## Tool index` in `readme.rs`: -``` +```markdown ### Key-value store (scratchpad) - `kv_set` — save a variable/state/summary/JSON under a store + key (upsert). @@ -862,7 +862,7 @@ Expected: PASS, all cases. Real output. `hyperdb-mcp/CHANGELOG.md` under `## [Unreleased]` → `### Added`: -``` +```markdown - `kv_get`, `kv_set`, `kv_delete`, `kv_list`, `kv_list_stores`, `kv_size`, `kv_pop`, `kv_clear` tools — a key-value scratchpad backed by the `hyperdb-api` KV store, routable to any database via the standard `database`/`persist` parameters. - `hyper://schema/kv` resource describing the KV table schema, durability rule, and LEFT JOIN enrichment pattern. ``` diff --git a/docs/superpowers/plans/2026-07-11-kv-mcp-llm-ergonomics.md b/docs/superpowers/plans/2026-07-11-kv-mcp-llm-ergonomics.md index f170e97..2e81acc 100644 --- a/docs/superpowers/plans/2026-07-11-kv-mcp-llm-ergonomics.md +++ b/docs/superpowers/plans/2026-07-11-kv-mcp-llm-ergonomics.md @@ -1245,7 +1245,7 @@ Replace `kv_set` (`server.rs:3123-3139`). Resolve the value from exactly one of Update the `#[tool(description = ...)]` on `kv_set` (`server.rs:3120-3121`) to document the new behavior. Append to the existing description: -``` +```text Returns {stored, created, value_bytes}; `created:false` means an existing value was overwritten. Pass overwrite=false to avoid clobbering (skips + returns stored:false, existed:true). Pass value_path= to store a file's contents server-side instead of `value` (exactly one of value/value_path; reads any server-readable path — no sandbox). ``` @@ -1346,7 +1346,7 @@ Replace the `kv_size` handler body in `hyperdb-mcp/src/server.rs:3189-3204` (the Update the `#[tool(description = ...)]` on `kv_size` (around `server.rs:3186-3187`) to document the `bytes` field. Extend the description to say: -``` +```text Returns {store, size, bytes} where `size` is the key count and `bytes` is the total `OCTET_LENGTH` of all values (0 for empty stores). ``` @@ -1729,7 +1729,7 @@ Replace the `kv_list` handler (`server.rs:3169-3184`) to accept `KvListParams` a Locate the `#[tool(description = ...)]` attribute on `kv_list` (a few lines above `:3169`). Append to the existing description: -``` +```text Pass values=true to return full (key, value) pairs as an `entries` array instead of just keys — useful for reading a whole store without N×kv_get. ``` From a388db8099839d063e7c5d17db2d5e64b9c48a04 Mon Sep 17 00:00:00 2001 From: Stefan Steiner Date: Sat, 5 Sep 2026 15:10:05 -0700 Subject: [PATCH 7/7] docs(agents): record that a nested markdownlint config replaces, not merges Adds a fourth trap to the markdownlint reminder. It belongs with the other three for the reason the section exists: markdownlint is not a CI gate, so a headless agent gets no feedback at all, and an agent adding a nested config elsewhere in the repo would have no reason to open docs/superpowers/.markdownlint.json and discover this. The full rationale stays in that file's comments; the reminder just names the trap and the fix. The failure mode is what makes it worth a bullet. Omitting `extends` does not make the document wrong, it makes the linter wrong -- it reports fewer or different findings while looking like it works, and here it would silently re-enable MD060, the one rule this repo has already been burned by when a formatter satisfying it stripped the README's badge links. Placed last so it sits directly above the paragraph explaining why MD060 is disabled. Mirrored the neighbouring bullets exactly rather than trusting the rendered view, which normalizes whitespace: on this list the `-` marker sits at column 0 and continuations at one space, so the bullets are a sibling top-level list rather than nested under item 3. Indenting a new bullet even one space makes it a badly-indented nested list and trips MD007, which has happened here before. Verified on both engines -- `markdownlint-cli2` latest and the 0.23.2 the installed extension (DavidAnson.vscode-markdownlint 0.62.1) declares -- 68 files, 0 issues, exit 0 on each. Latest currently resolves to 0.23.2, so there is no version skew between editor and CLI at all right now. --- AGENTS.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 64cc485..b6ec3c9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -519,7 +519,7 @@ All commit messages **must** follow the format `(): ` — upstream/main:` and re-lint — rather than assuming, or you will "fix" things that were never broken and miss the ones you introduced. - Three traps that have actually bitten: + Four traps that have actually bitten: - **Duplicate `### Fixed` / `### Added` siblings under one `## [Unreleased]`** (MD024). Changelogs here often already have the section further down. Merge @@ -533,6 +533,11 @@ All commit messages **must** follow the format `(): ` — new block. This corrupted 176 fences across 22 files once. Any such pass must track fence state; prefer `markdownlint-cli2 --fix`, which is safe, and note that it cannot fix MD040 because choosing a language needs judgement. +- **A nested `.markdownlint.json` replaces the root config rather than merging + with it**, so a new one must `extends` the root or every rule there reverts to + default. Dropping that line from `docs/superpowers/.markdownlint.json` turns 5 + findings into 92 — 70 of them the MD060 disabled just below. It hides well: it + makes the *linter* wrong, not the document. Beware format-on-save: a Markdown formatter reformatting tables to satisfy MD060 once stripped the README's badge links (`[![CI](img)](target)` became