Skip to content

Commit 30fb49f

Browse files
committed
feat: add bounded vector queries and deterministic native work regressions
1 parent aa10337 commit 30fb49f

48 files changed

Lines changed: 1812 additions & 363 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -332,12 +332,13 @@ Discovery exposes static operation documentation. Each invocation checks current
332332

333333
> **KeyLoad is a development preview.** Try it, build with it and [tell us what breaks](https://github.com/managedcode/KeyLoad/issues), but don't trust it with production data yet.
334334
335-
The original 104-task plan has **10 accepted, 94 in progress and 0 pending**.
335+
The original 104-task plan has **11 accepted, 93 in progress and 0 pending**.
336336
Accepted: **KL-005** backup/restore, **KL-007** storage codecs, **KL-010**
337337
document CRUD/CAS, **KL-012** batch/idempotency, **KL-013** document-only filter/query,
338338
**KL-016** committed projection
339339
outbox, **KL-024** time-series ordering/idempotency and **KL-025** reference
340-
ranges/aggregates, **KL-022** graph storage and **KL-023** bounded graph traversal.
340+
ranges/aggregates, **KL-022** graph storage, **KL-023** bounded graph traversal
341+
and **KL-020** three-node replicated apply.
341342
Each closure is bound to its original criteria and Linux
342343
operation evidence. Complete feature and cluster qualification remains open.
343344

‎docs/ADR/ADR-009-search-provider-boundaries.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,23 @@ flowchart LR
4848
Freshness --> Compare
4949
Compare --> Gate[Qualified semantics before delivery claims]
5050
```
51+
52+
53+
## TASK-KL028-SELECTED-PROVIDER-CAPABILITY-AUDIT-001
54+
55+
REQ-FTS-AUDIT-001 / AC-FTS-AUDIT-001 freezes original KL028 selected-provider audit: retain centrally pinned ZoneTree.FullTextSearch1.0.9/ZoneTree1.9.8 and native IndexOfTokenRecordPreviousToken<ulong,ulong> derived generation. No alternative provider selection. API/test-backed capability matrix:
56+
57+
|Capability|Actual ownership/API|Exact proof/boundary|
58+
|---|---|---|
59+
|Token postings and bounded candidates|NativeTextIndex.Open native primary index, configured synchronous native WAL; native range/token/record keys|NativeTextProjectionParityTests and hash-collision arguments; candidate completeness/revalidation remain KeyLoad-owned|
60+
|English/Ukrainian Unicode query|KeyLoad frozen canonical tokenizer→native token postings→authorized canonical documents|New NativeTextBilingualLifecycleTests exact hello/ПРИВІТ references/fullJSON/revision/redaction and one-branch rank1/61|
61+
|Ranking|KeyLoad exact lexical corpus scoring and RRF over canonical scoped cut|Existing native-vs-canonical Unicode scores/ties/hybrid; no native BM25 promise from postings API|
62+
|Delete and immutable command replay|Canonical authorized DeleteDocument atomic command, source-cut generation replacement|New delete removes only Ukrainian result; complete same-ID receipt bytes/store-position/allbytes stable; English unchanged|
63+
|Native reopen/rebuild|Dispose selected native projection and canonical ZoneTree, reopen canonical owner and same native projection root|New deleted output/English full literal retained, unknown principal denied with full-store no-effect, healthy continuation|
64+
|Authorization/freshness|Persisted canonical policy/readcut governs candidates|Existing NativeTextProjectionAuthorityTests revocation/hidden rows/field denial/stale revision; new unknown-principal denial|
65+
|Analyzer/stemming/stopwords/phrase/public grammar|Not exposed by current KeyLoad public text profile|Explicit unsupported/unavailable capabilities, no inferred provider parity or silently selectable analyzer|
66+
|Formats/migration|Current typed owned projection manifest/index generation; disposable rebuild only|Existing malformed/unknown/link native projection tests; no Lucene index compatibility, migration or legacy fallback|
67+
68+
AC pass requires actual new native normal/scalar test plus original existing parity/authority/restart outcomes bound to same-source compile identities. No small corpus performance/recall/license delivery claim; RF3/package signature/resource/endurance gates remain independent. New case uses real ZoneTree and selected native provider, no fake/tokenizer injection/product seam. Docs/ADR freeze precedes assembled implementation; root owns join/build/native execution. Rollback removes only audit regression/docs, stored formats/dependencies unchanged.
69+
70+
R2: independent original delete commandId/partition/position and single full literal mutation bytes (deleteDocument, collection, Ukrainian ID, revision2), exact outer immutable operation ID on first submit and replay, full replay receipt/allstore/position preserved; final healthy postdenial cut equality required. Source-only; native execution remains pending.

‎docs/ADR/ADR-039-official-mcp-agent-api.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -398,3 +398,10 @@ The observed archive portion does not close bounded streaming/peak memory,
398398
cluster-cut/reconciliation or AC-BACKUP-004; member denial does not qualify
399399
revocation/cancellation parity. This decision remains Accepted until all its
400400
required implementation and qualification exist.
401+
402+
403+
### KL014 cold bootstrap SDK qualification ownership
404+
405+
TASK-KL014-COLD-BOOTSTRAP-SDK-001 under REQ/AC-CLIENT-004/005 and REQ/AC-TEST-010 adds fixture-only operation proof, with no product API/format/topology change. Implement docs first, then ClientApi integration Cases/Helpers/Assertions/Processes; root joins, builds/format verifies, and executes actual new fresh Aspire RF3 case plus existing Unknown recovery. Rollback removes only this authored regression. Existing ClusterFixture owns startup/readiness/private profile/node resources/joined shutdown; CLI borrows discovered endpoint/private profile and owns only its original process/readers through existing AppHost process settlement policy. Source absent-root admission is distinct from clean Linux qualification. No alternate coordinator, image whitelist, trusted roles, receipt authority or migrations.
406+
407+
R3 fixture-only refinement captures a fixed typed actual pre-builder selected-root existence observation in ClusterFixture and ClusterReplication/Models; it does not expose a generic callback or new product diagnostics. The case asserts that actual pre-start observation plus full native startup/SDK/CLI/cleanup flows. Covered selection remains the original closed eleven, with existing strict ledger admission.

‎docs/ADR/ADR-090-graph-search.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,41 @@ Before scoped kill, freeze exactly one immutable addition CommandRequest (delta
8585
Submit frozen addition once. Definitive OwnershipLost and every other non-Unknown error fail; only actual UnknownWriteOutcome permits one bounded same-byte/same-ID receipt reconciliation, no loop/new GUID/client retry policy. Verify exact two mutation receipts, nonempty partition/incarnation/position and QuorumProcessDurable. Explicit later SDK/official-MCP administrative replay uses identical original command and requires byte-identical complete receipt. Genuine graph-search readers/MCP preserve full literal four document references/revisions/JSON/redaction/weighted ranks, and a bounded native canonical Traverse from original second seed returns exact four vertices/three full literal edge records. After original stopped node restart, all three nodes replay original receipt and return those complete graph literals without duplicate edge revision or derived document effect.
8686

8787
Ownership IntegrationTests/Search existing fixture/kill/restart/discovered clients and original error-preserving cleanup. No product/dependency/public contract/timeouts/whitelists/LocalImage catalog changes. Ordered docs→private tests→root guarded join/build/focused native original case/full180cohort; preserve original TRX/error and source/image receipts. Actual Linux rerun remains required; source repair is not qualification. Rollback removes only this readiness/oracle amendment and test helpers.
88+
89+
90+
## Adapter implementation contract — TASK-KL051-VECTOR-LINQ-ATTACHMENT-001
91+
92+
REQ-KL051-VECTOR-001 / AC-KL051-VECTOR-001 and REQ-QUERY-007 under ADR-090:
93+
`KeyLoadQuery<T>.Take(limit).AttachVector(field, vector, space, scope)` translates
94+
only the existing non-evaluating member path and constructs the same canonical
95+
GraphSearchRequest v1 consumed by GraphSearchAsync. The vector is an immutable
96+
attachment bounded by existing MaximumConstantArrayItems; scope is explicit
97+
bounded native GraphScope. No new planner, result materialization, wire model,
98+
trusted caller role or query dispatcher. Scalar Where/order/project/Explain
99+
state is rejected UnsupportedCapability rather than silently omitted. Take is
100+
preserved as the canonical search limit. Server validation remains responsible
101+
for finite values, dimension/profile, current persisted policy, scoped read cut,
102+
shared budget and cancellation. Named SQL, builder request, typed/JSON requests
103+
must return complete independent literal ranks and errors; SQL's existing
104+
parser detail remains distinct from native typed validation detail. Real
105+
ZoneTree wrong dimension/unsupported/overattachment/denied-principal operations
106+
must preserve complete store and position then produce the complete healthy
107+
literal result. Root must execute native normal/scalar and official RF3 public
108+
paths before original KL051 closure; authored source is not qualification.
109+
110+
Ordered join: freeze owning spec/ADR; Client QueryExecution builder/helper; native UnitTests wholeflows; root guarded integration/build/format/discovery/full suites/Linux RF3. Dependencies unchanged; rollback removes adapter API and tests without modifying stored data.
111+
112+
## TASK-KL051-VECTOR-SCALAR-STATE-002
113+
114+
REQ-KL051-VECTOR-001 / AC-KL051-VECTOR-001, REQ-QUERY-007 and ADR-090: supplement
115+
the existing Order whole flow with three actual builder Where, custom projection
116+
and Explain states. Construct each supported scalar state successfully before
117+
AttachVector; require exact UnsupportedCapability and no lowered request, unchanged
118+
native read diagnostics, complete canonical storage bytes and position. Then execute
119+
the valid C# attachment through the real ZoneTree SearchEngine and retain the existing
120+
independent full literal two-document ranks, revision, JSON, redaction, empty expansion
121+
and equivalent SQL/typed/JSON results. Preserve the same complete storage and cut after
122+
healthy continuation. No planner/public scope change. Authored cases require native
123+
normal/scalar execution and existing public RF3 gates; source is not qualification.
124+
125+
R467 typed seed repair: VectorAttachmentDocument is instantiated as each actual committed document. Default absent vector attachment is omitted by WhenWritingDefault; member remains supported by the existing field translator. Independent literal full JSON equality is checked before native commit. Native vector records, document bytes/revisions, SQL/typed/JSON rank oracles remain unchanged. No dummy/public type/suppression.

‎docs/Features/ClientApi.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -463,3 +463,12 @@ repair alone is not runtime qualification or original-task closure.
463463

464464

465465
TASK-MCP-R457-ROLLUP-READ-CORPUS / REQ-MCP-003 / AC-MCP-003: original complete normal and scalar R457/R458 each executed2946 cases and failed only the stale body-read corpus count28. The canonical independently authored corpus adds ReadSampleRollupRequest, making29 body-bearing reads; four no-body read contracts and28 mutation contracts remain unchanged. Update only this literal body-read expectation, preserving every canonical DTO native roundtrip, absent/null/unknown/wrong-case rejection and private-value exclusion. These are required ordinary protocol controls and do not contribute product functional coverage. Existing ClientApi ADRs cover this contract; no product schema, dispatch, bounds or authorization changes. Fresh native compilation, focused normal/scalar decoder execution and complete current suites remain required; preserve both original failures.
466+
467+
468+
### TASK-KL014-COLD-BOOTSTRAP-SDK-001
469+
470+
REQ/AC-CLIENT-004/005, original architecture KL014 and ADR039/117: one fresh owning ClusterFixture captures actual selected storage root and native path existence immediately before creating the AppHost builder, then starts actual Aspire-owned RF3 current image resources, waits existing readiness, uses its discovered endpoint and private local profile in the actual built CLI status child. Reuse original process reader/exit settlement policy from AppHost; no CLI build/restore, secret argument, manual Docker or deadline expansion. Persist non-admin scoped credentials through actual SDK admin calls, configure collection, create/read full literal document, compare complete native receipt bytes for same command replay, require changed-content Conflict and unchanged full document plus original replay bytes, then healthy new-ID revision2. Await original fixture shutdown and assert only its owned root removed; primary and cleanup failures remain aggregated.
471+
472+
This is cold data/bootstrap proof on native RF3, not proof of an unprovisioned clean operating-system machine or a single server without external services. The original clean-machine criterion additionally requires authenticated Linux runner tool/image prerequisites and actual execution evidence; topology remains mandated RF3. Existing SdkSubmitReturnedCancellationReusesCommittedReceipt remains separate mandatory UnknownWriteOutcome recovery proof, including its retained original failure. No task closure or PASS is claimed from authored source.
473+
474+
The fixed internal ClusterFixtureColdStartObservation records only selected Root plus actual directory/file existence after coverage root selection and argument construction, before native builder/startup. No callback, fault control or caller-selected root. The whole operation requires that observation absent and exact same actual root after startup. Existing covered RF3 profile remains exactly eleven selected cases; this new bootstrap case is normal RF3 qualification, not currently eligible for covered selection. Register its actual case through the existing ledger: normal mode is no-op; unexpected covered selection remains rejected, with no whitelist change or coverage suppression. Constructor random root does not establish selected-root identity.

‎docs/Features/ClusterReplication.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,3 +291,37 @@ Lifetime: the physical host owns bridge, real materializer worker, storage callb
291291
Whole flow: actual Aspire current-image RF3, persisted non-admin principal and document+queue scope, original SDK atomic Batch, original signed BeforeSubmit marker, native follower JournalFlushed marker carrying real entry index/term, accepted authenticated Append settlement while held, and absent canonical-origin outbound evidence. Original leader SDK receipt must be successful under unchanged deadlines. Release the exact canonical arm and join its owner; verify independent literal complete document (reference/revision/JSON/redaction/empty fields) and queue inspection metadata/body/headers through SDK and official MCP. Assert exact receipt effects and native-byte same-ID SDK/MCP replay, changed-payload conflict with unchanged public state, then distinct healthy queue continuation. This proves the full scoped public projections, not a complete raw-store image or power-loss durability.
292292

293293
Owned paths: Replication ClusterReplication real ApplyBatch/worker scope; Orleans ClusterReplication real GrainService incoming/outgoing transport; Server ClusterRouting existing private control codecs/lifecycle and observer composition; Server StorageRecovery physical host/native storage callback; AppHost ClusterRouting strict current owner reader; IntegrationTests ClusterRouting actual Aspire wave/probe/SDK/MCP helpers; UnitTests ClusterRouting bounded private codec-negative control. That unit codec case is supporting infrastructure, not a product coverage contributor. Root-only format/full build, genuine new census/source-image binding, focused native codec in normal/scalar, existing probe request/read/fault flows, new current-image RF3 operation and mandatory full normal/scalar/recovery/RF3 gates remain required. No numeric coverage or successful native execution is claimed. Rollback removes the same-current private diagnostics coherently; persisted database, replication/native binary/public JSON formats and authorities are unchanged.
294+
295+
## TASK-KL020-ORIGINAL-ACCEPTANCE-CLOSEOUT-001
296+
297+
The original KL-020 acceptance in architecture-v0.3.uk.md is independently
298+
qualified at source 4e18ba1ba29ae31970302e6bfa43ad9c04ead7e0 by the original
299+
Linux [RF3 job 112948482972](https://github.com/managedcode/KeyLoad/actions/runs/37666943488/job/112948482972).
300+
`ClusterTests.ReplicatedAtomicBatchSurvivesLeaderContainerKillAndMinorityRejectsWrites`
301+
passed through the actual fixture-owned Aspire three-voter topology. It commits
302+
an acknowledged QuorumProcessDurable document/event/queue/topic batch, kills the
303+
elected container, retries the frozen command ID and compares its original token
304+
and one document revision/one queue delivery, then observes processing and
305+
subscription/projection effects. It kills the second voter, rejects both a strong
306+
write and read, restores both voters, verifies the minority document remains
307+
absent and checks replicated revisions and checkpoints across all three nodes.
308+
This maps the three original task criteria to REQ/AC-REP-003.
309+
310+
The immutable original required RF3 TRX digest is
311+
`8c9eb6c1b6fad38a0dd8532ea8f375465623939ef7784d9bb9ba9e24f938f75c`;
312+
the original native RF3 test-image observation digest is
313+
`086f26313d9d9311bcb544bf33290a90e51ebd49dd3d1f2d56d06a95a1d34c72`.
314+
Its exact test/helper declaration hashes match original native PDB documents;
315+
the operation bodies remain unchanged. Fixture and product sources have later
316+
changes: this is an original task/source qualification, not current whole-module
317+
equivalence. Required RF3 was 173/183 passed with ten retained failures; covered
318+
RF3 was 0/11 due to selection admission. Consensus fault/unacknowledged-entry,
319+
power-loss, endurance, performance, coverage and broader feature gates remain
320+
open under their owning criteria. No failed report is promoted to full-suite PASS.
321+
322+
KL-021 remains open: the passed scenario performs ordinary quorum reads, not a
323+
read carrying the previous commit/session token. Current GetDocumentRequest has
324+
only EntityRef and the SDK GetAsync has no minimum CommitToken argument. Public
325+
token-bound failover read and wrong-incarnation rejection need a frozen ClientApi
326+
read-admission contract and actual SDK/MCP operation proof; internal token
327+
validation and receipt equality do not supply that missing public flow.

0 commit comments

Comments
 (0)