implement bounded ANN candidate and native CQRS qualification contracts

Retain original normal/scalar ANN development evidence, published Communication 10.2.7 provenance and strict full build/formatter results. Record Graph owner repair publication and unchanged C0 consumer gates as pending. Freeze RequestContext and Identity.Core integration without claiming runtime, RF3 or complete acceptance qualification.

Author: KSemenenko

AGENTS.md

@@ -29,6 +29,8 @@ ManagedCode packages are our projects. Fix dependency defects in their owning si
 - Treat relational integrity and unified SQL as first-class product workstreams alongside SIMD and operation efficiency. Specify typed rows, keys/constraints, joins, atomic boundaries, unsupported diagnostics and measured performance before advertising capability; preserve RF3 and node-local storage ownership. A unified database/file delivery requirement MUST NOT be interpreted as permission to discard replication journals or promise a single physical file without a qualified storage-format ADR.
 - Treat the owner's SIMD work (earlier written as SMID) as a first-priority product workstream and keep it visible in implementation planning and status. On 2026-10-02 the owner confirmed SIMD means vectorized CPU operations and directed .NET intrinsics first, with Rust considered only after profiling. Map its canonical slices, REQ/AC criteria, scalar correctness/portability contract and required ADRs before implementation; never claim acceleration without comparable GitHub measurements.
 - Orleans provides request isolation, cluster routing and activation movement. Keep the one-grain-per-request boundary and RF3 topology; treat Orleans Streams as a first-class architecture workstream to specify and qualify alongside persisted KeyLoad EventStreams. Keep their distinct delivery/restart contracts explicit, and define bounded resource, concurrency, backpressure, recovery and performance behavior for every operation.
+- Owner reiteration 2026-10-04 requires Orleans to own database execution and coordination, including search/index work and long operations, through the separate grain-per-request boundary and node-local storage owners. Use our centrally pinned ManagedCode.Communication CQRS APIs and their native IAsyncEnumerable result/operation streaming contracts for streamed results and long operations. Freeze typed commands, queries, progress/results, cancellation, bounded buffering, backpressure, authorization, terminal failures and restart/failover semantics in the feature requirements and ADR before implementation; qualify actual SDK/MCP consumption through Aspire RF3. Do not replace these owned CQRS APIs with a parallel dispatcher or unbounded materialization, and repair dependency defects in the owning Communication repository under the mandatory release policy.
+- Owner direction 2026-10-04 selects native Orleans RequestContext for bounded typed request/identity state propagation between grain calls and authorizes adopting our ManagedCode.Orleans.Identity packages for that context path. Reuse the owning native APIs; freeze the exact context contract, trust boundary, registration, cancellation/disposal restoration, concurrent-stream isolation and restart/migration behavior before integration. Verify server-authenticated request identity against the signed request and persisted authorization; propagated context does not authorize caller-supplied roles or replace committed ZoneTree state, node-local storage ownership or RF3 receipts. Keep secrets and user payloads out of diagnostic context, and repair any ManagedCode identity dependency defect in its owning repository under the mandatory release policy.
 - Optimize every operation and its shared serialization, validation, routing and storage execution paths against their correctness and fault contracts, using representative multi-node GitHub qualification to measure latency, throughput, allocations, memory, contention and backlog where applicable. The owner reiterated on 2026-10-03 that high performance and minimum practical memory use are required across the entire implementation, including shared hot paths. Prioritize avoidable allocations, retained memory and repeated work; verify latency, throughput, allocations and memory before/after without weakening correctness, authorization, bounds or durability. Architecture choices or local builds alone do not prove maximum scalability or performance; select further optimization from actual comparable measurements.
 - Use ZoneTree's native storage APIs correctly and Orleans for bounded parallel execution of independent operation work. Preserve node-local storage ownership, the ordered atomic commit/apply gate, scoped read cuts, cancellation and backpressure; qualify the resulting performance in real multi-node GitHub runs (owner direction 2026-10-02).
 - All KeyLoad-owned database models MUST use ZoneTree as their canonical storage foundation, including documents, relational rows, graphs, vectors/search, time series, blobs, queues and events. ZoneTree.FullTextSearch is the selected text-index implementation. Qualify native ZoneTree WAL below the unchanged Orleans RF3, node-local ownership and atomic-commit contracts; retain distinct replication and atomic-commit recovery journals until an explicit storage-format ADR and fault qualification prove any migration safe. Comparison databases retain their own real native storage.

Directory.Packages.props

@@ -8,9 +8,9 @@
     <PackageVersion Include="Cartograph.Catalog" Version="0.1.0-alpha" />
     <PackageVersion Include="ZoneTree" Version="1.9.8" />
     <PackageVersion Include="ZoneTree.FullTextSearch" Version="1.0.9" />
-    <PackageVersion Include="ManagedCode.Communication" Version="10.2.6" />
-    <PackageVersion Include="ManagedCode.Communication.AspNetCore" Version="10.2.6" />
-    <PackageVersion Include="ManagedCode.Communication.Orleans" Version="10.2.6" />
+    <PackageVersion Include="ManagedCode.Communication" Version="10.2.7" />
+    <PackageVersion Include="ManagedCode.Communication.AspNetCore" Version="10.2.7" />
+    <PackageVersion Include="ManagedCode.Communication.Orleans" Version="10.2.7" />
     <PackageVersion Include="ManagedCode.TimeSeries" Version="10.0.3" />
     <PackageVersion Include="ManagedCode.Storage.FileSystem" Version="10.0.13" />
     <PackageVersion Include="ManagedCode.Orleans.Graph" Version="10.0.6" />
@@ -20,6 +20,7 @@
     <PackageVersion Include="Microsoft.Orleans.Sdk" Version="10.3.1" />
     <PackageVersion Include="Microsoft.Orleans.Serialization" Version="10.3.1" />
     <PackageVersion Include="Microsoft.Orleans.CodeGenerator" Version="10.3.1" />
+    <PackageVersion Include="Microsoft.Orleans.TestingHost" Version="10.3.1" />
     <!-- Orleans' existing Roslyn dependency; analyzer tests select SDK compiler assets. -->
     <PackageVersion Include="Microsoft.CodeAnalysis.Common" Version="5.0.0" />
     <PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="5.0.0" />

docs/ADR/ADR-019-managed-ann.md

@@ -1,10 +1,10 @@
 # ADR-019: Managed-first ANN provider qualification
 
-Status: Proposed. No approximate-nearest-neighbor provider or index format is selected by this record.
+Status: Accepted for the bounded first-party managed HNSW candidate stage on 2026-10-04. Online projection, persisted index format and public ANN capability remain unqualified and gated by later contracts.
 
 ## Context and decision
 
-Exact vector scans are the correctness oracle and current source path. The product prefers a managed-first ANN implementation; a native provider adds deployment, license, persistence, concurrency, deletion, and memory-ownership risks. The current design identifies provider criteria but has not qualified a library or implementation.
+Exact vector scans are the correctness oracle and current public source path. The product prefers a managed-first ANN implementation; a native provider adds deployment, license, persistence, concurrency, deletion, and memory-ownership risks. Root selects an independently authored KeyLoad-owned packed managed HNSW candidate for the first computational stage, with no new package, native binary or project. This decision approves implementation of the bounded candidate and its independent real-store tests; it does not qualify recall, an online projection, a persisted format or a public capability.
 
 Select a provider only after a pinned-source audit and real-store comparison against exact search. Required evidence includes recall by filter/selectivity, update/delete/reinsert behavior, concurrent access, persistence/rebuild, memory accounting, and platform/license dependencies. ANN remains optional until the exact oracle and fallback semantics are measurable.
 
@@ -22,7 +22,8 @@ flowchart LR
 
 - Always scan exact vectors: dependable oracle but scaling cost remains.
 - Add an unreviewed native package: rejected until transitive binaries, licensing, and ownership are audited.
-- Managed HNSW or another candidate: research direction only; API fit and lifecycle remain open.
+- Audited Hnsw.Net 0.1.4: not admitted. Its first vector chunk allocates 256 MiB, search/build lack cancellation and work admission, and pooled visited state has no retained concurrency cap. A consumer-side wrapper cannot supply cancellation inside its synchronous unbounded algorithm. This is a third-party package, not a ManagedCode-owned dependency repair.
+- First-party packed managed HNSW: selected as the bounded computational candidate; lifecycle and public integration remain separate qualification stages.
 
 ## Related requirements and implementation contract
 
@@ -32,6 +33,31 @@ Related: `REQ-SEARCH-001/AC-SEARCH-001`, `REQ-SEARCH-003/AC-SEARCH-003`, `REQ-SE
 2. Add real-store property/differential tests for recall, mutation, deletion, filtered retrieval, concurrency, restart, and corrupted generation rejection.
 3. Implement projection generation and provider adapter in Search; never move node-local canonical vector or document-revision authority out of Core.
 4. Roll out behind an explicit capability setting; rollback disables the projection and rebuilds from canonical data without changing document state.
-5. Qualify pinned provider and workload on GitHub CI across supported platforms; publish exact source/package versions and resource/quality evidence.
+5. Qualify the exact candidate source and workload on Linux GitHub CI, retaining scalar portability checks; publish exact source/package versions and resource/quality evidence.
 
-No package or index format is approved here. Runtime tests/benchmarks run only in GitHub Actions. The current historical benchmark is not ANN qualification.
+No external ANN package or persisted index format is approved here. The root owner-authorized local development workflow applies: actual TUnit tests run through Aspire, and local results remain development evidence. Delivered-source Linux CI, recovery, Docker/Aspire RF3 and genuine GitHub comparison gates remain required. The current historical benchmark is not ANN qualification.
+
+## Accepted ordered candidate contract, 2026-10-04
+
+The complete computational API, bounds, requirements, acceptance criteria, slice ownership and rollout are frozen in [ManagedAnn](../Features/Search/ManagedAnn.md). Related tasks are KL-030/031/032/059/060; no task is closed by this decision.
+
+1. R1: implement the immutable packed computational candidate in new Query/Search files and independently authored real-ZoneTree metric, budget, filter, recall and rebuild tests. Root owns integration, all shared files, strict solution checks, Aspire execution and a stage commit. Public exact search and canonical stored data remain unchanged.
+2. R2: root must first accept an exact native-generated ZoneTree manifest/chunk and source-cut/outbox replay contract, including atomic publication, locks, leases, cancellation, corruption and crash recovery. Only then implement the node-local disposable projection. The current decision does not authorize workers to invent that format or lifecycle.
+3. R3: root must first accept versioned SQL/SDK/MCP approximation, completeness, fallback, eligibility and freshness metadata. Only then integrate an explicit ANN capability with persisted authorization in one scoped read cut and genuine RF3 tests. No silent approximation or branch truncation is permitted.
+4. R4: retain actual loaded corpora, quality/resource counters, normal/scalar reports and exact-source Linux qualification. Global performance still requires the mandated 100,000/1,000,000/5,000,000-record matched GitHub workloads; R1's 10,000-record recall control is not that evidence.
+
+The algorithm reference is the primary [HNSW paper](https://arxiv.org/abs/1603.09320v4), algorithms 1–5. Code is independently authored within KeyLoad's existing MIT ownership; no source code or package is imported. The immutable source ordinal seeds bounded geometric levels, base degree is at most twice Connections and upper degree at most Connections. Current metric-specific selection uses the unchanged exact similarity oracle: diversification for Cosine/Euclidean and the paper's simple highest-score selection for DotProduct under the refinement below.
+
+Vector-only PutVector updates can preserve DocumentRevision. R2 freshness therefore must bind committed source cuts and actual outbox positions, not only document revision. Logical index identity includes partition, collection, field and the complete VectorSpace; node identity/incarnation/data epoch and read-generation binding belong to the later physical projection contract. Orleans coordinates logical ownership and admission; open ZoneTree handles remain node-local.
+
+Rollback of R1 removes the unused candidate. R2/R3 rollback disables its derived capability and rebuilds from canonical ZoneTree data without changing acknowledged writes, native WAL, replication or atomic recovery journals. No public wire/data migration occurs in R1; later stages require their explicit upgrade contracts before implementation.
+
+Root accepted the R1 packing/reservation refinement in [ManagedAnn](../Features/Search/ManagedAnn.md#accepted-packing-and-reservation-refinement-2026-10-04) before the first runtime gate: actual-level upper offsets/edges, per-query visit bitmap, explicit conservative array/string/object accounting, reported build/search reservations and inclusive exact/excess/sparse admission. This corrects source-review findings without changing public search, canonical data or the unapproved R2/R3 contracts. Both workers retain their disjoint new-file scopes; root must review the revised source and execute every gate before any qualification claim.
+
+The first real Aspire R1 run passed17/19 cases and failed both10,000-record builds on the unchanged work cap before recall/deadline assertions. Root accepts the [insertion-work correction](../Features/Search/ManagedAnn.md#accepted-insertion-work-correction-2026-10-04): Algorithm1 establishes Connections new edges, keeps base2M/upperM maximum degree, reselects reciprocal neighbors only on overflow, and uses validated source ordinals for identical ID ordering with truthful integer-comparison charges. TASK-ANN-R1-PACKED-ALGORITHM owns these changes; TASK-ANN-R1-INDEPENDENT-TESTS retains the same independent cases and limits; TASK-ANN-R1-ROOT-JOIN retains the original failure reports and repeats strict checks and actual Aspire normal/scalar evidence. This is an implementation correction, not a qualified performance result or waived acceptance gate.
+
+The second original Aspire cohort also passed17/19 and hit the unchanged build work cap in FindWorst. Root accepts the [operation-local worst-slot cache](../Features/Search/ManagedAnn.md#accepted-worst-slot-cache-refinement-2026-10-04): reject against a validated cached worst, update on append, and fully recompute only after an accepted replacement. One scratch integer fits the declared fixed object reservation; graph state, arrays, work caps, metrics and independent tests remain unchanged. The same owned tasks must review reset/replacement invariants and repeat complete gates; no observed recall or performance improvement is claimed yet.
+
+The third original Aspire cohort remains17/19 and hits the same cap in reciprocal-link work. Root accepts the [bounded dual-heap search refinement](../Features/Search/ManagedAnn.md#accepted-bounded-dual-heap-refinement-2026-10-04), replacing repeated frontier/worst scans while preserving exact extraction/tie order and the diversified neighbor-selection rule. Three explicitly reserved int[E] heap arrays replace the old bool[E] processed array/cache; private admission formulas include every array before allocation. Root and independent review must prove map/remove/reset invariants and unchanged test outcomes. Reciprocal work, recall and deadline qualification remain open; the three original failures are retained and no cap/AC reduction is authorized.
+
+The fourth original report reaches and passes all10 Cosine/Euclidean cells, but remains17/19 because DotProduct builds exceed the unchanged work cap. Root accepts [simple DotProduct neighbor selection](../Features/Search/ManagedAnn.md#accepted-dotproduct-neighbor-selection-refinement-2026-10-04) for both new and overflowing reciprocal edges, with unchanged exact dot scores and deterministic ties. This selects Algorithm3 for that nonmetric similarity while preserving diversification for the two qualified metric controls. It removes pair-diversity work without changing bounds or ACs; the potential bridge/recall cost must pass the unchanged five DotProduct cells. Whole-source guard failure from concurrent benchmark edits and separate unchanged ANN/runtime proof remain explicit; all results are local development evidence.