Skip to content

Commit 156d732

Browse files
committed
Checkpoint shard catalog, placement and bounded partition query stages
1 parent 55fb4e3 commit 156d732

243 files changed

Lines changed: 10659 additions & 375 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
dotnet build src/KeyLoad.AppHost --no-restore --configuration Release
4343
dotnet build tests/KeyLoad.Analyzers.Tests --no-restore --configuration Release
4444
- name: Run analyzer tests
45-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers
45+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers --KeyLoadTests:ReportTrx=true
4646
- name: Save analyzer test results
4747
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
4848
if: always()
@@ -102,7 +102,7 @@ jobs:
102102
- name: Check repository rules
103103
run: node scripts/Features/RepositoryGovernance/verify.mjs
104104
- name: Test code analyzers
105-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers
105+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers --KeyLoadTests:ReportTrx=true
106106
- name: Test prompt termination on Aspire resource failures
107107
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/AspireFailure*/*' --KeyLoadTests:ResultsDirectory=TestResults/aspire-failure --KeyLoadTests:ReportTrx=true
108108
- name: Test Aspire recovery prerequisite ownership
@@ -116,13 +116,13 @@ jobs:
116116
- name: Test Aspire workflow entry contracts
117117
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit '--KeyLoadTests:Filter=/*/*/WorkflowLayoutAspireEntryTests/*' --KeyLoadTests:ResultsDirectory=TestResults/live-benchmark-workflow --KeyLoadTests:ReportTrx=true
118118
- name: Run unit tests
119-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit
119+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit --KeyLoadTests:ReportTrx=true
120120
- name: Run unit tests without CPU intrinsics
121121
if: ${{ !cancelled() && steps.build.outcome == 'success' }}
122-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit-scalar
122+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit-scalar --KeyLoadTests:ReportTrx=true
123123
- name: Test recovery after process crashes
124124
if: ${{ !cancelled() && steps.build.outcome == 'success' }}
125-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=recovery
125+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=recovery --KeyLoadTests:ReportTrx=true
126126
- name: Save test results
127127
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
128128
if: always()
@@ -162,6 +162,9 @@ jobs:
162162
- name: Build the genuine epoch-six RPC1 KeyLoad server image
163163
id: rpc1-images
164164
run: node scripts/Features/ClusterRouting/prepare-rpc1-server-image.mjs
165+
- name: Build the genuine epoch-seven interface-three KeyLoad server image
166+
id: interface3-images
167+
run: node scripts/Features/ClusterRouting/prepare-interface3-server-image.mjs
165168
- name: Configure Docker image references
166169
shell: bash
167170
env:
@@ -173,13 +176,21 @@ jobs:
173176
KEYLOAD_RPC1_SERVER_IMAGE: ${{ steps.rpc1-images.outputs.KEYLOAD_RPC1_SERVER_IMAGE }}
174177
KEYLOAD_RPC1_IMAGE_RECEIPT: ${{ steps.rpc1-images.outputs.KEYLOAD_RPC1_IMAGE_RECEIPT }}
175178
KEYLOAD_RPC1_SERVER_MANIFEST: ${{ steps.rpc1-images.outputs.KEYLOAD_RPC1_SERVER_MANIFEST }}
179+
KEYLOAD_INTERFACE3_SERVER_IMAGE: ${{ steps.interface3-images.outputs.KEYLOAD_INTERFACE3_SERVER_IMAGE }}
180+
KEYLOAD_INTERFACE3_IMAGE_RECEIPT: ${{ steps.interface3-images.outputs.KEYLOAD_INTERFACE3_IMAGE_RECEIPT }}
181+
KEYLOAD_INTERFACE3_SERVER_MANIFEST: ${{ steps.interface3-images.outputs.KEYLOAD_INTERFACE3_SERVER_MANIFEST }}
182+
KEYLOAD_INTERFACE3_SERVER_INVENTORY: ${{ steps.interface3-images.outputs.KEYLOAD_INTERFACE3_SERVER_INVENTORY }}
183+
KEYLOAD_INTERFACE3_SERVER_ARCHIVE: ${{ steps.interface3-images.outputs.KEYLOAD_INTERFACE3_SERVER_ARCHIVE }}
176184
run: |
177185
test -n "$KEYLOAD_SERVER_IMAGE" && test -n "$KEYLOAD_RUNNER_IMAGE"
178186
test -n "$KEYLOAD_PRIOR_SERVER_IMAGE" && test -n "$KEYLOAD_PRIOR_IMAGE_RECEIPT" && test -n "$KEYLOAD_PRIOR_SERVER_MANIFEST"
179187
test -n "$KEYLOAD_RPC1_SERVER_IMAGE" && test -n "$KEYLOAD_RPC1_IMAGE_RECEIPT" && test -n "$KEYLOAD_RPC1_SERVER_MANIFEST"
188+
test -n "$KEYLOAD_INTERFACE3_SERVER_IMAGE" && test -n "$KEYLOAD_INTERFACE3_IMAGE_RECEIPT" && test -n "$KEYLOAD_INTERFACE3_SERVER_MANIFEST"
189+
test -n "$KEYLOAD_INTERFACE3_SERVER_INVENTORY" && test -n "$KEYLOAD_INTERFACE3_SERVER_ARCHIVE"
180190
printf 'KeyLoad__ContainerImages__Server=%s\nBenchmarks__ContainerImages__LoadGenerator=%s\nKEYLOAD_IMAGE_RECEIPT=%s\n' "$KEYLOAD_SERVER_IMAGE" "$KEYLOAD_RUNNER_IMAGE" "$RUNNER_TEMP/keyload-images/image-receipt.json" >> "$GITHUB_ENV"
181191
printf 'KEYLOAD_PRIOR_SERVER_IMAGE=%s\nKEYLOAD_PRIOR_IMAGE_RECEIPT=%s\nKEYLOAD_PRIOR_SERVER_MANIFEST=%s\n' "$KEYLOAD_PRIOR_SERVER_IMAGE" "$KEYLOAD_PRIOR_IMAGE_RECEIPT" "$KEYLOAD_PRIOR_SERVER_MANIFEST" >> "$GITHUB_ENV"
182192
printf 'KEYLOAD_RPC1_SERVER_IMAGE=%s\nKEYLOAD_RPC1_IMAGE_RECEIPT=%s\nKEYLOAD_RPC1_SERVER_MANIFEST=%s\n' "$KEYLOAD_RPC1_SERVER_IMAGE" "$KEYLOAD_RPC1_IMAGE_RECEIPT" "$KEYLOAD_RPC1_SERVER_MANIFEST" >> "$GITHUB_ENV"
193+
printf 'KEYLOAD_INTERFACE3_SERVER_IMAGE=%s\nKEYLOAD_INTERFACE3_IMAGE_RECEIPT=%s\nKEYLOAD_INTERFACE3_SERVER_MANIFEST=%s\nKEYLOAD_INTERFACE3_SERVER_INVENTORY=%s\nKEYLOAD_INTERFACE3_SERVER_ARCHIVE=%s\n' "$KEYLOAD_INTERFACE3_SERVER_IMAGE" "$KEYLOAD_INTERFACE3_IMAGE_RECEIPT" "$KEYLOAD_INTERFACE3_SERVER_MANIFEST" "$KEYLOAD_INTERFACE3_SERVER_INVENTORY" "$KEYLOAD_INTERFACE3_SERVER_ARCHIVE" >> "$GITHUB_ENV"
183194
- name: Find Chrome for admin tests
184195
shell: bash
185196
run: |
@@ -192,7 +203,7 @@ jobs:
192203
- name: Build three-node database tests
193204
run: dotnet build tests/KeyLoad.IntegrationTests --no-restore --configuration Release
194205
- name: Test three-node database with .NET and MCP clients
195-
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=rf3
206+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=rf3 --KeyLoadTests:ReportTrx=true
196207
- name: Clean up Docker registry
197208
if: ${{ always() && (steps.images.outcome == 'success' || steps.images.outcome == 'failure') }}
198209
run: node scripts/Features/BenchmarkComparisons/cleanup-images.mjs

‎docs/ADR/ADR-019-managed-ann.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,3 +148,39 @@ development observations, strict checks and Aspire normal/scalar/full Linux
148148
gates. Rollback restores only the scan representation. Qualification remains
149149
pending until the original full quality, authorization, recovery and RF3 gates
150150
have evidence; this source-level work reduction is not a measured speedup claim.
151+
152+
153+
## Accepted ADR-019 amendment — ANN-016 adjacent budget check de-duplication
154+
155+
Accepted bounded stage: REQ/AC-ANN-016 and TASK-ANN-DUPLICATE-BUDGET-CHECKS in `REQ-AC-ANN-016.md`.
156+
157+
`AnnWorkBudget.Charge` begins by calling `Check`, then validates the charge and updates the existing work counter. In eight frozen Search source files, twelve sites call `Check` immediately before `Charge` with no intervening operation. Remove only those twelve calls. Retain every `Charge` and its original amount, so each charged action still observes cancellation/deadline immediately before its counter update. Preserve other explicit checks, particularly any path that can exit without reaching a charge.
158+
159+
The Luna worker owns only the eight listed ANN source files in the private packet. Root owns exact-base review, integration, strict checks, and same-source genuine Aspire normal/scalar observations with no changes to the corpus, caps, deadline, test admission, or assertions. This is an equivalent-check-count optimization candidate, not a measured speedup, timeout diagnosis, or qualification. Rollback restores the eight source files. Persistence, public transport, storage, and migration are N/A.
160+
161+
162+
## Accepted ADR-019 amendment — charged ANN inner-loop check de-duplication
163+
164+
Candidate stage REQ/AC-ANN-017 / TASK-ANN-INNERLOOP-CHECKS in
165+
`REQ-AC-ANN-017.md`.
166+
167+
The 10k Build report records 165M work units, 4.37M distances and 12.41M edge
168+
visits, but does not attribute their per-method source or prove the original
169+
deadline cause. Actual source walks HNSW edge lists, scores unseen neighbors,
170+
performs heap comparisons and diversifies / repairs reciprocal neighborhoods.
171+
These counters remain exact and unchanged in this stage.
172+
173+
In four binary-heap sift loops, a loop-entry `budget.Check()` is followed on
174+
every executed iteration by `IsBetter` / `IsWorse`, which calls
175+
`budget.Charge(1)` and therefore the same `ReadExecutionBudget.Check()` before
176+
its work update. In `Greedy`, each loop iteration similarly reaches charged
177+
`NeighborCount`. Remove only those five duplicated loop-entry checks. Preserve
178+
every comparison, distance, edge, mutation and charge. Keep the `SearchLayer`
179+
loop check because its `PopBest` sentinel may exit without charging; retain all
180+
other checks, cancellation/deadline paths and original bounds.
181+
182+
The source owner is restricted to `PackedAnnCandidateHeaps.cs` and
183+
`PackedAnnLayerSearch.cs`. Root owns cumulative source join with pending ANN
184+
patches, review, normal/scalar unchanged-corpus observations, strict gates and
185+
qualification. This is an unmeasured repeated-check optimization candidate,
186+
not a deadline diagnosis or speed claim. Rollback restores the five checks.

‎docs/ADR/ADR-039-official-mcp-agent-api.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,16 @@ then the existing capability/partition grain with a reloaded persisted principal
8484

8585
## Frozen catalog and wire contract
8686

87+
ADR-098 adds its direct and SQL graph-path tools as additive version-one reads;
88+
AC-MCP-001 independently verifies the complete 66-name catalog, typed schemas
89+
and effect hints. The two PMAP tools use strict generated request/result schemas:
90+
bind is `{ commandId, request }` with version, expectedRevision, complete
91+
partition and physicalShardId, while read is `{ request }` and returns the full
92+
same-view placement witness including ordered voters and both directory and row
93+
revisions. Persisted administrator authorization remains enforced by the existing
94+
request-grain path. Runtime RF3 confirmation remains pending.
95+
96+
8797
Outer arguments are strict: a body-bearing tool accepts only `request`; the four
8898
header-command-ID tools additionally require nonempty GUID `commandId`. No-body
8999
tools accept `{}`. Write identities inside canonical DTOs remain unchanged:
@@ -98,6 +108,10 @@ request.CommandId. MCP request IDs and Orleans actor IDs never replace them.
98108
| keyload_subscriptions_status | GetSubscriptionRequest | SubscriptionInfo | /v1/subscriptions/status; Subscription |
99109
| keyload_messages_inspect | InspectMessageRequest | MessageInspection or null | /v1/queues/inspect; Message |
100110
| keyload_graph_traverse | TraverseRequest | GraphTraversal | /v1/graph/traverse; Traverse |
111+
| keyload_graph_shortest_path | GraphShortestPathRequest | GraphShortestPathResult | /v1/graph/shortest-path; GraphShortestPath |
112+
| keyload_query_graph_path | SqlGraphPathRequest | GraphShortestPathResult | /v1/query/graph-path; SqlGraphPath |
113+
| keyload_admin_partition_placement_bind | BindAtomicPartitionPlacementRequest | bool | /v1/admin/partition-placement/bind; BindAtomicPartitionPlacement |
114+
| keyload_admin_partition_placement_read | AtomicPartitionPlacementReadRequest | AtomicPartitionPlacementResolution | /v1/admin/partition-placement/read; AtomicPartitionPlacement |
101115
| keyload_series_read | ReadSamplesRequest | SampleRecord array | /v1/series/read; Samples |
102116
| keyload_query_execute | QueryRequest | QueryPage | /v1/query; Query |
103117
| keyload_query_ast | AstQueryRequest | QueryPage | /v1/query/ast; AstQuery |

‎docs/ADR/ADR-040-static-site-threejs-evidence.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -456,3 +456,114 @@ retain all raw/archive/freshness/native/browser/coverage gates and historical
456456
formats exactly. New aggregate formats cannot be published until coupled tooling,
457457
closed inventories, test evidence and actual provider/live receipt exist. Required
458458
TASK-ISO-006/010/011/012 ownership/stages/rollback and tests are inADR056; statusAccepted.
459+
460+
461+
## Accepted native Chrome session admission (2026-10-05)
462+
463+
The following root-reviewed contract is accepted before source integration; runtime qualification remains pending.
464+
465+
# Private candidate: ADR-040 / TASK-SITE-CHROME-ADMISSION
466+
467+
## Decision
468+
469+
Treat a complete real Chrome session as a native bounded test resource. Use a
470+
separate capacity-one admission instance for Chrome sessions in each TUnit
471+
process. Do not share the existing capacity-two Node child pool: isolated-site
472+
build/projection work can precede browser start, and a shared pool would couple
473+
the two resource lifetimes. Reuse the existing native FIFO/cancellation and
474+
fail-closed unsettled-child semantics only through a distinct browser pool and
475+
an ownership path that joins the Chrome process, CDP socket and instrumentation
476+
before completing its lease.
477+
478+
The only current Chrome launch caller is
479+
`SiteBrowserChrome.StartAsync`; acquire before its version subprocess so every
480+
actual launch passes admission. Transfer the same lease with the original
481+
`SiteBrowserProcessStart` owner into the returned `SiteBrowserChrome`. Startup
482+
failure before process creation releases only after the original startup task
483+
and owned cleanup have settled. Once launched, only successful original CDP /
484+
coverage shutdown and observed original process exit settle the lease. Preserve
485+
the permit on cleanup failure/unjoined original work by poisoning the browser
486+
pool, which rejects queued sessions. Do not synthesize completion, detach an
487+
original task, or release on timeout.
488+
489+
`SiteBrowserSession.CompleteAsync` already completes/stops native coverage
490+
before disposing Chrome. Preserve that order. On failed sessions the original
491+
owned Chrome process teardown closes its instrumentation before releasing the
492+
permit. Preserve all current report, keyboard, numeric, accessibility,
493+
provenance, source-coverage and renderer assertions.
494+
495+
The real keyboard failure receives failure-only, strictly bounded observations
496+
from actual `Input.dispatchKeyEvent` acknowledgements and selected-control
497+
state after each known key. Do not record report values, payloads, query data or
498+
unbounded console/browser events. The assertion remains the same and the
499+
telemetry is diagnostic only.
500+
501+
## Ordered task, ownership and tests
502+
503+
`TASK-SITE-CHROME-ADMISSION` is tests/source work after root accepts this
504+
contract; root owns this ADR/feature edit, integration, full gates and final
505+
evidence.
506+
507+
1. Source owner: add a browser-session-specific pool in
508+
`tests/KeyLoad.SiteTests/Features/BenchmarkComparisons/Helpers/` with the
509+
exact one-active/64-queued/20-minute contract. Reuse
510+
`SiteHeavyChildAdmission` only as a separately instantiated pool, not its
511+
shared Node instance. Update `Helpers/SiteHeavyChildLease.cs` only to permit
512+
a new original child after the prior child and its output captures have
513+
settled; reject overlapping unsettled children. The browser version probe
514+
and Chrome child use the same reservation sequentially.
515+
2. Source owner: change only `Helpers/SiteBrowserChrome.cs` and
516+
`Processes/SiteBrowserProcess.cs` to acquire before the version process,
517+
track/settle its real output readers and original exit, transfer ownership
518+
with the actual Chrome process owner, then complete coverage/socket teardown
519+
and observe that same process before release. All failed/unjoined native
520+
cleanup fails closed.
521+
3. Source owner: add one real Chrome TUnit admission/cancellation/lifecycle
522+
case under `Cases/`; it uses the configured browser path and CDP, verifies
523+
the queued/canceled original caller and successor start after native owner
524+
settlement, with bounded task coordination and no sleep or synthetic
525+
provider/process.
526+
4. Source owner: add fixed-size, failure-only safe evidence in
527+
`Assertions/SiteIsolatedBrowserKeyboardAssertions.cs`, leaving ArrowUp,
528+
Enter, the exact selected value and independent row oracle intact.
529+
5. Root reviews the complete packet and immutable hashes, joins it, then runs
530+
the actual Aspire-owned strict build/format/governance and full TUnit/site /
531+
Chrome / native coverage gates. Browser resource admission alone makes no
532+
speed claim and does not diagnose the original keyboard miss.
533+
534+
Rollback restores the owned helper/callers/test and their contract together;
535+
there is no product, persisted-data, dependency, website rendering, workflow or
536+
publication migration.
537+
538+
# Proposed ADR-040 r2 amendment — original waiter withdrawal oracle
539+
540+
Accepted candidate delta: REQ/AC-BC-CHROME-001 clarification and bounded
541+
waiter-membership oracle in `REQ-AC-r2-amendment.md`.
542+
543+
The current native `SiteHeavyChildAdmission.Waiter` already stores the original
544+
caller token and the pending `LinkedList<Waiter>` is bounded to 64. Add only an
545+
internal read-only method that scans that list under its existing `_sync` lock
546+
for `CancellationToken.Equals` against the supplied unique linked token. The
547+
browser-specific `SiteBrowserSessionAdmission` wrapper is the only caller/API
548+
surface. The observation creates no new queue, registry, subscription, retained
549+
identity, timing mechanism or diagnostic output and does not change how waiters
550+
are admitted, cancelled, withdrawn, granted or rejected. The scan is bounded by
551+
the already enforced queue cap.
552+
553+
The real Chrome test asserts that its exact token is in the pending queue
554+
before cancellation and absent after the actual canceled acquisition has
555+
settled with the original token. Preserve aggregate counts and all real Chrome,
556+
CDP, coverage and original-process ownership checks from r1. No global
557+
serialization or weakened acceptance is introduced.
558+
559+
Implementation owns only the native helper, browser wrapper and existing
560+
browser lifecycle case enumerated in the feature amendment. Root owns review,
561+
join and actual Aspire/browser verification. Rollback removes only the
562+
observation and its exact test assertions. The observer proves membership and
563+
withdrawal for that token; it does not claim concurrent-launch performance,
564+
keyboard root cause or CI qualification.
565+
566+
567+
### Browser lease transfer at factory return
568+
569+
The reviewed browser session factory admits one actual browser lease before the version child, endpoint/Chrome startup and CDP/coverage setup. Its lexical `using` owns all failure paths. Immediately before a successful return it marks that same lease transferred to the returned native process owner; only the factory's scope disposal consumes this transfer marker, leaving the actual lease held until the returned owner settles its original process, CDP and HTTP resources. Existing non-browser callers never set this marker. This is an ownership-transfer marker, not a readiness, exit or coverage proof. Failed setup follows the existing resource joins and bounded pool poison rule. AC-ISO-009's real cancellation/successor/native PID regression remains the evidence; the keyboard assertion retains its exact predicate and prints only bounded observed failure diagnostics through native TUnit output.

0 commit comments

Comments
 (0)