Skip to content

Commit 6949fa0

Browse files
committed
Repair RF3 qualification and add EventStreams SDK/MCP proof
Disambiguate safe rejected-request diagnostics, restore failed fault fixtures, preserve exact ordered protocol bytes and fix Timescale Aspire resource identity. New EventStreams tests exercise actual SDK/MCP retry, replay and persisted authority. Runtime qualification remains GitHub-only and pending.
1 parent 3559225 commit 6949fa0

52 files changed

Lines changed: 2228 additions & 328 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
1+
AGENTS.md text eol=lf
2+
**/AGENTS.md text eol=lf
13
site/Features/BenchmarkComparisons/vendor/three/0.186.1/*.js whitespace=-space-before-tab

‎.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -659,6 +659,15 @@ _site/
659659
#plans
660660
*.plan.md
661661

662+
# Local task scaffolding; durable specifications remain in docs/.
663+
/*.brainstorm.md
664+
/*.acceptance.md
665+
666+
# Required source artifacts checked by RepositoryGovernance in CI.
667+
!/mcaf-governance.brainstorm.md
668+
!/mcaf-governance.acceptance.md
669+
!/mcaf-governance.plan.md
670+
662671
# KeyLoad local databases, credentials and qualification artifacts
663672
artifacts/
664673
data/

‎AGENTS.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,11 @@ Use .NET 10 and centrally pinned packages. Do not generate or commit packages.lo
1212

1313
ManagedCode packages are our projects. Fix dependency defects in their owning sibling repository, with regression tests, canonical patch release, successful GitHub publication and verified NuGet availability before updating KeyLoad. Preserve unrelated work; never force-push or bypass repository protections. The user authorizes scoped dependency repair commits, pushes and releases, and stable KeyLoad commits and pushes to main.
1414

15+
## System-critical SMID and operation efficiency
16+
- Treat the owner's SMID work as a first-priority product workstream and keep it visible in implementation planning and status. The term is not defined in the current repository or v0.3 architecture documents; do not guess its expansion or begin SMID-specific implementation until its canonical slice, REQ/AC criteria and required ADRs are mapped with the owner.
17+
- Orleans provides request isolation, cluster routing and activation movement. Keep the one-grain-per-request boundary and RF3 topology; treat Orleans Streams as a first-class architecture workstream to specify and qualify alongside persisted KeyLoad EventStreams. Keep their distinct delivery/restart contracts explicit, and define bounded resource, concurrency, backpressure, recovery and performance behavior for every operation.
18+
- Optimize each operation only against its correctness and fault contracts, using representative multi-node GitHub qualification to measure latency, throughput, allocations, memory, contention and backlog where applicable. Architecture choices or local builds alone do not prove maximum scalability or performance.
19+
1520
## MCAF repository workflow
1621

1722
Project: KeyLoad
@@ -377,6 +382,9 @@ For changes outside existing owner authorization, obtain direction before changi
377382
- Shared-checkout work that preserves unrelated changes and completes authorized delivery.
378383
- Real Docker/Aspire multi-node comparisons using the same data, oracle and workload contract.
379384
- All published test results and chart values MUST come from JSON produced by successful GitHub Actions runs. Include raw GitHub JSON links, measured source SHA, options, topology and acknowledgement/read guarantees. Missing or unsupported measurements MUST remain unavailable; never invent winners, zeros or sample performance values.
385+
- Website publication MUST use a separate GitHub Actions workflow that collects fresh JSON artifacts from the actual successful comparison job, verifies its run/job/revision/artifact provenance, and publishes those files. Performance figures MUST be read from that evidence; never hardcode them in website or README charts. A failed, skipped, incomplete, expired or unauthenticated comparison cannot refresh the published evidence.
386+
- The separate website workflow MUST trigger when `site/` changes and when the performance-benchmark producer workflow completes. Website completion is scoped to the site, its evidence ingestion, qualification and publication workflow; independently owned database and performance implementation MUST NOT be silently added to that task.
387+
- Owner direction on 2026-10-02 resolves the earlier website-source and producer-success policy conflict for REQ-BC-028: qualify the current trusted-main website separately from the measured-source checkout, and require the actual successful comparison job and its measurement steps. Unrelated database jobs or whole-workflow failure MUST NOT block this website-only delivery. Preserve accurate site/measured/control revisions and all website qualification gates. Earlier successful-run and equal-source guidance remains recorded as the superseded task boundary; never label a failed whole run successful.
380388
- Free community features only for comparison engines. No Enterprise images, license activation or paid cluster features.
381389
### Dislikes
382390
- Repeated permission questions for already authorized work.

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -233,17 +233,17 @@ Tests use TUnit and Microsoft.Testing.Platform and execute in GitHub Actions. De
233233

234234
The root `.editorconfig` is copied directly from Prostir. Every project enables SDK/static/style analysis with warnings as errors. Edit custom rules under `src/KeyLoad.Analyzers/Features/CodeQuality/` and add real-compilation cases under `tests/KeyLoad.Analyzers.Tests/Features/CodeQuality/`. They attach centrally to consumer projects and report in the IDE and build. Compiler SARIF 2.1 reports live under `artifacts/code-quality/<project>/<configuration>/<framework>/diagnostics.sarif`; CI retains them even when the build fails. See [CodeQuality](docs/Features/CodeQuality.md) for the rule catalog, authoring and applicability, and [current evidence](docs/implementation/code-quality.md) for actual gates.
235235

236-
On the joined 2026-10-02 working tree based on `9c570f8c33a7a9667507a8e1c0ca68860de3be45`, the full 25-project Release build passed with zero warnings/errors, canonical solution formatting passed, and repository governance passed. These checks cover source and static policy only. TUnit, process-recovery, Docker/Aspire RF3, MCP, and comparison qualification have not run for this exact source tree; the earlier successful main-branch CI run does not qualify it. Numeric coverage collection and its baseline are still not configured.
236+
Commit [`3559225a5f918160e46e32c9a812c3f71790e382`](https://github.com/managedcode/KeyLoad/commit/3559225a5f918160e46e32c9a812c3f71790e382) is on `main`. Its full 25-project Release build passed locally before delivery. Exact-SHA GitHub run [36988949282](https://github.com/managedcode/KeyLoad/actions/runs/36988949282) failed: analyzer tests passed 84/88, comparison tests 2/4, and RF3 integration tests 3/23. Governance also failed on all three OSes because its required plan file was ignored, and Windows normalized the preserved `AGENTS.md` prefix; the unit and process-recovery suites did not run because governance stopped those jobs. The first repair set is in progress; no exact-SHA qualification is claimed. Numeric coverage collection and its baseline are still not configured.
237237

238-
The comparison library/sole CLI host split under [ADR-043](docs/ADR/ADR-043-comparison-library-host.md), immutable harness under [ADR-044](docs/ADR/ADR-044-benchmark-immutable-contracts.md), owned PostgreSQL schema repair under [ADR-045](docs/ADR/ADR-045-postgres-schema-ownership.md), private storage owners under [ADR-046](docs/ADR/ADR-046-storage-private-owners.md) and genuine BenchmarkDotNet library under [ADR-047](docs/ADR/ADR-047-embedded-benchmark-host.md) are source joins awaiting qualification. The read-only product contract migration preserves JSON/base64 and fingerprints under [ADR-041](docs/ADR/ADR-041-read-only-public-collections.md). The 57 authored analyzer test methods and new resource regressions have not executed in CI for this source; numeric complexity rules are configured, while compatible coverage collection, container export and its baseline remain pending. See the [current source and gate evidence](docs/implementation/code-quality.md).
238+
The comparison library/sole CLI host split under [ADR-043](docs/ADR/ADR-043-comparison-library-host.md), immutable harness under [ADR-044](docs/ADR/ADR-044-benchmark-immutable-contracts.md), owned PostgreSQL schema repair under [ADR-045](docs/ADR/ADR-045-postgres-schema-ownership.md), private storage owners under [ADR-046](docs/ADR/ADR-046-storage-private-owners.md) and genuine BenchmarkDotNet library under [ADR-047](docs/ADR/ADR-047-embedded-benchmark-host.md) are source joins awaiting qualification. The read-only product contract migration preserves JSON/base64 and fingerprints under [ADR-041](docs/ADR/ADR-041-read-only-public-collections.md). The exact-SHA analyzer run executed 88 TUnit cases: 84 passed and four span/cohesion cases failed. Numeric complexity rules are configured, while compatible coverage collection, container export and its baseline remain pending. See the [current source and gate evidence](docs/implementation/code-quality.md) and exact run [36988949282](https://github.com/managedcode/KeyLoad/actions/runs/36988949282).
239239

240-
Memory and read-work repairs are in progress across storage, SQL/search, events, time series, graph, messaging, replication and transport. The [located repair inventory](docs/implementation/memory-performance.md) records authored changes and remaining proof separately. [Baseline CI 36936319423](https://github.com/managedcode/KeyLoad/actions/runs/36936319423) passed on the existing main SHA; it predates these dirty changes and does not qualify their correctness or performance.
240+
Memory and read-work repairs are in progress across storage, SQL/search, events, time series, graph, messaging, replication and transport. The [located repair inventory](docs/implementation/memory-performance.md) records authored changes and remaining proof separately. Earlier baseline CI [36936319423](https://github.com/managedcode/KeyLoad/actions/runs/36936319423) passed on its earlier SHA; candidate CI [36988949282](https://github.com/managedcode/KeyLoad/actions/runs/36988949282) failed and does not qualify correctness or performance.
241241

242242
All test qualification and load measurements run in GitHub Actions. The comparison harness uses deterministic JSON, float32 vectors and cyclic graphs, verifies the complete returned payload, and retains every measured attempt, including failures. Reports include useful throughput, p50/p95/p99, separate enqueue/receive/ACK timings and load-generator CPU/allocation/RSS. These resource metrics describe the client process. See the [comparison methodology](docs/implementation/comparative-benchmarks.md).
243243

244244
The [public benchmark lab](https://www.keyload.cloud/) displays verified CI reports with workload, scenario, measure and repetition controls. CI runs a correctness smoke and two measured profiles with 1 KiB/16 KiB documents, eight/four clients and three/five graph hops. GitHub Pages publishes their reports only after the complete CI workflow succeeds; [website operations](docs/implementation/website.md) describes provenance and the custom domain. Current external baselines are single-node and contracts differ; matched-durability, database resource budgets, Marten/Wolverine and scaling qualification remain planned. These development observations do not establish an equal-durability winner or production readiness.
245245

246-
The separate TimeSeries comparison profile is implemented under [ADR-050](docs/ADR/ADR-050-timeseries-timescale-comparison.md). Its CI test starts a digest-pinned, ephemeral TimescaleDB container, sends the same UTC samples through KeyLoad's RF3 .NET SDK and TimescaleDB's hypertable, and checks bucket results against the published `ManagedCode.TimeSeries` in-memory aggregation library. The library is not a persistence provider, and the Timescale container has no cross-run data volume; reports label each guarantee separately. The source builds, but Docker/Aspire execution and comparison artifacts remain pending exact-source GitHub Actions qualification.
246+
The separate TimeSeries comparison profile is implemented under [ADR-050](docs/ADR/ADR-050-timeseries-timescale-comparison.md). Its CI test starts a digest-pinned, ephemeral TimescaleDB container, sends the same UTC samples through KeyLoad's RF3 .NET SDK and TimescaleDB's hypertable, and checks bucket results against the published `ManagedCode.TimeSeries` in-memory aggregation library. The library is not a persistence provider, and the Timescale container has no cross-run data volume; reports label each guarantee separately. Candidate run 36988949282 reached the comparison suite but failed its Timescale resource-name collision and a container digest assertion that included Aspire-built KeyLoad images. Both source corrections are in progress; a new exact-SHA run is still required.
247247

248248
The product website redesign is in progress under [ADR-040](docs/ADR/ADR-040-static-site-threejs-evidence.md): a product introduction, a conceptual Three.js RF3 illustration and the complete evidence workspace. Its independent TUnit site suite runs in GitHub Actions against authentic historical reports. Website source, measured source and raw hashes stay distinct; the preview does not qualify current database changes or publish the pending nine-engine comparison profiles.
249249

‎docs/ADR/ADR-035-memory-performance.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -560,3 +560,73 @@ in prepared-storage-write.acceptance.md and prepared-storage-write.plan.md. Roll
560560
the coherent cache source unit without data conversion or weakened analysis. Exact-
561561
SHA GitHub unit/recovery/Docker RF3 SDK/MCP remains mandatory; source is not a measured
562562
allocation win, numeric coverage or power-loss proof.
563+
564+
## AC-MP-011 extension: per-operation RF3 budgets and Orleans cost
565+
566+
Owner direction makes whole-system operation efficiency and RF3 scalability a
567+
first-priority requirement. REQ-MP-005 / AC-MP-011 and
568+
REQ-RESOURCE-002 / AC-RESOURCE-002 therefore require each advertised operation
569+
family to have a measured workload, explicit numeric resource/latency/throughput
570+
budgets, and exact read/acknowledgement semantics. Budget values are derived from
571+
repeated same-topology baseline/candidate runs; they are not guessed from source
572+
limits or client-side measurements.
573+
574+
Ordered stages:
575+
576+
1. After a correctness-qualified candidate SHA exists, capture the matched RF3
577+
baseline for documents, events, messaging, reads/search, graph, time-series,
578+
blob, and administrative operations which are actually advertised.
579+
2. Add bounded low-cardinality per-operation and per-hop evidence for admission,
580+
request/read/command grains, quorum barrier, apply/store work and backpressure
581+
where each path uses those stages. Separate load-generator resources from each
582+
database node's CPU, allocations/GC and working set. Never label metrics with
583+
request IDs, tenants, users, credentials, keys or payloads.
584+
3. Add contract tests for metric names/labels and real RF3 SDK/MCP flows covering
585+
success, saturation/rejection, cancellation and slow-consumer behavior where
586+
applicable. Do not use metrics doubles as runtime proof.
587+
4. Repeat baseline and candidate profiles with identical source-independent
588+
workload inputs, topology, limits, warmup, concurrency and acknowledgement/read
589+
guarantees. Record numeric thresholds and raw artifact provenance before
590+
making performance claims.
591+
5. Update the operation matrix, README and status only from exact-SHA GitHub
592+
evidence; no local test/benchmark result can qualify the feature.
593+
594+
The implementation task is TASK-MP-011B in
595+
`memory-performance.plan.md`; it depends on TASK-MP-010's integrated correctness,
596+
build and governance joins. The lead is the sole owner of shared
597+
`src/KeyLoad.ServiceDefaults/` instrument registration,
598+
`src/KeyLoad.Orleans/Features/ClusterRouting/` hop instrumentation,
599+
`benchmarks/KeyLoad.Comparisons/Features/BenchmarkComparisons/` workload/report
600+
integration, shared contracts, ADR/acceptance/status evidence and final review.
601+
Feature owners implement their own operation budgets and correctness in the
602+
canonical feature slices. Planned tests are
603+
`tests/KeyLoad.UnitTests/Features/ResourceExecution/` for metric names, bounds and
604+
label privacy;
605+
`tests/KeyLoad.IntegrationTests/Features/ClusterRouting/ActivationMovementScenario.cs`
606+
for observed real RF3 movement and node-local storage identity;
607+
`tests/KeyLoad.ComparisonTests/Features/BenchmarkComparisons/` for matched
608+
operation profiles; and its `TimeSeries/` slice for repeated read/append/aggregate
609+
comparisons. Each advertised SDK/MCP operation family must join real caller
610+
success, authorization, overload/rejection, cancellation and applicable
611+
backpressure behavior to metric assertions without changing its public contract.
612+
613+
Persisted KeyLoad EventStreams and Orleans runtime Streams are separate contracts.
614+
The existing KeyLoad stream read/replay path is qualified through its persisted
615+
store and cursor. This amendment does not authorize a new Orleans pub/sub provider.
616+
That provider requires a separate REQ/AC and ADR covering provider durability,
617+
delivery/replay/ordering, restart, backpressure and its real RF3 tests before its
618+
files or task plan are added.
619+
620+
After tests-first source integration, verification order is enabled Release
621+
solution build, `dotnet format KeyLoad.slnx --verify-no-changes --no-restore`,
622+
repository governance, then the complete GitHub `ci.yml` exact-SHA run for TUnit,
623+
process recovery, RF3 SDK/MCP, repeated profiles, coverage and complexity evidence.
624+
Keep run/job URLs and raw per-node/per-operation JSON in the workflow artifacts.
625+
Numeric thresholds enter the operation matrix only after matched repeated
626+
baseline/candidate artifacts pass correctness with the same topology and
627+
acknowledgement/read guarantees. Local checks are development evidence only.
628+
629+
Rollback of instrumentation requires no persisted-data conversion and must not
630+
remove existing correctness budgets or request isolation. Rollback of an
631+
operation-level optimization is confined to that owning slice and must preserve
632+
its persisted format, exact outcomes and established budgets.

‎docs/ADR/ADR-036-orleans-foundation.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,21 @@ Preserved-policy reference conflict: root `AGENTS.md` still names “ADR-034 evi
88

99
Related requirements: REQ-REP-001 through 005, REQ-ROUTE-001 through 003, REQ-TEST-001 through 003; their same-numbered ACs are defined in the linked feature specs.
1010

11+
TASK-ROUTE-DIAGNOSTICS is an accepted internal observability refinement for
12+
REQ/AC-ROUTE-008 and AC-ROUTE-001/003. Stage/category/error enums and a GUID are
13+
the complete log contract; never pass raw exceptions, JSON, tokens, principal IDs
14+
or arbitrary runtime strings. Success uses only a stack-local stage enum. Existing
15+
KeyLoadException error code/detail is retained; private stage metadata is added
16+
only on failures. Root owns RequestGrain, CommandPartitionGrain,
17+
DatabaseReadGrain, GrainCommandExecutor, GrainRequestAuthority, GrainPayloadJson and GrainReplyFactory
18+
joins. A bounded worker owns only new ClusterRouting diagnostic helper/enum and
19+
UnitTests files. Ordered stages: author actual-provider privacy/reply regressions;
20+
add helpers; lead joins closed stage updates and error mapping; enabled build and
21+
format; full GitHub UnitTests/RF3 with exact SHA and fault log artifacts. No database,
22+
public API or persisted-format migration; rollback removes only this metadata/log
23+
path while retaining the original safe replies. The stage log helps locate a
24+
rejection and does not by itself qualify a repaired runtime failure.
25+
1126
Host integration contract: `Server/Features/StorageRecovery/PartitionHost` owns
1227
two physically separate ZoneTree stores, canonical `database` and replica
1328
`replica`, with one immutable voter configuration and incarnation. It constructs

0 commit comments

Comments
 (0)