Skip to content

Latest commit

 

History

History
71 lines (48 loc) · 16.4 KB

File metadata and controls

71 lines (48 loc) · 16.4 KB

Native CQRS result and operation streams

Status: C0 runtime-compatibility contract accepted by root under the owner's104-task scope,2026-10-04. Product stream implementation and qualification are pending. Canonical slice: ClusterRouting; parent ClusterRouting. Decision: ADR-082. Related slices: ClientApi, QueryExecution, Search, Authorization, ResourceExecution and StorageRecovery.

The owner requires Orleans execution/coordination and ManagedCode.Communication CQRS native IAsyncEnumerable results and long operations. KeyLoad currently returns a bounded Task through its unique request grain. Pages, cursors and persisted feeds are real existing contracts; they do not establish this new stream contract. Communication10.2.6 supplies ICommand, a capacity-one CqrsStream producer and native CqrsStreamChunk serializers. It does not supply an IQuery dispatcher. Keep typed KeyLoad read contracts and the canonical persisted command/receipt path.

flowchart LR
    Caller[SDK or official MCP request] --> Entry[Unique Orleans request grain]
    Entry --> CQRS[Communication native result and progress stream]
    CQRS --> Leaf[Authorized database capability grains]
    Leaf --> Owner[Node-local PartitionHost and ZoneTree]
    Owner --> RF3[Ordered commit and RF3 authority]
    CQRS --> Consumer[Bounded pull and terminal outcome]
    Proof[Real Graph and native enumeration tests] --> Gate[Contract and qualification gate]
    Gate --> Entry
Loading

Requirements and acceptance

Requirement Measurable acceptance Automated evidence
REQ-NCQRS-001: native enumeration preserves enforced Orleans caller/method transitions and separate request identity AC-NCQRS-001: a real Orleans cluster with the pinned Graph filters accepts an explicitly allowed nested grain call after an emitted chunk and awaited suspension; the same missing transition fails. The observed caller/method and request keys remain correct across independent concurrent streams. Product SDK/MCP RF3 requests must later prove the real signed request/read/command join. C0 NativeCqrsGraphTests and NativeCqrsIsolationTests; C1/C2 actual RF3 routing tests pending
REQ-NCQRS-002: native pull lifetime, cancellation and backpressure are bounded AC-NCQRS-002: actual Communication chunks cross Orleans with native batch size1; an unread bounded producer cannot grow an unbounded backlog. Cancellation after the first chunk and early enumerator disposal settle the real producer and release the enumerator within a declared10-second test cleanup bound; a following stream succeeds. No fake clock or programmable provider substitutes. C0 NativeCqrsLifetimeTests; later product admission/cancellation RF3 tests pending
REQ-NCQRS-003: one terminal outcome and safe failures have explicit authority AC-NCQRS-003: a KeyLoad-owned well-formed Create producer yields increasing sequences and exactly one Completed or Failed chunk; domain failure after progress retains a bounded safe code/detail. Transport failure before/after chunks and consumer cancellation remain distinguishable from committed command outcomes. Commands never report cancellation as proof that an acknowledged or uncertain write was undone. C0 NativeCqrsTerminalTests; C1 canonical receipt/privacy and C2 RF3 interruption tests pending
REQ-NCQRS-004: SDK/MCP consume the same native typed execution stream AC-NCQRS-004: genuine Aspire RF3 .NET SDK and official MCP clients consume results/progress, abort/retry and observe persisted grant/revocation errors through the same request/capability path; public result payload/frame/total/rate/time limits and cleanup are measured. Existing bounded final response compatibility drains that path once, without another dispatcher. C2 ClientApi/QueryExecution RF3 tests pending; Communication-owned HTTP bound regressions required before SSE admission
REQ-NCQRS-005: long work has a defined restart/failover contract AC-NCQRS-005: genuine process interruption and RF3 failover prove the chosen durable receipt/checkpoint or explicitly disposable operation contract, stable retry identity, fencing, no leaked storage lease and recovery after activation movement. ANN generation freshness/publication must additionally satisfy AC-ANN-007/008. C3 Search/StorageRecovery/ClusterRouting process and RF3 tests pending

Every requirement maps to its acceptance above. C0 qualifies only the native runtime mechanism; it cannot close the product half of AC-NCQRS-001–003 or AC-NCQRS-004/005. No source-only claim, skipped case, assertion relaxation or mock/fake/stub is accepted as runtime proof.

Accepted C0 test contract

Use a genuine Microsoft.Orleans.TestingHost10.3.1 TestCluster, centrally pinned to the existing Orleans family, with the actual ManagedCode.Orleans.Graph10.0.6 filters and Communication10.2.6 registration. This package is test-only in KeyLoad.UnitTests; no new product package, project, topology or public capability is introduced. Start and completely dispose the native cluster inside the TUnit process launched by the existing Aspire unit entry. It is an explicitly named routing-mechanism fixture, never a replacement for Docker RF3.

Test actors perform real native runtime observations: actual actor/caller/method identity, enumerator/producer entry and settlement counters, and typed chunks. They are not substitutes for storage or replica capabilities, are not registered in the product server and do not return invented database success. Use generated test-only stable Alias/Id records and native Communication chunk conversion. No caller-controlled expected response, mocked factory, fake filter, synthetic cancellation hook or alternate stream implementation.

Freeze per-test request GUIDs; allowed and denied transitions are method-specific. The allowed stream emits Started, awaits a real suspension, invokes the observation leaf, emits bounded Progress and returns its actual observation in one terminal Result. The denied stream uses the same native path with that edge absent: assert Started, the actual leaf-call attempt, native exception type/status, no leaf entry, successful allowed control and real finally settlement. Do not catch every InvalidOperationException and manufacture a safe Graph-denial Problem. Native dependency failure detail is not a public oracle; safe product conversion is required in C1. Mid-stream domain failure uses an explicitly constructed bounded typed Problem. Observe producer settlement in its real finally; separate client observation calls must not fabricate cleanup evidence. Concurrent streams use distinct keys and assert no caller/history or terminal/result identity crossover.

Inside the leaf, CaptureCurrentCaller describes the current leaf method because Graph's incoming filter sets it before invocation. Observe the originating edge using the actual CallHistory and native GetLatestObservedCall, including exact source/method and target/method. Bind source/target native grain IDs and leaf primary key to the genuine requested actors; an echoed request parameter or substring comparison alone is insufficient. Never construct or restore synthetic Graph authority to make a test pass.

Set native Orleans batch size1 on the actual enumerable. All test pulls, observer waits and teardown have finite real deadlines; no Timer-based fabricated elapsed budget or unbounded wait. Cancellation and early disposal must occur after actual first-chunk receipt. Both consumer and producer must settle before fixture disposal. A failed cleanup is a test failure; never swallow it as success. The cleanup deadline is a test envelope, not a production cancellation-latency promise.

Exact ownership: new tests/KeyLoad.UnitTests/Features/ClusterRouting/NativeCqrs*.cs only for the worker; root owns the central pin, test-project reference, this specification, ADR, shared status and every build/runtime/Git join. Existing tests and product files remain outside C0 write scope. Keep numeric source limits and native serializer analysis. Root verifies every diff and then runs the focused C0 cases through Aspire, full normal/scalar unit gates, build/formatter/governance and delivered-source Linux qualification.

C0 test construction uses the pinned native extension points: a narrowly targeted Orleans IConfigureGrainTypeComponents installs an IGrainActivator only for the three empty-constructor test grain implementations. Its actual factories construct those instances; Orleans still attaches their real context and owns scheduling, routing, activation and lifecycle. Disposal delegates to the native DefaultGrainActivator, and all other grain types keep their native activators. A custom TUnit DataSourceGeneratorAttribute obtains one per-test-session fixture through SharedDataSources.GetOrCreate with an explicit constructor factory; native initialization and disposal remain automatic. Ownership adds tests/KeyLoad.UnitTests/Features/ClusterRouting/Helpers/NativeCqrsActivation.cs and NativeCqrsDataSource.cs. These construction hooks do not inject authority, RequestContext, call history, results or producer observations.

Ordered product stages and blocked joins

  1. C0: prove actual Graph/native enumeration context, lifetime, failure and isolation behavior. A demonstrated ManagedCode defect must be repaired in its owning repository, with the prescribed patch release, successful publication and verified NuGet availability before KeyLoad consumes it.
  2. C1: the accepted NativeCqrsRequestV2 contract freezes the single native request entry, alias keyload.request.v2 and current RequestInterfaceVersion4 as distinct identifiers, Started/terminal types, signed purpose, authenticated peer envelope3 with separately preserved discovery-MAC2, cohort readiness/majority admission, cold rollout/rollback, byte/count/deadline bounds and canonical write-outcome semantics. Product writes wait for published dependencies and the unchanged six C0 Aspire oracles; current source is not yet application-RPC compatible merely because this contract exists.
  3. C2: root freezes versioned SDK/SSE and official MCP progress/abort contracts. Use actual Communication HTTP helpers after the owning transport-bounds repair is published and verified; preserve current persisted authorization and official MCP request tokens. No custom competing CQRS dispatcher or consumer-side parser workaround.
  4. C3: root freezes each long-operation receipt/checkpoint, node-local lease, restart/failover/fencing and ANN build/replay/publication contract before implementing it. Native IAsyncEnumerable alone is not durable execution, an outbox checkpoint or an atomicity guarantee.

Frontend N/A: runtime/API work has no requested UI. Benchmarks later measure chunk/admission/serialization/cancellation resource cost on genuine GitHub workloads. Rollback of C0 removes only test infrastructure. Product rollout/rollback stays blocked on C1–C3 contracts and required gates; no canonical stored data, native WAL, authorization, RF3 authority or public response format changes in C0.

Accepted owning Graph enumeration repair, 2026-10-04

The first actual Aspire C0 cohort fails all six tests before the intended producer observations. Its original reports and native cluster logs are retained. Pinned Orleans10.3.1 source proves that AsyncEnumerableGrainExtension.StartEnumeration calls the original IAsyncEnumerableRequest.SetTarget/InvokeImplementation/GetAsyncEnumerator directly, then retains that enumerator for native MoveNext/Dispose. The original application method bypasses the ordinary GrainMethodInvoker request.Invoke filter path. Graph10.0.6 and current owning10.0.8 have no adapter for this path; the default Orleans-module tracking skip loses the application method's caller context. This is an owning dependency defect, not permission to inject authority in KeyLoad tests.

Requirement Acceptance Owning automated evidence
REQ-GSE-001: native StartEnumeration enforces the original application transition AC-GSE-001: genuine Orleans cluster client and silo-origin calls admit allowed initial stream methods and reject absent client/method edges before application producer entry; permitted nested leaf calls after first and later pulls retain exact original caller/method and native grain IDs. Default system-target and ordinary non-stream behavior remain unchanged. NativeAsyncEnumerationPolicyTests; existing attribute/system-target suites
REQ-GSE-002: original application context follows the actual enumerator lifetime AC-GSE-002: actual GetAsyncEnumerator, later MoveNext and Dispose execute with the captured original caller/history; async suspension, errors, cancellation and early disposal restore the previous ambient context. Concurrent distinct streams have independent history branches, no crossed result identity and no history growth per MoveNext. NativeAsyncEnumerationContextTests and NativeAsyncEnumerationLifetimeTests; existing history isolation suite
REQ-GSE-003: the adapter preserves native runtime ownership with bounded registration state AC-GSE-003: the server-local wrapper delegates every native IRequest/IInvokable member and MaxBatchSize; Orleans keeps its request-id dictionary, scheduler, batching, cancellation, expiration and disposal. Factories are closed at startup from the same configured grain-interface assembly inventory; runtime lookup uses exact original generated MethodInfo metadata in an immutable finite catalog. An unregistered/open-generic stream method fails closed without per-call reflection, scans or lazy catalog growth. Catalog/delegation regressions and real batch-size1/multi-batch streams
REQ-GSE-004: the owning repair is delivered before KeyLoad consumption AC-GSE-004: owning formatter, full Release build and full TUnit suite pass; canonical patch commit/push/release succeeds at its exact SHA and ManagedCode.Orleans.Graph is verified on NuGet before KeyLoad's central pin changes. The unchanged six C0 oracles then pass through Aspire, followed by required full Linux/recovery/RF3 gates. Root owning release/feed receipts and original Aspire C0 reports

The repair intercepts only exact native IAsyncEnumerableGrainExtension.StartEnumeration with its original typed request argument. Outgoing and incoming tracking use that original request's generated interface/method/options, while retaining actual context.SourceId/TargetId and TargetContext.GrainInstance for native activation identity and MayInterleave resolution. Policy validation precedes wrapper attachment. Later native extension/system calls remain native; do not represent each MoveNext as another application invocation or append synthetic authority/history entries.

After successful incoming policy validation, fork the existing actual CallHistory once for that stream and capture its actual CurrentCallerContext. A server-only delegating IAsyncEnumerableRequest returns a scoped native enumerable. Scope InvokeImplementation, GetAsyncEnumerator, every actual MoveNextAsync and DisposeAsync, restoring both previous ambient values in finally. Reuse that isolated captured branch across sequential pulls; outgoing child calls retain the existing fork/restore behavior. Do not infer caller identity from history, share mutable branches between enumerators or add global stream/handle registries. Native Orleans remains the owner of the actual enumerator and source cancellation token. This is graph-transition enforcement; persisted database authorization remains separate and mandatory.

Root freezes this contract and ADR before write delegation. The query worker stages a scoped patch under /private/tmp against the clean owning Orleans.Graph checkout: new ManagedCode.Orleans.Graph/Features/AsyncEnumeration helpers, minimal joins in the two filters, RequestContextHelper and client/silo registration; corresponding real native tests under ManagedCode.Orleans.Graph.Tests/Features/AsyncEnumeration plus scoped README/owning feature documentation. No KeyLoad test/production edit, Orleans fork, transport serializer DTO, wildcard policy, suppressions, release, build or Git by the worker. Root reviews/applies, owns canonical formatter/build/tests, diagnoses actual failures and delivers the published patch. No KeyLoad storage-format conversion is involved. Product stream adoption remains disabled until its current contract and gates qualify; rollback must not bypass graph checks or retain a consumer workaround. Client and silo Graph versions must match before qualification.