Skip to content

Commit 6af5938

Browse files
committed
Clarify connection and request grain lifetimes
1 parent 8acae67 commit 6af5938

6 files changed

Lines changed: 95 additions & 2 deletions

File tree

‎AGENTS.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -589,3 +589,8 @@ A bounded website qualification candidate contains the20-project historical runt
589589
- Ingestion load qualification MUST include a one-client baseline and concurrent-client cases at 10 and 500 clients. Each case adds exactly 1,000,000 total distinct records across its clients, verifies acknowledged writes and the final stored records through real clients, and records actual client concurrency separately from native TUnit test concurrency. Bound client admission, cancellation and cleanup; retain the existing 100,000/1,000,000 dataset inventory and matched comparison contracts.
590590
- Owner direction2026-10-09 requires a primary-source-backed benchmark methodology and its actual implemented tests: one canonical comparable scenario set covers isolated individual operations and deterministic mixed operations at native1/3 nodes, with explicit setup/warmup/measurement/validation phases, repetitions, latency distributions, offered/completed load, errors, resource use, acknowledgement/durability and complete stored-result verification. Preserve serial measurement per job and genuine native clients/topologies; a methodology document alone does not finish the benchmark work.
591591
- Owner correction2026-10-09 requires all benchmark-stage builds, checks, tests and workload execution in GitHub Actions. Do not run benchmark development verification locally on the owner's computer. This scope-specific correction supersedes earlier local-development permission; retain original local failures as history and qualify the delivered source only from original Linux GitHub results.
592+
593+
## Connection and request grain lifetimes, owner clarification 2026-10-09
594+
595+
- Persistent client connections MUST have a distinct Orleans connection/session grain with bounded disposable session state and explicit disconnect/idle-expiry cancellation, joined cleanup and deactivation. Freeze the actual logical connection identity, transport ownership, limits and scheduling in ClientApi/QueryExecution requirements and the native-client ADR before implementation; a connection grain MUST NOT cache trusted roles or replace fresh persisted authorization for each operation. This is a required product target, not a claim that the native connection protocol is delivered.
596+
- Request and independently keyed read grains MUST retain no persistent operation state or completed request history. Keep request data call-scoped and bounded; after success, failure, cancellation, deadline or early stream disposal, join the original producer/capability cleanup and request native DeactivateOnIdle as part of settlement, without waiting for ordinary idle collection. Bound active work and admission/cleanup backlogs; total historical request count MUST NOT determine retained activation count. Native deactivation and CLR memory reclamation are asynchronous, so requesting deactivation alone MUST NOT be reported as measured activation or RAM recovery.

‎README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,14 +157,16 @@ Details: [composition guide](docs/Features/DatabaseComposition.md) · [transacti
157157

158158
```mermaid
159159
flowchart LR
160-
Callers["SDK · MCP · SQL · HTTP"] --> Request["One Orleans grain<br/>per request"]
160+
Callers["SDK · MCP · SQL · HTTP"] --> Request["Short-lived Orleans<br/>request grain"]
161161
Request --> Partition["Partition grains"]
162162
Partition --> N1[("Node 1<br/>ZoneTree")]
163163
Partition --> N2[("Node 2<br/>ZoneTree")]
164164
Partition --> N3[("Node 3<br/>ZoneTree")]
165165
```
166166

167-
Each authenticated call gets its own Orleans request grain. Partition grains route to node-local storage owners; moving a grain does not move its storage handles. Writes require a persisted majority of the three replicas. See the [architecture map](docs/Architecture.md).
167+
Each authenticated call gets its own request grain, with bounded call-local data and no persisted request state. After the operation and its stream cleanup settle, it requests deactivation immediately instead of waiting for ordinary idle collection. Completed requests are not kept as a history of active grains. Actual activation and memory recovery under load still require qualification.
168+
169+
Persistent connections will have a separate, bounded connection/session grain; that connection layer is still in development. Partition grains route to node-local storage owners; moving a grain does not move its storage handles. Writes require a persisted majority of the three replicas. See the [architecture map](docs/Architecture.md) and [grain lifetime contract](docs/Features/ClusterRouting/ExecutionPrimitives.md#connection-and-request-lifetimes).
168170

169171
## Why .NET, Orleans and ZoneTree
170172

‎docs/ADR/ADR-065-full-sql-client-compatibility.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,20 @@ family keeps the full SQL gate false. PostgreSQL wire3.0 is the first native
6565
client target, with explicit3.2 negotiation fixtures before advertising that
6666
version. Current HTTP/JSON plus official MCP remain their actual transports.
6767

68+
Owner clarification 2026-10-09 requires a separate Orleans connection/session
69+
grain for each accepted persistent logical client connection. Its bounded
70+
disposable session state is distinct from a fresh state-free operation grain;
71+
request/read activations request DeactivateOnIdle after their original work and
72+
cleanup settle. [ClientApi connection requirements](../Features/ClientApi.md#persistent-connection-ownership)
73+
own REQ/AC-CLIENT-CONNECTION-001/002 and TASK-CLIENT-CONNECTION;
74+
[ClusterRouting lifetime requirements](../Features/ClusterRouting/ExecutionPrimitives.md#connection-and-request-lifetimes)
75+
own REQ/AC-ORL-013 and TASK-ORL-REQUEST-LIFETIME. Freeze actual connection identity,
76+
transport callbacks, typed quotas, scheduling, cancellation and joined teardown
77+
before integration. A session must reload current persisted authorization for
78+
each operation and cannot retain storage views, trusted roles or completed
79+
request history. Native-client/MCP interoperability and activation/RAM recovery
80+
remain unqualified; this decision does not advertise a delivered session layer.
81+
6882
[Command inventory](../implementation/sql-client-commands-postgresql18.json)
6983
enumerates all183 entries from the official version18 command index. Family
7084
mapping is research inference;75 entries need explicit named coverage beyond

‎docs/Architecture.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,14 @@ The [documentation index](README.md) is the complete entry point for 25 canonica
141141

142142
Current mandatory policy requires an Orleans RF3 database, node-local PartitionHost storage ownership, separate request grains, distributed grain directory and activation migration, TUnit tests, Docker/Aspire RF3 execution and real .NET SDK plus official MCP SDK callers. Atomic partitions remain separate from physical replica placement. Credentials and trusted authorization are persisted server-side.
143143

144+
Request/read grains are short-lived operation owners with no persisted request
145+
state; current settlement asks for native deactivation after original work and
146+
cleanup. Persistent connections require a separate bounded connection/session
147+
grain, still unimplemented. [Connection ownership](Features/ClientApi.md#persistent-connection-ownership)
148+
and [request lifetimes](Features/ClusterRouting/ExecutionPrimitives.md#connection-and-request-lifetimes)
149+
distinguish these contracts and retain the open real-client, activation-count,
150+
backlog and RAM qualification gates.
151+
144152
Native Orleans execution/scheduling choices are mapped per grain and method in
145153
[ClusterRouting ExecutionPrimitives](Features/ClusterRouting/ExecutionPrimitives.md)
146154
and [ADR-110](ADR/ADR-110-native-orleans-execution-primitives.md). Bounded

‎docs/Features/ClientApi.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,32 @@ CALL/unknown roots with genuine Kestrel tests; TASK-SQLC-FULL owns native client
6161
interoperability. [Inventory](../implementation/sql-client-conformance.json)
6262
keeps both native and full SQL qualification false until actual evidence exists.
6363

64+
## Persistent connection ownership
65+
66+
Owner clarification 2026-10-09 requires one Orleans connection/session grain
67+
for each accepted persistent logical client connection, separately from each
68+
short-lived request grain. Current HTTP operations and the official MCP transport
69+
do not establish that this session layer exists. Native SQL and persistent MCP
70+
session integration remain implementation targets under
71+
[ADR-065](../ADR/ADR-065-full-sql-client-compatibility.md); request cleanup is owned
72+
by [ClusterRouting](ClusterRouting/ExecutionPrimitives.md#connection-and-request-lifetimes).
73+
74+
| Requirement | Measurable acceptance | Automated proof / current state |
75+
|---|---|---|
76+
| REQ-CLIENT-CONNECTION-001: an accepted persistent logical connection has one server-authenticated Orleans session identity and bounded disposable state; each operation retains its separate signed request identity and fresh persisted authorization. | AC-CLIENT-CONNECTION-001: real native SQL clients and official MCP clients where persistent sessions apply perform multiple operations through the same connection owner, while concurrent connections and requests have distinct isolated identities. Credential/policy revocation rejects the next operation and preserves stored state; reconnect obtains a fresh connection identity without recovering stale roles, payloads or results. | TASK-CLIENT-CONNECTION / planned ClientApi connection whole-flow tests and Aspire RF3 native-client/official MCP cases. No current implementation or passing evidence is claimed. |
77+
| REQ-CLIENT-CONNECTION-002: bound connection admission, retained state, in-flight work, disconnect/idle expiry and joined cleanup through validated typed options. | AC-CLIENT-CONNECTION-002: saturation has a fixed typed rejection without unbounded admission or buffers; disconnect/idle expiry cancels and joins original operations, releases session resources and requests native deactivation. Abrupt transport/silo loss, slow consumption and cancellation during cleanup preserve committed-write uncertainty and recover capacity within the frozen observation deadlines; native session activations return to baseline. | Same task and real transport/fault/lifecycle tests; exact numeric limits, protocol state, scheduling and teardown ownership must be frozen before implementation. Source/configuration alone is not acceptance. |
78+
79+
ClientApi owns transport/session registration and lifetime; QueryExecution owns
80+
actual SQL statement/portal/transaction semantics. The connection grain owns no
81+
ZoneTree handles, quorum receipt or persisted credential authority. Root owns
82+
shared contracts and the integration join. Implement in order: freeze logical
83+
identity, transport ownership, quotas and audited per-method scheduling; author
84+
real client success/revocation/overload/disconnect/fault tests; implement session
85+
routing plus original cancellation/cleanup joins; then qualify the complete
86+
mapped Linux Aspire RF3 scope. Session routing must not serialize unrelated
87+
operations accidentally or introduce blanket reentrancy. Rollback is scoped
88+
source rollback before protocol qualification, with no data migration or fallback.
89+
6490
REQ-CLIENT-010 / AC-CLIENT-010 / AC-DIAG-001..004 add bounded internal RF3
6591
dispatch evidence under [ADR-036](../ADR/ADR-036-orleans-foundation.md).
6692
Closed credential/read/command phases and failure categories correlate the

‎docs/Features/ClusterRouting/ExecutionPrimitives.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,42 @@ The linked adoption contracts own executable stages and reader rollout.
3030
Frontend: N/A, no new UI. Client schemas: N/A for these stages; existing SDK/MCP
3131
operation contracts remain the qualification entry points.
3232

33+
## Connection and request lifetimes
34+
35+
Owner clarification 2026-10-09 distinguishes a persistent connection/session from
36+
an individual operation. [ClientApi](../ClientApi.md#persistent-connection-ownership)
37+
and [ADR-065](../../ADR/ADR-065-full-sql-client-compatibility.md) own the required
38+
connection grain and its bounded disposable session state. That layer is not
39+
implemented. A connection is not the storage owner or an authorization cache;
40+
every operation still has a fresh signed GUID and persisted authorization.
41+
42+
The existing request and independently keyed read grains have no persisted
43+
request state. Payload, identity, reply and probe state must remain bounded and
44+
call-scoped, with no retained completed-request history, activation-owned storage
45+
handles, request timers or per-request caches. RequestGrain passes DeactivateOnIdle
46+
to NativeCqrsStreamLifetime; NativeCqrsStreamSettlement first awaits native
47+
producer disposal and then invokes that callback. DatabaseReadGrain requests the
48+
same deactivation in its capability finally. Success, rejection, failure,
49+
cancellation, deadline and early disposal must all settle original work before
50+
activation removal; a terminal frame alone is not completion of cleanup.
51+
52+
[Native Orleans DeactivateOnIdle](https://learn.microsoft.com/en-us/dotnet/api/orleans.grain.deactivateonidle?view=orleans-10.0)
53+
requests removal when the activation becomes idle, overriding ordinary idle
54+
collection. This is asynchronous runtime deactivation, not synchronous CLR
55+
memory reclamation. Existing source proves the cleanup request, not measured
56+
activation-count or RAM recovery after a burst.
57+
58+
The current validated GrainRoutingOptions admit at most 64 request producers and
59+
128 producer/capability frames per silo. Those are execution-frame bounds, not
60+
a bound on every activation awaiting creation, admission or deactivation.
61+
Ingress and cleanup backlogs therefore need their own bounded-flow proof.
62+
At stable load, concurrent requests are approximately completed requests/second
63+
times mean execution-plus-cleanup seconds; request/read activations and native
64+
transport overhead must be counted separately. For example, 10,000/s at 100 ms
65+
means roughly 1,000 concurrent requests before other overhead, not one retained
66+
activation for every historical request. This is arithmetic, not KeyLoad capacity
67+
or throughput evidence.
68+
3369
## Per-grain and method selection
3470

3571
| Current owner / method | Decision and concrete reason |
@@ -68,6 +104,7 @@ Implementation authorized on 2026-10-06; concrete staged contracts are in
68104
| REQ-ORL-010: review all official documentation capability families for useful KeyLoad applications. | AC-ORL-010: a version-aware linked inventory covers grain/runtime model, messaging, state/time, services/lifecycle, placement/directory/migration, serialization, hosting/configuration, observability, security/deployment and testing/resources. Each capability records actual source presence, concrete use, missing contract and priority or deferral; unknowns and legacy examples stay explicit. | TASK-ORL-CAPABILITY-REVIEW; primary-source/native-API review plus actual source search and static link/navigation validation are the explicit documentation-only evidence exception. No runtime or performance result is inferred. |
69105
| REQ-ORL-011: wake the existing native due service from canonical apply with bounded, joined disposable waiting. | AC-ORL-011: actual apply/racing registration/coalescing/cancellation/shutdown workflows settle safely; at most two scan pages per second, one page/job in flight and unchanged finite sweep/creator/quorum/restart single-effect contracts pass through actual SDK/MCP operations. | TASK-ORL-DUE-APPLY and ROOT-JOIN in RuntimeAdoption; new real-operation ClusterReplication/Messaging cases and existing DueCoordination RF3 gates. |
70106
| REQ-ORL-012: export native Orleans runtime telemetry with bounded fail-closed privacy. | AC-ORL-012: real signed write/read/failure/concurrent identity operations preserve state/outcome and trace parentage; exported points/spans/exemplars contain only the accepted fixed metadata, immutable privacy sentinels suppress unsafe spans, capture/providers flush and join. | TASK-ORL-TELEMETRY/ROOT-JOIN in RuntimeAdoption; new OrleansRuntimeTelemetry operation cases plus actual SDK/MCP RF3 qualification. |
107+
| REQ-ORL-013: request/read activations are disposable operation lifetimes, with immediate native deactivation after joined cleanup and no retained completed-request state. | AC-ORL-013: real SDK and official MCP success, rejection, cancellation, deadline and early-disposal flows settle original producers/capabilities and preserve stored effects or stable uncertain outcomes. Native request/read activation counts return to the pre-trial baseline within the frozen observation deadline, shorter than ordinary idle collection; repeated bounded bursts do not increase that baseline. Saturation rejects before unbounded activation/admission/cleanup backlog, and a subsequent authorized operation succeeds without prior payload/identity/result crossover. | TASK-ORL-REQUEST-LIFETIME / root integration owner; existing NativeCqrsRequestV2 settlement/work-owner cases cover mechanisms. A dedicated real Aspire RF3 RequestActivationLifetimeRf3Tests scope, native activation observations and isolated Linux load/RAM evidence remain planned; source callbacks and ProducerDisposed markers alone do not pass this criterion. |
71108

72109
Native scheduling still runs one turn at a time. Interleaving admits other turns
73110
while a method awaits; it does not parallelize a CPU loop. AlwaysInterleave can
@@ -133,6 +170,7 @@ flowchart LR
133170
|---|---|
134171
| TASK-ORL-AUDIT / read-only native API and grain reviewers | Inspect pinned version, official APIs and actual methods. Complete with source-backed findings; no source/provider/test edits. |
135172
| TASK-ORL-CONTRACT / root integration owner | Join both audits; own root policy, this specification, ADR-110, architecture/index/status links. Complete after policy diff preservation, link/diagram checks and static governance. This is the current stage. |
173+
| TASK-ORL-REQUEST-LIFETIME / root integration owner | First freeze native observation deadlines below ordinary idle collection and finite ingress/admission/cleanup bounds. Own matching ClusterRouting lifetime operation tests, any reproduced lifecycle repair and the Server admission join. Verify real stored outcomes and original cleanup, actual request/read activation return to baseline, saturation and following-operation isolation through Aspire RF3. Run heavy load/RAM trials only in isolated Linux GitHub jobs; no runtime gate closes from this documentation stage. |
136174
| TASK-ORL-WORKERS / ClusterRouting execution owner | Starts after exact pure operation, keys, quotas, call-graph transitions and oracle are frozen. Own new feature-local Grains/Contracts/Models and matching real-operation tests. No storage or request-grain replacement. |
137175
| TASK-ORL-CONTROL / native long-operation owner | Starts after NativeCqrs durable operation/control API and per-await state audit exist. Own only control contract and grain scheduling plus real identity/cancellation tests; public API changes require their owning ADR. |
138176
| TASK-ORL-HINTS / affected slice owner | Starts only when a concrete disposable hint and no-hint recovery path are frozen. Own that method and actual loss/duplicate/overload tests; no reliable protocol may be converted silently. |

0 commit comments

Comments
 (0)