Skip to content

Commit 3e84e3e

Browse files
committed
Add verified replica-prefix GC and physical owner registration
Local Release build and formatter pass. Native unit and scalar2991/2991, verified-prefix process recovery10/10 and all12 production discoveries pass. Preserve original LinuxRF3 and covered-session failures; exact-source Linux acceptance and numeric product coverage remain open.
1 parent c028cfe commit 3e84e3e

89 files changed

Lines changed: 17077 additions & 190 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/ADR/ADR-030-retention-paused-restore.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,3 +97,44 @@ Shared-ledger boundary: each first purge-through1 completes exactly two observed
9797

9898

9999
R441 actual native correction, TASK-KL098-TOPIC-RETENTION-CORRUPTION-001 / REQ/AC-EVENT-RETENTION-005 and AC-EVENT-RETENTION-002: original twelve Corruption failures and one nonadvancing policy-epoch failure remain retained. DatabaseEngine.ExecuteAndBuildOutcome excludes Corruption from stored domain outcomes. The genuine manager revocation advances persisted epoch1 to2 before replay denial; original success receipt and complete native bytes remain immutable after denied repeat. No product exception normalization, authorization weakening or new recovery claim. Root owns fresh native build/census/whole-operation re-execution; this source correction is not PASS evidence.
100+
101+
102+
## KL035 bounded replica-prefix GC contract (2026-10-08)
103+
104+
TASK-REP-PREFIX-GC-001 / REQ-REP-GC-001 / AC-REP-GC-001 freezes actual prefix reclamation. A node-local Materializer holds its existing apply owner, then SnapshotStore gate, then native log ProtocolGate and log lock. CURRENT published snapshot is reverified against native complete image, incarnation, format, checksum and committed cut before any eligible key deletion. One batch deletes actual canonical replica-entry keys with absolute Index<=snapshot.Index using the native replica store atomic Commit; it changes neither hardstate snapshot/term/vote/LastIndex/CommittedIndex nor canonical apply/docs/receipts. Only existing MaxAppendEntries and MaxAppendBytes bound range/work/key retention, checked before allocation; cancellation before commit publishes no effects. The same verified immutable image remains and all suffix entries are retained. Unpublished/corrupt/missing image fails closed before deletion.
105+
106+
The GC batch then awaits the existing REPLICA store Compact publication under the same owner, rewriting replica commands.wal with surviving live keys; CANONICAL Compact is not called. Native ZoneTree derived segments may retain older deleted bytes, and eventual maintainer segment reclamation is explicitly separate/unmeasured. This is actual key deletion plus replica-journal live-key rewrite, not read visibility cutoff or immediate whole-tree byte reclamation. Native compaction failure/poison and original ownership cleanup propagate; snapshot+tail remains at every valid recovery cut. Existing journals/formats/aliases/term/index semantics are unchanged. No discarded atomic journal, migration or compatibility fallback.
107+
108+
REQ-REP-GC-002 / AC-REP-GC-002: maintenance schedules at most one existing joined checkpoint/reclamation task. Bounded later ticks use actual remaining prefix keys as progress; no persisted/stale cursor and no unbounded reclaim loop. Reader results are existing independent native arrays; per-call image files are closed under snapshot gate. GC deletes no snapshot image or transfer file and never invalidates borrowed native read cuts. Compact uses its actual storage write gate/current-generation contract.
109+
110+
REQ-REP-GC-003 / AC-REP-GC-003: real current-format store/CrashHost flows create committed snapshot+new retainedtail/full literal original receipts, reclaim actual eligible keys, inspect exact hardstate/terms/index/suffix/canonical state, native dispose/reopen, fresh-target complete install+tail and exact retry with healthy next command. Corrupt/missing snapshot and canceled GC preserve full native state; repair then healthy GC follows. Real owned child kills at existing publication and replica-store native checkpoint/journal fault boundaries prove at least one complete verified snapshot+tail path. All actual original children/readers/filelocks are joined. This proves process-kill recovery only; power-loss/endurance/current Linux RF3 remain separate pending gates.
111+
112+
Ownership: Replication ClusterReplication/Contracts existing interfaces; Storage native log prefix collector/reclaimer and snapshot owner; Execution Materializer/Maintenance. No new provider, public SDK/SQL/MCP operation or wire format is introduced: this is existing node-local housekeeping, never caller-authorized database mutation bypass. RecoveryTests/ClusterReplication and CrashHost/ClusterReplication own real native cuts and wholeflow assertions. Root joins/builds/native tests/records actual new census. ADR030 remains Accepted until exact required evidence exists.
113+
114+
```mermaid
115+
flowchart LR
116+
V[Verify current complete image] --> D[Atomic eligible key deletion]
117+
D --> R[Replica journal live-key rewrite]
118+
R --> N[Next bounded tick: remaining actual keys]
119+
V --> P[Keep snapshot and committed tail]
120+
D --> P
121+
R --> P
122+
```
123+
124+
### Exact private GC verification map
125+
126+
`ReplicaPrefixGcProcessRecoveryTests.AcRepGc003ActualPrefixDeletionAndReplicaRewriteRetainVerifiedSnapshotTailRecovery` owns ten native argument rows: HeaderWritten/0, PayloadWritten/0, JournalFlushed/0, MutationApplied/0, MutationApplied/3, ApplyCompleted/0, SnapshotWritten/0, SnapshotFlushed/0, InstallPrepared/0, JournalSwapped/0. Existing synchronous native `CommitStage` observer pauses only the admitted replica-store deletion/rewrite position; the real original CrashHost child is killed and its readers joined before reopen. No new product fault callback is introduced. Original cuts before durable journal completion permit only complete prefix retained or complete prefix reclaimed; all later cuts require complete reclamation. Partial key deletion is rejected.
127+
128+
Each actual row verifies snapshot cut4+term1, retained tail5, immutable complete hardstate bytes, canonical inventory equivalence, all literal original put mutation receipts2–5 and byte-identical replay without position changes. A fresh empty native target installs the retained complete snapshot and actual surviving tail entry, applies them, then commits independently specified healthy revision5/cut6 and reopens. The same owned trial additionally checks original-token cancellation, missing native image (FileNotFoundException) and hash-corrupt image (Corruption), full unchanged replica/canonical records and position, exact image repair, successful reclamation and healthy tail. Native images, stores, original child/readers and file locks remain trial-owned through cleanup; failures retain original root/evidence. New case discovery counts/compiled identities and all Linux qualification remain root-owned and unobserved for this packet.
129+
130+
Code ownership adds feature-local `ReplicaPrefixReclaimer`, `ReplicaVerifiedPrefixReclamation`, `ReplicaCheckpointReclamation` and cohesive `ReplicaCheckpointPublication`; the latter moves existing state calculation only, preserving its original locks/guards. New crash/test types are under their actual ClusterReplication Contracts/Helpers/Processes/Assertions/Cases roles. No native aliases/field IDs, storage formats, public SDK/MCP/SQL routes, runtime deadlines or qualification statuses change.
131+
132+
An explicit reclamation call with an already-deleted prefix still verifies the current complete image and joins one native replica WAL rewrite, returning zero without a no-op atomic commit. This settles process interruption between deletion and rewrite. Original cancellation is checked again before that zero-deletion rewrite; once a positive deletion commits, the original rewrite is joined despite later cancellation. The normal maintenance trigger remains actual remaining prefix keys or its unchanged snapshot threshold. No persistent cursor/new recovery format is added.
133+
134+
Native GC negative-image setup drains actual FileStream buffers with caller-canceled FlushAsync, then joins RandomAccess.FlushToDisk on the same owned handle before testing corruption. This preserves the actual synchronous OS durability barrier; async buffer drain alone does not substitute for it. The API contract is documented by [Microsoft .NET10 RandomAccess](https://learn.microsoft.com/en-us/dotnet/api/system.io.randomaccess.flushtodisk?view=net-10.0) and [FileStream.FlushAsync](https://learn.microsoft.com/en-us/dotnet/api/system.io.filestream.flushasync?view=net-10.0). These are process-recovery fixture actions, not a power-loss qualification claim.
135+
136+
R573 retained ten original process-trial failures at the independent document-result oracle. The expected literal and recovered result are separate object graphs; their Orleans reference encoding is not the public value contract. Compare the complete public document result through exact JsonDefaults bytes, including the entire reference, revision, exact user JSON, redaction flag and ordered redacted fields. The independent literal mutation receipt comparison, byte-identical original native receipt replay, complete canonical and replica inventory bytes, hardstate and native snapshot/WAL checks remain unchanged. This changes no production serializer, persisted format or acceptance requirement; repaired native execution is still required.
137+
138+
R578 retained ten original failures at fresh-target append: the fixture created an independent random canonical signing key and correctly rejected the source's original signed native tail. A fresh receiving member must use the source RF3 cluster's actual configured canonical signing key, as production PartitionStores and existing real stored-cluster fixtures do. Pass that owned key through ZoneTreeStoreOptions at target creation and again at its genuine reopen; do not re-sign or re-normalize the original tail, copy an identity file, relax signature verification or substitute another operation. Native source-tail bytes and all receipt/state assertions remain unchanged. This fixture correction is not a production authority change or proof that the repaired flow passed.
139+
140+
R580 retained ten cleanup failures after the original operation assertions completed: the scalar-store cleanup searched for owner.lock at the trial root instead of its actual nested stores. GC cleanup must use existing bounded ReplicaProcessFiles ownership checks for all three known node roots (original, fresh target and negative boundary), including both canonical and replica stores of each, after original process/readers/materializers joined. Only after every exact owned file check succeeds may the existing bounded deletion remove the trial root. Observe every check/deletion failure and retain that original root; do not create a dummy root lock, skip missing store locks, relax file-release checks or expand cleanup deadlines.

‎docs/ADR/ADR-082-native-cqrs-streams.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -331,3 +331,9 @@ Reuse the existing RequestCqrsLifecycleEvidence owner and actual native startup/
331331
Guard owns exact stages: seed, malformed response headers, bounded native body read, problem validation, response owner disposal, warning wait, healthy SDK/MCP continuation. Instrument only actual awaits/calls with the existing synchronous SetStage; do not detach a reader or fabricate a completed phase. On failure first snapshot is taken before owned wave stop and terminal after original joins; actual original exceptions remain in existing failure order with cleanup errors. The original guard400, exact closed warning and real healthy follow-up assertions remain mandatory. No warning can be synthesized or accepted from another node/wave.
332332

333333
SCAT binds its existing original/mismatch/corrected actual waves to the same scoped evidence and records boundaries before native start/SDK-MCP assertion and scope disposal. Preserve every native membership/catalog fencing, denied-effect, corrected healthy operation, all-lock and cleanup gate. Source instrumentation is not a cause or pass: fresh root-owned actual Aspire runs must supply bounded context and original logs. No new LocalImage selection, product provider/seam, sleep/retry/timer/health gate or deadline increase. Root owns guard join, format/full build and genuine whole-operation/native Linux qualification. Rollback removes this observation-only task with its private test owner plumbing.
334+
335+
### TASK-CRS-C1-NATIVE-PHASE-EVIDENCE-001 (2026-10-08; frozen before code)
336+
337+
REQ/AC-CRS-DIAG-007 extends AC-CRS-DIAG-002/005/006 without changing logger, admission, quotas or timing. Original30fb run37694136259 guard400 succeeded but GuardWarningWait reached original12min cancellation with empty rejection artifact; canonical case canceled after2m13s without retained waiting phase. Neither proves provider/parser/consensus defect. Retain fixed-size per-native-resource actual line/prefix/accepted/malformed/oversize counts and observer first/last timestamps; no raw lines/payloads/credentials. Original closed rejection schema remains exact; separate native sidecar only after subscriptions settle under existing16KiB artifact bound. Existing real malformed HTTP→closed warning→healthy SDK/MCP operation verifies accepted native count matches its original rejection.
338+
339+
Canonical flow records closed phase before actual awaits, original SDK task status, token flags and whether an actual marker/receipt was observed. Save immutable pre-cleanup snapshot and terminal status with native file Flush, preserving primary/capture/cleanup failures. No identities/payloads copied; existing marker/ACK/read/replay assertions, deadlines, ownership and cleanup unchanged. This exposes next initiating boundary, not a consensus fix or pass. Integration owns ClusterRouting Helpers/Assertions; root builds/discovers/executes fresh proof. No product hook, alias, format, provider, selector or retry. ADR082 remains Accepted pending actual evidence.

0 commit comments

Comments
 (0)