|
| 1 | +# ClusterRouting: bounded partition record pages |
| 2 | + |
| 3 | +Status: first implementation stage for KL-036/071/072; complete movement remains |
| 4 | +unqualified. Decisions: [ADR-016](../../ADR/ADR-016-atomic-physical-placement.md) |
| 5 | +and [ADR-017](../../ADR/ADR-017-migration-tokens.md). Initial serving remains RF3. |
| 6 | + |
| 7 | +This stage reads exact canonical key/value bytes from an already owned committed |
| 8 | +`IKeyValueView`. The node-local caller retains that cut for all its pages. It |
| 9 | +changes no existing persisted format, public API, placement or writer authority. |
| 10 | + |
| 11 | +| Requirement | Acceptance criterion and real oracle | |
| 12 | +|---|---| |
| 13 | +| REQ-PMOVE-001: use a closed source-owned partition-family inventory | AC-PMOVE-001: actual partition-key constructors are covered; every selected family uses `KeySpace.Partition` with the four complete partition components. Unknown families, invalid partitions and foreign continuation keys fail before copying. Global catalog, principal/API keys, global outcomes, physical watermarks and shared blob accounting are explicitly excluded. | |
| 14 | +| REQ-PMOVE-002: bound retained bytes, records and examined work independently | AC-PMOVE-002: real native ZoneTree pages cover empty/inclusive/one-over bounds, exact keys/values/order and charged lookahead. Retained bytes include every returned key/value buffer and the independent continuation buffer. Checked reservation precedes every copy, including continuation; an overlarge record or examined-byte exhaustion throws typed BudgetExceeded with no successful partial page. No full Scan, payload decode or whole-store materialization. | |
| 15 | +| REQ-PMOVE-003: preserve caller-owned cut, cancellation and identity | AC-PMOVE-003: paginated reads inside one actual native view reproduce exact selected records; equal key text across tenant/database/domain stays isolated. Cancellation before/during traversal settles the original call with no successful partial image. Reopen preserves bytes; grains acquire no file owners. | |
| 16 | +| REQ-PMOVE-004: a family page cannot stand in for a complete movable image | AC-PMOVE-004: no installer or public move endpoint exists until partition outcomes, shared catalog/authorization, blob accounting, retained cursor/transfer state and derived-index readiness have an accepted complete protocol. Later process-kill/fencing/RF3 SDK/MCP gates remain mandatory. | |
| 17 | + |
| 18 | +The inventory covers documents/indexes/unique keys and epochs, graphs and |
| 19 | +adjacency, vectors/lineage/effects, samples/sequence/dedup/retention, blob heads/ |
| 20 | +uploads/parts/metadata, every queue index/body/metadata/counter/inbox, recurring |
| 21 | +schedules/sagas/capacity, subscription state/window/completion/inbox, both remote |
| 22 | +transfer endpoints, event streams/topics/identities/sequence/feed/snapshots, |
| 23 | +outbox/heads/consumers/receipts and visibility epochs. Enumerate actual current |
| 24 | +constructors, including dynamic family arguments. There is no universal encoded |
| 25 | +partition prefix: the family precedes the four partition components. |
| 26 | + |
| 27 | +## Frozen reader contract and ordered ownership |
| 28 | + |
| 29 | +TASK-PMOVE-PAGES: root owns architecture, source inventory and joins. Luna |
| 30 | +implementation owns only new internal Core ClusterRouting Queries/Contracts/ |
| 31 | +Validation files prefixed `PartitionRecord`. Exact contract: |
| 32 | +`PartitionRecordPageReader.Read(IKeyValueView view, PartitionRef partition, |
| 33 | +string family, int maxRecords, long maxRetainedBytes, long maxExaminedBytes, |
| 34 | +ReadOnlyMemory<byte> afterKey = default, CancellationToken cancellationToken = default)` |
| 35 | +returns an internal immutable `PartitionRecordPage` containing owned |
| 36 | +`ImmutableArray<KeyValueRecord> Records`, `bool HasMore`, `long RetainedBytes`, |
| 37 | +`long ExaminedBytes` and optional independently owned `ReadOnlyMemory<byte>` |
| 38 | +continuation. `PartitionRecordFamilies.All` is an immutable ordinal inventory. |
| 39 | +Use native VisitRange, charge its real observer before copying, verify the |
| 40 | +exclusive continuation belongs to this exact prefix and has canonical KeyCodec |
| 41 | +encoding, and preserve raw bytes. Validate the continuation's bounded length |
| 42 | +before decoding its key components; user value payloads are never decoded. |
| 43 | +Reserve and copy continuation independently from the last returned key; its |
| 44 | +bytes contribute to RetainedBytes and maxRetainedBytes. Native accounting and |
| 45 | +unexpected visitor stops fail closed. |
| 46 | +Positive bounds must be checked before traversal; overflowing counters fail |
| 47 | +closed. There is no serialization or inter-grain DTO in this first stage. |
| 48 | + |
| 49 | +TASK-PMOVE-PAGE-ORACLES: an independent Luna worker owns only new UnitTests |
| 50 | +ClusterRouting Cases/Helpers prefixed `PartitionRecord`. Derive the criteria |
| 51 | +above against actual ZoneTree/TestDatabase primitives, without mocks, fake view, |
| 52 | +skips or weakened limits. Preserve callbacks' borrowed lifetime. Root reviews |
| 53 | +the complete private packets, then executes Aspire normal/scalar tests. |
| 54 | + |
| 55 | +Complete ownership is a later root-owned stage. Current epoch7 outcome keys have |
| 56 | +no partition locator; their hash cannot recover one. Copying every global outcome |
| 57 | +or dropping outcomes is incorrect. Existing-store migration needs its exact |
| 58 | +accepted upgrade/rollback contract. Current persisted authorization remains |
| 59 | +authority and an acknowledged revocation must fence all serving groups. Source |
| 60 | +and destination group indexes are never directly comparable; explicit ownership |
| 61 | +epoch invalidation is the first candidate under KL-072, pending full freeze. |
| 62 | + |
| 63 | +Slice map: Core owns this internal pure reader in ClusterRouting; Server's |
| 64 | +StorageRecovery retains native views/files and Replication retains ordered |
| 65 | +commit authority. Later Orleans orchestration uses one request grain and native |
| 66 | +ManagedCode.Communication asynchronous streams with bounded handoff. Public |
| 67 | +contracts, SDK/MCP, frontend/admin controls are N/A for this internal stage; |
| 68 | +their later actual move contract must be specified and qualified. |
| 69 | + |
| 70 | +Verification: full strict Release build, formatter/governance, genuine Aspire |
| 71 | +unit and unit-scalar focused PartitionRecord cases for local development, then |
| 72 | +exact-source Linux CI. Later movement requires process cuts at every persisted |
| 73 | +state and Docker/Aspire RF3 SDK/MCP recovery, fencing, token and receipt oracles. |
| 74 | +Internal pages alone do not satisfy those movement acceptance criteria. |
| 75 | + |
| 76 | +```mermaid |
| 77 | +flowchart LR |
| 78 | + Owner[Node local committed view] --> Family[Closed family and full partition prefix] |
| 79 | + Family --> Budget[Reserve examined and retained bytes] |
| 80 | + Budget --> Page[Exact owned bounded page] |
| 81 | + Page --> Later[Later complete ownership and fenced transfer] |
| 82 | +``` |
0 commit comments