Skip to content

Freeze capsule sources and retain durable worker results - #365

Draft
ovasylenko wants to merge 11 commits into
mainfrom
review/capsule-export-cut
Draft

ovasylenko wants to merge 11 commits into
mainfrom
review/capsule-export-cut

Conversation

@ovasylenko

@ovasylenko ovasylenko commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Capsule export and delayed worker preparation can outlive their original execution. This draft freezes captured sources and binds remote work to its original run, activation, mode and continuity owner. Dispatch atomically publishes the actual prepared payload with its effect receipt. A new immutable success journal also preserves the winning result when an acknowledgement is lost and the hot task is collected.

The implementation includes:

  • Atomic source capture, seal, abort and acknowledgement, bounded recovery snapshots, physical retirement evidence, fresh destination identities and guarded source writes.
  • Early preparation fences and atomic effect/payload/queue admission, including fallback handlers, exact retry reservations, immutable specifications and cold deduplication. Initial ownership is enrolled lazily for actual remote work, preserving explicit SDK capsule export.
  • PostgreSQL migrations 129–134 and SQLite schema 65, with retained invocation, admission and accepted-success evidence and downgrade protection.
  • The winning task completion, original sealed output, original invocation and Completed attempt event commit together. Journal insertion failure rolls the completion back. Exact tenant/worker/claim replay acknowledges original acceptance after hot-task GC without accepting regenerated output. HTTP retains its existing hot-history integration retry.
  • Ciphertext privacy, payload fingerprints, redacted debug output, checked completion claim epochs, canonical tenant routing even for nullable or incorrect legacy task mirrors, transactional retry markers and awaited rollback.
  • Repair of the four API failure fixtures from the preceding CI run: seed the actual run and flat-step identity before capturing the dispatch. Invocation checks remain strict.

Latest checkpoint: 0ef2acf9c646994241878f1b8807152393162913. Exact-head CI is pending.

Local validation for the accepted-success checkpoint:

  • 2,145 engine unit tests and 29 distributed scenarios pass.
  • 384 storage cases pass with PostgreSQL required: 60 admission/outcome, 111 invocation, 20 resolution, 136 integration, 42 encryption and 15 migration checks. The outcome matrix covers plain/encrypted PostgreSQL and SQLite, concurrent differing payloads, rollback, original-claim/tenant replay, legacy tenant mirrors, overflow, downgrade retention and SQLite reopen after hot GC.
  • All 149 gRPC and 103 API coverage cases pass, including the lost-acknowledgement regression and four preceding CI failures. Total across these suites: 2,810 passing tests.
  • Strict all-target Clippy passes for storage, gRPC and API. Formatting and whitespace checks pass.
  • Prior admission checkpoint 55f5e6c also passed 177 source-cut, 701 mobile library and 20 embedded facade engine cases locally. Its exact-head CI finished with 24/25 passing; only the four API failure fixtures above failed. These prior results are not current-head CI validation.
  • Earlier capsule/local-migration work passed all 300 continuity E2E cases per PostgreSQL/SQLite backend; those results predate admission and the success journal.

This remains a draft. The journal proves acceptance, not workflow integration. Effect settlement still precedes acceptance in a separate transaction, and accepted paused results still need automatic integration without worker retries. Atomic scoped context/output/node/state application with an Applied receipt, bounded recovery, failure/delegation parity, pending-outcome snapshot handling, issued-claim/reporter authorization and explicit acknowledged archive/compaction remain merge gates.

Legacy identity-only queue APIs do not establish a coordinator protocol floor or unknown-legacy quarantine policy. Invariant audit reconciliation, full PostgreSQL u16 attempt compatibility, backup/key rotation, compensation invocation identity and independent offline-owner handling remain broader gates. The distributed/mobile review and physical lifetime review track these requirements. Passing local tests does not close them. No release is published by this PR.

Draft foundation for continued distributed/mobile/embedded review. Preserve source recovery points and physical identity fences; see the lifetime review for remaining suspension, ownership, archival and reconciliation gates.
Separate local filesystem/memory support from server S3 features and refresh the standalone fuzz graph without dependency upgrades. Validate production mobile outside dev feature unification, server artifact compatibility and all locked workspaces.
Validate original instance and block, replacement scope and optional tree node atomically in fenced resolution and legacy retry. Preserve parent-before-worker locking and outcome evidence. Exercise cross-tenant rejection and valid retry controls; repair export fixtures to configure artifacts and use fresh physical destinations.
Guard native writes, owner changes and active history deletion in PostgreSQL 131 and SQLite 62. Release capture atomically during abort, quarantine old state once, serialize downgrade checks and use existing identity/partial indexes for point admission. Cover controlled native update/delete, owner and downgrade waits, scope and JSON mismatches, rollback and GC. Configure cloud fixture artifacts; keep live migration snapshot and transfer/archive gates explicit in the draft review.
@ovasylenko ovasylenko changed the title Freeze capsule sources and retain physical execution retirement Freeze capsule sources and separate local migration recovery Oct 27, 2026
Resolve dependency guards in the namespace owning the locked source-cut table. A later search_path schema can contain another installation after the local dependency is removed.

Add a regression that downgrades with shadow guards and tables in a fallback schema and verifies they remain untouched. The isolated published-baseline candidate passes all 176 capsule/source-cut tests, formatting and strict test-target Clippy.
Capture run, activation, mode and full ownership before preparation; publish against the original fence and retain retry scope. Validate stranded receipt recovery, await rollback cleanup and preserve gRPC claims on rejected oversized outcomes. Atomic effect publication and durable outcome integration remain draft gates.
@ovasylenko ovasylenko changed the title Freeze capsule sources and separate local migration recovery Freeze capsule sources and fence distributed worker invocations Oct 27, 2026
Pin immutable actual dispatch identity separately from retry reservation, retain the first admitted payload, and protect admitted specifications across hot GC. Enroll initial ownership only during remote publication so pure local SDK execution preserves explicit export ownership. Outcome integration and coordinator protocol gates remain open.
@ovasylenko ovasylenko changed the title Freeze capsule sources and fence distributed worker invocations Freeze capsule sources and atomically admit worker dispatch Oct 27, 2026
@ovasylenko ovasylenko changed the title Freeze capsule sources and atomically admit worker dispatch Freeze capsule sources and retain durable worker results Oct 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant