Repository navigation
feat(core): commit a staged inventory (#445) - #1011
Conversation
Stage one left a finished inventory in a pending directory that nothing could use. commit_streamed_inventory_capture promotes its pages into the digest directory of the successor one page at a time, publishes the successor and advances the root binding, in the order and under the guarantees of commit_inventory_capture. promote_staged_inventory_fenced refuses before anything is read or written unless the caller is the committed owner of the exact root-named manifest and the successor is its capture successor, then reads each staged page back, confirms it against its reference, absorbs its entries into the inventory digest and stores it where the in-memory store would have put the same bytes. A page found wrong midway leaves an inert verified prefix and publishes nothing. The staged files are removed only after the root has committed, so a retry after a failed root commit still has them and adopts what is already durable. commit_inventory_capture and the new commit now share one core, which takes the step that stores the pages as an argument.
🤖 CodeAnt AI — Review Status
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Reviewer's GuideThis PR completes streaming capture stage two (a): it promotes staged pages into the successor’s durable inventory set, then publishes and binds that successor through a new root-committed API while retaining staged files for safe retries and sharing commit ordering with the existing in-memory capture path. Sequence diagram for committing a staged inventory capturesequenceDiagram
participant Caller
participant Authority
participant Root
participant Journal
participant StagedCapture
Caller->>Authority: commit_streamed_inventory_capture()
Authority->>Authority: check_operation_id()
Authority->>Root: acquire root_commit_mutex
Authority->>Root: read committed binding
Authority->>Journal: resolve_journal_key()
Authority->>Journal: promote_staged_inventory_fenced()
Journal->>Journal: assert_page_promote_authority()
Journal->>Journal: assert_root_named_manifest()
Journal->>Journal: assert_capture_successor()
loop each staged page
Journal->>StagedCapture: load_staged_page()
Journal->>Journal: store_page_at()
end
Journal->>Journal: publish_manifest_fenced()
Authority->>Root: advance binding as Capture
Root-->>Authority: committed
Authority->>StagedCapture: remove_files()
Authority-->>Caller: RootCommitted
Flow diagram for staged inventory promotion and retry safetyflowchart TD
A[Finished StreamedCapture in inventory/pending-*] --> B[commit_streamed_inventory_capture]
B --> C{Metadata and ownership checks pass?}
C -- No --> D[Refuse before reading or writing]
C -- Yes --> E[load_staged_page]
E --> F{Page matches its reference?}
F -- No --> G[Return Corrupt; keep staged files]
F -- Yes --> H[Absorb entries into inventory digest]
H --> I[store_page_at in successor digest directory]
I --> J{More pages?}
J -- Yes --> E
J -- No --> K{Inventory digest matches successor?}
K -- No --> L[Do not publish manifest]
K -- Yes --> M[publish_manifest_fenced]
M --> N[Commit root binding as Capture]
N --> O[Best-effort remove staged files]
N --> P[Retry can reuse staged files if root commit fails]
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
🏁 CodeAnt Quality Gate ResultsCommit: ✅ Overall Status: PASSEDQuality Gate Details
|
codeant-ai
Bot
added
the
size:XL
|
[check-pr-size] PR size is over the target tier (normal profile): 13 files, 1027 meaningful lines, 3 commits — limit ≤8 files / ≤400 lines / ≤6 commits. Consider splitting into smaller, independently reviewable PRs. |
![codescene-access[bot]](https://avatars.githubusercontent.com/in/3755875?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Warning Review limit reachedEnable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Next included review available in 44 seconds. View limit detailsLimit details: You’ve used the included review currently available. Your 68 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Review configuration: ⚙️ Run configuration
📒 Files selected for processing (5)
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (13)
Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour. 📝 WalkthroughWalkthroughThe change adds fenced promotion of staged inventory pages and a streamed capture commit that shares the existing capture commit sequence. Staged files remain available when the commit fails and are removed after root commitment. Key-routing staging remains separate work. ChangesStreamed Inventory Capture
Priority: ⬇️ Low Merge Risk: ⚪ Minimal · up to No concrete issue remains that needs resolution before merge; the documented promotion and retry behavior can proceed through normal checks.
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c0b776506b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
CodeAnt PR Risk: Medium Risk
Assessed commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ed in (#445) A finished StagedCapture did not remember its journal directory. Handed to a context for another directory that held an identical copy of the root-named manifest, every manifest check passed, the staged pages were read from the original directory by absolute path, and they were written and published into the other one. The finished handle now carries the directory it was staged in and the promotion refuses any other before it reads anything. With that binding in place the old capture-successor case, which promoted into another journal, is refused earlier than it intended; it now tests the relation the way it can actually fail, a journal that moved on after the capture began.
|
@codex review |
|
@CodeAnt-AI review |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
The maintainer's S2a acceptance text asks for a wrong context, a changed, missing or swapped staged page, a partial publish, a retry and exact page-set parity with the in-memory capture. The earlier head covered most of it; this adds what was missing: a promotion under another key, a swapped staged page (the three wrong-page cases are now one table), a failure between the promoted pages and the manifest that is retried with the same staged capture, and the parity of the committed manifest and of every stored page with what the in-memory capture builds from the same sealed pages.
|
@codex review |
|
@CodeAnt-AI review |
There was a problem hiding this comment.
Gates Passed
See analysis details in CodeScene
Quality Gate Profile: The Bare Minimum
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
Final disposition census — head
|
| Finding | Disposition |
|---|---|
Codex P2: a finished StagedCapture did not remember its journal directory, so a context for another directory that held an identical copy of the root-named manifest would have copied the staged pages across |
VALID_AND_FIXED in c9af414a — the handle carries its directory and the promotion refuses any other before reading anything; tested with exactly that identical-copy directory; mutation-checked. The old capture-successor case promoted into another journal and was refused earlier than it meant to, so it now tests the relation as it can fail (a journal that moved on after the capture began) and is mutation-checked again |
| CodeAnt (Major label): the directory binding compares path spellings, so a relative and an absolute spelling of one directory are refused | VALID_AND_DEFERRED_WITH_ACCEPTANCE_CRITERION — fails closed (refused before any read, the original spelling works); a canonicalisation is a file-system call outside DurableFs; the structural fix is S2b's session, which holds the directory value once for staging and promotion |
Acceptance list from the maintainer's rewrite of the QNB-11 S2a section (19:24, after the admission): wrong context, tampered / missing / swapped page, partial publish / crash, retry, exact committed page-set parity with the in-memory capture. The first head lacked a promotion under another key, a swapped staged page, a failure between the promoted pages and the manifest, and the parity check; 52418681 adds them, each mutation-checked.
Decisions flagged on the admission and not changed by review: one pass; the commit takes &StagedCapture; a second entry point beside commit_inventory_capture (they now share one core, commit_capture).
Disclosed limits of the proof: two checks have no test that fails without them through the public API (the staged references against the successor's page set, and the final inventory-digest comparison), because StreamedCapture::finish cannot produce a successor that disagrees with its staged references; both mutations were run and survive, and the evidence says so. The authority check and the root-named check overlap in the error class they give for a stale owner, so the authority check is pinned by asserting that nothing is read before it refuses.
Proof: the whole crate suite (50 passing results), clippy -D warnings, fmt --check, docs:check.
Scope: promotion and the composed commit only. No key-routed staging session (S2b), C2, reclamation, graphify. PRODUCTION_AUTHORITY_SWITCH_ALLOWED = NO.
1 participant
User description
What
Stage two (a) of the streaming capture: the step that makes a staged inventory reachable. Stage one (#1010) leaves the pages of a finished inventory in a private
inventory/pending-*directory; nothing could promote them, socommit_inventory_capture(every page in memory) was still the only way to capture.promote_staged_inventory_fencedload_staged_page(digest first, then open), the entries absorbed into the inventory digest,store_page_atintoinventory/<digest>/page-<i>/(an identical page is adopted, a different file never replaced). Finally the inventory digest must equal the successor'scommit_streamed_inventory_capturecommit_inventory_capture's order and guarantees: operation id, fence, root lock, committed binding, journal key routed from the root, the promotion,publish_manifest_fencedas the capture,BindingStep::Capture. The staged files are removed (best effort) only after the root has committed, so a retry after a failed root commit still has them and adopts pages and manifest; after a success the handle is spentcommit_inventory_captureand the new commit share one core (commit_capture) that takes the step storing the pages as an argument, so the order of operations under the root lock exists once. Memory stays one page.Decisions to review (mine, flagged on the admission): one pass, so a staged page found wrong midway leaves an inert verified prefix under the digest directory (as an I/O failure partway through the in-memory store already does) and no manifest is published, instead of two passes over the staged data; the commit takes
&StagedCaptureso a retry has the files; a second entry point besidecommit_inventory_capture, not a replacement.Boundary matrix
Corrupt, neverRecoveryRequired), an unroutable key epoch, no bound migration, a token that is not the successor's: each before a manifest is publishedNot in this PR
Stage two (b), the authority-level staging session that routes the journal key from the root (staging still takes the key from its caller), the C2 driver, reclaiming abandoned pending directories, inheriting unchanged pages, the write barrier. No production authority switch.
Verification
Local: on the journal alone, a promoted capture publishes and binds as a stored set that
verify_stored_inventoryaccepts with the same references and manifest, a repeated promotion adopts what is stored, a stale owner and another revision are refused before anything is read, a manifest the root does not name and a capture built over another phase are refused with nothing created, and a changed or missing staged page stops the promotion with only a verified prefix stored. With a real root, the commit names the successor, its pages verify and the staged files are gone; a stale token and an unbound migration are refused with the journal tree and the root unchanged and the staged files kept; a changed staged page refuses the commit before any manifest is published; a failed root commit is retried with the same staged capture, which writes nothing new into the journal. The key route refuses the new commit with a revoked or unregistered epoch and a route to another key, beside the other journal-owner operations. Mutation-checked: the root-named, capture-successor and promote-authority checks (the last one made observable by asserting that nothing is read before it refuses), the digest directory, the inventory digest absorption, keeping the staged files after a success, removing them before the root commit, and a fixed journal key in the shared core each fail the test that owns them. Disclosed: two checks have no test that fails without them through the public API (the staged references against the successor's page set, and the final inventory-digest comparison), becauseStreamedCapture::finishcannot produce a successor that disagrees with its staged references; both mutations were run and survive, and the evidence says so. The whole crate suite (50 passing results),cargo clippy --all-targets -D warnings,cargo fmt --check,pnpm run docs:check.Part of #359 and #445 (Linear QNB-11). Admission: #359 issuecomment-6067292231.
PRODUCTION_AUTHORITY_SWITCH_ALLOWED = NO.Summary by Sourcery
Enable streamed inventory captures to be promoted and committed page by page with fenced validation, durable retry behavior, and root binding.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests:
Chores:
Summary by cubic
Adds
commit_streamed_inventory_capture, which promotes a staged capture's pages into the digest directory, publishes the successor manifest, and advances the root binding — in the same order and under the same guarantees ascommit_inventory_capture. Both commits now share one core,commit_capture, which takes the page-storing step as an argument.promote_staged_inventory_fencedrefuses before any read or write unless the caller is the committed owner of the exact root-named manifest, the successor is its capture successor, and the capture belongs to the journal directory being promoted into. A finished capture now remembers the directory it was staged in, so a byte-identical copy of the root-named manifest in another directory can no longer trap the promotion into copying its pages elsewhere. Each staged page is then read back, confirmed against its reference, absorbed into the inventory digest, and stored one page at a time; an identical page already there is adopted, a different file is never replaced. A staged page found wrong midway leaves an inert verified prefix and publishes nothing. The staged files are removed only after the root has committed, so a retry after a failed root commit still has them and adopts what is already durable; after a success the handle is spent.Tests cover the promotion on the journal alone and the composed commit against a real root: refusal before any read or write, a wrong context holding an identical manifest, changed, swapped or missing staged pages, a partial publish retried with the same staged capture, a failed root commit retried without new journal writes, and exact page-set parity with the in-memory capture.
Written for commit 5241868. Summary will update on new commits.
Summary by CodeRabbit
CodeAnt-AI Description
Commit staged inventory captures with safe retries
What Changed
Impact
✅ Lower memory during inventory commits✅ Fewer lost captures after commit failures✅ Prevents incomplete inventories from being published💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.