feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul - #707
Draft
ScriptedAlchemy wants to merge 3447 commits into
Draft
feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul#707ScriptedAlchemy wants to merge 3447 commits into
ScriptedAlchemy wants to merge 3447 commits into
Conversation
🦋 Changeset detectedLatest commit: a2db02c The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
8 tasks
Owner
Author
|
Post-reopen correction pushed in a23f86a: source-neutral background reconcile no longer advances the scheduler epoch and cancel in-flight text activation; mounted hook overflow uses the explicit invalidating path. Evidence: RED reproduced 0/1, GREEN 1/1 for ordinary_background_reconcile_does_not_supersede_in_flight_text_work; cargo check --lib --locked passed. |
💡 Codex Reviewtracedecay/.github/workflows/release.yml Lines 408 to 411 in fc20c4f  Install the bundled Linux runtime beside the binary
For Linux targets, this archive now contains tracedecay/release-please-config.json Lines 22 to 25 in fc20c4f Advance the SDK version in release PRs
The new npm publication job derives its version from the packed tracedecay/dashboard/src/workspaces/delivery/DeliveryPage.tsx Lines 80 to 82 in fc20c4f Scope Delivery queries to the selected project
When the scope bar selects a project other than the active project, this query keeps the same cache key and continues fetching the unprefixed active-project route. The dashboard already provides tracedecay/src/tracedecay/lifecycle/registry.rs Lines 118 to 122 in fc20c4f Include Git remote identity in the registration digest
When AGENTS.md reference: AGENTS.md:L159-L161 tracedecay/plugin/cursor-native-extension/src/extension.ts Lines 187 to 189 in fc20c4f  Batch startup diagnostics instead of dropping the tail
At activation or language-client restart, ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 24, 2026
fix(clippy): clear the workspace clippy failures on #707
This was referenced Aug 25, 2026
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 25, 2026
Removing the re-export shims left their module doc comments orphaned in `context` and `graph`. The rest are pre-existing lints that only fire under `--all-features`, which compiles the `test-transport` support surface as non-test code: - the registered-test-server constructor gets the file's existing explicit `expect_used` allow, since `lib.rs` denies it only outside `cfg(test)`; - `http_application_registry` is read by the `cfg(test)` capacity journey and only reads as dead under `test-transport` alone, so the allow is gated to `not(test)` rather than renamed to a discard; - the hotpath cadence helpers keep `#[inline(always)]` with an explicit allow, preserving the author's intent instead of silently downgrading it. The `daemon_suite` restart wait is now the original helper recovered from the PR #707 head rather than a reconstruction: it additionally proves the runtime readiness receipt authorizes the exact configuration the SDK selected.
This was referenced Aug 26, 2026
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 26, 2026
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 26, 2026
The relocation fix that anchored this test above the package allocated a `PathBuf` only to hand out a reference, which `-D warnings` rejects as `unnecessary_to_owned` and which failed the Clippy gate on #707. `validate_requested_workload` takes `&Path`, and `CARGO_MANIFEST_DIR` is a literal, so `Path::new` yields a `&'static Path` with no allocation and no temporary to borrow from.
Owner
Author
|
#733 landed Clippy-red. Helper branch (not merged, shared checkout untouched):
Clears the #733 Clippy denials:
Adopt when you want: I will not land this on #707. |
Owner
Author
|
Clippy helper is now on the Codex worktrees (still not landed on this PR head):
Same change as |
Owner
Author
|
Isolated post-#721 verification (worktree Green (non-vacuous) on merge
Not green
Clippy helper remains on this tree at |
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 27, 2026
Ports the missing pieces of commit b1e41b712 ("fix(build): land the
batching commit's missing consumers") onto #707: HookCompletedReadinessDistributions
moved into tracedecay-agent-hosts with pub(crate) fields, so
input_rows_processed, input_rows_dropped_at_cap, and events_considered
are no longer reachable outside the crate. Adds the three public
accessors.
source_event and input_rows_received (also added by the source commit)
are not ported: nothing on this branch reads them through an accessor
today - the root-crate benchmark still compares via
serde_json::to_value(&readiness_distributions)["field"], which does
not need them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cursor
Bot
changed the base branch from
master
to
cursor/vendor-split-707-d6ed
August 27, 2026 05:09
cursor Bot
pushed a commit
that referenced
this pull request
Aug 27, 2026
Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
cursor
Bot
force-pushed
the
cursor/vendor-split-707-d6ed
branch
from
August 27, 2026 05:52
5d8f2a1 to
c7fcf3f
Compare
ScriptedAlchemy
added a commit
that referenced
this pull request
Aug 27, 2026
…d6ed chore(vendor): stacked vendor-only PR for #707 — generated vendor churn, land first
ScriptedAlchemy
changed the base branch from
cursor/vendor-split-707-d6ed
to
master
August 27, 2026 06:24
cursor Bot
pushed a commit
that referenced
this pull request
Aug 27, 2026
…rules Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
refactor(host-integration): remove duplicate evidence fixture
The filesystem payload authority only offered whole-buffer verified reads, so any consumer that wanted the pre-emission integrity proof had to hold the entire payload (up to the 64 MiB file ceiling) in memory first. Add `VerifiedPayloadStream`, a two-pass reader over one open handle: - `open` proves stable identity, byte count, SHA-256, and UTF-8 scalar count by reading the file once through a caller-provided window, then re-checks that the path still names the proven inode at the proven size. Holding a value is evidence the proof passed; there is no other constructor. - `emit` consumes the stream and re-reads the same handle from the start, handing each filled window to the consumer. The identity/size recheck runs before the first window and after the last, so a payload replaced or removed between proof and emission is refused before any byte crosses; the emitted bytes are also re-hashed against the proof, so a same-inode rewrite fails the emission instead of succeeding with mixed bytes. Peak transient memory is one window in either pass. The consumer's checkpoint runs before every read and its typed outcome (cancellation, deadline, budget, sink refusal) is returned verbatim through `PayloadStreamError::Consumer` instead of being flattened into an `LcmError`. `open_verified_payload_stream` applies the same payload-ref, directory, and containment checks as `read_verified_payload_content`. The incremental hash/UTF-8/char-count scanner is factored out of `scan_utf8_content` into `ContentScanner`, which the existing whole-buffer path now uses as well. Refs #1034.
`TemporalHydrationPort::read_authorized` is chunk-oriented, but the backend contract behind it, `read_bounded -> Zeroizing<Vec<u8>>`, materialized the whole authorized payload, verified it, then chunked the buffer into the sink. `max_chunk_bytes` bounded callback granularity, not peak memory: a file-backed payload cost the verified read buffer plus a second `bounded_copy` before the first chunk moved. Replace the contract with `open_bounded -> BoundedPayload`: - `Owned(Zeroizing<Vec<u8>>)` for row-engine payloads (summary text, the canonical projected message, inline raw content). The buffer the engine already produced is moved, not copied; the adapter proves it against the descriptor and chunks it in place. `bounded_copy` is deleted. - `File(VerifiedPayloadStream)` for payload files. The LCM authority proves identity, byte count, hash, and char count through one open handle inside a 64 KiB proof window, and the adapter emits by re-reading that same handle through one chunk-sized window. Pre-emission authentication is unchanged in shape: the adapter still refuses before any sink callback when the opened payload does not match the descriptor, then re-resolves the anchor on mutable backends. A file replaced or removed between proof and emission is refused before the first window; a same-inode rewrite fails the emission re-hash, and `hydrate_selected` discards the sink on any error. Cancellation, deadline, and work-budget checkpoints run before every window in both passes and return typed `Interrupted` outcomes; the stream and window are RAII so an interrupted run retains no handle or buffer. `BudgetExceeded` and `Unavailable` outcomes are preserved. `session_temporal.hydrate.verified_bytes` and `session_temporal.hydrate.emitted_bytes` gauges separate proof work from emission work so discarded proofs stay visible. Measured with a thread-local live-byte allocator over `read_after_recheck` for a 1,048,575 B payload (one under the default 1 MiB `hydration_payload_bytes`) in 64 KiB chunks, hashing sink, same fixture on both trees: before 2,100,264 B peak transient (2.00x payload) after 68,930 B peak transient (1.05x chunk window, 30.5x less) Refs #1034.
`session_message_from_hydrated_bytes` copied the hydrated bytes into a `String` before it had looked at the occurrence row, so every refusal (scope mismatch, missing row, role/session drift, duplicate occurrence) paid a full payload copy for nothing, and a successful reconstruction briefly held two owned copies beside the caller's hydrated buffer. Validate UTF-8 in place and construct the record's owned text once, only after the row binding has accepted the occurrence. The request still borrows the caller's `TemporalHydratedResult`, so the one construction on the success path remains; making that zero-copy would need `SessionPageReconstructionRequest` to take ownership through `tracedecay-temporal-query` and `tracedecay-session-runtime`. Refs #1034.
Source editing carried two descriptor-scoped file authorities with the same open/read path: `SourceEditCandidateAuthority` in tracedecay-usecases (read side, used by the state digest) and `SourceEditFileAuthority` in the composition root (read + atomic publish). Recovery and rollback lived in the root as `TraceDecay` methods reached through three pass-through `SourceEditRuntimePort` methods, so the crate that owns the journal did not own the code that restores what the journal records. Move the root file authority and the publish/rollback/postimage-commit authority into tracedecay-source-edit as the single owner. The state digest, crash recovery, caller-requested rollback, and every root edit primitive now go through that one authority. Delete the usecases duplicate (its tests fold into the unified module), drop the three recovery methods from the port and their `TraceDecay` pass-throughs, and remove the now-unused cap-fs-ext dependency from tracedecay-usecases. Journal bytes, digests, and hotpath labels are unchanged.
tracedecay-source-edit reached upward into tracedecay-usecases for `SourceEditRuntimePort`, the admitted graph handle, the task-local preview/apply plan authority, `PlannedSourceEditFile`, and its journal lock, so the crate that owns the journal depended on the broad use-case spine for the types it journals. Move the port, `SourceEditGraphReadV1`, `EditDiagnosticRecord`, `PlannedSourceEditFile`, and the plan capture/apply task-locals into tracedecay-source-edit as `port` and `plan`. The journal lock now uses runtime-core's `try_acquire_sidecar_lock` instead of a private fs2 copy. Repoint the composition root's port impl and edit primitives at the new owner, drop the tracedecay-usecases dependency from tracedecay-source-edit, and remove fs2 plus the emptied source-edit module surface from tracedecay-usecases, which keeps only its store runtime port and branch diagnostics. `cargo tree -p tracedecay-source-edit -e normal` no longer lists tracedecay-usecases. Journal serde shape and hotpath labels are unchanged.
…plan-reopened' into sol/layer-names-1075
The daemon opens every project's model lifecycle through the scoped owner (2f17bc8), whose acquisition imports the verified package into the shared artifact inventory. That path published the inventory's content address (`ModelArtifactManifestV1::artifact_identity_digest`, which also hashes host-derived resource ceilings) as the lifecycle `artifact_digest`, while the download/verify states before it, the private-root install path, shared-inventory discovery, and every projection identity and compatibility pin carry `catalog_package_digest`. `SemanticCompatibilityPinsV1::valid` requires the pins to equal the projection's `model_artifact_digest`, so the pins were canonical and the lifecycle was the side that drifted: `inspect_generation` and `evaluation_lifecycle_verification` compared the two and refused every evaluation with `inspect_generation.artifact_digest` whenever the byte-pinned FastEmbed fixture was installed (#1097). The lifecycle now names the catalog package it installed on every path. The inventory address stays private to the store: `installed_digest` is the inverse of `installed_directory`, and lease reconciliation, re-admission after restart, and rollback recover it from the install directory instead of reading it out of the lifecycle digest. Explicit local imports keep their inventory-addressed identity. Refs #1097
The cold-load resident bound samples the whole process, but each route of a project (and each project) owns its own session pool. Two linked worktrees publishing together each cold-loaded the same model at once, each observed the other's ~1.3 GB of growth on top of its own, and both were refused with `cold session load resident ceiling exceeded` against a 2 GiB ceiling that a single load fits under — the primary and linked projections of the retention journey failed within 2 ms of each other. Cold opens now take one process-wide turn, so the measured growth is attributable to the open it bounds and the transient peak is one load at a time. Waiting for the turn is admission, not load time: the load deadline and the recorded load duration start once the open begins, and the turn is released when the open returns or its own verdict fires. The synthetic kernel-sampler test reads growth from the lowest RSS it has seen, because with turn-taking it now starts right after another test's real model load returned and that test drops its session inside the window, which would otherwise hide the synthetic allocation behind the larger foreign release. Refs #1097
Batch splitting and vector preparation erased their `SemanticProjectionErrorV1` into the bare `Projection` failure, so a lifecycle marked failed by a projection reported only "semantic runtime Projection" and an operator could not tell a resident-ceiling refusal from a foreign chunk digest. Carry the error through `SemanticRuntimeScheduleFailureV1::projection`, which already exists for exactly this. Refs #1097
A linked worktree opened without the `sync.watch_linked_worktrees` opt-in carries `LinkedWorktreeDisabled` admission (f347a0a): the route serves but never indexes, and the daemon's own deferred owners answer that admission at spawn time (3b0d7c4). The production composition harness still awaited the route's first publication and exhausted its 20 s wait with "did not publish" for a generation that admission forbids. Answer the same typed state the daemon does and let reads report `linked_worktree_disabled`. Refs #1097
The activation journey injected its live install failure and retry through the `SemanticModelLifecycleOwnerV1` it resolved before the in-process daemon restart. That handle aliased the live daemon only while the owner was a process-wide singleton (`shared_lifecycle_owner`, removed in 2f17bc8); the session registry now owns one instance per composition, so the restarted daemon served a different owner and the public runtime state stayed `installed`. A retired composition's handle is not the daemon's lifecycle: resolve the owner the restarted registry retains before observing or mutating it. Refs #1097
The linked-worktree retention journey predates f347a0a and assumed a linked checkout is indexed automatically. Since then linked-worktree indexing is an explicit project opt-in (`sync.watch_linked_worktrees`, a daemon-restart setting), so the journey commits that opt-in through the public configuration authority before any composition mounts the linked route. Both checkouts share one project configuration, so each coordinated semantic transition converges before the sibling checkout swaps the profiles. Refs #1097
`CodeLexicalProjectionAdapterV1::new_published` had no production caller since search-eval moved to one-pass scoped partitioning through `new_admitted` (#1045); only the search-quality suite still used it. Delete it and give the suite a generation-backed helper over `new_admitted`, leaving one shared-source constructor. `new_admitted`, `new`, and `CodeLexicalProjectionBuildV1::new_admitted` now take the projection metadata and the qualified-names map as `impl Into<Arc<_>>`, and the adapter and build retain them as `Arc`s. search-eval's `build_query_projections` builds the metadata once and passes the corpus-wide names map through as one `Arc`, so every scoped projection shares one immutable copy instead of cloning the metadata and slicing the names per scope. Existing by-value callers compile unchanged. The packaged search-quality digests are untouched: `WORKLOAD_SHA256` and the search-eval `report_tests` pins pass as before. Closes #1102.
The `cobol` grammar vendored in `tokensave-large-treesitters` 0.5.0
(yutaro-sakamoto/tree-sitter-cobol, scanner as of `c7a36d7`) skips the
sequence-number area with `while (get_column() <= 5) advance()` and never
tests for end of input. Any token starting at column 0-5 with no line of
six or more columns before EOF reaches EOF, where `advance` is a no-op,
and spins forever: `"x"`, `"x\n"`, `"abcde"`, and a well-formed program
followed by a short trailing line all hang; `"abcdef"` and area-A/B code
parse.
Measured through `RetainedParseDocument::open("cobol", "x")` with a
250 ms budget: the parse never returned. tree-sitter polls the progress
callback only between parse actions, so the retained parser's cooperative
deadline cannot reach a scanner that never returns from one call. Record
the input class at the grammar boundary, the deadline's reach at its
definition, and gate the `end_line` equivalence shapes on the grammar fix
(upstream yutaro-sakamoto/tree-sitter-cobol#42, unmerged).
Refs #1104.
…sign-plan-reopened
The vendored tree-sitter-cobol external scanner in tokensave-large-treesitters 0.5.0 skips the sequence-number area with 'while (get_column() <= 5) advance()' and never tests for end of input, so any token starting in columns 0-5 with no six-column line before EOF spins forever, and tree-sitter never polls the progress callback inside a scanner call, so the retained parser's deadline cannot interrupt it. A COBOL file in that class pinned an indexer worker until restart. Pin the bundle to ScriptedAlchemy/tokensave-large-treesitters@102d010, which is v0.5.0 plus the upstream fix (yutaro-sakamoto/tree-sitter-cobol PR 42, still unmerged), and add cobol to the end_line shape test that was gated on this defect. The lockfile re-resolve also moves ten Windows-only windows-sys edges from 0.52 to 0.59; both versions were already locked, no package was added.
…esign-plan-reopened
`retrieval_profile_mutation_capability` collapsed every configuration error from the mutation-authority recheck onto the bare `Rejected`, so a refused coordinated transition told the operator only "semantic configuration transition rejected". The recheck compares the grant's configuration revision with the revision the semantic profile state pins; carry both and the authority's own error through `RejectedDetail`, which the daemon already renders into the problem message. This is what exposed the per-scope semantic state pinning a revision the sibling worktree's transition had already moved past (#1097 follow-up). Refs #1097
…ign-plan-reopened
…plan-reopened' into sol/layer-names-1075 # Conflicts: # Cargo.lock # crates/tracedecay-application/src/tracedecay/mod.rs # crates/tracedecay-application/tests/source_edit_candidate_authority.rs # crates/tracedecay-source-edit/Cargo.toml # crates/tracedecay-source-edit/src/digest.rs # crates/tracedecay-source-edit/src/execute.rs # crates/tracedecay-source-edit/src/journal.rs # crates/tracedecay-source-edit/src/lib.rs # crates/tracedecay-source-edit/src/plan.rs # crates/tracedecay-source-edit/src/reconcile.rs # crates/tracedecay-source-edit/src/rollback.rs # crates/tracedecay-source-edit/src/verify.rs # crates/tracedecay-usecases/Cargo.toml # crates/tracedecay-usecases/src/lib.rs # crates/tracedecay/src/tracedecay/edits/primitives.rs # crates/tracedecay/src/tracedecay/edits/rename.rs # crates/tracedecay/src/tracedecay/edits/rename/graph_evidence.rs # crates/tracedecay/src/tracedecay/edits/symbols.rs # crates/tracedecay/src/tracedecay/move_symbol/mod.rs # crates/tracedecay/src/tracedecay/project_runtime_port.rs
refactor(architecture): name contracts and application layers
…plan-reopened' # Conflicts: # crates/tracedecay/src/daemon/production_harness/generation_retention_test.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replacement review for #421 after its accidental merge was reverted from master in 52a9aab. This commit has the exact tree of #421 head d4c67d4. Do not merge until explicitly authorized.
Issues this PR fixes
Refs #753 (source-identity + restart journey tests landed on branch at eae9328; keep draft until full product journey/CI)
Refs #792 (journey acceptance pending)
Refs #837 (journey acceptance pending)
Refs #838 (journey acceptance pending)
Refs #855 (journey acceptance pending)
Fixes #862
Fixes #861
Fixes #810
Fixes #845
Fixes #856
Fixes #853
Fixes #799
Fixes #836
Fixes #830
Partially addressed here, tracked to completion on their own issues: #852, #800, #863, #843, #842, #848.
🤖 Generated with Claude Code
#887 dead-surface deletion inventory
build_statecompatibility wrapper was deleted; the liveget_gc_metasuppression was removed.clean_c_comment/clean_c_doc_commentare retained because feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 now has production callers in C, C++, GLSL, Go, and Objective-C extractors.cargo check --workspace --all-targets;cargo clippy --workspace --all-targets -- -D warnings; both pass on the integrated branch.#888/#890/#891/#892 workflow cleanup
using-tracedecaybootstrap was deleted, detailed mechanics moved to live tool help, and unique safety/domain rules remain narrowly routed.no_skill_needed, and keep only actual host size/syntax limits.