Skip to content

🤖 fix(history): recover excessively nested persisted tool payloads - #4337

Open
ThomasK33 wants to merge 2 commits into
ThomasK33/plan-review-depth-livefrom
ThomasK33/plan-review-depth-history
Open

ThomasK33 wants to merge 2 commits into
ThomasK33/plan-review-depth-livefrom
ThomasK33/plan-review-depth-history

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Sep 22, 2026

Copy link
Copy Markdown
Member

Summary

Recover persisted dynamic-tool inputs and outputs, including nestedCalls inputs and outputs whose JSON nesting exceeds the safety bound introduced in #4336. Reads substitute a shallow diagnostic in memory, so request building, partial promotion, and compaction do not recursively clone or serialize the offending payload.

This is the persisted-history prerequisite for #4317. It builds on #4336's limit and rejection diagnostic; the live-input guard remains unchanged.

Review focus

  1. valueExceedsDepth walks iteratively. Shallow payloads and ordinary strings remain unchanged, including reference identity.
  2. normalizePersistedMessage covers complete-message reads from history, archives, rewrite rows, and partial.json, including the provider-history scanner.
  3. Raw-byte preservation has a boundary: untouched chat rows and sealed-epoch rotation retain the original bytes. A rewrite targeting a damaged row persists the normalized payload. Promoting a partial writes the normalized row and deletes the raw partial; the original deep partial is not retained. No backup subsystem or new persisted state is added.

Validation

Native-Node tests load depth-2101 and depth-6000 rows, preserve their metadata, commit two consecutive compaction boundaries with tail copies, and compare archived raw bytes. A separate case verifies partial promotion and deletion of the raw partial. A real code_execution nested-call partial first reproduced a native stack overflow on the previous head; it now promotes and compacts with bounded payloads while preserving metadata and healthy sibling calls. Depth-256 data remains a control.

On 04b834f2774d2dc1d3174aa9ccb0837e193eb7ad: make static-check passed, 193 focused Bun tests passed, and all 17 native-Node integration cases passed. Bun 1.3.5, Node 22.19.0, and main's frozen AI SDK 7.0.107 dependencies were used. Combined remote UAT, code/security reviews, and required CI remain merge gates.

Compatibility

This is explicit recovery for over-deep dynamic-tool and nested-call input/output, not a general repair of arbitrary corrupt history. Ordinary payloads are not truncated. The original bytes remain available only where the existing history lifecycle retains them, as described above.


Generated with xum • Model: coder:openai/gpt-6-astra • Thinking: xhigh • Cost: $760.34

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-22T16:01:46.728529Z 04b834f New commits
🔒 Security Review Completed 2026-09-22T16:03:20.456148Z 04b834f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21c770f36e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/utils/tools/toolPayloadDepth.ts Outdated
…ws cannot brick compaction

An already-committed row with a deep dynamic-tool input/output is
re-serialized by every later compaction (boundary + tail copies) and by
partial promotion; V8's JSON.stringify/structuredClone and the SDK clone
overflow the stack, so the workspace can never compact again.

- toolPayloadDepth: explicit-stack `valueExceedsDepth` and
  `boundToolPayloadDepth(row)` replace over-deep `input`/`output` of
  dynamic-tool parts with the existing invalid-JSON diagnostic; same reference
  when nothing exceeds the bound, source row never mutated.
- normalizePersistedMessage composes the legacy metadata migration with that
  bound and replaces normalizeLegacyMuxMetadata at every full-row read in
  HistoryService and historyScanner (chat, archive, rewrite rows,
  partial.json). Untouched rows keep identity, so rewrites emit their raw bytes
  and sealed epochs still archive the original row byte-for-byte; a rewrite
  that targets the deep row itself, and partial promotion (which deletes
  partial.json), persist the flattened row.
- Node/Jest regression (persisted half): depth 2101 and 6000 rows load
  flattened with metadata intact and survive two consecutive
  persistBoundaryWithTailCopies; the archive retains the original deep raw
  line; a deep partial is recovered flattened, promoted, and partial.json is
  removed. Bun unit tests for the walker and the row bound.

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh -->

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…ol payload depth guard

Review finding (PR #4337): boundToolPayloadDepth only bounded a dynamic-tool
part's own input/output. code_execution persists its sub-calls under
part.nestedCalls[*] (NestedToolCallSchema) with their own input/output, so an
over-deep nested payload still reached JSON.stringify/structuredClone on
partial promotion and compaction rewrites.

- Bound the same two known payload keys on each persisted nested call.
  Every other nested field (toolCallId, toolName, state, failed, timestamp,
  workflowRun) is retained; nothing else on the row is walked or coerced.
  Same-reference semantics are kept at every level (row, part, nestedCalls
  array, healthy call) so untouched rows still rewrite byte-for-byte.
- Tests (red on the C2 base): unit — deep nested input/output replaced,
  healthy sibling call and shallow rows keep identity; Jest with the real
  HistoryService — a code_execution partial.json whose nested call carries a
  depth-6000 input reads bounded (parent input, healthy sibling and nested
  metadata intact), commitPartial promotes the bounded row and deletes
  partial.json, and a later boundary commit archives the promoted line
  byte-for-byte.

_Generated with `xum` • Model: `coder:anthropic/claude-fable-5-1` • Thinking: `xhigh`_

<!-- mux-attribution: model=coder:anthropic/claude-fable-5-1 thinking=xhigh -->

Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33
ThomasK33 force-pushed the ThomasK33/plan-review-depth-history branch from 21c770f to 04b834f Compare September 22, 2026 15:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant