[devhims]

Problem / Motivation

Visual questions lacked a consistent inspection workflow. Follow-up answers could confuse retrieved images with reported finding times or invent reasons for selecting frames. Ordinary answers could also include unnecessary narration about transcript reads and other tool use.

This PR depends on PR #211. Its base branch mirrors that PR's branch, currently at 48c27b8, which includes current main. Merge #211 first, then retarget this PR to main.

What changed

The visual-inspection skill guides locating subjects, using storyboard overviews, checking exact frames and reusing saved analysis. Frame requests record one basis category per timestamp, such as a spoken introduction or a storyboard overview, with the locating references it rests on. Free-text reasons are not accepted, so recorded selections carry no source content.

Structural records distinguish images supplied to analysts from timestamps mentioned in findings. Explicit workflow follow-ups use these records and identify unknown selection reasons. Ordinary answers present findings, citations and material uncertainty without unsolicited tool narration.

The separate semantic checker is removed entirely. Answers use existing schema and citation validation without a reviewer model call or duplicated reviewer prompt. Valid quote-only context and question memories are accepted without generated text.

The retained answer path uses the new inspection context:

%%{init: {'theme': 'base', 'themeVariables': {'actorBkg': '#e8eef6', 'actorTextColor': '#172b4d', 'actorBorder': '#52657a', 'signalColor': '#52657a', 'signalTextColor': '#52657a', 'labelBoxBkgColor': '#172b4d', 'labelTextColor': '#ffffff', 'sequenceNumberColor': '#ffffff'}}}%%
sequenceDiagram
    autonumber
    participant R as Research agent
    participant F as Finalizer (retained)
    participant S as Storage (retained)
    R->>F: Supply analyzed evidence and recorded inspection context
    F->>S: Save after structure and citation validation
    S->>S: Update quoted user memory separately

Loading

Tests

Platform build passed. Repository tests: 2,599 passed, 58 skipped. Separate auth integration: 170 passed. Regressions verify single-call finalization and retained history and memory behavior.

Live Fireworks replay passed all four original questions, with direct initial prose, working workflow follow-ups and no inspection reviewer calls. Replay used saved public assets and local SQLite.

Production rollout remains pending. Image interpretation and answer style depend on model behavior. Existing memory is not backfilled. Workflow and replay notes.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
video2ctx-web	Ready	Preview	Oct 20, 2026 12:29pm UTC

[devhims]

Independent review — 289c9b2

Verdict: CHANGES NEEDED. Reviewed against stacked base e1139b1. Independent local probes reproduce four groups of gaps despite the passing standard suite.

P2 — Unsupported inspection claims bypass the checker when records are absent

research-agent.ts:1141–1149, with the same predicate in visual-evidence-review.ts:16–17,31.

Both gates require existing visual coverage or inspection items. A mocked context-answer draft saying “I inspected frames at 3:20.” with no visual records reaches the finalize/persistence callback without selecting the review model. Absence of records is exactly when this positive inspection claim is unsupported; it should not exempt the claim from checking. Please make inspection-claim checking cover this case, while preserving legitimate hypothetical/uncertain answers. Add an end-to-end runtime regression for the no-record path.

Verification scope: real finalizer control flow with mocked models and persistence callback; no production DB write or live-model behavior was tested.

P2 — The checker receives less supporting context than the answer generator

visual-evidence-review.ts:38–39 and research-agent.ts:1136–1138.

Two prompt-capture probes confirm:

• Transcript analysis findings and their support text are removed, retaining only coverage/delivery. Those packets have no parallel excerpts containing the findings, so cited spoken context supporting a timestamp or selection explanation disappears from the checker input.
• An older assistant message explicitly retrieved by session.readHistory() appears in finalizer input but not reviewer input. Only the recent conversation window is forwarded, although the checker is instructed to verify what earlier answers cited against the actual messages.

Please pass the relevant cited transcript findings and already-prepared history/context to the checker with matching identities and explicit omission limits. Otherwise supported answers may be rejected or rewritten because their evidence was withheld from the reviewer. The deterministic input omissions are reproduced; actual live-model false rejection is a risk, not a claimed live reproduction.

P2 — Valid quote-only context/question memory updates are silently rejected

memory-updater.ts:137–148; new schema contract at lines 33–37.

The schema accepts a context upsert with a valid userQuote and omitted/default-empty text, and the new contract stores the quote rather than paraphrased text. Validation still gates the entire upsert on change.text. An independent probe rejects this schema-valid quoted update; adding irrelevant placeholder text accepts the exact same quote. A legitimate preference/correction can therefore be dropped rather than saved.

Please require text only for finding entries and validate context/question entries directly from userQuote. Add omitted/empty-text tests for both context and question updates.

P2 — Reviewer forgets the run's already-selected fallback model

research-agent.ts:234 and line 385 create the evidence reviewer without the run's modelFailover state, unlike the main/finalizer model construction. With the same configured primary, the checker retries a provider/model that this run has already abandoned, spending its small review deadline before reaching the healthy backup.

Three additional mock/fake-timer tests confirm the mechanism: constructor capture shows fallback:true reaching the finalizer but not the reviewer. With a stalled primary and a healthy backup that begins streaming after one second and completes after fifteen, inherited fallback succeeds in fifteen seconds without calling primary; fresh state retries primary, starts backup at ten seconds, and the review fails at its twenty-second deadline. No live provider failure was induced. Preserve known fallback selection and diagnostic reporting for the corresponding model profile while keeping the review's own deadline bounded.

Verification

• Clean root suite: 2,600 passed, 58 skipped, exit 0 (library105, platform2023, catalog11, runtime/account363, processor49, frames49).
• Separate auth integration: 170 passed. Unique standard total: 2,770 passed, 58 skipped.
• Focused rerun:160 passed; four independent probes plus three additional failover mock/fake-timer tests separately verified the gaps above. Overlapping reruns are not added to the standard total.
• Generator check and TypeScript passed using node --import tsx scripts/generate-agent-skills.ts --check and tsc --noEmit. Normal platform build's tsx CLI hit a sandbox Unix-socket EPERM, so this is equivalent component validation, not a claim that the normal command passed.
• Initial concurrent integration attempts were interrupted for resource contention; clean sequential runs succeeded.
• Verify CI 39114315346 and Vercel succeeded. These do not cover the reproduced gaps.

Scope and limits

Kiro also reviewed this head source-only and did not approve; no Kiro test counts are included above. The checker intentionally fails closed and adds a model call; those design choices alone are not classified as defects here. No fresh live provider/model/dashboard testing or production rollout verification. The author's live replay is reported evidence, not independently reproduced. Existing memory is intentionally not backfilled, and pending v1 updater jobs are skipped as documented. This remains stacked on PR211; after that lands, retarget to main and verify the actual resulting base/head. No implementation, push, merge or deployment performed.

[devhims]

Re-review — 696e307

Verdict: APPROVE for this exact head against stacked base e1139b1. All four previously reproduced P2 groups are resolved in independent regression checks. No new material code defect was established. This is scoped code approval, not production rollout verification.

Prior findings retested

1. A positive inspection claim with no records now invokes review and is rejected before finalize; absent inventory no longer bypasses the checker.
2. Reviewer receives the intact prepared evidence and shared answer context, including cited transcript findings, older retrieved messages, attribution IDs and omission flags, without new retrieval/admission.
3. Context/question memory upserts accept valid user quotes with omitted/empty text; findings still require text and citations. Existing safety and lifecycle fences remain intact.
4. Reviewer inherits known fallback selection and diagnostics through independent state. The healthy fifteen-second backup succeeds without retrying the stalled primary, and changing the reviewer deadline does not mutate the parent's deadline. Separate profile tests preserve configured DeepSeek/GPT primaries while honoring default/GLM fallback selection.

Fresh independent validation

• Full root suite: 2,616 passed, 58 skipped (library105, platform2039, catalog11, runtime/account363, processor49, frames49).
• Separate auth integration: 170 passed. Unique standard total: 2,786 passed, 58 skipped.
• 11 additional independent probes passed, comprising seven adapted prior regressions and four model-profile/deadline cases; separate from the standard count.
• Platform prebuild, generated-skill check and TypeScript passed. Used node --import tsx ... --check to avoid the established sandbox Unix-socket limitation of the tsx CLI; unrelated root web build was not independently run.
• Root and auth suites ran sequentially. Checkouts remain clean; no implementation or push.
• Verify CI 39116639097 and Vercel passed.

Kiro review and remaining limits

Kiro's source-only review agrees that all four settled P2s are fixed. After clarification, it revised its earlier latency/cost hold to approve with rollout verification; it has not observed the twenty-second checker cap being exceeded. We agree this is an explicit rollout-verification limit rather than an established code blocker: this revision documents that every substantive answer gets a bounded, fail-closed model check using the same prepared context. Largest-context live latency and incremental production cost have not been independently measured, and this approval does not promise they are acceptable under all workloads. Kiro ran no tests; the counts above are independent local results.

No fresh live provider/model/dashboard tests or production rollout verification. Mock tests establish control flow, input preservation and fallback behavior, not semantic reliability of the reviewer. Existing memory is not backfilled as documented.

This PR remains stacked on #211. Merge #211 first, then retarget to main and verify the actual base/head and checks. No merge or deployment performed.

[devhims]

Revised review — 0358914 (October 19, 2026)

Verdict: CHANGES NEEDED. Earlier approval for this head is withdrawn. Follow-up production-helper probes reproduce three material issues in the new inspection-record path, against stacked base e1139b1.

P2 — Sibling-branch inspection reasons enter the finalizer without provenance

platform/src/agents/runtime/session-evidence.ts:782–785 and research/research-agent.ts:905–911 filter the session-wide inventory by video ID, not the selected ancestor chain or originating run. A branch-A follow-up with no visual evidence in its selected ancestors receives branch B's recorded reason for choosing 3:20. Records contain no originating run/message IDs, and missing current-run operations are labelled inherited. inspection-records.ts:64 tells the finalizer to prefer these records over assistant history. The model can therefore attribute another branch's reason to the answer being discussed.

Scope is same authenticated user/session, sibling branches, not demonstrated cross-account leakage. Existing ancestor resolution is branch-scoped; this new path bypasses it. The identical base fixture made no structural read and received no sibling canary. Prompt contamination is reproduced, not a particular live model answer. Scope records to the relevant lineage or supply explicit provenance and enforce its use.

P2 — Every answer parses all stored evidence before filtering

session-evidence.ts:784–785 calls evidence(), whose unfiltered query at :234–246 materializes, JSON-parses and schema-parses every session packet. The finalizer invokes it at research-agent.ts:909 before distinguishing clarification/rejected intents. The 12k output cap does not bound preceding work.

Both clarification and rejected production-helper fixtures with zero video subjects parsed five full-transcript packets totaling 2,869,750 serialized bytes, then emitted zero inspection records. Base fixtures performed zero structural reads. Session growth, including PR211's redundant saved comment snapshots, increases CPU/heap/SQL work for even simple replies. No production OOM or latency figure was measured. Skip unnecessary paths and retrieve bounded relevant structural metadata without decoding all source payloads.

P2 — Source-bearing selection reasons bypass evidence admission

inspection-records.ts:39 projects frameCoverage.selections.reason unchanged. frame-selection.ts:4–8 permits six model-authored reasons up to 400 characters, with optional source refs; frame-tool validation checks timestamps/IDs, not semantic content. A real frame-tool fixture accepts a reason containing a transcript-derived numerical claim.

With the actual RunEvidenceLedger at its exhausted 22-credit reserve, a frame-analysis packet is withheld and the finalizer evidence array is empty, yet its free-text reason still arrives through the structural history projection. This path exists even without the session-wide method. Ordinary excerpts remain excluded: this is bounded source-bearing free text, not all transcript/image content. Do not assume agent-written explanations are source-free metadata. Enforce an explicit admission/provenance boundary for reasons while keeping truly structural times/counts separate.

Nonblocking corrections and feedback triage

• P3 omission counts: inspection-records.ts:51 sums omitted counts before deduplicating overlapping views. 100 unique packets produce 18 retained/82 omitted; merging that same view twice yields 18 retained/164 omitted. Deduplicate identities before computing omission counts.
• P3 quote fidelity: memory-updater.ts:144–148 accepts lowercased/whitespace-normalized matches but stores the model's spelling. FooBar→foobar and US→us are accepted despite not being exact substrings. Preserve matched original text for the new verbatim guarantee.
• The skill/tool gates differ, but normal classifier construction derives useStoryboard from visualEvidence; legacy inputs also align. Contradictory schema-valid inputs are not yet a demonstrated production path. Invariant tests would help.
• Explicit failed-storyboard→frame fallback and no-identical-retry wording were lost: worthwhile prompt hardening, without a reproduced live failure here.
• Tool limits remain in tool descriptions/schemas. Required visual completion and transcript-versus-image distinctions remain enforced; those were not simply lost with duplicated prompt text.
• The inspection-evidence repair comment at research-agent.ts:1101 is stale after semantic-checker removal. Extra message IDs in analyst prompts and single-use answerContext are cleanup concerns, not demonstrated correctness failures.

Verification and limits

Six targeted production-helper tests passed, plus three identical finalizer fixtures on head and three on base. Existing memory/router suites: 138 passed. An additional quote/gate helper probe reproduced the fidelity issue and confirmed ordinary legacy gate alignment. Production projection, finalization, frame validation, ancestor and billing helpers ran with fixture model/provider/storage inputs. No live provider/model call, production performance measurement, implementation, push or full-suite rerun. Earlier test totals below are historical, not evidence against these new findings. PR211's newly found search defects are tracked separately; this remains a stacked PR pending correct retarget/review.

---

Historical validation record (revised verdict above supersedes approval)

Re-review — 0358914

Verdict: APPROVE for this exact head against stacked base e1139b1. Independent review and Kiro's source review agree: the intentional checker removal is complete and the retained workflow/history/memory changes are coherent. No material blocker found.

Verified behavior

• Reviewer implementation, model role/factory, fallback helper, deadline, usage records, trace calls and semantic rejection stage are removed. Model configuration and failover implementation match the base.
• Successful finalization with and without visual records uses one finalizer model call and one cost record; no reviewer trace or new evidence retrieval occurs. Existing schema/citation validation and repair remain.
• Older message IDs, actual image-input positions and recorded selection reasons still reach the finalizer. Unadmitted observation text remains excluded.
• Quote-only context/question memory remains accepted without placeholder text; findings retain their evidence/text requirements.
• Selection notes remain tied to matching timestamps/assets, with latest matching notes preferred.
• Ordinary-answer guidance discourages tool narration while retaining citations and material uncertainty; explicit workflow questions retain structural records and unknown-reason guidance.

Independent validation

• Root suite: 2,599 passed, 58 skipped.
• Separate auth integration: 170 passed. Unique standard total: 2,769 passed, 58 skipped.
• Five additional independent intended-behavior probes passed; not added to the standard count.
• Platform prebuild, generated-skill check and TypeScript passed, using the build-equivalent invocation that avoids the known tsx CLI sandbox-socket limitation. Root web build was not independently run.
• Verify CI 39119349966 and Vercel passed for this head.
• Tests ran sequentially; checkouts remained clean. No implementation, push or deployment.

Nonblocking cleanup / limits

• research-agent.ts:1101 still has a stale comment saying inspection-evidence failures share the repair allowance; the semantic checker no longer exists.
• Optional prompt hardening: retain an explicit reminder to describe only supplied items using coverage/truncation counts when explaining workflow. Existing warnings and material-limitations guidance partly cover this; no regression was reproduced.
• Single-call probes cover valid first-pass answers, not a promise that schema repair or provider fallback never adds a call. Semantic accuracy and actual narration style remain model-dependent. No fresh live provider/model/dashboard tests were performed; the author's live replay is not an independent reproduction. Kiro's assessment was source-only.

PR211 must land first, then retarget this stacked PR to main and verify the actual resulting base/head and checks. This review does not merge or deploy either PR.

[devhims]

Verdict: CHANGES NEEDED for 3f7c3a2 against stacked base 9ba6ba8. The branch-scoping and session-wide scan fixes are improvements, but the earlier source-bearing selection-reason admission finding remains unresolved.

P2 — Historical selection reasons still bypass the current turn's evidence admission.
platform/src/agents/research/research-agent.ts:914-918 appends earlier turns' inspection/evidence packets independently of the admitted evidence array. platform/src/agents/runtime/inspection-records.ts:49 forwards their complete frameCoverage, including free-text selections[].reason. Calling those reasons agent notes does not make their contents source-free.

A current-head production-finalizer probe used the real RunEvidenceLedger with its 22-credit reserve exhausted. The historical analysis packet was withheld, finalizer evidence was [], receipts were [], and its ordinary visual observation was absent. Nevertheless a synthetic transcript-derived numerical claim appeared verbatim in inspectionRecords.items[0].frameCoverage.selections[0].reason. The claim was absent from both the earlier user and assistant messages, so this was not ordinary conversation-text reuse. A history-only route reproduced the same path without calling evidence admission at all. The real executeGetVideoFrames validator accepts that reason with a valid locating excerpt ID.

Impact: bounded source-bearing text reaches the answer model through supposedly structural context despite the underlying source packet being withheld. This is not a cross-account leak, a demonstrated live answer, or a measured billing overcharge. Preserve free structural timestamps/counts, but enforce an explicit admission/provenance boundary for source-bearing reason text. Do not regard the previous P2 as resolved merely because the storage lookup is now branch-scoped.

Prior fixes checked:

• Session-wide inspectionRecords()/evidence() lookup is removed. History is selected through the conversation ancestor chain; the new runtime regression excludes another conversation's records.
• Clarification/rejected finalizer probes neither read the global packet inventory nor emit inspection records. The former full-session scan path is gone.
• One-pass record deduplication now precedes omission counting; repeated views do not double-count omitted records.
• Context/question quotes preserve the matched user's spelling, with whitespace normalization documented; memory/inspection tests pass.
• Visual skill injection now follows the actual enabled visual tool set, and failed-storyboard fallback guidance is restored.

Independent verification:

• Full root suite: 2,637 passed / 58 skipped (library105, platform2034, catalog11, Workers389, processor49, frames49).
• Separate auth/payment integration: 170 passed. Unique standard total 2,807 passed / 58 skipped.
• Six additional reviewer probes reproduced the two reason-admission paths and checked real frame-input acceptance, clarification/rejected behavior and deduplicated omission counts. A 22-test memory/inspection rerun also passed; overlapping reruns are not added to totals.
• Platform TypeScript, generated-skill and generated-documentation checks, and whitespace checks passed before reviewer-only probes were added.
• GitHub Verify 39410925191 and Vercel passed for this exact head.

Dependency/readiness: this remains stacked on PR211 at 9ba6ba8, whose unrelated-deletion search omission is separately unresolved and whose integration with current main has conflicts. The PR description still names older base e1139b1; update it to the actual stack and re-review the eventual main-based tree. No fresh live provider/model/dashboard testing. Kiro sign-in remains unresolved, so no new Kiro agreement is claimed. Only local test probes were added; no production implementation, push, merge or deployment performed.

[devhims]

Independent re-review — 19194a2

Verdict: APPROVE this exact head against stacked base 48c27b8d3753f9734fbc84b4d52aa2ebce4eea54. The remaining P2 is fixed; no material source finding remains. This is not authorization to merge the stack now: PR211 must land first, then retarget this PR and verify its resulting base/head and checks. PR211's separate Vercel authorization blocker is not resolved by this review. No ready-for-review state change or new Kiro agreement is claimed.

Prior admission finding resolved

• platform/src/agents/runtime/frame-selection.ts:27–47 replaces arbitrary reasons with fixed basis categories, enforces locating-reference counts, rejects reason-only inputs, and strips free-text/unknown fields from current and legacy stored selections.
• get-video-frames.ts:41–49 checks locating references against delivered excerpts for the same video. An independent actual-tool probe verifies extra free-text reason is discarded and wrong-video references rejected.
• Real-finalizer probes with the actual exhausted 22-credit RunEvidenceLedger now withhold the historical source claim from inspectionRecords as well as evidence. The history-only route also strips it. Legacy saved-selection reuse preserves timestamps/reference IDs without the source-bearing reason.
• An actual Durable Object history probe still excludes sibling-branch inspection records. Earlier branch scoping, deduplicated omission counts and enabled-visual-tool skill routing remain intact.

P3 — Combined analysis has contradictory operation labels (nonblocking)

platform/src/agents/runtime/inspection-records.ts:43 calls evidenceDeliveryOperation(record) without the packet's artifact types. A newly combined get_video_frames analysis is therefore labelled operationThisTurn=retrieval even though savedRecord=analysis and inputMeaning explicitly confirms the images were supplied to the analyst. The ordinary admitted-evidence delivery label correctly says image_analysis after the base's fix. Pass artifact types here too, as finalizationEvidenceForModel does.

An independent probe reproduces the inconsistent fields, not a live model misstatement, missing analysis or billing error. Correct analysis proof remains present in the other fields, so this is a consistency cleanup rather than a merge-blocking defect.

Independent verification

• Full root suite: 2,746 passed, 61 skipped (library105/2 skipped; platform2115/59 skipped; catalog11; Workers417; processor49; frames49).
• Separate auth/payment integration: 170 passed. Combined standard total: 2,916 passed, 61 skipped.
• Nine additional reviewer probes passed: eight finalizer/schema/tool/label checks and one actual Durable Object lineage check. Not added to the standard totals.
• Library build, generated-skill/documentation checks, final platform TypeScript and full-PR whitespace checks passed. Checkout clean after removing reviewer-only probes.
• The tsx CLI initially hit a sandbox IPC error; the supported node --import tsx invocation succeeded. No source workaround was introduced.
• Verify CI39422614327 and Vercel succeeded for this exact head.

Limits and dependency

No fresh live model/provider/dashboard, production-performance, local browser E2E, web-build or Docker-build verification. The author's historical replay was not independently rerun; structural tests do not guarantee correct image interpretation or answer style. Skipped tests remain unverified. Kiro sign-in is unresolved, so writer/reviewer alignment for this head is not claimed. Keep the stacked dependency and eventual main-based recheck explicit. No implementation, push, merge, deployment or repository settings change performed.