Skip to content

[bug] Return only the final assistant answer in public run message text #2114

Description

@zerob13

Summary

The public runs.get/run get message text field concatenates internal reasoning, action text, full tool responses, errors, and the final answer. It should represent the final assistant answer only.

Priority: P1

Observed behavior

Sanitized live runs showed both failure modes:

  • A prompt requesting one exact short answer returned internal reasoning followed by that answer.
  • A file-inspection task returned reasoning, the complete tool response, and the final answer in one public text string.

This is not merely a formatting issue. Machine consumers cannot determine which portion is the answer, and tool output can contain local information that was never intended to become the public result.

Root cause

messageText() in src/main/cli/runService.ts calls
buildAssistantResponseMarkdown() from
src/main/agent/deepchat/runtime/sessionUpdates.ts.

That renderer-oriented helper intentionally extracts text from every block type:

  • action content;
  • tool-call responses;
  • error content;
  • reasoning/content blocks.

The repository already has the correct answer-oriented projection:
projectFinalAssistantAnswer() in
src/shared/lib/assistantDeliverySegments.ts.

Related contracts and documentation

  • docs/architecture/local-control-plane/spec.md requires deterministic machine output, bounded/redacted public views, and stable terminal records for external harnesses.
  • It also states that a genuinely redacted public view should have an explicit canonical contract rather than reuse an unsafe internal shape.
  • docs/architecture/durable-execution-journal/spec.md deliberately avoids duplicating raw tool responses into journal facts because they may be sensitive and have a different lifecycle.

Using a display/debug projection as the public answer violates those boundaries.

Impact

  • JSON consumers receive an unstable mixture of process and result.
  • Internal reasoning can be exposed unintentionally.
  • Full tool output can leak local file contents or command output to higher-level automation.
  • Benchmark answer scoring becomes invalid.
  • Large tool responses consume the public text limit and can truncate the actual final answer.

Proposed direction

  • Use projectFinalAssistantAnswer() for public assistant text.
  • Keep the internal renderer preview path unchanged.
  • If process visibility is required, add an explicit, bounded, opt-in field or trace surface with its own redaction contract. Do not concatenate it into text.
  • Define behavior for runs that terminate without a final answer: empty answer plus structured terminal reason, not an arbitrary earlier block.

Acceptance criteria

  • A message containing reasoning + tool response + final content exposes only final content in public text.
  • Tool output and action/permission copy never appear in public answer text.
  • A final-answer-less failure exposes a structured reason without promoting reasoning/tool text to the answer.
  • UTF-8 truncation applies to the final answer, not the internal transcript projection.
  • Renderer previews and persisted structured blocks remain unchanged.
  • Regression tests cover reasoning, multiple tools, actions, errors, and no-final-answer cases.

User benefit

Scripts receive a clean, predictable answer, while local execution details remain private unless explicitly requested through a dedicated diagnostic surface.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions