Skip to content

Migrate agents off ::: fenced-block parsing → approval-gated tool calls #123

@drewstone

Description

@drewstone

Problem

The agent products (insurance/tax/legal/creative/gtm) emit structured side-effects as custom fenced blocks in free text:::proposal, :::followup, :::client, :::filing, etc. — which a post-process.ts regex parses into approval-queue rows / calendar entries / DB writes.

This is structured-output-smuggled-through-prose, and it's brittle:

  • The model can describe instead of emit. Observed in insurance-agent: the model wrote "I'll route this via :::proposal" in backticks → regex matched nothing → the approval queue stayed silently empty. We patched it with prompt discipline + a tolerant regex, but that's papering over the mechanism.
  • Weak/cheap models don't comply with the exact fence format; format drifts across model swaps.
  • Two parallel mechanisms for the same concern. These products already do real tool calls (integration_invoke) whose writes return "approval-required" through the runtime tool-loop's approval-stop. So there are TWO human-in-the-loop paths: the robust tool-call approval path AND the brittle :::-parse path. We maintain both.
  • Portability liability. For products meant to be sold/forked, a bespoke ::: regex contract is something a buyer must learn and that breaks on model changes. Tool-calling is the portable standard.

The proper / SOTA pattern

Express each regulated side-effect as a tool call with an approval gate:

  • The agent calls e.g. propose_swap({...}) / contact_lead({...}) as a function/MCP tool.
  • The runtime captures structured args directly (schema-enforced — no regex, no "described vs emitted").
  • The human-in-the-loop gate is the tool-call approval boundary: the call is paused, routed to a certified/authorized human, executed on approval. This is the standard agent HITL pattern and it reuses the runtime tool-loop's existing approval-stop.
  • The existing executor + approval queue + authorization checks (e.g. insurance-agent's requiresCertifiedAgent) are reused — they just get invoked from the tool path instead of the parse path.

Keep rendering/annotation blocks as-is (:::openui, :::citation) — those are display, not side-effects, so inline blocks are fine.

Migration recipe (per agent)

  1. Define the regulated side-effects as tools (schema per type).
  2. Route tool calls through the runtime tool-loop with permission: ask/approval-stop so they pause for human approval.
  3. On approval, dispatch to the EXISTING executor (no change to the side-effect logic / authz gate).
  4. Render the approval card from the captured tool-call args (replaces parsing prose).
  5. Delete the :::proposal/:::followup/:::client (+ legacy :::filing) branches from post-process.ts; keep :::openui/:::citation.
  6. Update the system prompt: "call the tool" instead of "emit the block."
  7. Tests: the regulated action materializes into the approval queue via the tool path; the authz gate still blocks unapproved/uncertified execution.

Tracking — per agent

  • insurance-agent (reference implementation — drives the pattern)
  • tax-agent
  • legal-agent (also has legacy :::filing → tool)
  • creative-agent
  • gtm-agent

Substrate

The runtime tool-loop + approval-stop already exists in @tangle-network/agent-runtime (the ChatEngine tool loop, and integration_invoke writes already return approval-required). This migration consolidates the products onto that primitive. If a shared proposeAction/approval-tool helper would reduce per-product boilerplate, add it here.

Motivation trace: insurance-agent hit the described-vs-emitted failure during onboarding prep; tolerant-parser + prompt patch shipped as a stopgap (insurance-agent), but the right fix is this migration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions