Prowl Research Agent

Runbook-driven research agent. It follows structured research runbooks (SaaS competitor teardowns, ads & creative research, subscription-app audits, market sizing, equity research, SEO, AI visibility and more), calls the Prowl MCP tool bank (440+ data tools: SEO, ads libraries, reviews, scraping, app intelligence, LLM cross-checks), stores extracted claims in an evidence ledger, and produces reports with source citations and verification disclosures.

It is a CLI for humans and has two MCP surfaces for agents:

• the Fabric service (prowl-research-service, HTTP MCP at http://127.0.0.1:18764/mcp, the connection fabric-agent.json declares) — 7 tools: research.run, research.tools.call, research.tutorial, research.list_runbooks, research.artifact.get, fabric.job.get, fabric.job.cancel (see Fabric service);
• the legacy local stdio MCP server (prowl-research mcp) — 4 tools: research.run, research.list_runbooks, research.get_status, research.get_report.

Quick start for a new teammate

Install

git clone https://github.com/PROWL-AI/research-agent.git
cd research-agent
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Configure

Two keys, both via environment (key names only — values never in the repo):

• PROWL_API_KEY — your Prowl API key (prowl_…), billed per tool call. Get one at https://prowl.chat.
• OPENROUTER_API_KEY — the agent's own LLM (planning, synthesis, writing). Any OpenAI-compatible endpoint works: set RESEARCH_LLM_BASE_URL / RESEARCH_LLM_API_KEY / RESEARCH_LLM_MODEL to override.

Other environment knobs (all optional):

• PROWL_MCP_URL — Prowl MCP endpoint (default https://prowl.chat/mcp).
• RESEARCH_LLM_MODEL_STRONG — the strong-tier model (planning, writing, repair; default xiaomi/mimo-v2.6-pro); RESEARCH_LLM_MODEL covers the cheap tier (claim extraction, transforms, citation checks; default deepseek/deepseek-v4.1-flash); RESEARCH_JUDGE_MODEL the eval judge (default anthropic/claude-haiku-5.5 — another vendor than the default writer, so a report is not graded by the family that wrote it; keep that true if you change the writer). Defaults apply to new installs and the environment: an installed service keeps the models saved in its settings. Judge scores before and after the judge default changed (2026-10-17, from the strong model) are not comparable. Prices for the cost estimate are in research_agent/pricing.py (OpenRouter catalog, read 2026-10-17; scripts/check_prices.py re-reads it).
• RESEARCH_MAX_LLM_USD — the run's LLM spend ceiling (see "How it works"). Claim extraction and citation checks send reasoning: {enabled: false} to OpenRouter: on the cheap tier reasoning could spend the whole 2000-token output budget before any JSON, and an answer cut that way is not retried, because the same request is cut the same way (B-050, measured 2026-10-18/20).
• RESEARCH_RUNS_DIR — where runs are written, for the stdio MCP server and every CLI command alike (run, status, report, rewrite, export, prune). Default ./runs relative to the process; set it explicitly in MCP client configs.
• RESEARCH_NO_SUBAGENTS=1 — force sequential execution (no worker fan-out).
• RESEARCH_PROWL_CALL_TIMEOUT_S — read timeout in seconds for each Prowl MCP call (default 180; a hung server fails the call instead of parking it).

CLI subcommands: run <runbook> [--run-id <id>] [--max-llm-usd <usd>] [--key value …], list-runbooks, status <run_id>, report <run_id>, rewrite <run_id>, export <run_id> [--format html|md], prune [--older-than 30d] [--keep-last N] [--yes], validate [--online], mcp.

Exit codes: run exits 1 when the run ends partial (budget or failure — the report says why); rewrite exits 1 when lint issues remain; validate exits 0 when all runbooks are valid, 1 when a runbook fails validation, 2 on usage or network/config errors (CI can tell "runbook invalid" apart from "catalog unreachable"); other input and usage errors exit 2. list-runbooks, status, report, export, prune exit 0 on success; mcp serves on stdio until stopped.

runs/ grows forever unless you prune it: prune lists runs older than --older-than (by checkpoint created_at, default 30d) without deleting anything; pass --yes to actually delete. --keep-last N always spares the N newest runs, and any run whose .lock is held by a live process is skipped.

MCP

Register the agent as an MCP server (stdio) in any MCP client:

{
  "mcpServers": {
    "research-agent": {
      "command": "prowl-research",
      "args": ["mcp"],
      "env": { "PROWL_API_KEY": "prowl_…", "OPENROUTER_API_KEY": "…" }
    }
  }
}

The stdio server exposes four tools: research.run, research.list_runbooks, research.get_status, research.get_report. It is the legacy local surface; the Fabric bundle points at the HTTP service, which exposes seven (listed at the top of this file).

Proving tool call (verifies install and registration; needs no keys and makes no billed calls):

research.list_runbooks()

Expected: the runbook list (saas-competitor-teardown, …) with inputs and budgets. research.get_status and research.get_report are key-free too — only research.run requires the keys above (it fails fast, naming the missing env var, before spending anything).

Runs take minutes, so for long runs prefer the job pattern: call research.run with wait=false (the default) to get {run_id, status: "running"} back immediately, then poll research.get_status(run_id) until the checkpoint shows complete, partial, failed, or interrupted. failed/interrupted require inspecting the checkpoint; a report may not exist. For complete/partial, fetch the result with research.get_report(run_id). Use wait=true only when the caller can block for the whole run.

Next to status, every terminal state carries a terminal_code saying why the run stopped (null while it is live). The set is closed:

terminal_code	status	meaning
completed	complete (or partial when a disclosed omission remains, e.g. a step excluded because its tool left the catalog)	every runnable step ran and its evidence was saved
budget_tools / budget_usd / budget_minutes	partial	a runbook budget ran out; the report marks what is missing
budget_llm	partial	the next LLM call would cross max_llm_usd (minus the writer reserve for research stages), or a report-stage call (writer, repair, citation check) was refused; the report is written from what was gathered, or is the evidence ledger listed
step_failed	partial	a required step failed: refused while the run went on to the writer (B-049), or a refusal that stopped the run (empty wallet, the service's own limit, five refusals of required steps in a row), or a worker crashed; stop_reason says which and names the steps
evidence_incomplete	partial	evidence is missing with no other stop: a claim extraction failed (the resume finishes it from the saved raw, no new paid call), or a required step's raw could not be written (the resume fetches it again — a new, paid call; an optional step is skipped_on_error instead)
cancelled	partial	any cancellation of the owning task: a caller or service cancel, the service timeout (job error timeout), Ctrl-C, a client closing its session or a graceful service shutdown
error	failed	an exception escaped the run; stop_reason carries it
interrupted	interrupted	the owning process died without a terminal write (a hard kill or crash); written by reconciliation, resumable

Run state has exactly one writer: the process holding the run's lock (runs/<id>/.lock). The orchestrator records cancelled/error itself before releasing the lock; a second process on the same run is refused (run … is locked by another process) and writes nothing — on stdio MCP the refusal is returned synchronously, also for wait=false. get_status reports active (some process holds the run lock) and orphaned (running, no live owner anywhere). interrupted is written by reconciliation — at stdio server start, at CLI run/status, and at Fabric service start — and only for a running checkpoint whose lock is free. A run whose version-1 checkpoint cannot be migrated without guessing is reported with status: needs_attention and the reason in stop_reason: its report stays readable and exportable, it can be pruned, and research.run refuses to resume it. get_status also reports has_checkpoint: false while a run accepted by this server is still starting, or when it failed or was cancelled before it took the run lock (the status then comes from the in-process job and disappears with it).

First real run (makes billed Prowl calls; see budget limits below):

prowl-research run saas-competitor-teardown --competitors example.com,rival.com

Expected: a markdown report under runs/<id>/report.md plus an HTML export, with a source log and confidence ladder.

Develop

pytest                                  # unit tests + runbook validation (offline)
python scripts/validate_runbooks.py --online  # frontmatter + tool names vs live Prowl catalog (needs PROWL_API_KEY)
python evals/judge.py --run runs/<id>   # 5-dimension quality rubric for a finished run (exit 0 pass, 1 fail, 2 judge error)
python evals/calibrate.py               # the judge against planted-defect fixtures (offline; --live spends)

The judge's verdict is vetoed by defects the code proves — a citation of no claim; a quotation no cited claim backs; a sentence whose metric figure none of its cited claims carries (a sentence the writer marked "derived" excepted); a conflicting claim cited on a line without the upper-case CONFLICT tag and outside a conflict register (rows under a "Source log" heading are provenance, not findings); a citation tagged VERIFIED whose claim the ledger does not hold as verified; a partial run that does not say "partial" before its first section — so a lenient model cannot pass them. An uncited figure lowers the citation score instead. Each judge LLM call is checked by estimate against --max-usd (default 0.50) before it is sent and recorded in <run>/judge-ledger.json, never in the run's checkpoint. The judge needs only an LLM key.

Authoring a new runbook: docs/runbook-authoring.md. Architecture decisions and inherited traps: docs/knowledge-pack.md.

How it works

runbook (SKILL.md)  →  brief  →  plan  →  parallel research sub-agents
                     (data segments; transforms stay on the lead)
                     →  evidence ledger  →  writer + citation-repair loop
                     →  citation verification pass  →  markdown + HTML

• Runbooks ask for sourced numbers and triangulation. The writer consumes the evidence ledger and runs citation repair/verification, but this is not a guarantee that every fact is true or has two independent sources. Inspect the report's verification disclosures and source evidence before relying on it.

• The repair pass fixes what the judge would veto. Besides the citation lint, the writer's repair pass gets the defects the eval judge proves by code (research_agent/agent/report_checks.py): a VERIFIED tag on a claim the ledger does not hold as verified, a conflict cited without its [CONFLICT] tag, a quotation no cited claim holds, a figure no cited claim carries. lint_issues_before/lint_issues_after count both (B-057). The repair pass sends only the lines at fault and patches the lines it gets back (L<n>: …), so a long report is not rewritten whole (B-059). The citation lint also flags drafting residue inside a citation (a question mark, a note or a nested bracket, such as [C186? no, [C237]]) as malformed_citation, which the repair pass rewrites to plain claim ids (B-068).

• One refused step does not end the run. A tool that is refused (no results, not found, a provider, scope or argument error) marks that step failed (or skipped_on_error when the plan marked it optional) with the server's reason, and the run goes on; with a refused required step it ends partial / step_failed, and the writer is told which steps were refused and why, so the report says what is missing. What still stops the run: an empty Prowl wallet (insufficient_funds), the service's own call limit or an unavailable call journal, a call whose outcome is unknown, five refusals of required steps in a row across the run (optional steps neither count nor reset it), and every budget. A budget stop keeps its own terminal_code; its stop_reason also names the refused steps. A resume asks every failed step again (a new attempt, which the provider may bill again). After an empty wallet or the service's own limit, the other workers send no further step (B-092), and a worker crash is recorded when it happens, so the others see the stop at once (B-091).

• A value only an earlier step returns is a placeholder. The plan is written before any tool runs, so an argument such as an app's numeric store id is planned as {{headspace_apple_id}}, with a transform before the step that names it. That transform ends its note with a fenced bindings block; a value is kept only when a raw artifact of the run states it verbatim (checkpoint.bindings), and the step sends the filled arguments (bound_arguments; arguments keeps the plan as written). A placeholder nobody resolved is never sent: the step is skipped_on_error (or failed, when required) with not sent: no earlier step resolved {{name}}, and it does not count toward the refusals in a row (B-065).

• A whole earlier result is a reference. A step that needs an earlier step's whole output (a scraped page for an analysis tool) writes {{output:<step name in snake_case>}}; the stored raw artifact of the completed data step of that name (sha256-checked; JSON compact and without the server's billing and timing) is sent in its place, cut at 16 000 characters per step, shared by its references, with a [cut: N more characters] marker (bindings.OUTPUT_CAP). The referencing step runs one level after the step it names while the other steps keep running beside the producer, and a name two steps share leaves it unresolved (names two steps). The reference is judged as the plan wrote it, so provider text inside the output is never taken for template text; a reference to a step that did not complete is not sent (B-074).

• Template text and known-refused values are not sent either. Live runs showed the planner writing other shapes than {{name}} ([substitute landing_page_url of ...], <video_id from youtube_search ...>, {youtube_search.latest_video_id}, PLACEHOLDER_FROM_STEP_22); providers billed each one, and an LLM-answer tool even "completed" on the literal text. Such a step is skipped_on_error (or failed) with not sent: an argument holds template text, not a value: ... (B-070). A value a provider is known to refuse while its schema is silent (research_agent/agent/tool_limits.py, each limit with its source: tiktok_ads_library takes country only as all or a European code) is told to the planner and, if planned anyway, not sent: not sent: <tool> does not take country='US' (a known provider limit) (B-071). Neither counts as a refusal, the writer reports such a step as a gap of the run (not the source's answer), in a partial run and in a complete one, and is told what a limited source covers (TikTok: Europe only); an accepted value is sent in the provider's spelling; research.tools.call refuses a known-refused value before a job exists and again at dispatch.

• Runbooks declare their tool allowlist and budget (max_tool_calls / max_usd / max_minutes). On budget exhaustion the agent emits a partial report that marks what is missing — it never fails silently. The budget is read from the run's attempt ledger: every dispatched call is recorded before it is sent (so in-flight calls count against max_tool_calls), and its cost is stored when it returns or fails — a billed failure costs money too. Spend = server-reported actual costs + estimates for calls without one (the catalog price, else the runbook's per-call share of max_usd, never zero); stats.cost_source says server, mixed, estimate, or null before any tool attempt. A step that reads an earlier step's whole output keeps one tool call: unrelated steps of its block yield it first (skipped_budget, the reason says so), never its producer or a step whose outcome is unknown, and the budget_tools stop lands after the block (B-086). They yield only while the calls left still reach one such step after its pending producers (a producer that will not be sent is not counted); with fewer, plan order stands (B-087), and nothing yields once another worker has stopped the run (B-090). The runbook budget caps Prowl tool calls; the agent's own LLM spend (planning, claim extraction, transforms, writing, repair, citation checks) is capped separately by max_llm_usd: --max-llm-usd / MCP max_llm_usd, else RESEARCH_MAX_LLM_USD, else the cap a resumed run recorded, else the runbook's budget.max_llm_usd, else USD 5. It is a check before every LLM HTTP attempt (retries included), against an estimate of that call: the last call it admits can overshoot by its actual cost minus its estimate, so stats.llm_budget.spend_usd can end slightly above the cap. Each attempt is recorded in the attempt ledger with the provider's usage.cost (OpenRouter is asked for it), else a price-table estimate (research_agent/pricing.py; an unlisted model is priced at USD 15 / 75 per 1M tokens, above every listed model — never 0); an HTTP error response (429, 5xx) is recorded as unbilled. Research stages stop at the cap minus a writer reserve (25%, at most USD 1) with terminal_code=budget_llm, and once a research call is refused no further tool is dispatched; the writer, repair and citation passes may spend the reserve — a refusal there also ends the run partial / budget_llm — and if even that is gone the report is the evidence ledger as gathered. A resume reads the spend and cap already recorded (also after a re-plan) and keeps an existing report when it gathered nothing new and cannot afford a writer. rewrite --max-llm-usd spends from the same ledger. stats.llm_budget reports cap, reserve, spend, cost source and refused stages; stats.llm_usage the tokens.

• Evidence statuses (ledger schema 2). A claim's status is recomputed over its whole subject group on every change and on load, so it never depends on arrival order. verified needs either:

  • span — the claim's quote is found in the step's whole stored raw (its values or its JSON, sha256-checked on resume; a bare number is not a quote) and the value sits inside the quote on number boundaries (1200 is not in 11,200). A span-verified claim is one source saying it; or
  • corroboration — two agreeing claims from two different steps with distinct known sources: different publishers (registrable domain of a URL the raw or the request actually names — an extractor-supplied URL that the raw does not contain is dropped) or different upstream data providers (delivery wrappers removed: google_trends and serpapi_google_trends are one Google Trends source).

  Never a source: a transform note and an LLM-answer tool (Perplexity, Gemini, Google AI mode/overview, *_responses) — their own text never span-verifies and they add no independence; a web search/scrape claim without a URL; a claim whose quote was not found. Two values that disagree make every valued claim of the group conflict (a missing unit is compatible with any unit; $ and USD are one unit); a claim without a value never confirms or resolves one, and a chart over a conflicting claim is rejected. The extractor's own word never verifies; a v1 ledger's verbatim claim without a quote loads as assumed with the reason in verification.detail. Reports written before schema 2 keep their old tags — regenerate them with rewrite_report, which also re-saves the ledger; judge status shares are not comparable across this change. Events: claim.verified, claim.conflict, claim.quote_not_found (with run_id; under the service they go to ~/Library/Logs/prowl-research/events.log, rotated at 5 × 5 MB, beside the service's own job.*, service.* and config.changed events with their severity and job id; the person-facing text stays on the dashboard).

• Charts are rendered deterministically from ledger data, never described by the LLM.

Current verification and resume limits

The deep agent audit adds seven defects: lifecycle ownership (KA-06 — wave 1), hidden artifact/extraction failures and budget accounting across mixed receipts/resume (KA-10…KA-12 — wave 2), and false evidence verification (KA-07…KA-09 — wave 3, ledger schema 2). The ported contracts live in tests/test_ka_contracts.py. The historical probe file stays as the audit's cut: 11 of its 12 cases pass; its success control asserts a cwd-relative raw_ref, which wave 2 replaced with a run-dir-relative one. The 2026-10-10 handoff links the remediation plan and records the audited source revision and checks (its repair tasks are history; the handoff's first entry is current). The offline suite is .venv/bin/python -m pytest tests/ -q (network-free); its pass count is a fact about one revision, so it is recorded with that revision in the handoff rather than here. A green run does not establish live provider quality or Fabric host acceptance.

Resume works from the checkpoint's step records (schema version 2, wave 2): every step has an immutable step_id assigned when the plan is made, so a tool leaving the catalog excludes that step by id and shifts nothing. A step's state is pending, dispatching, outcome_unknown, raw_saved, completed, failed, skipped_on_error, skipped_budget or excluded. A resume runs pending and failed steps, finishes raw_saved steps from their saved raw without a new paid call, and re-sends an outcome_unknown call with the same idempotency key (<run_id>:<step_id>:<attempt>) so the server can replay it instead of billing twice. A client read timeout counts as an unknown outcome, never as a refusal, and a budget stop leaves such a step outcome_unknown so its key survives. The Prowl server deduplicates a re-sent key only within its retry window (15 minutes by default): a resume after that may be charged again, and the re-send is still recorded as one attempt — the budget then undercounts by that call. A step migrated from version 1 whose dispatch may have reached the server is first re-sent under its v1 key (<run_id>:<index>:<tool>). Version-1 checkpoints are migrated on load; a v1 state that cannot be mapped without guessing is reported needs_attention (above) and never resumed. The upgrade is one-way for a run directory: the first save of a migrated run keeps its v1 file as checkpoint.v1.json, but a pre-v2 build reading the new checkpoint.json re-plans (and re-bills) the run. Restart stdio sessions and reinstall the Fabric service before resuming runs a v2 build has touched; a build refuses a checkpoint whose schema_version is newer than it understands.

Status and checkpoint shape (schema 2). research.get_status returns completed_steps as step ids (not indexes), planned_steps without excluded steps, step_counts per state, skipped_steps entries carrying step_id, index, step, tool, reason and state (while a run is live, steps still in flight are not listed), and counters with data_calls, attempted_calls, cost_usd, cost_estimate_usd, spend_usd and cost_unknown_attempts (the same keys appear in a run's stats). stats.cost_source is server, mixed, estimate or null (no tool attempt yet). checkpoint.json (also what research.artifact.get serves) holds plan_raw, steps and attempts; it no longer persists plan, completed_steps, skipped_steps or counters. Crash injection at every durable write is covered by tests/test_durability.py.

Finance/subscription live acceptance remains unverified. With several workers, the USD check can still overshoot by the in-flight calls' unknown cost (each counts at its estimate); max_usd is not a cap on the separate LLM provider spend.

Fabric service and operator dashboard

The separate prowl-research-service entry point adds durable jobs, observed tool/LLM receipts, human decisions and a service-owned dashboard for Fabric Dashboards. It reuses the existing research core. See Fabric setup and architecture, dashboard design, the current handoff and verification (waves 1 to 5 history: Fabric handoff). Legacy CLI/stdio and their existing run directories remain separate entry points. Other agents on this machine reach it through the prowl gateway profile with the service's own agent token from loopback (tier T0, no delegation token needed). The callee side of Fabric hub delegation tokens (DEC-0039) is built, but a presented token is refused until an issuer is configured, and the hub publishes no keys yet (KA-G03). An ended research job is continued with research.run + resumeJobId without paying again for what was already paid (FABRIC.md).

License

MIT — see LICENSE.