diff --git a/AGENTS.md b/AGENTS.md index 4cf332f82..1fa97ad67 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -272,3 +272,14 @@ communities load. Do not expose credentials in logs or responses. If the identit or community loading cannot be verified, report that limitation before presenting the build as ready to test. Follow the worktree bootstrap instructions above; do not hardcode identity values into tracked files. + +## Inspect a live agent session + +When asked to inspect or debug an Agent conversation, open its live session viewer +for the human: + +```sh +bin/node scripts/session-debug.mjs '' +``` + +Keep it running for live updates. See [details](docs/session-debugging.md) if needed. diff --git a/docs/communities.md b/docs/communities.md index 66479b7a3..c9e8713bf 100644 --- a/docs/communities.md +++ b/docs/communities.md @@ -114,7 +114,10 @@ community origin and viewer; channel IDs alone are not sufficient keys. ## Session lifetime Selecting a saved membership lazily acquires its session. Sessions survive page -navigation and switching; startup opens only the selected one. Opened communities +navigation and switching; startup opens only the selected one. Once its agent +list loads, Agents2 also opens each joined community where the viewer has an +agent, through `open`, which connects without selecting so those agents keep +running, and retries one whose connection failed while it is not selected. Opened communities retain their existing per-session query/cache budgets. Arbitrary destinations mean there is **no longer a two-session maximum**: retained session count grows with communities opened until app disposal. This slice does not add background eviction diff --git a/docs/session-debugging.md b/docs/session-debugging.md new file mode 100644 index 000000000..88b337812 --- /dev/null +++ b/docs/session-debugging.md @@ -0,0 +1,110 @@ +# Inspect a live agent session + +From a buzz-app checkout: + +```sh +bin/node scripts/session-debug.mjs '' +``` + +The command opens a private loopback URL in your browser and follows Codex or +Claude JSONL records once a second. Leave the command running. Ctrl+C stops it. +It does not resume an agent, execute transcript content, or send data to a relay. +For a known file use `--file /absolute/session.jsonl`. `--no-open` prints the URL +without opening it. A partial last JSONL line is retried on the next refresh; +malformed complete records produce a visible error, not silent omission. + +The left rail is one vertical wall-time chart for the whole session, with time +running downward. Active time is proportional: inference, tools, inference +overlapping a tool, and **unattributed** gaps (a run is active but no model or +tool activity is recorded). **Idle** time between runs, when the agent waits for +the next prompt, collapses to a labeled break. The legend totals each category; +overlap is counted once, never added to wall time. Tool time is call-to-result +elapsed time, not subprocess CPU time. Beside each inference, a thin mark shows +its new (uncached) tokens; hover it for the full context window and cache share. +Hover or focus a bar segment for its timing and tool names; click either to jump +the conversation to that record. + +Native transcripts do not expose exact provider request start times. Inference +bars are explicitly estimated from the preceding logged input/result through +observed model output, and can include queue, prefill and dispatch overhead. With +no usable output boundary, the chart shows an untimed inference and leaves that +time unattributed. An explicit running turn extends to now; legacy sessions with +no completion record end at their last recorded message. Claude Code's zero-usage +`` replies and queue bookkeeping are not counted as activity. The chart +uses UTC to match the transcript timestamps. + +Each inference shows new-input, cache-read, cache-write (when nonzero), and output +tokens. Token counts are the default cost view. Optional dollar estimates appear +only when rates are supplied; missing prices do not produce placeholder warnings. +For Codex, cached tokens are part of input and are subtracted to find new input. +Claude already reports new input separately. Neither provider's cache usage is +counted twice. + +The conversation stays on the right, under one line naming the harness, model +and reasoning effort (when the transcript records it). Messages the agent posted +to Buzz are highlighted; plain assistant text was never sent. Search filters the +conversation. The view follows new records while it is scrolled to the bottom +and leaves your position alone otherwise; expanded disclosures survive updates. +These are recorded transcript views, not exact reconstructed HTTP inference +requests. When multiple sessions match a thread, only one is shown; open a +specific one with `--file`. Thread discovery checks user delivery envelopes, +rather than matches in tool outputs that merely quoted a thread. Channel-scoped +sessions and DMs may require `--file`, and can contain multiple Buzz threads. + +## Recorded fields and cost + +**Prompt** opens what the model received before its first response. Instructions +are tagged with the role they were sent as (System, Developer, User, or Settings +for run configuration) and render as Markdown: the system prompt first, then +the rest in recorded order. Tools show one row per tool, opening to its +description and JSON schema. Context injected later, such as Claude Code's +reminders, stays in the conversation as faint rows. + +Codex supplies `session_meta.base_instructions`, `dynamic_tools`, developer/user +messages, response items, lifecycle events, and token usage when supported by its +CLI version. Run settings (approval, sandbox, collaboration mode, effort) come from +`turn_context` and show again when they change. Codex does not record its native +tool schemas, such as shell and apply_patch. The viewer prefers per-response `token_usage_record`, deduplicates +response IDs, and falls back to changes in cumulative token counts. It does not +add those two sources together. The chart uses timestamp intervals to account for +wall time; recorded `duration_ms` remains available in raw records. + +Claude's native transcript supplies messages and usage. Claude Code also writes a +`prompt_snapshot` with the system prompt and tool schemas, shown once per change, +and the rendered text and role of each reminder it injects (environment, skills, +CLAUDE.md, hook context). Request framing, such as billing and SDK system blocks, +beta headers and deferred tool schemas, is not recorded. Content blocks can repeat +the same message ID and usage; the viewer counts each response once. Legacy turn +boundaries are inferred from user messages; tool results do not start turns. +Without a completion event the viewer does not pretend that the last token's +arrival proves completion. Inferred boundaries are labeled. + +Dollar costs are unavailable unless reported or a rate file is supplied. They are +never represented as zero merely because the account uses a subscription. Claude +result totals are process-cumulative; the viewer derives differences within each +captured launch and labels the result. Exact billing can differ from API estimates. + +To estimate costs, create a JSON object keyed by exact model ID with USD per +million token rates, then pass `--rates /path/to/rates.json`: + +```json +{"your-model-id":{"input":2,"output":10,"cached":0.2,"written":2.5}} +``` + +These are illustrative rates, not current pricing. Supply all four rates. No +models or prices are guessed. The UI labels estimates. For Claude, input excludes +cache reads and writes; for Codex those are subtracted from inclusive input before +pricing. Models with tiered context, service tiers, or cache TTL pricing need rates +appropriate to the inspected usage; the simple estimator does not infer tiers. + +## Storage and older exports + +Discovery reads `$CODEX_HOME/sessions`, `$CODEX_HOME/archived_sessions` (default +`~/.codex`) and `$CLAUDE_CONFIG_DIR/projects` (default `~/.claude`). Compressed +`.jsonl.zst` rollouts must be decompressed to a local file first and opened with +`--file`. The server is bound to 127.0.0.1, uses an unguessable path, rejects foreign +Host/Origin requests, and serves only its three assets and selected transcript. + +For standalone snapshots, the existing +[Codex](view-codex-session.md) and [Claude](view-claude-session.md) exporters remain +available. A snapshot does not update. diff --git a/docs/view-claude-session.md b/docs/view-claude-session.md index a23949e92..c0ab51fc3 100644 --- a/docs/view-claude-session.md +++ b/docs/view-claude-session.md @@ -1,5 +1,10 @@ # View a Claude Code session started by Buzz +For the live viewer, run `bin/node scripts/session-debug.mjs ''` +from the repository root. See [session debugging](session-debugging.md) for wall +time, token use and live updates. The commands below remain useful for +standalone snapshots and manual transcript inspection. + This guide is for agents investigating a Buzz conversation. All commands run non-interactively and read the saved transcript without invoking Claude or tools. diff --git a/docs/view-codex-session.md b/docs/view-codex-session.md index c5b47ae74..00ed57a45 100644 --- a/docs/view-codex-session.md +++ b/docs/view-codex-session.md @@ -1,5 +1,10 @@ # View a Codex session started by Buzz +For the live viewer, run `bin/node scripts/session-debug.mjs ''` +from the repository root. See [session debugging](session-debugging.md) for wall +time, token use and live updates. The commands below remain useful for +standalone snapshots and manual transcript inspection. + This guide is for agents investigating a Buzz conversation. All commands run non-interactively and read the saved transcript without invoking Codex or tools. diff --git a/scripts/export-claude-session.mjs b/scripts/export-claude-session.mjs index b1d49a3ec..ac2cab8bf 100644 --- a/scripts/export-claude-session.mjs +++ b/scripts/export-claude-session.mjs @@ -1,5 +1,6 @@ #!/usr/bin/env node -import { entry, exportSession } from "./session-viewer.mjs"; +import { pathToFileURL } from "node:url"; +import { entry, exportSession, text } from "./session-viewer.mjs"; // Tool results are a string or a list of text/image content blocks. const resultText = (content) => @@ -16,7 +17,7 @@ const target = (input) => .map((key) => input?.[key]) .find((value) => typeof value === "string") ?? ""; -exportSession((records) => { +export function buildClaude(records) { const messages = records.filter((record) => record?.message != null); const results = new Map( messages.flatMap((record) => @@ -29,7 +30,79 @@ exportSession((records) => { ); const entries = []; let model; - for (const { timestamp, type, isMeta, message } of messages) { + // Claude Code writes `prompt_snapshot`s in pairs: the first, before a + // response, lacks the CLI prefix and tools that the second fills in. Show the + // pair as one, at the first, and again only when the prompt or tools change. + let system; + let tools; + const deferred = records.some( + (record) => record?.attachment?.type === "deferred_tools_delta", + ); + for (const record of records) { + const { timestamp, type, isMeta, message, attachment } = record; + if (type === "attachment" && attachment?.type === "prompt_snapshot") { + const key = JSON.stringify(attachment.systemPrompt); + const blocks = [ + attachment.cliPrefix, + ...(attachment.systemPrompt ?? []), + ].filter((block) => typeof block === "string" && block.trim()); + const prompt = blocks.join("\n\n"); + const row = (at) => + entry.instructions(at, "System prompt", prompt, { + scope: "system", + blocks, + }); + if (prompt && key !== system?.key) { + system = { key, prefixed: !!attachment.cliPrefix, at: entries.length }; + entries.push(row(timestamp)); + } else if (prompt && attachment.cliPrefix && !system.prefixed) { + system.prefixed = true; + entries[system.at] = row(entries[system.at].timestamp); + } + // `schema` is the wire definition; name and description show apart. + const defs = (attachment.tools ?? []).map((tool) => { + const { + name: _, + description: __, + ...schema + } = tool.schema ?? tool.input_schema ?? {}; + return { name: tool.name, description: tool.description, schema }; + }); + if (defs.length && JSON.stringify(defs) !== tools) { + tools = JSON.stringify(defs); + const row = (at) => + entry.tools(at, `Tools · ${defs.length}`, defs, { + note: deferred + ? "Deferred tools load on demand; only their names are recorded, in the deferred tools reminder." + : null, + }); + if (system && !system.tools) + entries.splice(system.at + 1, 0, row(entries[system.at].timestamp)); + else entries.push(row(timestamp)); + } + if (system && defs.length) system.tools = true; + continue; + } + // Other attachments that reached the model carry their rendered text. + if (type === "attachment") { + const body = (record.rendered ?? []) + .map((part) => text(part?.content)) + .join("\n\n") + .trim(); + if (body) + entries.push( + entry.instructions( + timestamp, + [record.renderedRole, attachment?.type?.replaceAll("_", " ")] + .filter(Boolean) + .join(" · ") || "context", + body, + { scope: record.renderedRole }, + ), + ); + continue; + } + if (message == null) continue; if (message.model && message.model !== model) { model = message.model; entries.push(entry.model(timestamp, model)); @@ -46,7 +119,7 @@ exportSession((records) => { if (body) entries.push( isMeta - ? entry.instructions(timestamp, "meta", body) + ? entry.instructions(timestamp, "meta", body, { scope: "user" }) : entry.user(timestamp, body, buzzRequest(body)), ); for (const block of blocks) @@ -60,6 +133,12 @@ exportSession((records) => { else if (block?.type === "thinking") { if (block.thinking?.trim()) entries.push(entry.thinking(timestamp, block.thinking.trim())); + } else if ( + block?.type === "tool_use" && + /(^|__)buzz__send$/.test(block.name) && + typeof block.input?.text === "string" + ) { + entries.push(entry.sent(timestamp, block.input.text)); } else if (block?.type === "tool_use") { const result = results.get(block.id); entries.push( @@ -88,4 +167,7 @@ exportSession((records) => { ], entries, }; -}); +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) + exportSession(buildClaude); diff --git a/scripts/export-codex-session.mjs b/scripts/export-codex-session.mjs index 1761c48a0..a1ae74a3d 100644 --- a/scripts/export-codex-session.mjs +++ b/scripts/export-codex-session.mjs @@ -1,4 +1,5 @@ #!/usr/bin/env node +import { pathToFileURL } from "node:url"; import { entry, exportSession, text } from "./session-viewer.mjs"; const contentText = (content) => @@ -17,6 +18,14 @@ const isContext = (value) => /^(# AGENTS\.md instructions||)/.test( value, ); +// Name injected context by its role and leading tag, as `developer · skills instructions`. +const contextLabel = (role, body) => + `${role} · ${ + /^# AGENTS\.md/.test(body) + ? "AGENTS.md" + : (body.match(/^<([A-Za-z][\w.-]*)>/)?.[1].replace(/[_.]/g, " ") ?? + "instructions") + }`; // Buzz turns end with `Request: ""`; show that instead of context. function buzzRequest(value) { const match = value.match(/^Request: (".*")$/m); @@ -27,7 +36,15 @@ function buzzRequest(value) { } } -exportSession((records) => { +function jsonString(literal) { + try { + return JSON.parse(literal); + } catch { + return undefined; + } +} + +export function buildCodex(records) { const meta = records.find((record) => record?.type === "session_meta")?.payload ?? {}; const outputs = new Map( @@ -37,7 +54,28 @@ exportSession((records) => { ); const entries = []; let model; + let settings; for (const { timestamp, type, payload: item } of records) { + // Approval, sandbox, collaboration mode and effort, shown when they change. + if (type === "turn_context" && item) { + const { turn_id: _, root_turn_id: __, ...rest } = item; + const next = JSON.stringify( + { + ...rest, + model_provider: meta.model_provider, + context_window: meta.context_window, + }, + null, + 2, + ); + if (next !== settings) + entries.push( + entry.instructions(timestamp, "Run settings", next, { + scope: "settings", + }), + ); + settings = next; + } if (type === "turn_context" && item?.model && item.model !== model) { model = item.model; entries.push( @@ -53,11 +91,9 @@ exportSession((records) => { if (!body) continue; if (item.role === "developer" || isContext(body)) entries.push( - entry.instructions( - timestamp, - item.role === "developer" ? "instructions" : "context", - body, - ), + entry.instructions(timestamp, contextLabel(item.role, body), body, { + scope: item.role, + }), ); else if (item.role === "user") entries.push(entry.user(timestamp, body, buzzRequest(body))); @@ -67,19 +103,36 @@ exportSession((records) => { if (summary) entries.push(entry.thinking(timestamp, summary)); } else if (item.call_id && !item.type.endsWith("_output")) { const input = item.input ?? item.arguments ?? item.action ?? ""; - // Code Mode runs nested tools from an `exec` script; name them. - const nested = new Set( - [...text(input).matchAll(/tools\.([A-Za-z0-9_]+)/g)].map((m) => m[1]), - ); const output = outputs.get(item.call_id); - entries.push( + const tool = (name) => entry.tool(timestamp, { - name: item.name || item.type, - detail: [...nested].join(", "), + name, input, result: output && outputText(output.output), - }), - ); + }); + // Code Mode runs nested tools from an `exec` script: name the calls in + // order, and show each Buzz send as the message it posted. + let group = []; + const flush = () => { + if (group.length) entries.push(tool([...new Set(group)].join(", "))); + group = []; + }; + const calls = [ + ...text(input).matchAll( + /tools\.([A-Za-z0-9_]+)\(\s*(?:\{\s*"?text"?\s*:\s*("(?:[^"\\]|\\.)*"))?/g, + ), + ]; + for (const [, name, literal] of calls) { + const body = name === "buzz__send" && literal && jsonString(literal); + if (body === undefined || body === false || body === "") + group.push(name); + else { + flush(); + entries.push(entry.sent(timestamp, body)); + } + } + flush(); + if (!calls.length) entries.push(tool(item.name || item.type)); } else if (!item.type.endsWith("_output")) { const { encrypted_content: _, ...rest } = item; entries.push(entry.other(timestamp, item.type, rest)); @@ -95,4 +148,7 @@ exportSession((records) => { ], entries, }; -}); +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) + exportSession(buildCodex); diff --git a/scripts/session-debug-client.js b/scripts/session-debug-client.js new file mode 100644 index 000000000..e3090bcf7 --- /dev/null +++ b/scripts/session-debug-client.js @@ -0,0 +1,771 @@ +const $ = (id) => document.getElementById(id); +const htmlEscape = (value) => + String(value ?? "").replace( + /[&<>"']/g, + (c) => + ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[ + c + ], + ); +const number = (n) => Number(n ?? 0).toLocaleString(); +const money = (n) => `$${n.toFixed(4)}`; +let state, + signature = "", + pending = false; +const duration = (ms) => { + const s = Math.max(0, ms) / 1000; + if (s < 1) return `${Math.round(s * 1000)}ms`; + if (s < 60) return `${s.toFixed(s < 10 ? 2 : 1)}s`; + return `${Math.floor(s / 60)}m ${Math.floor(s % 60)}s`; +}; +const compact = (n) => + n >= 1e6 + ? `${(n / 1e6).toFixed(n < 1e7 ? 2 : 1)}M` + : n >= 1000 + ? `${(n / 1000).toFixed(n < 10000 ? 1 : 0)}k` + : String(n); +// Dense duration for the summary line: 0.3s · 47s · 2m43 · 25m · 1h12. +const brief = (ms) => { + const s = Math.max(0, ms) / 1000; + if (s < 10) return `${s.toFixed(1)}s`; + if (s < 60) return `${Math.round(s)}s`; + if (s < 600) + return `${Math.floor(s / 60)}m${String(Math.floor(s % 60)).padStart(2, "0")}`; + if (s < 3600) return `${Math.round(s / 60)}m`; + return `${Math.floor(s / 3600)}h${String(Math.floor((s % 3600) / 60)).padStart(2, "0")}`; +}; +const clock = (ms) => new Date(ms).toISOString().slice(11, 19); +const kindNames = { + inference: "Inference", + tool: "Tools", + overlap: "Inference + tool", + unknown: "Unattributed", + idle: "Idle", +}; +const kindNotes = { + unknown: "Run active, but no model or tool activity recorded", + idle: "No run active; waiting for the next prompt", +}; +// One vertical time axis for the whole session. Active time is proportional; +// idle waits between runs collapse to a fixed-height break so a 25-minute pause +// can't flatten seconds of work. Each painted segment remembers which +// inference/tool items were active in it, for the hover readout. +const IDLE_HEIGHT = 30; +const PAD = 14; +const MIN_SCALE = 0.00225; +let chart = { segments: [], marks: [] }, + chartHtml = ""; +function runSegments(run) { + const items = run.items.filter( + (i) => (i.kind === "inference" || i.kind === "tool") && i.end > i.start, + ); + const edges = [ + ...new Set([run.start, run.end, ...items.flatMap((i) => [i.start, i.end])]), + ] + .filter((t) => t >= run.start && t <= run.end) + .sort((a, b) => a - b); + const out = []; + for (let k = 1; k < edges.length; k++) { + const a = edges[k - 1], + b = edges[k]; + const active = items.filter((i) => i.start < b && i.end > a); + const inference = active.some((i) => i.kind === "inference"); + const tool = active.some((i) => i.kind === "tool"); + const kind = + inference && tool + ? "overlap" + : inference + ? "inference" + : tool + ? "tool" + : "unknown"; + const key = active.map((i) => i.id).join("|"); + const last = out.at(-1); + if (last && last.key === key && last.end === a) last.end = b; + else out.push({ kind, key, start: a, end: b, items: active, run }); + } + return out; +} +// Min-height segments inflate the layout, so refit once with the overflow removed. +function layoutChart(timing, height) { + const first = layoutAt(timing, height, null); + if (first.height <= height || first.scale <= MIN_SCALE) return first; + return layoutAt( + timing, + height, + first.scale * + (1 - (first.height - height) / Math.max(1, first.height - first.fixed)), + ); +} +function layoutAt(timing, height, fitted) { + const perRun = timing.runs.map((run) => ({ run, segs: runSegments(run) })); + const activeMs = timing.runs.reduce((n, r) => n + (r.end - r.start), 0); + const breaks = timing.runs.filter( + (r, i) => i && r.start > timing.runs[i - 1].end, + ).length; + const fixed = PAD * 2 + breaks * IDLE_HEIGHT; + // px per ms: fit the active time to the rail, within readable bounds. + const scale = Math.max( + MIN_SCALE, + fitted ?? + Math.min(0.06, Math.max(240, height - fixed) / Math.max(1, activeMs)), + ); + const segments = [], + marks = [], + runs = []; + let y = PAD; + for (const [index, { run, segs }] of perRun.entries()) { + const previous = timing.runs[index - 1]; + if (previous && run.start > previous.end) { + segments.push({ + kind: "idle", + start: previous.end, + end: run.start, + items: [], + run, + y, + h: IDLE_HEIGHT, + }); + y += IDLE_HEIGHT; + } + const top = y; + for (const s of segs) { + s.y = y; + s.h = Math.max(3, (s.end - s.start) * scale); + segments.push(s); + y += s.h; + } + if (segs.length) { + segs[0].cap = "cap-top"; + segs.at(-1).cap = `${segs.length === 1 ? "cap-top " : ""}cap-bottom`; + } + runs.push({ run, y: top, h: y - top }); + for (const item of run.items) + if (item.kind === "inference") { + const at = + segs.find((s) => item.start >= s.start && item.start < s.end) ?? + segs.at(-1); + marks.push({ + item, + run, + y: at + ? at.y + + ((item.start - at.start) / Math.max(1, at.end - at.start)) * at.h + : top, + }); + } + } + return { segments, marks, runs, height: y + PAD, scale, fixed }; +} +const context = (t) => t.fresh + t.cached + t.written; +function renderSummary(timing) { + const tokens = { fresh: 0, cached: 0, written: 0, output: 0 }; + let inferences = 0, + peak = 0; + for (const run of timing.runs) + for (const i of run.items) + if (i.kind === "inference") { + inferences++; + peak = Math.max(peak, context(i.tokens)); + for (const k of Object.keys(tokens)) tokens[k] += i.tokens[k]; + } + const costs = state.turns.map((t) => t.cost ?? t.estimate); + const cost = costs.some((c) => c != null) + ? costs.reduce((a, c) => a + (c ?? 0), 0) + : null; + const input = context(tokens); + // Sub-second overlap stays in the tooltip; it would only crowd the line. + const parts = ["inference", "tool", "overlap", "unknown", "idle"] + .filter( + (k) => + timing.totals[k] >= (k === "overlap" ? 1000 : 1) || + k === "inference" || + k === "tool", + ) + .map( + (k) => + `${brief(timing.totals[k])}`, + ) + .join(""); + const tokenTitle = `${number(tokens.fresh)} new input · ${number(tokens.cached)} cache read · ${number(tokens.written)} cache write · ${number(tokens.output)} output · ${inferences} model calls · peak context ${number(peak)}`; + return `
${brief(timing.end - timing.start)}${parts}
${compact(input + tokens.output)} tokens · ${input ? Math.round((tokens.cached / input) * 100) : 0}% cached${cost == null ? "" : ` · ${money(cost)}`}
`; +} +function renderTiming(timing, force = false) { + if (!timing?.runs.length) { + $("summary").textContent = "Waiting for timing records…"; + $("timeline").replaceChildren(); + chartHtml = ""; + return; + } + const summary = renderSummary(timing); + if ($("summary").innerHTML !== summary) $("summary").innerHTML = summary; + const scroller = document.querySelector(".timeline-scroll"); + const scroll = scroller.scrollTop; + const atEnd = scroll + scroller.clientHeight >= scroller.scrollHeight - 40; + // Lay out 1.5× the visible height: taller segments are easier to hover, at + // the cost of some scrolling. + chart = layoutChart(timing, scroller.clientHeight * 1.5); + // Marks show each call's new (uncached) tokens; the full window is on hover. + const fresh = (t) => t.fresh + t.written; + const maxFresh = Math.max(1, ...chart.marks.map((m) => fresh(m.item.tokens))); + const last = chart.segments.at(-1); + const html = `
${chart.runs + .map( + (r) => + `${clock(r.run.start).slice(0, 5)}`, + ) + .join("")}${chart.segments + .map((s, i) => + s.kind === "idle" + ? `` + : ``, + ) + .join("")}${chart.marks + .map( + (m, i) => + ``, + ) + .join("")}
`; + if (force || chartHtml !== html) { + $("timeline").innerHTML = html; + chartHtml = html; + } + if (atEnd) scroller.scrollTop = scroller.scrollHeight; + else scroller.scrollTop = scroll; +} +function tipRow(color, value, label, detail) { + const row = document.createElement("div"); + row.className = "tip-row"; + const key = document.createElement("i"); + key.className = `key key-${color}`; + const strong = document.createElement("strong"); + strong.textContent = value; + const name = document.createElement("span"); + name.textContent = label; + row.append(key, strong, name); + if (detail) { + const small = document.createElement("small"); + small.textContent = detail; + row.append(small); + } + return row; +} +function inferenceRow(item) { + return tipRow( + "inference", + item.estimated ? duration(item.end - item.start) : "untimed", + "Inference", + ); +} +// Token mark readout: the whole context window as one bar (cached prefix, +// then this call's new tokens), with new/out counts beneath. +function contextReadout(item) { + const t = item.tokens; + const size = context(t); + const added = t.fresh + t.written; + const bar = document.createElement("div"); + bar.className = "ctx-bar"; + const cached = document.createElement("i"); + cached.className = "ctx-cached"; + cached.style.width = `${size ? (t.cached / size) * 100 : 0}%`; + const fresh = document.createElement("i"); + fresh.className = "ctx-new"; + fresh.style.width = `${size ? (added / size) * 100 : 0}%`; + bar.append(cached, fresh); + const stats = document.createElement("div"); + stats.className = "ctx-stats"; + const stat = (value, label, className) => { + const span = document.createElement("span"); + if (className) span.className = className; + const strong = document.createElement("strong"); + strong.textContent = value; + span.append(strong, ` ${label}`); + return span; + }; + stats.append( + stat(`+${compact(added)}`, "new"), + stat(compact(t.output), "out"), + ); + if (item.estimate != null) stats.append(stat(money(item.estimate), "")); + const total = document.createElement("span"); + total.className = "ctx-total"; + total.textContent = compact(size); + total.title = `${number(t.cached)} cached · ${number(t.fresh)} input · ${number(t.written)} cache write`; + stats.append(total); + return [bar, stats]; +} +function showTip(target, x, y) { + const tip = $("tip"); + const seg = chart.segments[target.dataset.seg]; + const mark = chart.marks[target.dataset.mark]; + if (!seg && !mark) return hideTip(); + if (mark) tip.replaceChildren(...contextReadout(mark.item)); + else { + const head = document.createElement("div"); + head.className = "tip-head"; + head.textContent = + seg.kind === "idle" + ? `${clock(seg.start)} – ${clock(seg.end)}` + : clock(seg.start); + const rows = seg.items.length + ? seg.items.map((item) => + item.kind === "inference" + ? inferenceRow(item) + : tipRow( + "tool", + item.open ? "running" : duration(item.end - item.start), + item.name ?? "tool", + ), + ) + : [ + tipRow( + seg.kind, + duration(seg.end - seg.start), + kindNames[seg.kind], + kindNotes[seg.kind], + ), + ]; + tip.replaceChildren(head, ...rows); + } + tip.classList.toggle("compact", !!mark); + tip.hidden = false; + const box = tip.getBoundingClientRect(); + tip.style.left = `${Math.min(innerWidth - box.width - 8, x + 16)}px`; + tip.style.top = `${Math.max(8, Math.min(innerHeight - box.height - 8, y - box.height / 2))}px`; +} +function hideTip() { + $("tip").hidden = true; +} +// Jump the conversation to the record nearest the hovered activity. +function jumpTo(target) { + const seg = chart.segments[target.dataset.seg]; + const item = + chart.marks[target.dataset.mark]?.item ?? + seg?.items.find((i) => i.kind === "tool") ?? + seg?.items[0]; + const at = item + ? item.kind === "inference" + ? item.end + : item.start + : seg?.start; + if (at == null || !state?.entries) return; + let best = null; + for (const e of state.entries) { + if (e.kind === "instructions") continue; + const node = document.getElementById(`row-${e.id}`); + const t = Date.parse(e.timestamp); + if (!node || !Number.isFinite(t)) continue; + if (!best || Math.abs(t - at) < Math.abs(best.t - at)) best = { node, t }; + } + if (!best) return; + best.node.scrollIntoView({ block: "center", behavior: "smooth" }); + best.node.classList.remove("flash"); + void best.node.offsetWidth; + best.node.classList.add("flash"); +} +// Hover targets: a context mark, or the bar column widened by 8px each side so +// 2px segments don't need precise aim. Elsewhere on the rail shows nothing. +let hot = null; +function targetAt(event) { + const mark = event.target.closest("[data-mark]"); + if (mark) return mark; + const bar = $("timeline").querySelector(".seg"); + if (!bar) return null; + const box = bar.getBoundingClientRect(); + const pad = 8; + if (event.clientX < box.left - pad || event.clientX > box.right + pad) + return event.target.closest(".idle-break"); + const y = event.clientY - $("timeline").getBoundingClientRect().top; + const index = chart.segments.findIndex((s) => y >= s.y && y < s.y + s.h); + return $("timeline").querySelector(`[data-seg="${index}"]`); +} +function setHot(node) { + if (hot === node) return; + hot?.classList.remove("hot"); + hot = node; + hot?.classList.add("hot"); +} +$("timeline").addEventListener("pointermove", (event) => { + const target = targetAt(event); + setHot(target); + if (target) showTip(target, event.clientX, event.clientY); + else hideTip(); +}); +$("timeline").addEventListener("pointerleave", () => { + setHot(null); + hideTip(); +}); +$("timeline").addEventListener("focusin", (event) => { + const target = event.target.closest("[data-seg],[data-mark]"); + if (!target) return; + const box = target.getBoundingClientRect(); + showTip(target, box.right, box.top + box.height / 2); +}); +$("timeline").addEventListener("focusout", hideTip); +$("timeline").addEventListener("click", (event) => { + const target = targetAt(event); + if (target) jumpTo(target); +}); +// Minimap viewport: the conversation's on-screen time range as a band on the +// rail, like an editor scrollbar map. An entry spans until the next one starts; +// each band edge interpolates within the entry it crosses, so it glides with +// the scroll instead of stepping entry by entry. +function timeToY(t) { + const segs = chart.segments; + if (!segs.length) return null; + if (t <= segs[0].start) return segs[0].y; + for (const s of segs) + if (t < s.end) + return t <= s.start + ? s.y + : s.y + ((t - s.start) / Math.max(1, s.end - s.start)) * s.h; + const last = segs.at(-1); + return last.y + last.h; +} +function visibleEntries(top, bottom) { + const rows = []; + for (const node of $("content").children) { + const t = Date.parse(node.dataset.time); + if (!Number.isFinite(t)) continue; + const box = node.getBoundingClientRect(); + rows.push({ t, top: box.top, bottom: box.bottom }); + if (box.top > bottom) break; + } + const first = rows.findLastIndex((row) => row.top <= top); + return rows.slice(Math.max(0, first)); +} +function timeAtLine(rows, line) { + const end = chart.segments.at(-1)?.end ?? rows.at(-1).t; + let i = rows.findLastIndex((row) => row.top <= line); + if (i < 0) i = 0; + const row = rows[i], + next = rows[i + 1]; + const to = next ? next.t : Math.max(row.t, end); + const span = (next ? next.top : row.bottom) - row.top; + const share = + span > 0 ? Math.min(1, Math.max(0, (line - row.top) / span)) : 0; + return row.t + share * Math.max(0, to - row.t); +} +function updateViewport(reveal) { + const band = document.querySelector(".chart .viewport"); + if (!band) return; + const top = document.querySelector(".toolbar").getBoundingClientRect().bottom; + const rows = visibleEntries(top, innerHeight); + band.hidden = !rows.length || !chart.segments.length; + if (band.hidden) return; + const y0 = timeToY(timeAtLine(rows, top)), + y1 = Math.max(y0, timeToY(timeAtLine(rows, innerHeight))); + band.style.top = `${y0 - 3}px`; + band.style.height = `${Math.max(6, y1 - y0 + 6)}px`; + if (!reveal) return; + // Follow continuously: keep the band inside the rail with a margin, moving + // the rail only as far as the band itself moved past that margin. + const scroller = document.querySelector(".timeline-scroll"); + const margin = Math.min(48, scroller.clientHeight / 6); + const view = scroller.clientHeight - 2 * margin; + if (y1 - y0 > view || y0 < scroller.scrollTop + margin) + scroller.scrollTop = y0 - margin; + else if (y1 > scroller.scrollTop + scroller.clientHeight - margin) + scroller.scrollTop = y1 - scroller.clientHeight + margin; +} +let viewportFrame = 0; +addEventListener( + "scroll", + () => { + if (viewportFrame) return; + viewportFrame = requestAnimationFrame(() => { + viewportFrame = 0; + updateViewport(true); + }); + }, + { passive: true }, +); +addEventListener( + "resize", + () => state?.timing && renderTiming(state.timing, true), +); + +function render() { + if (!state || state.waiting) { + $("content").textContent = state?.message ?? "Loading…"; + return; + } + const nearBottom = + window.scrollY + innerHeight >= document.documentElement.scrollHeight - 100; + const open = new Set( + [...document.querySelectorAll("details[open]")].map((d) => d.dataset.key), + ); + const focused = document.activeElement; + const focusedKey = focused?.closest("details")?.dataset.key; + const scroll = window.scrollY; + const query = $("search").value.toLowerCase(); + const { harness, model, effort } = state.header ?? {}; + const scope = `${htmlEscape(harness)}${model ? `${htmlEscape(model)}` : ""}${effort ? `${htmlEscape(effort)}` : ""}`; + if ($("scope").innerHTML !== scope) $("scope").innerHTML = scope; + $("scope").title = state.sessionId ?? ""; + renderTiming(state.timing); + renderSetup(state.setup); + const rows = state.entries + .filter((e) => + `${e.label} ${e.preview} ${e.html}`.toLowerCase().includes(query), + ) + .map((e) => ({ + id: e.id, + time: e.timestamp, + html: `${e.kind === "tool" || e.kind === "instructions" ? "" : `

${htmlEscape(e.label)}

`}${e.html}`, + })); + // Reconcile entries by stable transcript position: live polling must not close + // a disclosure, reset selection, or steal focus while the person reads. + const container = $("content"); + const existing = new Map([...container.children].map((n) => [n.id, n])); + let previous = null; + for (const row of rows) { + const id = `row-${row.id}`; + let node = existing.get(id); + if (!node) { + node = document.createElement("section"); + node.className = "entry"; + node.id = id; + } + if (row.time) node.dataset.time = row.time; + else delete node.dataset.time; + if (node.dataset.html !== row.html) { + node.innerHTML = row.html; + node.dataset.html = row.html; + node.querySelectorAll("details").forEach((d, i) => { + d.dataset.key ||= `${id}-${i}`; + d.open = open.has(d.dataset.key); + }); + } + if ( + previous + ? previous.nextElementSibling !== node + : container.firstElementChild !== node + ) + container.insertBefore( + node, + previous ? previous.nextElementSibling : container.firstElementChild, + ); + previous = node; + existing.delete(id); + } + for (const node of existing.values()) node.remove(); + if (!rows.length) { + const empty = document.createElement("p"); + empty.className = "empty"; + empty.textContent = "No matching records."; + container.append(empty); + } + if (focusedKey && !document.contains(focused)) + [...container.querySelectorAll("details")] + .find((d) => d.dataset.key === focusedKey) + ?.querySelector("summary") + ?.focus({ preventScroll: true }); + if (nearBottom) window.scrollTo(0, document.documentElement.scrollHeight); + else window.scrollTo(0, scroll); + updateViewport(false); +} +// Prompt setup: what the model was given before its first response, apart +// from the conversation. Transcript text is untrusted: everything is escaped +// before the small Markdown subset below adds its own tags. +const SCOPES = { + system: "System", + developer: "Developer", + user: "User", + settings: "Settings", +}; +const BOUNDARY = "__SYSTEM_PROMPT_DYNAMIC_BOUNDARY__"; +const inline = (text) => + htmlEscape(text) + .replace(/`([^`\n]+)`/g, "$1") + .replace(/\*\*([^*\n]+)\*\*/g, "$1"); +// Headings, lists, fenced code, tag-only lines, inline code and bold. No links +// or raw HTML. +function markdown(source) { + const out = []; + let para = [], + list = [], + code = null; + const flush = () => { + if (para.length) out.push(`

${para.map(inline).join("
")}

`); + if (list.length) + out.push( + `
    ${list.map(([depth, text]) => `
  • ${inline(text)}
  • `).join("")}
`, + ); + para = []; + list = []; + }; + for (const line of String(source ?? "").split("\n")) { + if (code) { + if (/^\s*```/.test(line)) { + out.push( + `
${htmlEscape(code.join("\n"))}
`, + ); + code = null; + } else code.push(line); + continue; + } + const heading = line.match(/^(#{1,6})\s+(.+)$/); + const item = line.match(/^(\s*)(?:[-*]|\d+[.)])\s+(.+)$/); + if (/^\s*```/.test(line)) { + flush(); + code = []; + } else if (heading) { + flush(); + const level = Math.min(6, heading[1].length + 2); + out.push(`${inline(heading[2])}`); + } else if (/^\s*<\/?[A-Za-z][\w.-]*>\s*$/.test(line)) { + flush(); + out.push(`
${htmlEscape(line.trim())}
`); + } else if (item) { + if (para.length) flush(); + list.push([Math.min(3, Math.floor(item[1].length / 2)), item[2]]); + } else if (!line.trim()) flush(); + else if (list.length && /^\s/.test(line)) + list.at(-1)[1] += ` ${line.trim()}`; + else { + if (list.length) flush(); + para.push(line); + } + } + if (code) + out.push( + `
${htmlEscape(code.join("\n"))}
`, + ); + flush(); + return out.join(""); +} +// Pretty JSON with keys, strings, numbers and literals told apart. +function json(value, indent = "") { + const next = `${indent} `; + if (Array.isArray(value)) + return value.length + ? `[\n${value.map((v) => next + json(v, next)).join(",\n")}\n${indent}]` + : "[]"; + if (value && typeof value === "object") { + const fields = Object.entries(value); + return fields.length + ? `{\n${fields.map(([k, v]) => `${next}${htmlEscape(JSON.stringify(k))}: ${json(v, next)}`).join(",\n")}\n${indent}}` + : "{}"; + } + const kind = + typeof value === "string" + ? "j-str" + : typeof value === "number" + ? "j-num" + : "j-lit"; + return `${htmlEscape(JSON.stringify(value))}`; +} +const parsed = (text) => { + try { + return JSON.parse(text); + } catch { + return undefined; + } +}; +function setupBody(item) { + if (item.blocks) + return item.blocks + .map((block) => + block === BOUNDARY + ? `
${htmlEscape(block)}
` + : `
${markdown(block)}
`, + ) + .join(""); + const value = item.scope === "settings" && parsed(item.body); + return value ? `
${json(value)}
` : markdown(item.body); +} +let setupKey = ""; +function renderSetup(setup) { + const key = JSON.stringify(setup ?? null); + if (key === setupKey) return; + setupKey = key; + const { instructions = [], tools = [] } = setup ?? {}; + const count = tools.reduce((n, group) => n + group.tools.length, 0); + $("prompt-toggle").hidden = !instructions.length && !count; + $("tab-tools").textContent = `Tools · ${count}`; + const items = instructions + .map((item) => { + const scope = SCOPES[item.scope] ? item.scope : "other"; + const label = item.label.replace(/^(system|developer|user) · /, ""); + const at = Date.parse(item.timestamp); + return `
${SCOPES[scope] ?? "Context"}${htmlEscape(label)}${Number.isFinite(at) ? `${clock(at)}` : ""}
${setupBody(item)}
`; + }) + .join(""); + $("prompt-instructions").innerHTML = items + ? `

What the model received before its first response. Later context stays in the conversation.

${items}` + : `

No instructions recorded.

`; + $("prompt-tools").innerHTML = + tools + .map( + (group) => + `${group.note ? `

${htmlEscape(group.note)}

` : ""}${group.tools + .map( + (tool) => + `
${htmlEscape(tool.name)}${htmlEscape(String(tool.description ?? "").split("\n")[0])}
${markdown(tool.description)}
${tool.schema ? `
${json(tool.schema)}
` : ""}
`, + ) + .join("")}`, + ) + .join("") || `

No tool definitions recorded.

`; +} +function showPrompt(open) { + $("prompt").hidden = !open; + $("prompt-toggle").setAttribute("aria-expanded", String(open)); + if (open) $("prompt-close").focus(); + else $("prompt-toggle").focus(); +} +function showTab(name) { + for (const tab of ["instructions", "tools"]) { + $(`tab-${tab}`).setAttribute("aria-selected", String(tab === name)); + $(`prompt-${tab}`).hidden = tab !== name; + } +} +$("prompt-toggle").addEventListener("click", () => + showPrompt($("prompt").hidden), +); +$("prompt-close").addEventListener("click", () => showPrompt(false)); +$("tab-instructions").addEventListener("click", () => showTab("instructions")); +$("tab-tools").addEventListener("click", () => showTab("tools")); +addEventListener("keydown", (event) => { + if (event.key === "Escape" && !$("prompt").hidden) showPrompt(false); +}); +$("search").addEventListener("input", render); +$("theme").addEventListener("click", () => { + document.documentElement.dataset.theme = + (document.documentElement.dataset.theme || + (matchMedia("(prefers-color-scheme:dark)").matches + ? "dark" + : "light")) === "dark" + ? "light" + : "dark"; +}); +function setStatus(kind, text, detail) { + const status = $("status"); + status.dataset.state = kind; + status.title = detail; + if (status.textContent !== text) status.textContent = text; +} +async function poll() { + if (pending) return; + pending = true; + try { + const response = await fetch(new URL("state", location.href), { + cache: "no-store", + }); + const next = await response.json(); + if (!response.ok) throw new Error(next.error ?? "Could not read session"); + const nextSignature = JSON.stringify(next); + setStatus("live", "Live", `Checked ${new Date().toLocaleTimeString()}`); + if (signature !== nextSignature) { + state = next; + signature = nextSignature; + render(); + } + } catch (error) { + setStatus("error", "Reconnecting…", error.message); + } finally { + pending = false; + } +} +await poll(); +setInterval(poll, 1000); diff --git a/scripts/session-debug.css b/scripts/session-debug.css new file mode 100644 index 000000000..bc247d709 --- /dev/null +++ b/scripts/session-debug.css @@ -0,0 +1,1056 @@ +/* Standalone diagnostic renderer; no application host or React runtime. One + quiet surface: a tinted rail for the wall-time chart beside a borderless + conversation column. Chart hues are categorical slots 1–3 of the dataviz + reference palette (validated in light and dark); unattributed and idle stay + neutral so they never read as work. */ +:root { + color-scheme: light dark; + --base: #fafaf9; + --rail: #f4f4f3; + --panel: #fff; + --inset: #ebebea; + --text: #1c1c1c; + --muted: #62625f; + --faint: #9a9a96; + --line: #e4e4e2; + --accent: #457b9d; + --hover: #ededec; + --error: #e63946; + --shadow: 0 8px 28px rgb(20 20 18 / 0.12), 0 1px 3px rgb(20 20 18 / 0.08); + --k-inference: #457b9d; + --k-tool: #1d3557; + --k-unknown: #a1a19d; + --k-cached: #a8dadc; + --mono: ui-monospace, "SF Mono", Menlo, monospace; + font: + 100% / 1.5 system-ui, + -apple-system, + sans-serif; +} +@media (prefers-color-scheme: dark) { + :root:not([data-theme="light"]) { + --base: #111111; + --rail: #161616; + --panel: #1d1d1d; + --inset: #262626; + --text: #ececea; + --muted: #a3a3a0; + --faint: #6e6e6b; + --line: #2a2a2a; + --accent: #a8dadc; + --hover: #262626; + --error: #e63946; + --shadow: 0 8px 28px rgb(0 0 0 / 0.5), 0 0 0 1px rgb(255 255 255 / 0.06); + --k-inference: #a8dadc; + --k-tool: #457b9d; + --k-unknown: #6e6e6b; + --k-cached: #2f4f68; + } +} +:root[data-theme="dark"] { + color-scheme: dark; + --base: #111111; + --rail: #161616; + --panel: #1d1d1d; + --inset: #262626; + --text: #ececea; + --muted: #a3a3a0; + --faint: #6e6e6b; + --line: #2a2a2a; + --accent: #a8dadc; + --hover: #262626; + --error: #e63946; + --shadow: 0 8px 28px rgb(0 0 0 / 0.5), 0 0 0 1px rgb(255 255 255 / 0.06); + --k-inference: #a8dadc; + --k-tool: #457b9d; + --k-unknown: #6e6e6b; + --k-cached: #2f4f68; +} +:root[data-theme="light"] { + color-scheme: light; +} +* { + box-sizing: border-box; +} +body { + margin: 0; + background: var(--base); + color: var(--text); + font-size: 0.875rem; + -webkit-font-smoothing: antialiased; +} +input, +button { + font: inherit; + color: inherit; +} +button, +summary { + cursor: pointer; +} +:focus-visible { + outline: 2px solid var(--accent); + outline-offset: 2px; +} +.workspace { + display: grid; + grid-template-columns: 16rem minmax(0, 1fr); + min-height: 100dvh; +} + +/* ── Rail ─────────────────────────────────────────────────────────────── */ +aside { + display: flex; + flex-direction: column; + min-width: 0; + height: 100dvh; + position: sticky; + top: 0; + background: var(--rail); + border-right: 1px solid var(--line); +} +.rail-head { + padding: 0.875rem 1rem 0.625rem; + font-size: 0.75rem; + color: var(--muted); + font-variant-numeric: tabular-nums; +} +.rail-title { + display: flex; + align-items: center; + justify-content: space-between; + margin-bottom: 0.625rem; +} +.rail-title strong { + color: var(--text); + font-size: 0.8125rem; + font-weight: 600; +} +#status { + display: inline-flex; + align-items: center; + gap: 0.375rem; + font-size: 0.6875rem; + color: var(--faint); +} +#status::before { + content: ""; + width: 0.375rem; + height: 0.375rem; + border-radius: 50%; + background: var(--faint); +} +#status[data-state="live"]::before { + background: var(--accent); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 22%, transparent); +} +#status[data-state="error"] { + color: var(--error); +} +#status[data-state="error"]::before { + background: var(--error); +} +.rail-total { + display: flex; + align-items: baseline; + gap: 0.5rem; + white-space: nowrap; +} +.rail-total strong { + color: var(--text); + font-size: 1.0625rem; + font-weight: 600; + letter-spacing: -0.01em; + margin-right: 0.125rem; +} +.rail-total span { + display: inline-flex; + align-items: center; + gap: 0.25rem; + color: var(--text); + cursor: help; +} +.key { + display: inline-block; + flex-shrink: 0; + width: 0.4375rem; + height: 0.4375rem; + border-radius: 2px; +} +.key-inference { + background: var(--k-inference); +} +.key-tool { + background: var(--k-tool); +} +/* Overlap is both at once: stripes of the two hues, not a third color. */ +.key-overlap { + background: repeating-linear-gradient( + 135deg, + var(--k-inference) 0 2px, + var(--k-tool) 2px 4px + ); +} +.key-unknown { + background: repeating-linear-gradient( + 135deg, + var(--k-unknown) 0 1.5px, + transparent 1.5px 3.5px + ); +} +.key-idle { + background: none; + border: 1.5px dotted var(--faint); + border-radius: 50%; +} +.rail-tokens { + margin-top: 0.25rem; + cursor: help; +} +.rail-tokens strong { + color: var(--text); + font-weight: 600; +} +.timeline-scroll { + position: relative; + overflow: auto; + min-height: 0; + flex: 1; + scrollbar-width: none; + border-top: 1px solid var(--line); +} +.timeline-scroll:focus-visible { + outline-offset: -2px; +} + +/* The chart: time label · bar · context marks. */ +.chart { + position: relative; + --bar-left: 3.25rem; + --bar-width: 0.5rem; + --tok-left: 4.5rem; + --tok-room: calc(100% - var(--tok-left) - 1rem); +} +.run-label { + position: absolute; + left: 0; + width: calc(var(--bar-left) - 0.5rem); + text-align: right; + font-size: 0.6875rem; + line-height: 1; + color: var(--faint); + font-variant-numeric: tabular-nums; +} +.run-track { + position: absolute; + left: var(--bar-left); + width: var(--bar-width); + border-radius: 999px; + background: var(--inset); +} +.seg, +.idle-break { + position: absolute; + padding: 0; + border: 0; + margin: 0; + cursor: inherit; +} +.seg { + left: var(--bar-left); + width: var(--bar-width); + transition: + transform 0.12s ease, + filter 0.12s ease; +} +.seg.cap-top { + border-top-left-radius: 999px; + border-top-right-radius: 999px; +} +.seg.cap-bottom { + border-bottom-left-radius: 999px; + border-bottom-right-radius: 999px; +} +.seg.key-unknown { + background: repeating-linear-gradient( + 135deg, + color-mix(in srgb, var(--k-unknown) 75%, transparent) 0 1.5px, + transparent 1.5px 4px + ); +} +.seg.hot, +.seg:focus-visible { + transform: scaleX(1.75); + filter: saturate(1.15) brightness(1.08); + z-index: 1; +} +.seg:focus-visible { + outline-offset: 3px; +} +.seg.live { + animation: pulse 1.6s ease-in-out infinite; +} +@keyframes pulse { + 50% { + opacity: 0.4; + } +} +.idle-break { + left: 0; + right: 0; + background: none; + display: flex; + align-items: center; + color: var(--faint); + font-size: 0.6875rem; + font-variant-numeric: tabular-nums; +} +.idle-break::before { + content: ""; + position: absolute; + left: calc(var(--bar-left) + var(--bar-width) / 2 - 1px); + top: 6px; + bottom: 6px; + border-left: 2px dotted color-mix(in srgb, var(--faint) 70%, transparent); +} +.idle-break span { + margin-left: var(--tok-left); +} +.idle-break.hot, +.idle-break:hover { + color: var(--muted); +} +/* Context window readout: cached prefix + this call's new tokens. */ +.ctx-bar { + display: flex; + width: 12rem; + height: 0.375rem; + border-radius: 999px; + overflow: hidden; + background: var(--inset); + margin: 0.125rem 0 0.375rem; +} +.ctx-bar i { + display: block; + height: 100%; +} +.ctx-bar .ctx-cached { + background: var(--k-cached); +} +.ctx-bar .ctx-new { + background: var(--k-inference); + min-width: 2px; +} +.ctx-stats { + display: flex; + gap: 0.625rem; + color: var(--muted); + font-variant-numeric: tabular-nums; +} +.ctx-stats strong { + color: var(--text); + font-weight: 600; +} +.ctx-stats .ctx-total { + margin-left: auto; + color: var(--faint); +} +/* New (uncached) tokens per call; an 8px transparent hit box around a 2px line. */ +.tok { + position: absolute; + left: var(--tok-left); + width: calc(var(--tok-room) * var(--share)); + min-width: 3px; + height: 8px; + margin-top: -3px; + padding: 3px 0; + background: none; + border: 0; + cursor: pointer; +} +.tok i { + display: block; + height: 2px; + border-radius: 1px; + background: color-mix(in srgb, var(--k-inference) 70%, var(--rail)); +} +.tok.hot i { + background: var(--k-inference); + height: 3px; + margin-top: -0.5px; +} +@media (prefers-reduced-motion: reduce) { + .seg { + transition: none; + } + .seg.live { + animation: none; + } +} + +/* Viewport band: the conversation's visible range, minimap style. */ +.viewport { + position: absolute; + left: 0.375rem; + right: 0.375rem; + min-height: 6px; + border-radius: 0.375rem; + background: color-mix(in srgb, var(--text) 6%, transparent); + pointer-events: none; +} +#tip.compact { + min-width: 0; +} + +/* Tooltip: values lead, labels follow; keys are short strokes. */ +#tip { + position: fixed; + z-index: 10; + min-width: 11rem; + max-width: 20rem; + padding: 0.5rem 0.75rem 0.625rem; + border-radius: 0.625rem; + background: color-mix(in srgb, var(--panel) 92%, transparent); + backdrop-filter: blur(12px); + box-shadow: var(--shadow); + font-size: 0.75rem; + pointer-events: none; +} +.tip-head { + color: var(--faint); + font-size: 0.6875rem; + margin-bottom: 0.375rem; + font-variant-numeric: tabular-nums; +} +.tip-row { + display: grid; + grid-template-columns: 0.625rem auto 1fr; + column-gap: 0.5rem; + align-items: center; +} +.tip-row + .tip-row { + margin-top: 0.375rem; +} +.tip-row .key { + width: 0.625rem; + height: 2px; + border: 0; + border-radius: 1px; +} +.tip-row .key-unknown { + height: 0.5rem; +} +.tip-row .key-idle { + height: 0; + border-top: 2px dotted var(--faint); + background: none; +} +.tip-row strong { + font-weight: 600; + font-variant-numeric: tabular-nums; +} +.tip-row span { + color: var(--muted); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.tip-row small { + grid-column: 2 / -1; + color: var(--muted); + font-size: 0.6875rem; + font-variant-numeric: tabular-nums; +} + +/* ── Conversation ─────────────────────────────────────────────────────── */ +main { + min-width: 0; + width: 100%; + max-width: 60rem; + margin: 0 auto; + padding: 0 2rem 4rem; +} +.toolbar { + display: flex; + gap: 0.5rem; + align-items: center; + position: sticky; + top: 0; + z-index: 3; + margin: 0 -0.5rem 0.5rem; + padding: 0.75rem 0.5rem; + background: color-mix(in srgb, var(--base) 86%, transparent); + backdrop-filter: blur(10px); +} +#search, +.toolbar button { + height: 2rem; + border: 1px solid var(--line); + border-radius: 0.5rem; + background: var(--panel); + font-size: 0.8125rem; +} +/* Compact until used: focus or a query widens it toward the session line. */ +#search { + flex: 0 1 10rem; + min-width: 6rem; + padding: 0 0.75rem; + transition: flex-basis 0.18s ease-out; +} +#search:focus, +#search:not(:placeholder-shown) { + flex-basis: 26rem; +} +#search:focus-visible { + outline: none; + border-color: var(--accent); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 18%, transparent); +} +@media (prefers-reduced-motion: reduce) { + #search { + transition: none; + } +} +#search::placeholder { + color: var(--faint); +} +.toolbar button:hover { + background: var(--hover); +} +#theme { + flex: none; + width: 2rem; + padding: 0; + color: var(--muted); +} +/* Session identity: harness, then model and effort as quiet chips. */ +#scope { + display: flex; + flex: 1; + align-items: center; + gap: 0.375rem; + min-width: 0; + margin: 0; + overflow: hidden; + white-space: nowrap; + font-size: 0.75rem; + color: var(--muted); +} +#scope strong { + color: var(--text); + font-weight: 600; + margin-right: 0.25rem; +} +#scope .chip { + padding: 0.0625rem 0.5rem; + border-radius: 999px; + background: var(--inset); + font: 0.6875rem / 1.6 var(--mono); +} +#scope .effort { + color: var(--accent); + background: color-mix(in srgb, var(--accent) 12%, transparent); +} +details.tool .status { + color: var(--error); + font-family: system-ui, sans-serif; +} +.muted { + color: var(--muted); +} +.accent { + color: var(--muted); +} +.entry { + padding: 0.375rem 0; + scroll-margin-top: 5rem; + overflow-wrap: anywhere; + border-radius: 0.5rem; +} +.entry h2 { + display: flex; + align-items: center; + gap: 0.5rem; + margin: 0.875rem 0 0.25rem; + font-size: 0.6875rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; + color: var(--muted); +} +.entry h2::before { + content: ""; + width: 0.375rem; + height: 0.375rem; + border-radius: 50%; + background: var(--k-inference); +} +.entry:has(.user) h2::before { + background: var(--text); +} +.entry:has(.thinking) h2::before { + background: var(--faint); +} +.time { + font-size: 0.6875rem; + color: var(--faint); + font-variant-numeric: tabular-nums; + margin-bottom: 0.25rem; +} +pre { + font: 0.75rem / 1.55 var(--mono); + white-space: pre-wrap; + overflow-wrap: anywhere; + margin: 0.375rem 0; +} +/* Prose reads as prose; code and payloads stay monospace. */ +.assistant > pre, +.user > pre { + font: + 0.875rem / 1.6 system-ui, + -apple-system, + sans-serif; +} +.assistant.thinking > pre { + color: var(--muted); + font-style: italic; +} +.card.user { + padding: 0.75rem 1rem; + border-radius: 0.75rem; + background: var(--panel); + box-shadow: 0 0 0 1px var(--line); +} +.model { + font-size: 0.75rem; + color: var(--faint); +} +.model .time { + display: none; +} +.title { + font-weight: 600; + font-size: 0.8125rem; + margin: 0.25rem 0; +} +.output { + padding: 0.5rem 0.75rem; + background: var(--inset); + border-radius: 0.5rem; + margin: 0.5rem 0 0.25rem; +} +details { + margin: 0.25rem 0; + border-radius: 0.5rem; +} +summary { + overflow-wrap: anywhere; +} +details details { + background: var(--inset); + padding: 0.125rem 0.75rem; +} +details details > summary { + font: 0.75rem / 1.55 var(--mono); + white-space: pre-wrap; + color: var(--muted); + padding: 0.375rem 0; +} +/* Tool calls: one quiet row keyed with the chart's tool color. */ +.entry:has(> details.tool) { + padding: 0; +} +details.tool { + margin: 0; +} +details.tool > summary { + display: block; + padding: 0.3125rem 0.625rem; + margin: 0 -0.625rem; + border-radius: 0.5rem; + font: 0.75rem / 1.5 var(--mono); + color: var(--muted); + list-style: none; +} +details.tool > summary::-webkit-details-marker { + display: none; +} +details.tool > summary::before { + content: ""; + display: inline-block; + vertical-align: 0.0625rem; + margin-right: 0.5rem; + width: 0.375rem; + height: 0.375rem; + border-radius: 2px; + background: var(--k-tool); +} +/* Context injected mid-conversation: present, but well below the conversation. + `:where` keeps hover and open brightening it like any tool row. */ +details.tool:where(.context) > summary { + padding-block: 0.0625rem; + font-size: 0.625rem; + color: var(--faint); +} +details.tool:where(.context) > summary::before { + width: 0.25rem; + height: 0.25rem; + background: var(--line); +} +details.tool:where(.context) > summary .accent { + color: inherit; +} +details.tool.pending > summary::before { + background: none; + box-shadow: inset 0 0 0 1.5px var(--k-tool); +} +details.tool.error > summary { + color: var(--error); +} +details.tool.error > summary::before { + background: var(--error); +} +details.tool > summary:hover { + background: var(--hover); + color: var(--text); +} +details.tool[open] > summary { + color: var(--text); +} +details.tool > .details-body { + margin: 0.25rem 0 0.75rem; + padding: 0.25rem 0 0.25rem 0.875rem; + border-left: 1px solid var(--line); +} +details[open] > :is(.details-body, pre) { + animation: reveal 0.15s ease-out; +} +@keyframes reveal { + from { + opacity: 0; + transform: translateY(-0.2rem); + } + to { + opacity: 1; + transform: none; + } +} +@media (prefers-reduced-motion: reduce) { + details[open] > .details-body, + details[open] > pre { + animation: none; + } +} +.error { + color: var(--error); +} +.empty { + padding: 3rem 1rem; + text-align: center; + color: var(--muted); +} +#content { + min-height: 10rem; +} +.entry.flash { + animation: flash 1.4s ease-out; +} +@keyframes flash { + from { + background: color-mix(in srgb, var(--accent) 16%, transparent); + } +} +@media (max-width: 48rem) { + .workspace { + grid-template-columns: 1fr; + } + aside { + height: auto; + max-height: 55dvh; + position: static; + border-right: 0; + border-bottom: 1px solid var(--line); + } + main { + padding: 0 1rem 3rem; + } +} +/* Posted to Buzz: the agent's side of the conversation, tinted with its hue. */ +.card.sent { + padding: 0.75rem 1rem; + border-radius: 0.75rem; + background: color-mix(in srgb, var(--k-inference) 9%, var(--panel)); + box-shadow: 0 0 0 1px color-mix(in srgb, var(--k-inference) 22%, transparent); +} +.card.sent > pre { + font: + 0.875rem / 1.6 system-ui, + -apple-system, + sans-serif; +} +.entry:has(.sent) h2 { + color: var(--k-inference); +} + +/* Prompt setup: a drawer over the conversation's right side. */ +#prompt-toggle { + flex: none; + padding: 0 0.75rem; + color: var(--muted); +} +#prompt-toggle[aria-expanded="true"] { + color: var(--text); + border-color: var(--accent); +} +#prompt { + position: fixed; + inset: 0 0 0 auto; + z-index: 10; + width: min(44rem, 92vw); + overflow-y: auto; + overscroll-behavior: contain; + background: var(--panel); + border-left: 1px solid var(--line); + box-shadow: var(--shadow); + padding: 0 1.5rem 3rem; +} +.prompt-head { + display: flex; + align-items: center; + justify-content: space-between; + position: sticky; + top: 0; + z-index: 1; + margin: 0 -1.5rem; + padding: 0.75rem 1.5rem; + background: color-mix(in srgb, var(--panel) 90%, transparent); + backdrop-filter: blur(10px); + border-bottom: 1px solid var(--line); +} +.tabs { + display: flex; + gap: 0.25rem; +} +.prompt-head button { + height: 2rem; + padding: 0 0.75rem; + border: 1px solid transparent; + border-radius: 0.5rem; + background: none; + font-size: 0.8125rem; + color: var(--muted); +} +.prompt-head button:hover { + background: var(--hover); + color: var(--text); +} +.tabs [aria-selected="true"] { + background: var(--inset); + color: var(--text); + font-weight: 600; +} +#prompt-close { + width: 2rem; + padding: 0; + font-size: 1.125rem; +} +.prompt-head button:focus-visible { + outline: none; + border-color: var(--accent); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 18%, transparent); +} +.prompt-note { + margin: 0.75rem 0; + font-size: 0.75rem; + color: var(--faint); +} +/* One section per recorded item, keyed by the role it was sent as. */ +details.setup-item { + margin: 0 0 0.75rem; + border: 1px solid var(--line); + border-radius: 0.625rem; + background: var(--base); + padding: 0; +} +#prompt details.setup-item > summary { + display: flex; + align-items: center; + gap: 0.5rem; + padding: 0.625rem 0.875rem; + list-style: none; + cursor: pointer; +} +#prompt details.setup-item > summary::-webkit-details-marker { + display: none; +} +.setup-label { + flex: 1; + font-weight: 600; +} +.setup-time { + font-size: 0.6875rem; + color: var(--faint); + font-variant-numeric: tabular-nums; +} +.scope { + flex: none; + padding: 0.0625rem 0.4375rem; + border-radius: 0.3125rem; + font-size: 0.625rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; + color: var(--scope); + background: color-mix(in srgb, var(--scope) 12%, transparent); + --scope: var(--muted); +} +.scope-system { + --scope: var(--k-tool); +} +.scope-developer { + --scope: var(--accent); +} +.scope-user { + --scope: var(--text); +} +.scope-settings, +.scope-other { + --scope: var(--faint); +} +:root[data-theme="dark"] .scope-system { + --scope: var(--k-inference); +} +@media (prefers-color-scheme: dark) { + :root:not([data-theme="light"]) .scope-system { + --scope: var(--k-inference); + } +} +/* Recorded Markdown, read as a document. */ +.md { + padding: 0.25rem 0.875rem 0.5rem; + font-size: 0.8125rem; + line-height: 1.6; +} +details.setup-item > .md { + border-top: 1px solid var(--line); +} +.md-block + .md-block { + border-top: 1px dashed var(--line); + margin-top: 0.5rem; + padding-top: 0.5rem; +} +.md-boundary { + margin: 0.75rem 0; + padding: 0.25rem 0; + border-block: 1px solid var(--line); + font: 0.625rem / 1.5 var(--mono); + color: var(--faint); + text-align: center; +} +.md :is(h3, h4, h5, h6) { + margin: 1rem 0 0.25rem; + font-size: 0.8125rem; + font-weight: 650; +} +.md h3 { + font-size: 0.9375rem; +} +.md p { + margin: 0.375rem 0; +} +.md ul { + margin: 0.25rem 0; + padding-left: 1.125rem; +} +.md li.depth-1 { + margin-left: 1rem; +} +.md li.depth-2, +.md li.depth-3 { + margin-left: 2rem; +} +.md code { + font: 0.75rem var(--mono); + padding: 0 0.25rem; + border-radius: 0.25rem; + background: var(--inset); +} +.md-code { + padding: 0.5rem 0.75rem; + border-radius: 0.5rem; + background: var(--inset); +} +.md-code code { + padding: 0; + background: none; +} +.md-tag { + margin: 0.5rem 0 0.25rem; + font: 0.6875rem var(--mono); + color: var(--faint); +} +/* Tool definitions: name and summary line, opening to description and schema. */ +details.tool-def { + margin: 0; + border-bottom: 1px solid var(--line); + border-radius: 0; + padding: 0; + background: none; +} +#prompt details.tool-def > summary { + display: flex; + gap: 0.75rem; + align-items: baseline; + padding: 0.5rem 0.25rem; + cursor: pointer; + list-style: none; +} +#prompt details.tool-def > summary::-webkit-details-marker { + display: none; +} +#prompt details.tool-def > summary:hover { + background: var(--hover); +} +.tool-name { + flex: none; + font: 600 0.75rem var(--mono); +} +.tool-desc { + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + font-size: 0.75rem; + color: var(--muted); +} +details.tool-def[open] > .md { + padding-inline: 0.25rem; +} +.json { + margin: 0 0 0.75rem; + padding: 0.625rem 0.75rem; + border-radius: 0.5rem; + background: var(--inset); + color: var(--faint); +} +.md .json { + margin: 0.25rem 0; +} +.j-key { + color: var(--accent); +} +.j-str { + color: var(--text); +} +.j-num, +.j-lit { + color: var(--k-tool); +} +:root[data-theme="dark"] :is(.j-num, .j-lit) { + color: var(--k-inference); +} +@media (prefers-color-scheme: dark) { + :root:not([data-theme="light"]) :is(.j-num, .j-lit) { + color: var(--k-inference); + } +} diff --git a/scripts/session-debug.html b/scripts/session-debug.html new file mode 100644 index 000000000..77208f0dc --- /dev/null +++ b/scripts/session-debug.html @@ -0,0 +1,38 @@ + + + + + + Agent session + + + +
+ +
+

+
+
+ +
+ + + diff --git a/scripts/session-debug.mjs b/scripts/session-debug.mjs new file mode 100644 index 000000000..cb3b9cc7e --- /dev/null +++ b/scripts/session-debug.mjs @@ -0,0 +1,277 @@ +#!/usr/bin/env node +// One-line, read-only, loopback session inspector. No model or relay connection. +import { createServer } from "node:http"; +import { randomBytes } from "node:crypto"; +import { readFile, readdir, stat } from "node:fs/promises"; +import { homedir } from "node:os"; +import { resolve, join, dirname } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { spawn } from "node:child_process"; +import { parseLines, sessionModel } from "./session-model.mjs"; + +export function threadId(value) { + if (/^[a-f0-9]{64}$/i.test(value)) return value.toLowerCase(); + try { + const url = new URL(value); + if (url.protocol !== "buzz:" || url.hostname !== "message") + throw new Error(); + const id = url.searchParams.get("thread") || url.searchParams.get("id"); + if (/^[a-f0-9]{64}$/i.test(id)) return id.toLowerCase(); + } catch {} + throw new Error( + "Use a Buzz thread ID, buzz://message link, or --file /absolute/session.jsonl", + ); +} +async function* files(root) { + let entries; + try { + entries = await readdir(root, { withFileTypes: true }); + } catch (e) { + if (e.code === "ENOENT") return; + throw e; + } + for (const e of entries) { + if (e.isDirectory()) yield* files(join(root, e.name)); + else if (e.isFile() && e.name.endsWith(".jsonl")) yield join(root, e.name); + } +} +export function ownsThread(records, id) { + return records.some((r) => { + const message = + r.type === "response_item" && r.payload?.role === "user" + ? r.payload + : r.type === "user" + ? r.message + : null; + if (!message) return false; + const body = + typeof message.content === "string" + ? message.content + : (message.content ?? []) + .filter((b) => b.type === "text" || b.type === "input_text") + .map((b) => b.text) + .join("\n"); + // Inspect the delivery envelope, not quoted history or tool output. + for (const m of body.matchAll(/\s*([^<]+)\s*<\/context>/g)) { + try { + if (JSON.parse(m[1]).session_thread === id) return true; + } catch {} + } + return ( + new RegExp(`^Thread root: ${id}\\s*$`, "m").test(body) || + new RegExp(`^Event ID: ${id}\\s*$`, "m").test(body) + ); + }); +} +export async function createInspector({ + target, + file, + provider, + roots, + traceRoot, + rates = {}, +}) { + const home = homedir(); + roots ??= [ + join(process.env.CODEX_HOME || join(home, ".codex"), "sessions"), + join(process.env.CODEX_HOME || join(home, ".codex"), "archived_sessions"), + join(process.env.CLAUDE_CONFIG_DIR || join(home, ".claude"), "projects"), + ]; + traceRoot ??= join(home, ".buzz", "session-debug"); + const known = new Map(); + let candidates = file ? [resolve(file)] : []; + let scanned = 0; + const token = randomBytes(24).toString("hex"); + const base = `/${token}`; + const assets = dirname(fileURLToPath(import.meta.url)); + let busy; + async function inspect(selected, through, turn) { + if (!file && Date.now() - scanned > 3000) { + const found = []; + for (const root of roots) + for await (const path of files(root)) { + const info = await stat(path).catch(() => null); + if (!info) continue; + const stamp = `${info.size}:${info.mtimeMs}`; + let match = known.get(path); + if (match?.stamp !== stamp) { + const source = await readFile(path, "utf8"); + const matches = + source.includes(target) && + ownsThread(parseLines(source, true), target); + match = { stamp, matches }; + known.set(path, match); + } + if (match.matches) found.push(path); + } + candidates = found.sort(); + scanned = Date.now(); + } + if (!candidates.length) + return { + waiting: true, + message: "Waiting for a local session for this thread…", + candidates: [], + }; + const chosen = + selected && candidates.includes(selected) ? selected : candidates.at(-1); + const records = parseLines(await readFile(chosen, "utf8"), true); + const id = + records.find((r) => r.type === "session_meta")?.payload?.id ?? + records.find((r) => r.sessionId)?.sessionId; + let trace = []; + if (/^[\w-]+$/.test(id ?? "")) { + try { + trace = parseLines( + await readFile(join(traceRoot, `${id}.jsonl`), "utf8"), + true, + ); + } catch (e) { + if (e.code !== "ENOENT") throw e; + } + } + const model = sessionModel(records, { + file: chosen, + provider, + trace, + rates, + }); + const response = model.turns + .flatMap((t) => t.requests) + .find((r) => r.id === through); + const until = + response?.timestamp ?? model.turns.find((t) => t.id === turn)?.end; + if (until) { + const snapshot = sessionModel( + records.filter((r) => !r.timestamp || r.timestamp <= until), + { + file: chosen, + provider, + trace: trace.filter((r) => r.timestamp <= until), + rates, + }, + ); + model.entries = snapshot.entries; + model.raw = snapshot.raw; + model.snapshotAt = until; + } + return { ...model, candidates }; + } + const server = createServer(async (req, res) => { + const origin = `http://127.0.0.1:${server.address().port}`; + const url = new URL(req.url, origin); + res.setHeader("Cache-Control", "no-store"); + res.setHeader("X-Content-Type-Options", "nosniff"); + res.setHeader("Referrer-Policy", "no-referrer"); + res.setHeader( + "Content-Security-Policy", + "default-src 'none'; script-src 'self'; style-src 'self' 'unsafe-inline'; connect-src 'self'; base-uri 'none'; frame-ancestors 'none'; form-action 'none'", + ); + if ( + req.method !== "GET" || + req.headers.host !== `127.0.0.1:${server.address().port}` || + (req.headers.origin && req.headers.origin !== origin) || + req.headers["sec-fetch-site"] === "cross-site" + ) { + res.writeHead(403); + res.end(); + return; + } + try { + if (url.pathname === `${base}/state`) { + // Serialize scans, but never share a different selection's response. + const job = (busy ?? Promise.resolve()) + .catch(() => {}) + .then(() => + inspect( + url.searchParams.get("session"), + url.searchParams.get("through"), + url.searchParams.get("turn"), + ), + ); + busy = job; + const state = await job; + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify(state)); + } else { + const name = new Map([ + [`${base}/`, "session-debug.html"], + [`${base}/session-debug-client.js`, "session-debug-client.js"], + [`${base}/session-debug.css`, "session-debug.css"], + ]).get(url.pathname); + if (!name) { + res.writeHead(404); + res.end(); + return; + } + res.setHeader( + "Content-Type", + name.endsWith(".js") + ? "text/javascript" + : name.endsWith(".css") + ? "text/css" + : "text/html", + ); + res.end(await readFile(join(assets, name))); + } + } catch (error) { + res.writeHead(500, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ error: error.message })); + } + }); + await new Promise((accept, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", accept); + }); + return { server, url: `http://127.0.0.1:${server.address().port}${base}/` }; +} +async function main() { + const args = process.argv.slice(2); + if (!args.length || args.includes("--help")) { + console.log( + 'Usage: bin/node scripts/session-debug.mjs [--no-open] [--rates rates.json]\n bin/node scripts/session-debug.mjs --file session.jsonl [--no-open]\nRates: {"exact-model-id":{"input":2,"output":10,"cached":0.2,"written":2.5}} in USD per million tokens.', + ); + return; + } + const take = (flag) => { + const i = args.indexOf(flag); + if (i < 0) return; + if (!args[i + 1] || args[i + 1].startsWith("--")) + throw new Error(`${flag} requires a value`); + return args.splice(i, 2)[1]; + }; + const file = take("--file"), + rateFile = take("--rates"); + const noOpen = args.includes("--no-open"); + if (noOpen) args.splice(args.indexOf("--no-open"), 1); + if (args.length !== (file ? 0 : 1)) + throw new Error("Unexpected arguments; use --help"); + const rates = rateFile + ? JSON.parse(await readFile(resolve(rateFile), "utf8")) + : {}; + const { server, url } = await createInspector({ + file, + target: file ? undefined : threadId(args[0]), + rates, + }); + console.log( + `Session viewer: ${url}\nLive local view. Ctrl+C stops the server.`, + ); + if (!noOpen) { + const command = + process.platform === "darwin" + ? "open" + : process.platform === "win32" + ? "explorer.exe" + : "xdg-open"; + const child = spawn(command, [url], { stdio: "ignore" }); + child.on("error", () => console.log("Open the URL above in a browser.")); + } + for (const signal of ["SIGINT", "SIGTERM"]) + process.on(signal, () => server.close(() => process.exit(0))); +} +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) + main().catch((e) => { + console.error(e.message); + process.exitCode = 1; + }); diff --git a/scripts/session-model.mjs b/scripts/session-model.mjs new file mode 100644 index 000000000..a2358b6d4 --- /dev/null +++ b/scripts/session-model.mjs @@ -0,0 +1,385 @@ +// Provider transcripts remain the source of truth. No replay or model requests. +import { buildCodex } from "./export-codex-session.mjs"; +import { buildClaude } from "./export-claude-session.mjs"; +import { entry } from "./session-viewer.mjs"; +import { wallTiming } from "./session-timing.mjs"; + +export function parseLines(source, live = false) { + const lines = source.split("\n"); + return lines.flatMap((line, i) => { + if (!line.trim()) return []; + try { + return [JSON.parse(line)]; + } catch { + if (live && i === lines.length - 1) return []; + throw new Error(`Invalid JSON on line ${i + 1}`); + } + }); +} + +const ms = (stamp) => + Number.isFinite(Date.parse(stamp)) ? Date.parse(stamp) : null; +const amount = (value) => + typeof value === "number" && Number.isFinite(value) && value >= 0 ? value : 0; +const usage = (u = {}) => ({ + input: amount(u.input_tokens), + output: amount(u.output_tokens), + cached: amount(u.cached_input_tokens ?? u.cache_read_input_tokens), + written: amount(u.cache_write_input_tokens ?? u.cache_creation_input_tokens), +}); +function total(requests) { + const out = { input: 0, output: 0, cached: 0, written: 0 }; + for (const r of requests) + for (const k of Object.keys(out)) out[k] += r.usage[k]; + return out; +} +// Never expose encrypted reasoning blobs as though they were readable context. +function publicRecord(value) { + if (Array.isArray(value)) return value.map(publicRecord); + if (value && typeof value === "object") + return Object.fromEntries( + Object.entries(value) + .filter(([key]) => !["encrypted_content", "signature"].includes(key)) + .map(([key, v]) => [key, publicRecord(v)]), + ); + return value; +} + +export function sessionModel( + records, + { provider, file, trace = [], rates = {} } = {}, +) { + provider ??= records.some((r) => r.type === "session_meta") + ? "codex" + : "claude"; + const base = + provider === "codex" ? buildCodex(records) : buildClaude(records); + const meta = records.find((r) => r.type === "session_meta")?.payload ?? {}; + const turns = []; + let active; + let model = "", + effort = null; + const start = (id, timestamp, inferred = false) => { + if (active && !active.end) { + active.end = timestamp; + active.status = "boundary inferred"; + } + active = { + id, + start: timestamp, + end: null, + status: inferred ? "completion not recorded" : "running", + inferred, + model, + requests: [], + cost: null, + }; + turns.push(active); + return active; + }; + const requests = new Map(); + let previousTotals; + const hasUsageRecords = records.some((r) => r.type === "token_usage_record"); + const hasLifecycle = records.some((r) => r.payload?.type === "task_started"); + for (const [index, r] of records.entries()) { + const p = r.payload ?? {}; + if (r.type === "turn_context") { + model = p.model ?? model; + effort = + p.effort ?? p.collaboration_mode?.settings?.reasoning_effort ?? effort; + if (active) active.model = model; + } + if (provider === "codex") { + if (r.type === "event_msg" && p.type === "task_started") + start(p.turn_id ?? `turn-${index}`, r.timestamp); + if ( + !hasLifecycle && + r.type === "response_item" && + p.role === "user" && + /Request:/.test(JSON.stringify(p.content)) + ) + start(`turn-${index}`, r.timestamp, true); + if ( + r.type === "event_msg" && + ["task_complete", "turn_aborted"].includes(p.type) && + active + ) { + const t = turns.find((t) => t.id === p.turn_id) ?? active; + t.end = r.timestamp; + t.durationMs = p.duration_ms; + t.status = p.type === "turn_aborted" ? "interrupted" : "complete"; + } + if (r.type === "token_usage_record" && active) { + const t = turns.find((t) => t.id === p.turn_id) ?? active; + requests.set(p.response_id ?? `usage-${index}`, { + id: p.response_id ?? `usage-${index}`, + turnId: t.id, + timestamp: r.timestamp, + model: t.model, + usage: usage(p.usage), + raw: publicRecord(p), + }); + } + if ( + r.type === "event_msg" && + p.type === "token_count" && + p.info?.total_token_usage + ) { + const u = usage(p.info.total_token_usage); + if ( + !hasUsageRecords && + active && + (!previousTotals || + Object.keys(u).some((k) => u[k] !== previousTotals[k])) + ) { + const delta = Object.fromEntries( + Object.keys(u).map((k) => [ + k, + Math.max(0, u[k] - (previousTotals?.[k] ?? 0)), + ]), + ); + requests.set(`usage-${index}`, { + id: `usage-${index}`, + turnId: active.id, + timestamp: r.timestamp, + model, + usage: delta, + inferred: true, + }); + } + previousTotals = u; + } + } else { + const content = r.message?.content; + const blocks = Array.isArray(content) + ? content + : [{ type: "text", text: content }]; + const body = blocks + .filter((b) => b.type === "text") + .map((b) => b.text ?? "") + .join("\n"); + if ( + r.type === "user" && + !r.isMeta && + !/^<(task-notification|system-reminder|local-command|command-name)/.test( + body.trim(), + ) && + body && + !blocks.some((b) => b.type === "tool_result") + ) { + // Steering remains in the current burst of work, not a fabricated new completion. + if (!/new-message-arrived-while-you-were-working/.test(body) || !active) + start(r.uuid ?? `turn-${index}`, r.timestamp, true); + } + // Claude Code writes zero-usage "" replies (for example "No + // response requested.") when it resumes; no model request happened. + if ( + r.type === "assistant" && + active && + r.message?.model !== "" + ) { + model = r.message.model ?? model; + active.model = model; + if (r.message.usage) { + const id = r.message.id ?? r.requestId ?? `usage-${index}`; + // Claude writes multiple content blocks with the same message ID and usage. + requests.set(id, { + id, + turnId: active.id, + timestamp: r.timestamp, + model, + usage: usage(r.message.usage), + raw: publicRecord(r.message.usage), + }); + } + } + if (r.type === "system" && r.subtype === "turn_duration" && active) { + active.durationMs = r.durationMs; + active.end = r.timestamp; + active.status = "complete"; + } + } + } + const context = []; + if (provider === "codex") { + if (meta.base_instructions) + context.push( + entry.instructions( + meta.timestamp, + "Base instructions", + typeof meta.base_instructions === "string" + ? meta.base_instructions + : (meta.base_instructions.text ?? + JSON.stringify(meta.base_instructions)), + // Codex 0.156 sends these as the first developer message, with no + // top-level `instructions`. + { scope: "developer" }, + ), + ); + // Namespaced tools are flattened to `namespace.name`. + const tools = (meta.dynamic_tools ?? []) + .flatMap((t) => + t.type === "namespace" + ? (t.tools ?? []).map((x) => ({ ...x, name: `${t.name}.${x.name}` })) + : [t], + ) + .map((t) => ({ + name: t.name, + description: t.description, + schema: t.inputSchema ?? t.parameters, + })); + if (tools.length) + context.push( + entry.tools( + meta.timestamp, + `Dynamic tools · ${tools.length} (native tools not recorded)`, + tools, + { + note: "Codex does not record its native tools, such as shell and apply_patch.", + }, + ), + ); + } + let previousCost = 0; + for (const r of trace) { + if (r.type === "context") previousCost = 0; + if (r.type === "context") + context.push( + entry.instructions( + r.timestamp, + "Recorded Buzz system prompt additions", + r.payload.systemPrompt, + ), + ); + if (r.type === "tools") + context.push( + entry.instructions( + r.timestamp, + "Recorded Buzz tool definitions", + JSON.stringify(r.payload, null, 2), + ), + ); + if (r.type === "result") { + const t = [...turns] + .reverse() + .find((t) => ms(t.start) <= ms(r.timestamp)); + if (t) { + t.end = r.timestamp; + t.durationMs = r.payload.duration_ms; + t.inferred = false; + const cumulative = r.payload.total_cost_usd; + if (typeof cumulative === "number" && cumulative >= previousCost) { + t.cost = (t.cost ?? 0) + cumulative - previousCost; + previousCost = cumulative; + } + t.costLabel = "reported total difference"; + t.status = r.payload.is_error ? "error" : "complete"; + } + } + } + for (const t of turns) { + t.requests = [...requests.values()].filter((r) => r.turnId === t.id); + t.usage = total(t.requests); + t.durationMs ??= + t.end && ms(t.start) !== null + ? Math.max(0, ms(t.end) - ms(t.start)) + : null; + const estimate = t.requests.map((r) => { + r.estimate = null; + const rate = rates[r.model]; + if (!rate) return null; + const u = r.usage; + const input = + provider === "codex" + ? Math.max(0, u.input - u.cached - u.written) + : u.input; + if ( + ![rate.input, rate.output, rate.cached, rate.written].every( + (x) => typeof x === "number" && Number.isFinite(x) && x >= 0, + ) + ) + return null; + r.estimate = + (input * rate.input + + u.output * rate.output + + u.cached * rate.cached + + u.written * rate.written) / + 1e6; + return r.estimate; + }); + t.estimate = + estimate.length && estimate.every((v) => v !== null) + ? estimate.reduce((a, b) => a + b, 0) + : null; + } + const identifiers = new Map(); + // Model switches are summarized in the header instead. + const all = [ + ...context, + ...base.entries.filter((e) => e.kind !== "model"), + ].map((e) => { + const key = `${e.timestamp ?? "meta"}-${e.kind}-${e.label}`; + const occurrence = identifiers.get(key) ?? 0; + identifiers.set(key, occurrence + 1); + return { + ...e, + id: `${key}-${occurrence}`, + turnId: + [...turns].reverse().find((t) => ms(t.start) <= ms(e.timestamp))?.id ?? + null, + }; + }); + // Context sent before the first response is the prompt setup, shown apart + // from the conversation; later context stays inline. + const first = all.findIndex((e) => + ["assistant", "tool", "sent"].includes(e.kind), + ); + const setup = all + .slice(0, first < 0 ? all.length : first) + .filter((e) => e.kind === "instructions"); + return { + ...base, + provider, + file, + sessionId: meta.id ?? records.find((r) => r.sessionId)?.sessionId, + header: { + harness: provider === "codex" ? "Codex" : "Claude Code", + model: + turns + .flatMap((t) => t.requests) + .reverse() + .find((r) => r.model && r.model !== "")?.model ?? model, + effort, + }, + entries: all.filter((e) => !setup.includes(e)), + setup: { + // The system prompt leads, as the request's `system` field does. + instructions: setup + .filter((e) => !e.tools) + .sort((a, b) => Number(!!b.blocks) - Number(!!a.blocks)) + .map(({ id, timestamp, label, scope, preview, blocks }) => ({ + id, + timestamp, + label, + scope: scope ?? null, + body: preview, + blocks: blocks ?? null, + })), + tools: setup + .filter((e) => e.tools) + .map(({ id, label, note, tools }) => ({ + id, + label, + note: note ?? null, + tools, + })), + }, + turns, + timing: wallTiming(records, turns, provider), + raw: records.map(publicRecord), + limitations: + provider === "codex" + ? "Recorded base instructions, dynamic tools, run settings, messages and usage. Native tool schemas and exact post-compaction inference requests are not recorded. Encrypted reasoning is omitted." + : "Recorded system prompt and tool snapshots (when Claude Code writes them), injected context as rendered, messages and per-response usage. Request framing, such as billing and SDK system blocks, beta headers and deferred tool schemas, is not recorded. Without completion records, duration is unknown or an inferred boundary.", + }; +} diff --git a/scripts/session-timing.mjs b/scripts/session-timing.mjs new file mode 100644 index 000000000..51be89b00 --- /dev/null +++ b/scripts/session-timing.mjs @@ -0,0 +1,261 @@ +// Wall-clock accounting for the diagnostic viewer. Model request starts are not +// in the native transcripts: inference windows are explicitly elapsed estimates, +// bounded by the preceding input/result and observed model output. Tool windows +// use call/result timestamps. Partition their union, never sum overlapping work. +const time = (value) => { + const n = Date.parse(value); + return Number.isFinite(n) ? n : null; +}; + +export function tokenBreakdown(usage, provider) { + const { input = 0, output = 0, cached = 0, written = 0 } = usage ?? {}; + const fresh = + provider === "codex" ? Math.max(0, input - cached - written) : input; + return { + fresh, + cached, + written, + output, + total: fresh + cached + written + output, + }; +} + +export function partitionTime(start, end, intervals) { + const clipped = intervals + .map((i) => ({ + ...i, + start: Math.max(start, i.start), + end: Math.min(end, i.end), + })) + .filter((i) => i.end > i.start); + const edges = [ + ...new Set([start, end, ...clipped.flatMap((i) => [i.start, i.end])]), + ].sort((a, b) => a - b); + const segments = []; + for (let i = 1; i < edges.length; i++) { + const a = edges[i - 1], + b = edges[i]; + if (b <= a) continue; + const kinds = new Set( + clipped.filter((s) => s.start < b && s.end > a).map((s) => s.kind), + ); + const kind = kinds.size > 1 ? "overlap" : ([...kinds][0] ?? "unknown"); + const last = segments.at(-1); + if (last?.kind === kind && last.end === a) last.end = b; + else segments.push({ kind, start: a, end: b }); + } + return segments; +} + +function toolLabel(p) { + if (p.name !== "exec") return p.name ?? p.type; + const nested = [ + ...new Set( + [...String(p.input ?? p.arguments ?? "").matchAll(/tools\.([\w]+)/g)].map( + (m) => m[1].replace(/^buzz__/, ""), + ), + ), + ]; + return nested.length + ? `${nested.slice(0, 2).join(" + ")}${nested.length > 2 ? ` +${nested.length - 2}` : ""}` + : "exec"; +} + +function isActivity(r, provider) { + if (provider === "codex") + return r.type === "response_item" || r.type === "event_msg"; + return ( + r.type === "user" || + (r.type === "assistant" && r.message?.model !== "") + ); +} + +export function wallTiming(records, turns, provider, now = Date.now()) { + const result = []; + const allRequests = new Map( + turns.flatMap((t) => t.requests.map((r) => [r.id, r])), + ); + const inference = new Map(); + const calls = new Map(); + let ready = null, + pending = null; + const codexUsage = records.some((r) => r.type === "token_usage_record"); + for (const [index, r] of records.entries()) { + const at = time(r.timestamp); + if (at === null) continue; + const p = r.payload ?? {}; + if (provider === "codex") { + if (r.type === "event_msg" && p.type === "task_started") { + ready = at; + pending = null; + } + if (r.type === "response_item") { + if (p.type?.endsWith("_output")) { + const call = calls.get(p.call_id); + if (call) call.end = Math.max(call.start, at); + ready = at; + } else if (p.role === "user") ready = at; + else if ( + p.type === "reasoning" || + p.role === "assistant" || + p.call_id + ) { + pending ??= { start: ready ?? at, end: at, known: ready !== null }; + pending.end = Math.max(pending.end, at); + if (p.call_id) + calls.set(p.call_id, { + id: p.call_id, + kind: "tool", + name: toolLabel(p), + start: at, + end: null, + }); + } + } + const id = + r.type === "token_usage_record" + ? (p.response_id ?? `usage-${index}`) + : !codexUsage && r.type === "event_msg" && p.type === "token_count" + ? `usage-${index}` + : null; + if (id && allRequests.has(id)) { + inference.set(id, { + id, + kind: "inference", + start: pending?.start ?? at, + end: pending?.end ?? at, + estimated: !!pending?.known, + }); + pending = null; + ready = Math.max(ready ?? at, at); + } + } else { + const blocks = Array.isArray(r.message?.content) ? r.message.content : []; + if (r.type === "user") { + ready = at; + for (const b of blocks) + if (b.type === "tool_result") { + const call = calls.get(b.tool_use_id); + if (call) call.end = Math.max(call.start, at); + } + } + if (r.type === "assistant" && r.message?.model !== "") { + const id = r.message.id ?? r.requestId ?? `usage-${index}`; + const span = inference.get(id) ?? { + id, + kind: "inference", + start: ready ?? at, + end: at, + estimated: ready !== null, + }; + span.end = Math.max(span.end, at); + inference.set(id, span); + ready = at; + for (const b of blocks) + if (b.type === "tool_use" && !calls.has(b.id)) + calls.set(b.id, { + id: b.id, + kind: "tool", + name: b.name, + start: at, + end: null, + }); + } + } + } + for (const [index, t] of turns.entries()) { + const start = time(t.start); + if (start === null) continue; + const next = time(turns[index + 1]?.start) ?? Infinity; + // Only conversation activity marks a run as busy. Bookkeeping written when + // the next prompt is queued (queue-operation, attachment, synthetic replies) + // would otherwise stretch an idle wait into the previous run. + const lastObserved = records.reduce((last, r) => { + const at = time(r.timestamp); + return at !== null && at >= start && at < next && isActivity(r, provider) + ? Math.max(last, at) + : last; + }, start); + // A missing completion is not proof the provider is still running. Only an + // explicit active lifecycle extends to now; older Claude logs stop at data. + const end = Math.max( + start, + t.status === "running" + ? Math.min(next, Math.max(lastObserved, now)) + : !t.inferred && time(t.end) !== null + ? Math.min(next, time(t.end)) + : lastObserved, + ); + const items = []; + for (const r of t.requests) { + const span = inference.get(r.id) ?? { + id: r.id, + kind: "inference", + start: time(r.timestamp) ?? start, + end: time(r.timestamp) ?? start, + estimated: false, + }; + items.push({ + ...span, + start: Math.min(end, Math.max(start, span.start)), + end: Math.min(end, Math.max(start, span.start, span.end)), + tokens: tokenBreakdown(r.usage, provider), + estimate: r.estimate, + model: r.model, + }); + } + for (const c of calls.values()) + if (c.start >= start && c.start < Math.min(next, end + 1)) { + items.push({ + ...c, + end: Math.min(end, c.end ?? end), + open: c.end === null, + }); + } + const segments = partitionTime(start, end, items); + for (const [i, s] of segments.entries()) + if (s.kind === "unknown") + items.push({ + ...s, + id: `gap-${i}`, + name: "Unattributed", + estimated: false, + }); + items.sort((a, b) => a.start - b.start || a.end - b.end); + result.push({ + id: t.id, + start, + end, + observedOnly: t.inferred, + ongoing: t.status === "running", + segments, + items, + recordedDurationMs: t.durationMs, + }); + } + const start = result[0]?.start ?? null; + const end = result.at(-1)?.end ?? null; + const idle = result + .slice(1) + .map((r, i) => ({ kind: "idle", start: result[i].end, end: r.start })) + .filter((s) => s.end > s.start); + const segments = + start === null + ? [] + : partitionTime(start, end, [ + ...result.flatMap((r) => r.segments), + ...idle, + ]); + // No active run in this period: the agent was idle, waiting for a prompt. It + // is neither a slow model nor a tool, and stays separate from unattributed + // gaps inside a run. + for (const s of segments) + if ( + s.kind === "unknown" && + !result.some((r) => r.start < s.end && r.end > s.start) + ) + s.kind = "idle"; + const totals = { inference: 0, tool: 0, overlap: 0, unknown: 0, idle: 0 }; + for (const s of segments) totals[s.kind] += s.end - s.start; + return { start, end, segments, totals, runs: result }; +} diff --git a/scripts/session-viewer.mjs b/scripts/session-viewer.mjs index 7e9286334..787e34d42 100644 --- a/scripts/session-viewer.mjs +++ b/scripts/session-viewer.mjs @@ -13,44 +13,72 @@ export const escapeHtml = (value) => ], ); export const text = (value) => - typeof value === "string" ? value : JSON.stringify(value, null, 2); + typeof value === "string" ? value : (JSON.stringify(value, null, 2) ?? ""); const time = (timestamp) => `
${escapeHtml(timestamp?.slice(11, 19))}
`; // Long text shows a preview; clicking toggles the full text, as in Pi's viewer. function expandable(value, lines = 10) { const all = text(value).split("\n"); if (all.length <= lines) return `
${escapeHtml(all.join("\n"))}
`; - return ``; + return `
${escapeHtml(all.slice(0, Math.max(1, lines)).join("\n"))} … (${all.length} lines)
${escapeHtml(all.join("\n"))}
`; } // Entry builders: kind drives filters, label/preview the sidebar, html the card. export const entry = { model: (timestamp, label) => ({ + timestamp, kind: "model", label: "model", preview: label, html: `
${time(timestamp)}Switched to model: ${escapeHtml(label)}
`, }), - instructions: (timestamp, label, body) => ({ + // Model-visible context outside the conversation: one quiet, collapsed row. + // A one-line body, once unwrapped from ``, also shows inline. + // `extra.scope` is the role it was sent as: system, developer or user. + instructions: (timestamp, label, body, extra = {}) => { + const brief = body.replace(/<\/?system-reminder>/g, "").trim(); + return { + timestamp, + kind: "instructions", + label, + preview: body, + html: `
${escapeHtml(label)}${/^.{1,120}$/.test(brief) ? ` ${escapeHtml(brief)}` : ""}
${time(timestamp)}
${escapeHtml(body)}
`, + ...extra, + }; + }, + // Tool definitions as recorded, `[{ name, description, schema }]`, one + // disclosure per tool. + tools: (timestamp, label, tools, extra = {}) => ({ + timestamp, kind: "instructions", label, - preview: body, - html: `
${time(timestamp)}
${escapeHtml(label)}
${expandable(body, 2)}
`, + tools, + ...extra, + preview: tools.map((tool) => tool.name).join(", "), + html: `
${escapeHtml(label)}
${time(timestamp)}${tools + .map( + ({ name, description, schema }) => + `
${escapeHtml(name)}
${escapeHtml([description, schema && text(schema)].filter(Boolean).join("\n\n"))}
`, + ) + .join("")}
`, }), // A Buzz turn shows its request, with the full turn context collapsed. user: (timestamp, body, request) => ({ + timestamp, kind: "user", label: "user", preview: request ?? body, html: `
${time(timestamp)}${request === undefined ? expandable(body, 20) : `
${escapeHtml(request)}
Buzz turn
${expandable(body, 0)}`}
`, }), assistant: (timestamp, body) => ({ + timestamp, kind: "assistant", label: "assistant", preview: body, html: `
${time(timestamp)}
${escapeHtml(body)}
`, }), thinking: (timestamp, body) => ({ + timestamp, kind: "assistant", label: "thinking", preview: body, @@ -58,12 +86,22 @@ export const entry = { }), // `result` is undefined while pending; `detail` names the call's target. tool: (timestamp, { name, detail = "", input, result, error = false }) => ({ + timestamp, kind: "tool", label: name, preview: detail || text(input), - html: `
${time(timestamp)}
${escapeHtml(name)} ${escapeHtml(detail)}
${expandable(input, 20)}${result === undefined ? "" : `
${expandable(result)}
`}
`, + html: `
${escapeHtml(name)}${detail ? ` ${escapeHtml(detail)}` : ""}${error ? ` failed` : ""}
${time(timestamp)}${expandable(input, 20)}${result === undefined ? "" : `
${expandable(result)}
`}
`, + }), + // A message the agent posted to Buzz through its send tool. + sent: (timestamp, body) => ({ + timestamp, + kind: "sent", + label: "posted to Buzz", + preview: body, + html: `
${time(timestamp)}
${escapeHtml(body)}
`, }), other: (timestamp, label, value) => ({ + timestamp, kind: "other", label, preview: text(value), diff --git a/src/app/services.ts b/src/app/services.ts index 287dd7e29..7193908fb 100644 --- a/src/app/services.ts +++ b/src/app/services.ts @@ -76,7 +76,7 @@ export function createServices() { subscribe: communities.subscribe, }); const relay = communities.relay; - const agents2 = new Agents2Service(ctx, relay); + const agents2 = new Agents2Service(ctx, communities); ctx.effect(() => bindAgentMentions(agentControl, communities)); ctx.effect(() => bindTeamTextSync(agentControl, communities)); const notifications = new NotificationsService( diff --git a/src/bundled/agents2/Agents2Page.test.tsx b/src/bundled/agents2/Agents2Page.test.tsx index 1e54dbb7e..3eb719e8a 100644 --- a/src/bundled/agents2/Agents2Page.test.tsx +++ b/src/bundled/agents2/Agents2Page.test.tsx @@ -49,7 +49,7 @@ const agent = (name: string, n: number): Agent => ({ }); function setup(agents: Agent[], { shown = true } = {}) { - let snapshot: AgentsSnapshot = { status: "ready", agents }; + let snapshot: AgentsSnapshot = { status: "ready", agents, running: agents }; const listeners = new Set<() => void>(); const type = { key: "test/echo", @@ -73,6 +73,7 @@ function setup(agents: Agent[], { shown = true } = {}) { return () => listeners.delete(listener); }, find: (pubkey: string) => snapshot.agents.find((a) => a.pubkey === pubkey), + relay: () => undefined, register: vi.fn(), create: vi.fn(async () => { // `shown: false`: it was made for a community no longer selected. diff --git a/src/bundled/builderlab/index.test.tsx b/src/bundled/builderlab/index.test.tsx index 900d6310d..c07d2ad43 100644 --- a/src/bundled/builderlab/index.test.tsx +++ b/src/bundled/builderlab/index.test.tsx @@ -6,7 +6,10 @@ import { act, cleanup, render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; import { HostService } from "../../features/host/service"; import { SettingsCardsService } from "../../features/settings/service"; -import { Agents2Service } from "../../features/agents2/service"; +import { + Agents2Service, + type AgentCommunities, +} from "../../features/agents2/service"; import type { RelayData } from "../../features/relay/service"; import { Agents2Page } from "../agents2/Agents2Page"; import * as builderlab from "./index"; @@ -42,7 +45,12 @@ afterEach(() => { }); function provideAgents2(root: Context, relay: RelayData) { - return new Agents2Service(root, relay, { + const communities = { + snapshot: () => ({ memberships: [], selected: null }), + subscribe: () => () => {}, + open: () => relay, + } as unknown as AgentCommunities; + return new Agents2Service(root, communities, { list: async () => [], create: vi.fn(), rename: vi.fn(), diff --git a/src/bundled/claude-code/claude.ts b/src/bundled/claude-code/claude.ts index 048b44225..c163acc52 100644 --- a/src/bundled/claude-code/claude.ts +++ b/src/bundled/claude-code/claude.ts @@ -89,7 +89,7 @@ export class ClaudeProcess { private constructor( private readonly process: HostProcess, resume: string | undefined, - private readonly tools: ToolServer | undefined, + private tools: ToolServer | undefined, ) { this.sessionId = resume; this.exited = process.exited.then((code) => { @@ -104,6 +104,14 @@ export class ClaudeProcess { } static async start(spawn: Spawn, launch: ClaudeLaunch) { + const claude = await ClaudeProcess.spawn(spawn, launch); + await claude.initialize(launch); + return claude; + } + + /** Starts `claude` without its prompt. This is most of what a start costs, + * and fixes nothing about the agent it serves but its folder and model. */ + static async spawn(spawn: Spawn, launch: Omit) { let claude: ClaudeProcess | undefined; const early: string[] = []; const earlyErrors: string[] = []; @@ -150,8 +158,14 @@ export class ClaudeProcess { ); for (const data of earlyErrors) claude.readError(data); for (const data of early) claude.read(data); - // Loads settings, tools and MCP servers now rather than on the first message. - await claude.request({ + return claude; + } + + /** Gives a started process its prompt, and its tools to whoever it now + * works for. Loads settings and tools now rather than on the first message. */ + async initialize(launch: Pick) { + if (launch.tools) this.tools = launch.tools; + await this.request({ subtype: "initialize", appendSystemPrompt: launch.systemPrompt, ...(launch.tools @@ -165,7 +179,6 @@ export class ClaudeProcess { } : {}), }); - return claude; } get running() { diff --git a/src/bundled/claude-code/index.ts b/src/bundled/claude-code/index.ts index 17c510d96..a31de4171 100644 --- a/src/bundled/claude-code/index.ts +++ b/src/bundled/claude-code/index.ts @@ -11,23 +11,24 @@ export const TYPE_ID = "claude-code"; /** A turn's run returns once Claude has it, so this bounds only the hand-over. */ const HANDOVER_MS = 2 * 60_000; -export const inject = ["agents2", "host", "relay"]; +export const inject = ["agents2", "host"]; export const apply: PluginModule["apply"] = (ctx) => { // Only the desktop app can start `claude`, and setup and file uploads run // `bash` and `base64`, which stock Windows lacks; no unusable type elsewhere. if (!isTauri() || !/Mac|Linux/i.test(navigator.platform)) return; - const { agents2, host, relay } = ctx; + const { agents2, host } = ctx; const typeKey = `${ctx.pluginOwner?.id ?? "buzz.claude-code"}/${TYPE_ID}`; const spawn = host.spawn?.bind(host); const setup = new ClaudeSetup(spawn); - const runtime = new ClaudeRuntime(host, relay); + const runtime = new ClaudeRuntime(host, agents2.relay); const sync = () => { const snapshot = agents2.snapshot(); - // Agents leaving view, the app signing out included, stop their processes. + // Agents that stop running, the app signing out included, stop their + // processes; switching communities does not. if (snapshot.status === "loading") return; - runtime.sync(snapshot.agents.filter((agent) => agent.type === typeKey)); + runtime.sync(snapshot.running.filter((agent) => agent.type === typeKey)); }; - // Agents come and go with the community; a spare session is warmed for each. + // One spare session waits for whichever running agent needs it next. if (spawn) { const unsubscribe = agents2.subscribe(sync); sync(); diff --git a/src/bundled/claude-code/runtime.test.ts b/src/bundled/claude-code/runtime.test.ts index 437c2410f..00e8eac8b 100644 --- a/src/bundled/claude-code/runtime.test.ts +++ b/src/bundled/claude-code/runtime.test.ts @@ -58,7 +58,7 @@ function setup(thread: readonly EventData[] = []) { getItem: (key: string) => data.get(key) ?? null, setItem: (key: string, value: string) => void data.set(key, value), } as Storage; - const runtime = new ClaudeRuntime(host, relay, storage); + const runtime = new ClaudeRuntime(host, () => relay, storage); const published: { kind: number; content: string; tags: string[][] }[] = []; const publish = vi.fn(async (template: (typeof published)[number]) => { published.push(template); @@ -294,7 +294,8 @@ it("shows the whole thread to a session started over from a lost one", async () ); await deliver({ type: "mention", event: mention }); await flush(20); - const [lost, fresh] = claudes(); + const [lost] = claudes(); + const fresh = claudes().find((process) => process.prompts.length); expect(lost?.options.args).toContain("lost"); expect(lost?.prompts).toEqual([]); expect(fresh?.prompts[0]).toContain(' { expect(claude?.prompts[0]).toContain("Timer: watch/daily"); }); -it("warms a spare for each agent of its type and stops those that go away", async () => { +it("keeps one spare for all its agents, and none once they are gone", async () => { const { runtime, claudes } = setup(); - const agent = { pubkey: self, config: DEFAULT_CONFIG } as Agent; - runtime.sync([agent]); + const agent = (pubkey: string) => + ({ pubkey, config: DEFAULT_CONFIG }) as Agent; + runtime.sync([agent(self), agent(alice)]); await flush(10); expect(claudes()).toHaveLength(1); + expect(claudes()[0]?.options.cwd).toBe(DEFAULT_CONFIG.workspace); runtime.sync([]); await flush(10); expect(claudes().every((process) => process.killed)).toBe(true); }); + +it("replaces the spare when no agent runs where it was started", async () => { + const { runtime, claudes } = setup(); + runtime.sync([{ pubkey: self, config: DEFAULT_CONFIG } as Agent]); + await flush(10); + runtime.sync([ + { pubkey: self, config: { ...DEFAULT_CONFIG, model: "sonnet" } } as Agent, + ]); + await flush(10); + const [old, next] = claudes(); + expect(old?.killed).toBe(true); + expect(next?.options.args).toEqual( + expect.arrayContaining(["--model", "sonnet"]), + ); + expect(claudes()).toHaveLength(2); +}); diff --git a/src/bundled/claude-code/runtime.ts b/src/bundled/claude-code/runtime.ts index 874fc8251..c0f2b271e 100644 --- a/src/bundled/claude-code/runtime.ts +++ b/src/bundled/claude-code/runtime.ts @@ -25,7 +25,7 @@ import { timerPrompt, turnPrompt, } from "./prompt"; -import { AgentSessions, localSessions } from "./sessions"; +import { AgentSessions, localSessions, Spare } from "./sessions"; export type Config = Readonly<{ /** A Claude model or alias; empty uses Claude Code's own default. */ @@ -67,6 +67,12 @@ type Entry = { contexts: Map; }; +/** Where an agent's processes run, and with which model. */ +const where = ({ workspace, model }: Config) => ({ + cwd: workspace, + ...(model.trim() ? { model: model.trim() } : {}), +}); + export class ClaudeRuntime { private readonly agents = new Map(); private readonly reading = new Map>(); @@ -77,10 +83,16 @@ export class ClaudeRuntime { string, ReturnType >(); + /** One process started ahead of need, for any agent's next new conversation. */ + private readonly spare = new Spare( + (id, options) => this.spawn(id, options), + (_conversation, message) => respond(undefined, {}, message), + ); constructor( private readonly host: Host, - private readonly relay: RelayData, + /** The owner's connection to each agent's own community. */ + private readonly relay: (pubkey: string) => RelayData | undefined, private readonly storage: Storage = globalThis.localStorage, ) {} @@ -91,11 +103,12 @@ export class ClaudeRuntime { return spawn; } - /** Brings the runtime in line with this type's agents: new ones get a warm - * session, changed settings restart idle sessions, removed agents stop. */ + /** Brings the runtime in line with this type's agents: changed settings + * restart idle sessions, removed agents stop, and a spare waits while any + * agent is here. */ sync(agents: readonly Agent[]) { const current = new Set(agents.map((agent) => agent.pubkey)); - // Out of view (another community, offline) or deleted: either way its + // Deleted, signed out or its community disconnected: either way its // processes stop. Saved sessions stay so it resumes when it is back. for (const [pubkey, entry] of this.agents) if (!current.has(pubkey)) { @@ -105,12 +118,13 @@ export class ClaudeRuntime { for (const agent of agents) { const next = config(agent.config); const entry = this.agents.get(agent.pubkey); - if (!entry) this.entry(agent.pubkey, next).sessions.warm(); + if (!entry) this.entry(agent.pubkey, next); else if (JSON.stringify(entry.config) !== JSON.stringify(next)) { entry.config = next; entry.sessions.reconfigure(); } } + this.spare.warm(...agents.map((agent) => where(config(agent.config)))); this.notify(); } @@ -135,6 +149,7 @@ export class ClaudeRuntime { } dispose() { + this.spare.dispose(); for (const entry of this.agents.values()) entry.sessions.dispose(); this.agents.clear(); } @@ -149,6 +164,9 @@ export class ClaudeRuntime { return; const entry = this.entry(agent.pubkey, settings); entry.handle = agent; + // Read while the turn's prompt is built, not after: a new session's prompt + // needs it, and a spare is only faster if nothing else is awaited. + void this.memory(agent.pubkey, entry); if (trigger.type === "timer") { const done = entry.sessions.deliver( `timer/${trigger.slug}`, @@ -242,8 +260,8 @@ export class ClaudeRuntime { channelId: string, fresh = false, ) { - const snapshot = this.relay.snapshot(); - const session = snapshot.status === "ready" ? snapshot.session : undefined; + const snapshot = this.relay(self)?.snapshot(); + const session = snapshot?.status === "ready" ? snapshot.session : undefined; const summary = session?.channels.list().channels.find((item) => item.id === channelId) ?? session?.channels.get?.(channelId); @@ -327,8 +345,6 @@ export class ClaudeRuntime { const tools = (cwd: string): ToolServer => async (conversation, message) => { - // A spare starts before any delivery: it can list the tools, and has a - // handle to call them with by the time it is given a turn. const handle = entry.handle; const client = handle && { ...appClient(handle, { @@ -347,6 +363,7 @@ export class ClaudeRuntime { }; entry.sessions = new AgentSessions({ spawn: (id, options) => this.spawn(id, options), + spare: this.spare, store: localSessions(this.storage, pubkey), onChange: () => this.notify(), fingerprint: () => @@ -357,12 +374,11 @@ export class ClaudeRuntime { entry.config.scope, ]), launch: async () => { - const { model, workspace, instructions, scope } = entry.config; + const { workspace, instructions, scope } = entry.config; const memory = await this.memory(pubkey, entry); return { - cwd: workspace, + ...where(entry.config), tools: tools(workspace), - ...(model.trim() ? { model: model.trim() } : {}), systemPrompt: systemPrompt({ scope, cwd: workspace, @@ -406,8 +422,8 @@ export class ClaudeRuntime { return reading; } private async readMemories(pubkey: string): Promise { - const snapshot = this.relay.snapshot(); - if (snapshot.status !== "ready") throw new Error("Buzz is not connected"); + const snapshot = this.relay(pubkey)?.snapshot(); + if (snapshot?.status !== "ready") throw new Error("Buzz is not connected"); const view = snapshot.session.agentMemories.open(pubkey); try { const done = new Promise((resolve) => { diff --git a/src/bundled/claude-code/sessions.test.ts b/src/bundled/claude-code/sessions.test.ts index b96e5724a..478c70500 100644 --- a/src/bundled/claude-code/sessions.test.ts +++ b/src/bundled/claude-code/sessions.test.ts @@ -1,11 +1,12 @@ import { afterEach, expect, it, vi } from "vitest"; import { fakeSpawn, flush } from "./claude-testing"; +import type { ToolServer } from "./claude"; import { AgentSessions, IDLE_MS, localSessions, - MAX_LIVE, type SavedSession, + Spare, type SessionStore, } from "./sessions"; @@ -31,25 +32,35 @@ function sessions( hold: boolean; /** Holds each launch after it reads the settings, as a memory read does. */ launched: Promise; + /** The shared spare its new conversations look for. */ + spare: Spare; + tools: ToolServer; }> = {}, ) { const fake = fakeSpawn(options.known); const store = options.store ?? memoryStore(); let model = "opus"; let ids = 0; + const spawn = (id: string, spawnOptions?: object) => { + // The fake records each process as the call begins. + const process = fake.spawn(id, spawnOptions); + const started = fake.processes.at(-1); + if (options.hold && started) started.hold = true; + return process; + }; const pool = new AgentSessions({ - spawn: (id, spawnOptions) => { - // The fake records each process as the call begins. - const process = fake.spawn(id, spawnOptions); - const started = fake.processes.at(-1); - if (options.hold && started) started.hold = true; - return process; - }, + spawn, + ...(options.spare ? { spare: options.spare } : {}), store, launch: async () => { const current = model; await options.launched; - return { cwd: "/work", systemPrompt: "prompt", model: current }; + return { + cwd: "/work", + systemPrompt: "prompt", + model: current, + ...(options.tools ? { tools: options.tools } : {}), + }; }, fingerprint: () => model, newId: () => `new-${++ids}`, @@ -59,6 +70,7 @@ function sessions( return { pool, store, + spawn, claudes, live: () => claudes().filter((process) => !process.killed), setModel: (next: string) => { @@ -67,21 +79,165 @@ function sessions( }; } -it("starts a new conversation on the warm spare and warms another", async () => { - const { pool, store, claudes } = sessions(); - pool.warm(); +/** Sessions with a spare, spawned like their own processes. */ +function withSpare(options: Parameters[0] = {}) { + let ids = 0; + let parts: ReturnType | undefined; + const spare = new Spare( + (id, spawnOptions) => { + if (!parts) throw new Error("not ready"); + return parts.spawn(id, spawnOptions); + }, + async (_conversation, message) => ({ from: "spare", message }), + () => `spare-${++ids}`, + ); + parts = sessions({ ...options, spare }); + return { ...parts, spare }; +} + +const tool = (process: { emit(message: object): void }, id: string) => + process.emit({ + type: "control_request", + request_id: id, + request: { + subtype: "mcp_message", + server_name: "buzz", + message: { jsonrpc: "2.0", id: 1, method: "tools/list" }, + }, + }); +const answered = ( + process: { received: Record[] }, + id: string, +) => + process.received.find( + (message) => + message.type === "control_response" && + (message.response as { request_id?: string }).request_id === id, + )?.response; + +it("starts a new conversation on the shared spare and warms another", async () => { + const { pool, spare, store, claudes } = withSpare(); + spare.warm({ cwd: "/work", model: "opus" }); await flush(); expect(claudes()).toHaveLength(1); + // Started before any agent asked: no prompt yet. + expect(claudes()[0]?.received).toEqual([]); await expect(pool.deliver("c/root", "hello", 10)).resolves.toEqual({ ok: true, }); await flush(); expect(claudes()).toHaveLength(2); - expect(claudes()[0]?.prompts).toEqual(["hello"]); - expect(store.get("c/root")).toMatchObject({ id: "new-1", seen: 10 }); + const [used] = claudes(); + expect(used?.received[0]).toMatchObject({ + type: "control_request", + request: { subtype: "initialize", appendSystemPrompt: "prompt" }, + }); + expect(used?.prompts).toEqual(["hello"]); + expect(store.get("c/root")).toMatchObject({ id: "spare-1", seen: 10 }); expect(pool.snapshot()).toEqual([{ key: "c/root", busy: false }]); }); +it("hands the spare's tools to the agent that takes it", async () => { + const agentTools = vi.fn(async () => ({ from: "agent" })); + const { pool, spare, claudes } = withSpare({ tools: agentTools }); + spare.warm({ cwd: "/work", model: "opus" }); + await flush(); + const [process] = claudes(); + if (!process) throw new Error("no spare"); + // Claude lists the tools as it starts, before any agent has the spare. + tool(process, "early"); + await flush(); + expect(answered(process, "early")).toMatchObject({ + response: { mcp_response: { from: "spare" } }, + }); + await pool.deliver("c/root", "hi", 1); + tool(process, "later"); + await flush(); + expect(answered(process, "later")).toMatchObject({ + response: { mcp_response: { from: "agent" } }, + }); +}); + +it("serves whichever agent starts the next conversation", async () => { + const one = withSpare(); + const two = sessions({ spare: one.spare }); + one.spare.warm({ cwd: "/work", model: "opus" }); + await flush(); + await one.pool.deliver("c/a", "for one", 1); + await flush(); + await two.pool.deliver("c/b", "for two", 2); + // Each took the spare (all started by the first's spawn); neither agent + // started a process of its own. + expect(one.claudes().map((process) => process.prompts)).toEqual([ + ["for one"], + ["for two"], + [], + ]); + expect(two.claudes()).toEqual([]); +}); + +it("starts a conversation elsewhere afresh, and the spare follows it", async () => { + const { pool, spare, claudes } = withSpare(); + spare.warm({ cwd: "/other", model: "opus" }, { cwd: "/work", model: "opus" }); + await flush(); + const [other] = claudes(); + await pool.deliver("c/root", "hi", 1); + await flush(); + expect(other?.killed).toBe(true); + expect(other?.prompts).toEqual([]); + const used = claudes().filter((process) => process.prompts.length); + expect(used[0]?.options.args).toContain("new-1"); + const next = claudes().find((process) => + process.options.args?.includes("spare-2"), + ); + expect(next?.options.cwd).toBe("/work"); + expect(next?.killed).toBe(false); +}); + +it("does not wait for a spare started elsewhere", async () => { + // The spare for another folder never finishes starting. + const spare = new Spare( + () => new Promise(() => undefined), + async () => ({}), + ); + const { pool, claudes } = sessions({ spare }); + spare.warm({ cwd: "/other", model: "opus" }); + await expect(pool.deliver("c/root", "hi", 1)).resolves.toEqual({ ok: true }); + expect(claudes()[0]?.prompts).toEqual(["hi"]); +}); + +it("keeps a spare that fits any agent, and replaces one that fits none", async () => { + const { spare, claudes } = withSpare(); + const work = { cwd: "/work", model: "opus" }; + const other = { cwd: "/other", model: "opus" }; + spare.warm(work); + await flush(); + spare.warm(other, work); + await flush(); + expect(claudes()).toHaveLength(1); + spare.warm(other); + await flush(); + expect(claudes()).toHaveLength(2); + expect(claudes()[0]?.killed).toBe(true); + expect(claudes()[1]?.options.cwd).toBe("/other"); +}); + +it("starts no spare again after one dies, until a conversation starts", async () => { + const { pool, spare, claudes } = withSpare(); + const where = { cwd: "/work", model: "opus" }; + spare.warm(where); + await flush(); + claudes()[0]?.exit(1, "not signed in"); + await flush(); + spare.warm(where); + await flush(); + expect(claudes()).toHaveLength(1); + await pool.deliver("c/root", "hi", 1); + await flush(); + // One for the conversation, and a spare again. + expect(claudes()).toHaveLength(3); +}); + it("continues a conversation in its own process, apart from others", async () => { const { pool, claudes } = sessions(); await pool.deliver("c/one", "first", 1); @@ -93,14 +249,11 @@ it("continues a conversation in its own process, apart from others", async () => }); it("steers a conversation that is still working instead of queueing", async () => { - const { pool, claudes } = sessions(); - pool.warm(); - await flush(); - const process = claudes()[0]; - if (!process) throw new Error("no spare"); - process.hold = true; + const { pool, claudes } = sessions({ hold: true }); const first = pool.deliver("c/root", "first", 1); await flush(); + const process = claudes()[0]; + if (!process) throw new Error("no process"); const second = pool.deliver("c/root", "plain", 2, { steer: "steer" }); await flush(); expect(process.prompts).toEqual(["first", "steer"]); @@ -154,47 +307,13 @@ it("starts over when a saved session no longer loads", async () => { expect(store.get("c/root")).toMatchObject({ id: "new-1", seen: 9 }); }); -it("stops the longest-idle conversation to stay within its process limit", async () => { - const { pool, live } = sessions(); - for (let index = 0; index < MAX_LIVE + 2; index++) - await pool.deliver(`c/${index}`, "hi", index); - await flush(); - expect(live().length).toBeLessThanOrEqual(MAX_LIVE); - const keys = pool.snapshot().map((session) => session.key); - expect(keys).not.toContain("c/0"); - expect(keys).toContain(`c/${MAX_LIVE + 1}`); -}); - -it("says it is busy rather than start a process past its limit", async () => { +it("runs any number of conversations at once", async () => { const { pool, live } = sessions({ hold: true }); - pool.warm(); - for (let index = 0; index < MAX_LIVE; index++) + for (let index = 0; index < 12; index++) void pool.deliver(`c/${index}`, "hi", index); await flush(); - expect(live()).toHaveLength(MAX_LIVE); - await expect(pool.deliver("c/extra", "hi", 9)).resolves.toEqual({ - ok: false, - error: `Claude Code is already working in ${MAX_LIVE} conversations; try again when one finishes`, - }); - expect(live()).toHaveLength(MAX_LIVE); - expect(pool.snapshot().map((session) => session.key)).not.toContain( - "c/extra", - ); -}); - -it("counts conversations still starting toward its limit", async () => { - let release!: () => void; - const launched = new Promise((resolve) => { - release = resolve; - }); - const { pool, live } = sessions({ hold: true, launched }); - const results = Array.from({ length: MAX_LIVE + 1 }, (_, index) => - pool.deliver(`c/${index}`, "hi", index), - ); - await expect(results[MAX_LIVE]).resolves.toMatchObject({ ok: false }); - release(); - await flush(); - expect(live()).toHaveLength(MAX_LIVE); + expect(live()).toHaveLength(12); + expect(pool.snapshot().every((session) => session.busy)).toBe(true); }); it("starts again with settings saved while a conversation's process started", async () => { @@ -217,17 +336,15 @@ it("starts again with settings saved while a conversation's process started", as expect(used[0]?.options.args).toContain("sonnet"); }); -it("restarts idle processes and the spare when settings change", async () => { +it("restarts idle processes when settings change", async () => { const { pool, claudes, setModel } = sessions(); await pool.deliver("c/root", "hi", 1); await flush(); - const [conversation, spare] = claudes(); + const [conversation] = claudes(); setModel("sonnet"); pool.reconfigure(); await flush(); expect(conversation?.killed).toBe(true); - expect(spare?.killed).toBe(true); - expect(claudes().at(-1)?.options.args).toContain("sonnet"); await pool.deliver("c/root", "again", 2); const resumed = claudes().at(-1); expect(resumed?.options.args).toEqual( @@ -236,16 +353,37 @@ it("restarts idle processes and the spare when settings change", async () => { }); it("stops every process when disposed", async () => { - const { pool, claudes } = sessions(); - pool.warm(); + const { pool, spare, claudes } = withSpare(); + spare.warm({ cwd: "/work", model: "opus" }); await pool.deliver("c/root", "hi", 1); await flush(); pool.dispose(); + spare.dispose(); await flush(); + expect(claudes()).toHaveLength(2); expect(claudes().every((process) => process.killed)).toBe(true); await expect(pool.deliver("c/root", "late", 2)).resolves.toMatchObject({ ok: false, }); + expect(claudes()).toHaveLength(2); +}); + +it("starts no spare for a conversation that starts after its agents are gone", async () => { + let release!: () => void; + const launched = new Promise((resolve) => { + release = resolve; + }); + const { pool, spare, live } = withSpare({ launched }); + spare.warm({ cwd: "/work", model: "opus" }); + const done = pool.deliver("c/root", "hi", 1); + await flush(); + // As the runtime does when its last agent goes, mid-launch. + pool.dispose(); + spare.warm(); + release(); + await expect(done).resolves.toMatchObject({ ok: false }); + await flush(); + expect(live()).toEqual([]); }); it("keeps saved sessions per agent in storage", () => { diff --git a/src/bundled/claude-code/sessions.ts b/src/bundled/claude-code/sessions.ts index ca1ea59f5..0b80ab674 100644 --- a/src/bundled/claude-code/sessions.ts +++ b/src/bundled/claude-code/sessions.ts @@ -1,17 +1,16 @@ // One agent's Claude sessions: one per conversation (a thread, or a whole DM), // each continuing across messages and app restarts. A conversation in use keeps // its process; a quiet one is stopped and resumed from its saved session when it -// is next addressed. A spare process waits for the next new conversation, so -// that starts without a spawn. +// is next addressed. One spare process, shared by every agent, waits for the +// next new conversation, so that starts without a spawn. import { ClaudeProcess, type ClaudeLaunch, type Settled, type Spawn, + type ToolServer, } from "./claude"; -/** Processes an agent keeps, the spare included. */ -export const MAX_LIVE = 6; /** A conversation's process stops after this long without a message. */ export const IDLE_MS = 15 * 60_000; /** Saved sessions per agent; the least recently used are forgotten. */ @@ -32,6 +31,8 @@ export type SessionStore = { export type SessionsOptions = Readonly<{ spawn: Spawn; + /** Where a new conversation looks for a process already started. */ + spare?: Spare; store: SessionStore; /** How to start a process with the agent's current settings. */ launch(): Promise>; @@ -53,7 +54,6 @@ type Live = { export class AgentSessions { private readonly live = new Map(); private readonly opening = new Map>(); - private spare: Promise | undefined; private disposed = false; private readonly now: () => number; private readonly newId: () => string; @@ -75,22 +75,6 @@ export class AgentSessions { })); } - /** Starts a spare process unless one exists or the agent is at its limit. */ - warm() { - if (this.disposed || this.spare || this.count() >= MAX_LIVE) return; - const spare = this.start(undefined).catch((error) => { - console.warn("Claude Code could not start a spare session", error); - return undefined; - }); - this.spare = spare; - void spare.then((live) => { - // A spare that dies, or was started with old settings, is not kept. - live?.process.exited.then(() => { - if (this.spare === spare) this.spare = undefined; - }); - }); - } - /** Sends `text` into the conversation `key`, and resolves when the session * has answered it. `steer` replaces `text` when the session is mid-turn; * `fresh` builds it again for a new session when the saved one is lost. */ @@ -122,22 +106,10 @@ export class AgentSessions { return this.answer(key, live, live.process.busy ? steer : text); } const saved = this.options.store.get(key); - // A new conversation takes the spare's place; any other start needs room, - // reserved here, before anything is awaited. - if ((saved || !this.spare) && !this.room()) - return { - ok: false, - error: `Claude Code is already working in ${MAX_LIVE} conversations; try again when one finishes`, - }; - const starting = (async () => - saved - ? this.start(saved.id) - : ((await this.claimSpare()) ?? this.start(undefined)))().catch( - (error) => { - console.warn("Claude Code could not start", error); - return undefined; - }, - ); + const starting = this.start(saved?.id).catch((error) => { + console.warn("Claude Code could not start", error); + return undefined; + }); this.opening.set(key, starting); live = await starting; if (this.opening.get(key) === starting) this.opening.delete(key); @@ -158,51 +130,47 @@ export class AgentSessions { return delivered; } - /** Stops idle processes started with old settings and replaces the spare; - * busy ones stop when they next fall idle. */ + /** Stops idle processes started with old settings; busy ones stop when they + * next fall idle. */ reconfigure() { const fingerprint = this.options.fingerprint(); for (const [key, live] of this.live) if (live.fingerprint !== fingerprint && !live.process.busy) this.drop(key, live); - const spare = this.spare; - this.spare = undefined; - void spare?.then((live) => live?.process.kill()); - this.warm(); } dispose() { this.disposed = true; for (const [key, live] of this.live) this.drop(key, live); - void this.spare?.then((live) => live?.process.kill()); - this.spare = undefined; } private async start(resume: string | undefined): Promise { // Read before `launch` reads the settings, so it never describes newer ones. const fingerprint = this.options.fingerprint(); const launch = await this.options.launch(); - const process = await ClaudeProcess.start(this.options.spawn, { - ...launch, - ...(resume ? { resume } : { sessionId: this.newId() }), - }); + const process = resume + ? await ClaudeProcess.start(this.options.spawn, { ...launch, resume }) + : await this.fresh(launch); if (fingerprint === this.options.fingerprint() || this.disposed) return { process, fingerprint, used: this.now() }; // Settings saved while it started: it would work with the old ones. void process.kill(); return this.start(resume); } - private async claimSpare() { - const spare = this.spare; - this.spare = undefined; - const live = await spare; - if ( - live?.process.running && - live.fingerprint === this.options.fingerprint() - ) - return live; - void live?.process.kill(); - return undefined; + /** A process for a new session: the spare when it fits, else a new one. */ + private async fresh(launch: Omit) { + const spare = await this.options.spare?.take(launch); + if (spare) + try { + await spare.initialize(launch); + return spare; + } catch { + void spare.kill(); + } + return ClaudeProcess.start(this.options.spawn, { + ...launch, + sessionId: this.newId(), + }); } private adopt(key: string, live: Live, seen: number) { live.process.conversation = key; @@ -212,7 +180,6 @@ export class AgentSessions { void live.process.exited.then(() => { if (this.live.get(key) === live) this.drop(key, live); }); - this.warm(); } private async answer(key: string, live: Live, text: string) { const settled = await live.process.send(text); @@ -240,25 +207,6 @@ export class AgentSessions { at: live.used, }); } - /** Processes running or starting, the spare included. */ - private count() { - return this.live.size + this.opening.size + (this.spare ? 1 : 0); - } - /** Makes room for one more process by stopping the longest-idle - * conversation, else the spare. False when every process is working. */ - private room() { - while (this.count() >= MAX_LIVE) { - const idle = [...this.live] - .filter(([, live]) => !live.process.busy) - .sort(([, a], [, b]) => a.used - b.used)[0]; - if (idle) this.drop(...idle); - else if (this.spare) { - void this.spare.then((live) => live?.process.kill()); - this.spare = undefined; - } else return false; - } - return true; - } private drop(key: string, live: Live) { clearTimeout(live.timer); if (this.live.get(key) === live) { @@ -269,6 +217,92 @@ export class AgentSessions { } } +/** Where a process runs and its model: all a spare is started with. */ +type Where = Pick; +const place = ({ cwd, model }: Where) => JSON.stringify([cwd, model ?? ""]); + +/** One process started ahead of need, for whichever agent next starts a + * conversation where it runs and with its model. Starting is most of the wait + * before a new conversation's first reply; giving the process its prompt is not. + * It is kept only where the runtime's agents run, and follows the latest new + * conversation among those places, the likeliest place for the next. */ +export class Spare { + private next: + | { place: string; process: Promise } + | undefined; + /** Where a spare is wanted: set by the runtime alone. */ + private wanted: readonly Where[] = []; + /** Not started again after it failed to start or died, until a conversation + * asks for one, so a `claude` that cannot run is not started over and over. */ + private stopped = false; + + constructor( + private readonly spawn: Spawn, + /** Answers the tool server before any agent has it: it lists the tools. */ + private readonly tools: ToolServer, + private readonly newId: () => string = () => crypto.randomUUID(), + ) {} + + /** Wants a spare for one of `wheres`, or none when there are none. */ + warm(...wheres: Where[]) { + this.wanted = wheres; + if (!wheres.length) this.stopped = false; + this.fill(wheres); + } + + /** The spare, when it was started for `where`. Another is started for + * `where` when a spare is wanted there, else where one is wanted. */ + async take(where: Where) { + const next = this.next; + this.next = undefined; + this.stopped = false; + const here = this.wanted.some((wanted) => place(wanted) === place(where)); + this.fill(here ? [where] : this.wanted); + if (next?.place !== place(where)) { + void next?.process.then((process) => process?.kill()); + return undefined; + } + const process = await next.process; + if (process?.running) return process; + void process?.kill(); + return undefined; + } + + dispose() { + this.warm(); + } + + /** Keeps the spare when it fits one of `wheres`, else starts one for the + * first; with none, keeps none. */ + private fill(wheres: readonly Where[]) { + const current = this.next; + if (current && wheres.some((where) => place(where) === current.place)) + return; + this.next = undefined; + void current?.process.then((process) => process?.kill()); + const [first] = wheres; + if (this.stopped || !first) return; + const next = { + place: place(first), + process: ClaudeProcess.spawn(this.spawn, { + ...first, + sessionId: this.newId(), + tools: this.tools, + }).catch((error) => { + console.warn("Claude Code could not start a spare session", error); + return undefined; + }), + }; + this.next = next; + void next.process.then(async (process) => { + await process?.exited; + if (this.next !== next) return; + this.next = undefined; + this.stopped = true; + }); + } +} + /** Sessions saved in local storage, by agent and then conversation. */ export function localSessions( storage: Storage, diff --git a/src/bundled/claude-code/tabs.test.tsx b/src/bundled/claude-code/tabs.test.tsx index 5d1a128ea..d20ce78e5 100644 --- a/src/bundled/claude-code/tabs.test.tsx +++ b/src/bundled/claude-code/tabs.test.tsx @@ -61,12 +61,12 @@ function applied({ loggedIn = true } = {}) { const ctx = { pluginOwner: { id: "buzz.claude-code", revision: "r1" }, host: { spawn, request: vi.fn() }, - relay: { snapshot: () => ({ status: "signed-out" }) }, agents2: { register: (registered: AgentType) => { type = registered; }, - snapshot: () => ({ status: "ready", agents: [] }), + snapshot: () => ({ status: "ready", agents: [], running: [] }), + relay: () => undefined, subscribe: () => () => undefined, }, effect: vi.fn(), @@ -91,7 +91,7 @@ async function choose(label: string, option: string) { } it("is an agent type on the desktop app", () => { - expect(inject).toEqual(["agents2", "host", "relay"]); + expect(inject).toEqual(["agents2", "host"]); const { type } = installed(); expect(type).toMatchObject({ id: "claude-code", title: "Claude Code" }); expect(type.defaults()).toEqual({ config: DEFAULT_CONFIG }); diff --git a/src/bundled/codex/README.md b/src/bundled/codex/README.md index 72edcd631..dd92278f5 100644 --- a/src/bundled/codex/README.md +++ b/src/bundled/codex/README.md @@ -2,8 +2,10 @@ A bundled desktop plugin, on by default on macOS and Linux, using the native process host introduced in #731. Agents2 owns the identity, admission, configuration, and signed Buzz actions. This plugin -owns Codex work after delivery handover. It starts one `codex app-server` per agent -and uses independent Codex threads for Buzz conversations. +owns Codex work after delivery handover. It starts one `codex app-server` for all +agents and uses an independent Codex thread for each agent's Buzz conversation. +Each thread carries its agent's workspace, instructions and tools, so one crash +fails every agent's current turn; the next mention reopens the server. ## Try it @@ -47,7 +49,10 @@ current turn finishes. Expect the marker. Thread scope is the default. Channel scope shares a session across channel threads; DMs always share a conversation. Workspace changes start a fresh session. Bindings persist locally under the community and agent identity. Earlier bindings without dynamic tools are ignored; new conversations start fresh. Switching -communities, removing an agent, or disabling the plugin ends its server. +communities does not end an agent's work. Removing the agent, leaving or +disconnecting from its community interrupts its turns and ends shells they left +running; the server ends when no +Codex agent that has worked is still running, or when the plugin is disabled. If a saved Codex thread no longer exists, the next mention starts a fresh one. To inspect a saved session, see [View a Codex session](../../../docs/view-codex-session.md). @@ -76,8 +81,9 @@ workspace and a simulated relay. It sends no Buzz messages. It checks native she and file tools, the native coding prompt, conversation context, default steering, idle follow-ups, process shutdown, saved-thread reuse, changed and cleared instructions, missing-thread recovery, and disabled inherited MCP tools. It also -checks real dynamic Buzz callbacks, memory, workspace file uploads, and normal -final sends without aborted-turn markers. Set +checks real dynamic Buzz callbacks, memory, workspace file uploads, normal +final sends without aborted-turn markers, and two agents sharing one server with +their own workspace, instructions and replies. Set `BUZZ_CODEX_TEST_MODEL` to another available model if needed. It prints only sanitized outcomes; it does not save raw protocol or session logs. diff --git a/src/bundled/codex/index.test.ts b/src/bundled/codex/index.test.ts index 8bb490d70..8ca97b59a 100644 --- a/src/bundled/codex/index.test.ts +++ b/src/bundled/codex/index.test.ts @@ -2,6 +2,7 @@ import * as React from "react"; import { afterEach, beforeEach, expect, it, vi } from "vitest"; import type { AgentType } from "../../features/agents2/service"; +import { CodexRuntime } from "./runtime"; import { apply } from "./index"; const desktop = vi.hoisted(() => ({ tauri: true })); @@ -23,13 +24,12 @@ function applied() { react: React, pluginOwner: { id: "buzz.codex", revision: "r1" }, host: { spawn: vi.fn(), request: vi.fn() }, - relay: { snapshot: () => ({ status: "signed-out" }), subscribe }, - communityReader: { snapshot: () => ({}), subscribe }, agents2: { register: (registered: AgentType) => { type = registered; }, - snapshot: () => ({ status: "ready", agents: [] }), + snapshot: () => ({ status: "ready", agents: [], running: [] }), + relay: () => undefined, subscribe, }, effect: vi.fn(), @@ -48,3 +48,30 @@ it("registers only where the desktop app can run codex and base64", () => { desktop.tauri = false; expect(applied()).toBeUndefined(); }); + +it("stops agents that stop running even when the agent list could not be read", () => { + const sync = vi.spyOn(CodexRuntime.prototype, "sync"); + const agent = { pubkey: "agent", type: "buzz.codex/codex" }; + let state = { status: "ready", agents: [agent], running: [agent] }; + let notify = () => undefined; + apply({ + react: React, + pluginOwner: { id: "buzz.codex" }, + host: { spawn: vi.fn(), request: vi.fn() }, + agents2: { + register: () => undefined, + snapshot: () => state, + relay: () => undefined, + subscribe: (listener: () => undefined) => { + notify = listener; + return () => undefined; + }, + }, + effect: vi.fn(), + } as never); + expect(sync).toHaveBeenLastCalledWith([agent]); + // A failed refresh keeps the list; leaving the community then empties it. + state = { ...state, status: "error", agents: [], running: [] }; + notify(); + expect(sync).toHaveBeenLastCalledWith([]); +}); diff --git a/src/bundled/codex/index.ts b/src/bundled/codex/index.ts index 1cdb8405d..37195bef2 100644 --- a/src/bundled/codex/index.ts +++ b/src/bundled/codex/index.ts @@ -3,7 +3,7 @@ import type { PluginModule } from "../../plugins/api"; import { defaults, config } from "./config"; import { CodexRuntime } from "./runtime"; import { createTabs } from "./tabs"; -export const inject = ["react", "agents2", "host", "relay", "communityReader"]; +export const inject = ["react", "agents2", "host"]; export const apply: PluginModule["apply"] = (ctx) => { // Only the desktop app can start `codex`, and file uploads run `base64`, // which stock Windows lacks; no unusable type elsewhere. @@ -14,36 +14,19 @@ export const apply: PluginModule["apply"] = (ctx) => { (async () => { throw new Error("Codex agents run only in the desktop app"); }); - const runtime = new CodexRuntime(start, ctx.relay); + const runtime = new CodexRuntime(start, ctx.agents2.relay); const typeKey = `${ctx.pluginOwner?.id ?? "buzz.codex"}/codex`; - let selection = ""; + // Agents that stop running stop their turns, and the last one stops the + // server; switching communities or a reconnect does not. const sync = () => { - const community = ctx.communityReader.snapshot(); - const selected = JSON.stringify([community.selected, community.viewer]); - if (selected !== selection) { - runtime.dispose(); - selection = selected; - } const snapshot = ctx.agents2.snapshot(); - const relay = ctx.relay.snapshot(); - if (relay.status === "disconnected") { - runtime.sync([], ""); - return; - } - if (snapshot.status !== "ready" || relay.status !== "ready") return; - runtime.sync( - snapshot.agents.filter((agent) => agent.type === typeKey), - relay.scope ?? "", - ); + if (snapshot.status === "loading") return; + runtime.sync(snapshot.running.filter((agent) => agent.type === typeKey)); }; const offAgents = ctx.agents2.subscribe(sync); - const offRelay = ctx.relay.subscribe(sync); - const offCommunity = ctx.communityReader.subscribe(sync); sync(); ctx.effect(() => () => { offAgents(); - offRelay(); - offCommunity(); runtime.dispose(); }); const { CodexTab, SettingsTab } = createTabs(ctx.react, start, runtime); diff --git a/src/bundled/codex/live.mjs b/src/bundled/codex/live.mjs index fe197614b..dceae90ae 100644 --- a/src/bundled/codex/live.mjs +++ b/src/bundled/codex/live.mjs @@ -2,7 +2,7 @@ import assert from "node:assert/strict"; import { spawn } from "node:child_process"; import { access, mkdtemp, readFile, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; +import { homedir, tmpdir } from "node:os"; import { join } from "node:path"; import { build } from "rolldown"; import { JSDOM } from "jsdom"; @@ -20,6 +20,7 @@ const { CodexRuntime } = await import(join(workspace, "runtime.mjs")); const { AppServer } = await import(join(workspace, "rpc.mjs")); const children = new Set(); const notices = []; +let servers = 0; const sent = []; let acceptedSteers = 0; const nativeSpawn = async (_id, options = {}) => { @@ -46,6 +47,7 @@ const nativeSpawn = async (_id, options = {}) => { exited, }; } + servers++; const child = spawn( "codex", [ @@ -58,7 +60,8 @@ const nativeSpawn = async (_id, options = {}) => { `mcp_servers={"fixture.tools"={command="sh",args=["-c",${JSON.stringify(`touch '${join(workspace, "mcp-must-not-start")}'`)}]}}`, ], { - cwd: options.cwd, + // Like the desktop host, expand ~ in a process's working directory. + cwd: options.cwd?.replace(/^~(?=\/|$)/, homedir()), detached: true, env: Object.fromEntries( Object.entries(process.env).filter( @@ -169,7 +172,7 @@ const relay = { }, }), }; -const runtime = new CodexRuntime(nativeSpawn, relay, store); +const runtime = new CodexRuntime(nativeSpawn, () => relay, store); runtime.sync([{ pubkey }], scope); let id = 0; const delivery = (content) => { @@ -469,6 +472,52 @@ try { console.log( "PASS real dynamic callbacks, resumed tools, shared reads, memory, workspace file read/upload and final send without duplicate replies", ); + const other = "9".repeat(64); + const otherWorkspace = await mkdtemp(join(tmpdir(), "buzz-codex-other-")); + const otherPublished = []; + runtime.sync([{ pubkey }, { pubkey: other }]); + const before = servers; + const first = delivery( + "Reply exactly FIRST_AGENT_OK. No coding tools. Send the reply with buzz.send.", + ); + const second = delivery( + "Run pwd, then reply with its output. Send the reply with buzz.send.", + ); + await runtime.run(first); + await runtime.run({ + ...second, + agent: { + ...second.agent, + pubkey: other, + name: "Other", + publish: async (event) => { + otherPublished.push(event); + return { ...event, id: "8".repeat(64) }; + }, + }, + config: { + ...second.config, + workspace: otherWorkspace, + instructions: `${second.config.instructions} Every final answer MUST contain OTHER_AGENT.`, + }, + }); + await condition( + () => + published.length === 8 && + otherPublished.length === 1 && + runtime.sessions(pubkey)[0]?.status === "Idle" && + runtime.sessions(other)[0]?.status === "Idle", + "two agents on one server", + ); + // The previous step's server is still open and serves both agents. + assert.equal(servers, before); + assert.match(published.at(-1).content, /FIRST_AGENT_OK/); + assert.doesNotMatch(published.at(-1).content, /OTHER_AGENT/); + assert.match(otherPublished[0].content, /OTHER_AGENT/); + assert(otherPublished[0].content.includes(otherWorkspace.split("/").at(-1))); + console.log( + "PASS two agents share one server with their own workspace, instructions and replies", + ); } finally { await inspect?.close(); runtime.dispose(); diff --git a/src/bundled/codex/runtime.test.ts b/src/bundled/codex/runtime.test.ts index 7931b6875..b2d6599e0 100644 --- a/src/bundled/codex/runtime.test.ts +++ b/src/bundled/codex/runtime.test.ts @@ -186,7 +186,7 @@ function fixture( storage.set(key, value); }, } as Storage; - const runtime = new CodexRuntime(spawn, relay, store); + const runtime = new CodexRuntime(spawn, () => relay, store); runtimes.push(runtime); const events: EventData[] = []; const query = vi.fn(async (filters: readonly Record[]) => @@ -226,7 +226,7 @@ function fixture( upload, remember, }; - runtime.sync([{ pubkey } as Agent], scope); + runtime.sync([{ pubkey } as Agent]); let id = 0; const delivery = (content: string, threadRoot = root): Delivery => { const result: Delivery = { @@ -258,7 +258,8 @@ function fixture( events.push(result.trigger.event as EventData); return result; }; - const selection = { selected: "community", viewer: owner }; + /** Agents2's running agents, whichever community is selected. */ + const running = [{ ...agent, type: "buzz.codex/codex" }]; const listeners: (() => void)[] = []; let runPlugin: (delivery: Delivery) => Promise = (delivery) => runtime.run(delivery); @@ -269,25 +270,9 @@ function fixture( apply({ react: React, host: { spawn }, - communityReader: { - snapshot: () => selection, - subscribe: (listener: () => void) => { - listeners.push(listener); - return () => {}; - }, - }, - relay: { - ...relay, - subscribe: (listener: () => void) => { - listeners.push(listener); - return () => {}; - }, - }, agents2: { - snapshot: () => ({ - status: snapshot.status === "ready" ? "ready" : "loading", - agents: [{ ...agent, type: "buzz.codex/codex" }], - }), + snapshot: () => ({ status: "ready", agents: [], running }), + relay: () => relay, subscribe: (listener: () => void) => { listeners.push(listener); return () => {}; @@ -370,7 +355,7 @@ function fixture( memoryView, remember, upload, - selection, + running, runPlugin: (delivery: Delivery) => runPlugin(delivery), notify: () => listeners.forEach((listener) => { @@ -436,6 +421,135 @@ it("hands over promptly and runs conversations independently on one server", asy tags: expect.arrayContaining([["e", root, "", "reply"]]), }); }); +/** A second agent on the same runtime, with its own identity and publisher. */ +function second(f: ReturnType) { + const other = "9".repeat(64); + const base = f.delivery("unused").agent; + const agent = { + ...base, + pubkey: other, + name: "Other", + publish: vi.fn(async (event: unknown) => ({ + ...(event as object), + id: "8".repeat(64), + })), + } as unknown as Delivery["agent"]; + f.runtime.sync([{ pubkey } as Agent, { pubkey: other } as Agent]); + return { + agent, + delivery: (content: string) => ({ + ...f.delivery(content, "7".repeat(64)), + agent, + }), + }; +} +const contents = (publish: { mock: { calls: unknown[][] } }) => + publish.mock.calls.map(([event]) => (event as { content: string }).content); +const servers = (f: ReturnType) => + f.spawn.mock.calls.filter(([id]) => id === "app-server"); +it("serves every agent from one server and routes each tool call to its own agent", async () => { + const f = fixture(); + const b = second(f); + await f.runtime.run(f.delivery("@Codex first")); + await f.runtime.run(b.delivery("@Other second")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(2)); + expect(servers(f)).toHaveLength(1); + expect(f.starts()[1]?.params).toMatchObject({ + additionalContext: { + "buzz.instructions": { value: expect.stringContaining("Other") }, + }, + }); + await f.complete(1, "FROM OTHER"); + await f.complete(0, "FROM CODEX"); + await vi.waitFor(() => expect(f.publish).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(b.agent.publish).toHaveBeenCalledTimes(1)); + expect(contents(f.publish)).toEqual(["FROM CODEX"]); + expect(contents(b.agent.publish as never)).toEqual(["FROM OTHER"]); + expect(f.runtime.sessions("9".repeat(64))[0]?.status).toBe("Idle"); +}); +it("stops a removed agent's turn without ending the server the others share", async () => { + const f = fixture(); + const b = second(f); + await f.runtime.run(f.delivery("work")); + await f.runtime.run(b.delivery("other work")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(2)); + const removed = (f.starts()[1]?.params as { threadId: string } | undefined) + ?.threadId; + f.runtime.sync([{ pubkey } as Agent]); + await vi.waitFor(() => + expect( + f.sent.some( + (wire) => + wire.method === "thread/unsubscribe" && + (wire.params as { threadId: string }).threadId === removed, + ), + ).toBe(true), + ); + for (const method of ["turn/interrupt", "thread/backgroundTerminals/clean"]) + expect(f.sent.find((wire) => wire.method === method)?.params).toMatchObject( + { threadId: removed }, + ); + expect(f.process.end).not.toHaveBeenCalled(); + expect(f.runtime.sessions("9".repeat(64))).toHaveLength(0); + await f.complete(0, "STILL HERE"); + await vi.waitFor(() => expect(f.publish).toHaveBeenCalledTimes(1)); + expect(contents(f.publish)).toEqual(["STILL HERE"]); + expect(b.agent.publish).not.toHaveBeenCalled(); + f.runtime.sync([]); + await f.exits.promise; + expect(f.process.end).toHaveBeenCalled(); +}); +it("ends a stopped agent's leftover shells and lets it start again only after letting go", async () => { + const f = fixture(); + const b = second(f); + await f.runtime.run(f.delivery("work")); + await f.runtime.run(b.delivery("start a dev server")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(2)); + const thread = (f.starts()[1]?.params as { threadId: string } | undefined) + ?.threadId; + await f.complete(1, "SERVER RUNNING"); + await vi.waitFor(() => expect(b.agent.publish).toHaveBeenCalledTimes(1)); + await f.runtime.run(b.delivery("busy again")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(3)); + const index = (method: string) => + f.sent + .map( + (wire) => + wire.method === method && + (wire.params as { threadId: string }).threadId === thread, + ) + .lastIndexOf(true); + const before = index("thread/unsubscribe"); + f.runtime.sync([{ pubkey } as Agent]); + f.runtime.sync([{ pubkey } as Agent, { pubkey: "9".repeat(64) } as Agent]); + await f.runtime.run(b.delivery("back")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(4)); + expect(index("thread/unsubscribe")).toBeGreaterThan(before); + expect(index("thread/backgroundTerminals/clean")).toBeGreaterThan( + index("thread/unsubscribe"), + ); + expect(index("thread/resume")).toBeGreaterThan( + index("thread/backgroundTerminals/clean"), + ); + expect(f.process.end).not.toHaveBeenCalled(); + await f.complete(3, "BACK"); + await f.complete(0, "STILL HERE"); + await vi.waitFor(() => expect(f.publish).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(b.agent.publish).toHaveBeenCalledTimes(2)); + expect(contents(b.agent.publish as never)).toEqual([ + "SERVER RUNNING", + "BACK", + ]); +}); +it("reopens the shared server after it exits", async () => { + const f = fixture(); + await f.runtime.run(f.delivery("work")); + await vi.waitFor(() => expect(f.starts()).toHaveLength(1)); + f.exits.resolve(3); + await vi.waitFor(() => expect(f.publish).toHaveBeenCalledTimes(1)); + await f.runtime.run(f.delivery("again")); + await vi.waitFor(() => expect(servers(f)).toHaveLength(2)); +}); it("serializes owner follow-ups and routes tool replies to the latest accepted request", async () => { const f = fixture(); await f.runtime.run(f.delivery("first")); @@ -655,11 +769,11 @@ it("publishes unexpected exit and empty-completion failures instead of silently content: expect.stringContaining("Codex exited"), }); }); -it("ends the server on community change and suppresses a late final answer", async () => { +it("ends the server when its agent stops running and suppresses a late final answer", async () => { const f = fixture(); await f.runtime.run(f.delivery("work")); await vi.waitFor(() => expect(f.starts()).toHaveLength(1)); - f.runtime.sync([], "other-community"); + f.runtime.sync([]); f.complete(); await f.exits.promise; expect(f.process.end).toHaveBeenCalled(); @@ -880,7 +994,7 @@ it("delivers a plain follow-up after startup without waiting for the running tur await vi.waitFor(() => expect(f.publish).toHaveBeenCalledTimes(1)); }); -it("keeps plugin work alive through relay reconnect and disposes on real community change", async () => { +it("keeps plugin work alive through relay reconnect and ends it when its agent stops running", async () => { const f = fixture([], undefined, true); try { await f.runPlugin(f.delivery("work")); @@ -896,9 +1010,7 @@ it("keeps plugin work alive through relay reconnect and disposes on real communi expect(f.process.end).not.toHaveBeenCalled(); await f.runPlugin(f.delivery("more work")); await vi.waitFor(() => expect(f.starts()).toHaveLength(2)); - f.selection.selected = "other-community"; - f.snapshot.status = "connecting"; - f.snapshot.scope = undefined; + f.running.length = 0; f.notify(); f.complete(); await f.exits.promise; diff --git a/src/bundled/codex/runtime.ts b/src/bundled/codex/runtime.ts index 5804a7177..7f0f9cef7 100644 --- a/src/bundled/codex/runtime.ts +++ b/src/bundled/codex/runtime.ts @@ -9,6 +9,7 @@ import { appClient } from "../../buzz-mcp/app-client"; import { TOOLS, callTool } from "../../buzz-mcp/tools"; import { config, + defaults, absoluteWorkspace, validWorkspace, type Config, @@ -30,6 +31,8 @@ import { } from "./rpc"; type Request = { + /** The agent's community and owner, as `origin:viewer`. */ + scope: string; event: EventData; agent: AgentHandle; settings: Config; @@ -40,6 +43,8 @@ type Request = { }; type Saved = { threadId: string; workspace: string }; type Active = { + /** The server this turn runs on; steering must reach the same process. */ + rpc: AppServer; threadId?: string; turnId?: string; ready: Promise; @@ -58,11 +63,17 @@ type Lane = { type Entry = { storageKey: string; abort: AbortController; - rpc: AppServer; - opening: Promise; + /** Settles once the same agent's previous run has let go of its threads. */ + after: Promise; lanes: Map; saved: Record; }; +type Server = { + rpc: AppServer; + opening: Promise; + /** Threads this process has loaded, and the agent each belongs to. */ + threads: Map; +}; export type SessionView = { key: string; status: string; detail: string }; const input = (text: string) => [{ type: "text", text, text_elements: [] }]; const message = (error: unknown) => @@ -74,15 +85,19 @@ function textOf(event: EventData, name: string) { } /** Agents2 owns admission/identity. This plugin owns work after hand-over, and - * uses one app-server per agent with independent Codex threads per conversation. */ + * uses one app-server for every agent. Each conversation is its own Codex + * thread carrying its agent's workspace, instructions and tools, and tool + * calls find their agent by thread and turn. */ export class CodexRuntime { private entries = new Map(); + private server: Server | undefined; + private stopping = new Map>(); private views = new Map(); private listeners = new Set<() => void>(); - private scope = ""; constructor( private readonly spawn: Spawn, - private readonly relay: RelayData, + /** The owner's connection to each agent's own community. */ + private readonly relay: (pubkey: string) => RelayData | undefined, private readonly storage: Storage = localStorage, ) {} subscribe = (listener: () => void) => { @@ -94,6 +109,8 @@ export class CodexRuntime { sessions = (pubkey: string): readonly SessionView[] => this.views.get(pubkey) ?? EMPTY; private status(pubkey: string, key: string, status: string, detail = "") { + // A stopped agent's draining turns don't bring its sessions back. + if (!this.entries.has(pubkey)) return; const others = this.sessions(pubkey).filter((view) => view.key !== key); this.views.set(pubkey, [ ...others.slice(-31), @@ -101,28 +118,73 @@ export class CodexRuntime { ]); for (const listener of this.listeners) listener(); } - sync(agents: readonly Agent[], scope: string) { - if (scope !== this.scope) { - this.dispose(); - this.scope = scope; - } + sync(agents: readonly Agent[]) { const present = new Set(agents.map((agent) => agent.pubkey)); for (const [pubkey, entry] of this.entries) if (!present.has(pubkey)) { entry.abort.abort(); - void entry.rpc.close(); this.entries.delete(pubkey); this.views.delete(pubkey); + if (this.server) this.stop(this.server, pubkey, entry); } + if (!this.entries.size) this.close(); for (const listener of this.listeners) listener(); } + /** Other agents share the server, so a stopped agent's turns interrupt + * themselves. Then the shells its finished turns left are ended, which its + * own process used to do by exiting. Until then, the same agent starting + * again waits rather than resuming a thread that is still letting go. */ + private stop(server: Server, pubkey: string, entry: Entry) { + const stopped = (async () => { + await Promise.all( + [...entry.lanes.values()].map((lane) => + lane.tail.catch(() => undefined), + ), + ); + for (const [threadId, owner] of server.threads) + if (owner === pubkey) { + server.threads.delete(threadId); + await server.rpc + .request("thread/backgroundTerminals/clean", { threadId }) + .catch(() => undefined); + } + })(); + this.stopping.set(pubkey, stopped); + void stopped.finally(() => { + if (this.stopping.get(pubkey) === stopped) this.stopping.delete(pubkey); + }); + } dispose() { - for (const entry of this.entries.values()) { - entry.abort.abort(); - void entry.rpc.close(); - } + for (const entry of this.entries.values()) entry.abort.abort(); this.entries.clear(); this.views.clear(); + this.stopping.clear(); + this.close(); + } + private close() { + void this.server?.rpc.close(); + this.server = undefined; + } + private open() { + if (this.server) return this.server; + const rpc = new AppServer((call) => this.tool(call)); + // Threads set their own cwd; the process just needs a directory that exists. + const server: Server = { + rpc, + opening: rpc.open(this.spawn, defaults.workspace), + threads: new Map(), + }; + // An exit fails current work and permits the next ordinary mention to reopen. + rpc.onMessage((wire) => { + if (wire.method === "connection/closed" && this.server === server) + this.server = undefined; + }); + void server.opening.catch(() => { + if (this.server === server) this.server = undefined; + void rpc.close(); + }); + this.server = server; + return server; } private entry(request: Request) { const pubkey = request.agent.pubkey; @@ -131,7 +193,7 @@ export class CodexRuntime { let saved: Record = {}; try { const stored: unknown = JSON.parse( - this.storage.getItem(this.storageKey(pubkey)) ?? "{}", + this.storage.getItem(this.storageKey(request)) ?? "{}", ); if (stored && typeof stored === "object" && !Array.isArray(stored)) saved = Object.fromEntries( @@ -145,34 +207,19 @@ export class CodexRuntime { } catch { /* Start fresh after malformed local data. */ } - const rpc = new AppServer((call) => this.tool(entry, call)); const entry: Entry = { - storageKey: this.storageKey(pubkey), + storageKey: this.storageKey(request), abort: new AbortController(), - rpc, + after: this.stopping.get(pubkey) ?? Promise.resolve(), saved, lanes: new Map(), - opening: rpc.open(this.spawn, request.settings.workspace), }; - // An exit fails current work and permits the next ordinary mention to reopen. - rpc.onMessage((wire) => { - if ( - wire.method === "connection/closed" && - this.entries.get(pubkey) === entry - ) { - this.entries.delete(pubkey); - } - }); - void entry.opening.catch(() => { - if (this.entries.get(pubkey) === entry) this.entries.delete(pubkey); - void rpc.close(); - }); this.entries.set(pubkey, entry); return entry; } - private storageKey(pubkey: string) { + private storageKey(request: Request) { // Resume restores the persisted tool schema; pre-tools bindings start fresh. - return `buzz.codex.sessions.v3:${this.scope}:${pubkey}`; + return `buzz.codex.sessions.v3:${request.scope}:${request.agent.pubkey}`; } private save(request: Request, entry: Entry, key: string, saved: Saved) { entry.saved[key] = saved; @@ -197,8 +244,8 @@ export class CodexRuntime { "Codex timers are not supported yet; use mentions or event watches.", ); if (trigger.event.pubkey !== agent.owner) return; - const snapshot = this.relay.snapshot(); - if (snapshot.status !== "ready") + const snapshot = this.relay(agent.pubkey)?.snapshot(); + if (snapshot?.status !== "ready") throw new Error("Connect to the agent's community first"); const channelId = delivery.channelId; if (!channelId) throw new Error("The event has no unambiguous channel"); @@ -210,6 +257,7 @@ export class CodexRuntime { channel?.channelType === "dm" ? undefined : rootOf(trigger.event); const root = settings.scope === "channel" ? undefined : toolRoot; const request: Request = { + scope: snapshot.scope ?? "", event: trigger.event, agent, settings, @@ -234,7 +282,7 @@ export class CodexRuntime { const entry = this.entry(request); let lane = entry.lanes.get(key); if (!lane) { - lane = { tail: Promise.resolve() }; + lane = { tail: entry.after }; entry.lanes.set(key, lane); } if (lane.active) { @@ -254,7 +302,7 @@ export class CodexRuntime { this.start(request, text, key, lane, entry); return; } - await entry.rpc.request("turn/steer", { + await active.rpc.request("turn/steer", { threadId: active.threadId, expectedTurnId: active.turnId, input: input( @@ -308,8 +356,8 @@ export class CodexRuntime { } private async history(request: Request, signal: AbortSignal) { signal.throwIfAborted(); - const snapshot = this.relay.snapshot(); - if (snapshot.status !== "ready" || snapshot.scope !== this.scope) + const snapshot = this.relay(request.agent.pubkey)?.snapshot(); + if (snapshot?.status !== "ready" || snapshot.scope !== request.scope) throw new Error("The community changed"); return conversationHistory(request.event, request.conversation, (filters) => snapshot.session.read(filters, { signal }), @@ -328,8 +376,9 @@ export class CodexRuntime { ], }); } - private async tool(entry: Entry, call: ToolCall): Promise { - const active = [...entry.lanes.values()] + private async tool(call: ToolCall): Promise { + const active = [...this.entries.values()] + .flatMap((entry) => [...entry.lanes.values()]) .map((lane) => lane.active) .find( (active) => @@ -362,8 +411,8 @@ export class CodexRuntime { check, signal: active.signal, memories: async () => { - const snapshot = this.relay.snapshot(); - if (snapshot.status !== "ready" || snapshot.scope !== this.scope) + const snapshot = this.relay(request.agent.pubkey)?.snapshot(); + if (snapshot?.status !== "ready" || snapshot.scope !== request.scope) throw new Error("Buzz is not connected to this community"); const view = snapshot.session.agentMemories.open(request.agent.pubkey); const abort = () => view.dispose(); @@ -448,12 +497,14 @@ export class CodexRuntime { let ready!: () => void; let notReady!: (error: unknown) => void; const turnAbort = new AbortController(); + const server = this.open(); const signal = AbortSignal.any([ entry.abort.signal, turnAbort.signal, AbortSignal.timeout(29 * 60_000), ]); const active: Active = { + rpc: server.rpc, ready: new Promise((resolve, reject) => { ready = resolve; notReady = reject; @@ -477,7 +528,7 @@ export class CodexRuntime { void completed.catch(() => undefined); const abort = () => fail(signal.reason); signal.addEventListener("abort", abort, { once: true }); - const off = entry.rpc.onMessage((wire) => { + const off = server.rpc.onMessage((wire) => { const p = wire.params as | { threadId?: string; @@ -518,7 +569,7 @@ export class CodexRuntime { } }); try { - const rpc = await entry.opening; + const rpc = await server.opening; signal.throwIfAborted(); // The process host expands ~/.buzz and creates it; Codex's RPC needs // an absolute path rather than shell-style home notation. @@ -611,6 +662,7 @@ export class CodexRuntime { }); } active.threadId = started.thread.id; + server.threads.set(active.threadId, agent.pubkey); signal.throwIfAborted(); await rpc.request("thread/name/set", { threadId: active.threadId, @@ -660,23 +712,21 @@ export class CodexRuntime { this.status(agent.pubkey, key, "Idle"); } } catch (error) { + const ended = active.finished; active.finished = true; notReady(error); await active.steering; - if ( - signal.aborted && - active.threadId && - active.turnId && - !entry.abort.signal.aborted - ) { + // Other agents share the server, so even a stopped agent interrupts + // its turn rather than ending the process. + if (signal.aborted && !ended && active.threadId && active.turnId) { try { - await entry.rpc.request("turn/interrupt", { + await server.rpc.request("turn/interrupt", { threadId: active.threadId, turnId: active.turnId, }); - await this.terminals(entry.rpc, active.threadId); + await this.terminals(server.rpc, active.threadId); } catch { - await entry.rpc.close(); + await server.rpc.close(); } } if (!entry.abort.signal.aborted) { @@ -692,10 +742,10 @@ export class CodexRuntime { off(); // Release the subscription before the next resume: loaded-thread rejoin // ignores instruction/config overrides while a client is subscribed. - if (active.threadId && !entry.abort.signal.aborted) - await entry.rpc + if (active.threadId) + await server.rpc .request("thread/unsubscribe", { threadId: active.threadId }) - .catch(() => entry.rpc.close()); + .catch(() => server.rpc.close()); if (lane.active === active) delete lane.active; } } diff --git a/src/bundled/codex/tabs.test.tsx b/src/bundled/codex/tabs.test.tsx index 4a7aacd24..0034f5c92 100644 --- a/src/bundled/codex/tabs.test.tsx +++ b/src/bundled/codex/tabs.test.tsx @@ -11,7 +11,6 @@ import { import { afterEach, expect, it, vi } from "vitest"; import type { AgentViewProps } from "../../features/agents2/service"; import type { HostProcessOptions } from "../../features/host/service"; -import type { RelayData } from "../../features/relay/service"; import { defaults, type Config } from "./config"; import { createTabs } from "./tabs"; import { CodexRuntime } from "./runtime"; @@ -72,7 +71,7 @@ function fixture() { processes.push(process); return process; }); - const runtime = new CodexRuntime(spawn, {} as RelayData, {} as Storage); + const runtime = new CodexRuntime(spawn, () => undefined, {} as Storage); const tabs = createTabs(React, spawn, runtime); const props = (settings = defaults, save = vi.fn(async () => {})) => ({ diff --git a/src/features/agents2/service.test.ts b/src/features/agents2/service.test.ts index 19fe94cb7..6a48bf26e 100644 --- a/src/features/agents2/service.test.ts +++ b/src/features/agents2/service.test.ts @@ -5,7 +5,12 @@ import type { LiveBatch, LiveListener } from "../relay/incoming"; import type { RelayData, RelaySnapshot } from "../relay/service"; import type { RelaySession } from "../relay/session"; import type { AgentIdentity, AgentsNative } from "./native"; -import { Agents2Service, type AgentType, type Delivery } from "./service"; +import { + Agents2Service, + type AgentCommunities, + type AgentType, + type Delivery, +} from "./service"; import { memoryStorage } from "./test-fakes"; // Cleanup's channel step has its own coverage (relay-removal); here only its @@ -33,7 +38,7 @@ const event = (id: string, patch: Partial = {}) => ...patch, }) as unknown as RelayEvent; -function fakeRelay() { +function fakeRelay(origin = "https://relay.example.test") { const live = new Set(); const changes = new Set<() => void>(); const archived = new Set(); @@ -76,8 +81,8 @@ function fakeRelay() { status: "ready", generation: 1, viewer, - origin: "https://relay.example.test", - scope: `https://relay.example.test:${viewer}`, + origin, + scope: `${origin}:${viewer}`, session, }; const connected_ = snapshot; @@ -88,6 +93,7 @@ function fakeRelay() { changes.add(listener); return () => changes.delete(listener); }, + retry: vi.fn(), } as unknown as RelayData, /** Drops the connection, or restores it with a new session. */ connect(connected: boolean) { @@ -96,6 +102,16 @@ function fakeRelay() { : { status: "disconnected", generation: 2, session }; for (const listener of changes) listener(); }, + /** Fails the connection, as a first connect that timed out does. */ + fail() { + snapshot = { status: "error", generation: 4, session, error: "down" }; + for (const listener of changes) listener(); + }, + /** Replaces the session, as after the transport was lost. */ + reconnecting() { + snapshot = { status: "connecting", generation: 3, session }; + for (const listener of changes) listener(); + }, /** Drops and restores the socket inside the same session, as live.ts does. */ recover() { for (const next of ["retrying", "connected"]) { @@ -107,6 +123,39 @@ function fakeRelay() { for (const listener of live) listener(batch); }, archives, + origin, + }; +} + +/** The viewer's joined communities, one selected; `open` connects any of them. */ +function fakeCommunities(...joined: ReturnType[]) { + let relays = joined; + const changes = new Set<() => void>(); + let selected: string | null = relays[0]?.origin ?? null; + return { + communities: { + snapshot: () => ({ + viewer, + selected, + memberships: relays.map(({ origin }) => ({ id: origin, name: origin })), + }), + subscribe: (listener: () => void) => { + changes.add(listener); + return () => changes.delete(listener); + }, + open: vi.fn( + (id: string) => relays.find((fake) => fake.origin === id)?.relay, + ), + } as unknown as AgentCommunities, + select(next: string | null) { + selected = next; + for (const listener of changes) listener(); + }, + leave(origin: string) { + relays = relays.filter((fake) => fake.origin !== origin); + if (selected === origin) selected = null; + for (const listener of changes) listener(); + }, }; } @@ -171,8 +220,10 @@ async function setup({ subscribe: () => () => {}, }); const fake = fakeRelay(); + const second = fakeRelay("https://other.example.test"); + const client = fakeCommunities(fake, second); const native = fakeNative(identities); - const service = new Agents2Service(ctx, fake.relay, native, storage); + const service = new Agents2Service(ctx, client.communities, native, storage); const plugin = ctx.extend({ pluginOwner: Object.freeze({ id: "example", revision: "one" }), }); @@ -206,8 +257,15 @@ async function setup({ run, emit: fake.emit, connect: fake.connect, + fail: fake.fail, + relay: fake.relay, + reconnecting: fake.reconnecting, recover: fake.recover, archives: fake.archives, + second, + select: client.select, + leave: client.leave, + communities: client.communities, ctx, }; } @@ -424,6 +482,121 @@ it("claims an agent as it comes into view, and releases it as it leaves, without await vi.waitFor(() => expect(native.claim).toHaveBeenCalledWith(bot)); }); +it("keeps running an agent while another community is selected, and gives it only its community's events", async () => { + const { service, run, emit, second, select } = await setup(); + await service.create({ type: "example/echo", name: "Echo" }); + select(second.origin); + expect(service.find(bot)).toBeUndefined(); + expect(service.snapshot().running.map((agent) => agent.pubkey)).toEqual([ + bot, + ]); + expect(service.relay(bot)?.snapshot().origin).toBe( + "https://relay.example.test", + ); + second.emit({ events: [event("2", { tags: [["p", bot]] })] }); + const mention = event("1", { tags: [["p", bot]] }); + emit({ events: [mention] }); + await vi.waitFor(() => expect(run).toHaveBeenCalledTimes(1)); + expect(run.mock.calls[0]?.[0].trigger).toEqual({ + type: "mention", + event: mention, + }); + await settle(); + expect(run).toHaveBeenCalledTimes(1); +}); + +it("connects an agent's community at start without selecting it", async () => { + const { service, run, second, communities } = await setup({ + identities: [ + { + pubkey: bot, + relay: "wss://other.example.test", + owner: viewer, + type: "example/echo", + name: "Echo", + deleted: false, + }, + ], + }); + expect(communities.open).toHaveBeenCalledWith("https://other.example.test"); + expect(service.find(bot)).toBeUndefined(); + second.emit({ events: [event("1", { tags: [["p", bot]] })] }); + await vi.waitFor(() => expect(run).toHaveBeenCalledTimes(1)); +}); + +it("opens no community for an agent list that arrives after it stopped", async () => { + const ctx = new Context(); + ctx.provide("pluginStatus", { + isActive: () => true, + subscribe: () => () => {}, + }); + const client = fakeCommunities( + fakeRelay(), + fakeRelay("https://other.example.test"), + ); + client.select(null); + const native = fakeNative(); + let list!: (identities: AgentIdentity[]) => void; + native.list.mockReturnValueOnce( + new Promise((resolve) => { + list = resolve; + }), + ); + const service = new Agents2Service( + ctx, + client.communities, + native, + memoryStorage(), + ); + await vi.waitFor(() => expect(list).toBeDefined()); + await ctx.fiber.dispose(); + list([ + { + pubkey: bot, + relay: "wss://other.example.test", + owner: viewer, + type: "example/echo", + name: "Echo", + deleted: false, + }, + ]); + await settle(); + expect(client.communities.open).not.toHaveBeenCalled(); + expect(service.snapshot().running).toEqual([]); +}); + +it("keeps an agent running while its community reconnects, and stops it on disconnect or leave", async () => { + const { service, connect, reconnecting, leave } = await setup(); + await service.create({ type: "example/echo", name: "Echo" }); + const running = () => service.snapshot().running.map((agent) => agent.pubkey); + reconnecting(); + expect(service.find(bot)).toBeUndefined(); + expect(running()).toEqual([bot]); + connect(false); + expect(running()).toEqual([]); + connect(true); + expect(running()).toEqual([bot]); + leave("https://relay.example.test"); + expect(running()).toEqual([]); + expect(service.relay(bot)).toBeUndefined(); +}); + +it("tries a failed unselected community again every so often, and leaves the selected one's Retry to the viewer", async () => { + vi.useFakeTimers(); + const { service, relay, fail, second, select } = await setup(); + await service.create({ type: "example/echo", name: "Echo" }); + fail(); + await vi.advanceTimersByTimeAsync(10_000); + expect(relay.retry).not.toHaveBeenCalled(); + select(second.origin); + await vi.advanceTimersByTimeAsync(5_000); + expect(relay.retry).toHaveBeenCalledTimes(1); + await vi.advanceTimersByTimeAsync(20_000); + expect(relay.retry).toHaveBeenCalledTimes(1); + await vi.advanceTimersByTimeAsync(10_000); + expect(relay.retry).toHaveBeenCalledTimes(2); +}); + it("is not woken by reactions, deletions or DMs", async () => { const { service, run, emit } = await setup(); await service.create({ type: "example/echo", name: "Echo" }); @@ -639,7 +812,12 @@ it("keeps an agent created while identities are still loading", async () => { native.list.mockImplementationOnce( () => new Promise((resolve) => (release = resolve)), ); - const service = new Agents2Service(ctx, fakeRelay().relay, native, storage); + const service = new Agents2Service( + ctx, + fakeCommunities(fakeRelay()).communities, + native, + storage, + ); ctx .extend({ pluginOwner: Object.freeze({ id: "example", revision: "one" }) }) .agents2.register({ diff --git a/src/features/agents2/service.ts b/src/features/agents2/service.ts index dfe86bfa9..9c9fe592c 100644 --- a/src/features/agents2/service.ts +++ b/src/features/agents2/service.ts @@ -2,8 +2,8 @@ // type (its summary line, peek view, settings tabs and `run`); each agent made // from it has its own native-held key, an `agent-attention/v1` configuration the // app owns and edits, and a config blob the type owns. The app delivers matching live events to -// `run` from the stream the owner already receives, so no plugin opens a socket -// or a REQ. +// `run` from the stream the owner already receives in the agent's community, +// selected or not, so no plugin opens a socket or a REQ. import { Service, type Context } from "@deepseek-ai/cordis"; import type { ComponentType } from "react"; import { @@ -11,7 +11,8 @@ import { type Contribution, } from "../../plugins/contributions"; import { removeAgentFromChannels } from "../agents/relay-removal"; -import { relayOrigin } from "../communities/destination"; +import { communityDestination, relayOrigin } from "../communities/destination"; +import type { Communities } from "../communities/service"; import type { RelayEvent } from "../relay/events"; import type { LiveBatch } from "../relay/incoming"; import { relayPartition } from "../relay/partition"; @@ -48,7 +49,7 @@ import { type SkippedObject, } from "./store"; -/** One agent in the selected community. */ +/** One of the viewer's agents, in the community at `relay`. */ export type Agent = Readonly<{ pubkey: string; name: string; @@ -154,7 +155,11 @@ export type AgentType = { export type RegisteredAgentType = Contribution; export type AgentsSnapshot = Readonly<{ status: "unavailable" | "loading" | "ready" | "error"; + /** The selected community's agents, while it is connected. */ agents: readonly Agent[]; + /** Every agent kept running here, whichever community is selected: the + * viewer's agents in each joined community not disconnected. */ + running: readonly Agent[]; error?: string; }>; export type Agents2 = { @@ -164,6 +169,9 @@ export type Agents2 = { subscribe(listener: () => void): () => void; /** The agent of this pubkey in the selected community, if it is an Agents2 agent. */ find(pubkey: string): Agent | undefined; + /** The owner's connection to a running agent's community, which need not be + * the selected one: read the agent's conversations through this. */ + relay(pubkey: string): RelayData | undefined; create(input: Readonly<{ type: string; name: string }>): Promise; save(pubkey: string, change: AgentChange): Promise; /** Deletes its key at once. Leaving its channels and archiving it follow in @@ -186,6 +194,8 @@ const RATE_LIMIT = 60; const RATE_WINDOW_MS = 60_000; const TIMEOUT_LIMIT_MS = 30 * 60_000; const TIMER_TICK_MS = 5_000; +/** How often a failed connection to an unselected community is tried again. */ +const RETRY_MS = 30_000; const EMPTY = Object.freeze({}); const blank = (pubkey: string): AgentRecord => ({ pubkey, @@ -207,6 +217,12 @@ const bounded = (set: Set, id: string) => { if (set.size > SEEN_LIMIT) set.delete(set.values().next().value as string); }; +/** What Agents2 needs of the communities: the viewer, the selected one, and + * a connection to any joined one without selecting it. */ +export type AgentCommunities = Pick< + Communities, + "snapshot" | "subscribe" | "open" +>; type Binding = { scope: string; origin: string; @@ -214,6 +230,23 @@ type Binding = { session: RelaySession; /** Aborts when this binding is replaced, so no work outlives its session. */ controller: AbortController; + stopLive: () => void; +}; +/** A community the viewer has an agent in, and its session while live. */ +type Bound = { + relay: RelayData; + stop: () => void; + binding?: Binding | undefined; + /** When its failed connection was last tried again. */ + retried?: number; +}; +/** The origin a relay URL or community ID names, if it is a valid one. */ +const originOf = (value: string) => { + try { + return communityDestination(value).url; + } catch { + return undefined; + } }; type Job = { trigger: Trigger; channelId?: string }; type WatchTrigger = Extract; @@ -253,8 +286,10 @@ export class Agents2Service extends Service implements Agents2 { private identities: readonly AgentIdentity[] = []; private records: Record; private state: AgentsSnapshot; - private binding: Binding | undefined; - private stopLive: (() => void) | undefined; + /** By origin, so each agent listens in its own community, selected or not. */ + private bound = new Map(); + /** Agents whose community was live at the last update. */ + private connected = new Set(); private runners = new Map(); /** Counts list requests, so only the latest one is applied. */ private listed = 0; @@ -264,7 +299,7 @@ export class Agents2Service extends Service implements Agents2 { constructor( ctx: Context, - private readonly relay: RelayData, + private readonly communities: AgentCommunities, private readonly native: AgentsNative | undefined = nativeAgents(), private readonly storage: Storage = globalThis.localStorage, ) { @@ -274,6 +309,7 @@ export class Agents2Service extends Service implements Agents2 { this.state = Object.freeze({ status: native ? "loading" : "unavailable", agents: [], + running: [], }); ctx.effect(() => { const stops = [ @@ -281,27 +317,31 @@ export class Agents2Service extends Service implements Agents2 { this.update(); this.notify(); }), - relay.subscribe(() => this.bind()), + communities.subscribe(() => { + this.watch(); + this.update(); + }), ]; - this.bind(); + this.watch(); void this.refresh(); // Wakes each runner, which starts any timer that is due. const tick = setInterval(() => { for (const runner of this.runners.values()) void this.drain(runner); + this.reconnect(); }, TIMER_TICK_MS); return () => { clearInterval(tick); + // A list still being read is stale: it must not reopen communities. + this.listed++; for (const stop of stops) stop(); - this.stopLive?.(); - this.stopLive = undefined; - this.binding?.controller.abort(); - this.binding = undefined; + for (const origin of [...this.bound.keys()]) this.unbind(origin); for (const runner of this.runners.values()) this.retire(runner); this.runners.clear(); // Clear what is shown without reconciling, which would make new runners. this.state = Object.freeze({ ...this.state, agents: Object.freeze([]), + running: Object.freeze([]), }); this.notify(); }; @@ -338,10 +378,17 @@ export class Agents2Service extends Service implements Agents2 { }; find = (pubkey: string) => this.state.agents.find((agent) => agent.pubkey === pubkey); + relay = (pubkey: string) => { + const origin = originOf(this.agent(pubkey)?.relay ?? ""); + return origin ? this.bound.get(origin)?.relay : undefined; + }; async create({ type, name }: Readonly<{ type: string; name: string }>) { const native = this.native; - const binding = this.binding; + const selected = this.communities.snapshot().selected; + const binding = selected + ? this.bound.get(originOf(selected) ?? "")?.binding + : undefined; const kind = this.types().find((entry) => entry.key === type); if (!native) throw new Error("Agents run only in the desktop app"); if (!binding) throw new Error("Connect to a community first"); @@ -401,7 +448,7 @@ export class Agents2Service extends Service implements Agents2 { async remove(pubkey: string) { if (!this.native) throw new Error("Agents run only in the desktop app"); await this.native.remove(pubkey); - // Drops its settings, and starts its cleanup if its community is open. + // Drops its settings, and starts its cleanup if its community is connected. await this.refresh(); } @@ -442,16 +489,33 @@ export class Agents2Service extends Service implements Agents2 { config: record.config, }); } - private inScope(identity: AgentIdentity, binding: Binding) { - try { - return ( - identity.owner === binding.viewer && - relayPartition(relayOrigin(identity.relay), binding.viewer) === - binding.scope - ); - } catch { - return false; - } + /** The live binding of the agent's community, if it has one now. */ + private bindingOf({ relay, owner }: Pick) { + const origin = originOf(relay); + const binding = origin ? this.bound.get(origin)?.binding : undefined; + return binding?.scope === relayPartition(origin ?? "", owner) + ? binding + : undefined; + } + /** Whether the agent stays running: it is the viewer's, and its community is + * joined and not disconnected. A reconnect does not stop it. */ + private alive({ relay, owner }: AgentIdentity) { + const origin = originOf(relay); + const entry = origin ? this.bound.get(origin) : undefined; + return ( + owner === this.communities.snapshot().viewer && + !!entry && + entry.relay.snapshot().status !== "disconnected" + ); + } + /** A running agent, in any community. */ + private agent(pubkey: string) { + return this.state.running.find((agent) => agent.pubkey === pubkey); + } + /** The agent, while its community's session is live to run it in. */ + private ready(pubkey: string) { + const agent = this.agent(pubkey); + return agent && this.bindingOf(agent) ? agent : undefined; } /** Reads the agent list from native. Every change is made there first and @@ -485,17 +549,17 @@ export class Agents2Service extends Service implements Agents2 { error: message(error), }); } + this.watch(); this.update(); this.notify(); this.sync(); } /** Brings the relay in line with the saved identities: publishes each name - * its community lacks, and finishes Delete for the open community. Each step + * its community lacks, and finishes Delete where connected. Each step * is safe to repeat; what fails is tried again on the next refresh or connect. */ private sync() { const native = this.native; - const binding = this.binding; for (const identity of this.identities) if (!identity.deleted) native @@ -506,8 +570,10 @@ export class Agents2Service extends Service implements Agents2 { error, ), ); - else if (binding && this.inScope(identity, binding)) - void this.cleanup(identity.pubkey, binding); + else { + const binding = this.bindingOf(identity); + if (binding) void this.cleanup(identity.pubkey, binding); + } } /** As harness Delete does: leave every channel, then archive the identity so * it drops out of member lists and mention suggestions. The owner signs both, @@ -554,8 +620,59 @@ export class Agents2Service extends Service implements Agents2 { } } - private bind() { - const snapshot = this.relay.snapshot(); + /** Connects the selected community, where agents are made, and each one + * the viewer has an agent in, deleted ones included so their cleanup can + * finish; lets go of the rest. */ + private watch() { + const { viewer, selected } = this.communities.snapshot(); + const origins = new Set(); + const shown = viewer && originOf(selected ?? ""); + if (shown) origins.add(shown); + for (const identity of this.identities) { + const origin = identity.owner === viewer && originOf(identity.relay); + if (origin) origins.add(origin); + } + // Left, or left and joined again with a new session. + for (const [origin, entry] of this.bound) + if (!origins.has(origin) || this.communities.open(origin) !== entry.relay) + this.unbind(origin); + for (const origin of origins) { + if (this.bound.has(origin)) continue; + const relay = this.communities.open(origin); + if (!relay) continue; + this.bound.set(origin, { + relay, + stop: relay.subscribe(() => this.bind(origin)), + }); + this.bind(origin); + } + } + /** Tries each unselected community whose connection failed again, since + * nobody is there to press its Retry; its agents wait on it. */ + private reconnect() { + const selected = originOf(this.communities.snapshot().selected ?? ""); + const now = Date.now(); + for (const [origin, entry] of this.bound) + if ( + origin !== selected && + entry.relay.snapshot().status === "error" && + now - (entry.retried ?? 0) >= RETRY_MS + ) { + entry.retried = now; + entry.relay.retry(); + } + } + private unbind(origin: string) { + const entry = this.bound.get(origin); + this.bound.delete(origin); + entry?.stop(); + entry?.binding?.stopLive(); + entry?.binding?.controller.abort(); + } + private bind(origin: string) { + const entry = this.bound.get(origin); + if (!entry) return; + const snapshot = entry.relay.snapshot(); const live = snapshot.status === "ready" && !snapshot.cached && @@ -570,24 +687,29 @@ export class Agents2Service extends Service implements Agents2 { controller: new AbortController(), } : undefined; - if ( - this.binding?.session === live?.session && - this.binding?.scope === live?.scope - ) - return; - this.stopLive?.(); - this.binding?.controller.abort(); - this.stopLive = live && this.listen(live.session); - this.binding = live; + const changed = + entry.binding?.session !== live?.session || + entry.binding?.scope !== live?.scope; + if (changed) { + entry.binding?.stopLive(); + entry.binding?.controller.abort(); + entry.binding = live && { + ...live, + stopLive: this.listen(live.session, origin), + }; + } + // Disconnecting stops its agents even when no binding changed. this.update(); - if (live) this.sync(); + if (changed && live) this.sync(); } /** Delivers the session's live events, and retries what failed whenever its * connection recovers, which happens within the same session. */ - private listen(session: RelaySession) { + private listen(session: RelaySession, origin: string) { let connected = session.live.snapshot().status === "connected"; - const stopEvents = session.subscribeLive((batch) => this.dispatch(batch)); + const stopEvents = session.subscribeLive((batch) => + this.dispatch(batch, origin), + ); const stopStatus = session.live.subscribe(() => { const now = session.live.snapshot().status === "connected"; if (now && !connected) this.sync(); @@ -599,22 +721,20 @@ export class Agents2Service extends Service implements Agents2 { }; } - // Joins identities, records and the connection into the visible agents, and - // keeps one runner per saved agent. Runners outlive edits and reconnects; one is - // retired only when its agent is deleted, and its run is aborted when the - // agent's type is replaced. + // Joins identities, records and the connections into the running and visible + // agents, and keeps one runner per saved agent. Runners outlive edits and + // reconnects; one is retired only when its agent is deleted, and its run is + // aborted when the agent's type is replaced. private update() { - const binding = this.binding; - const wasShown = new Set(this.state.agents.map((agent) => agent.pubkey)); const identities = this.identities.filter((identity) => !identity.deleted); - const agents: Agent[] = []; + const running: Agent[] = []; for (const identity of identities) { - if (!binding || !this.inScope(identity, binding)) continue; + if (!this.alive(identity)) continue; const record = this.recordOf(identity); // Reuse the frozen agent while nothing visible changed, so subscribers see // a stable snapshot. - const prior = this.find(identity.pubkey); - agents.push( + const prior = this.agent(identity.pubkey); + running.push( prior && prior.name === identity.name && prior.attention === record.attention && @@ -625,6 +745,10 @@ export class Agents2Service extends Service implements Agents2 { : this.view(identity, record), ); } + const selected = originOf(this.communities.snapshot().selected ?? ""); + const agents = running.filter( + (agent) => originOf(agent.relay) === selected && this.bindingOf(agent), + ); const types = new Map(this.types().map((type) => [type.key, type])); for (const [pubkey, runner] of this.runners) if (!identities.some((identity) => identity.pubkey === pubkey)) { @@ -643,26 +767,33 @@ export class Agents2Service extends Service implements Agents2 { void this.drain(runner); } } - const same = - agents.length === this.state.agents.length && - agents.every((agent, index) => agent === this.state.agents[index]); - if (!same) { + const same = (next: readonly Agent[], prior: readonly Agent[]) => + next.length === prior.length && + next.every((agent, index) => agent === prior[index]); + if ( + !same(agents, this.state.agents) || + !same(running, this.state.running) + ) { this.state = Object.freeze({ ...this.state, agents: Object.freeze(agents), + running: Object.freeze(running), }); this.notify(); } - // Resume runners whose queue paused while their agent was out of view, and - // claim or release each agent as it comes into or leaves view, so a copy of - // the app takes over without waiting for an event or tick. - const shown = new Set(agents.map((agent) => agent.pubkey)); + // Resume runners whose queue paused while their community was not live, + // and claim or release each agent as its community goes live or not, so a + // copy of the app takes over without waiting for an event or tick. + const ready = new Set( + running.filter((agent) => this.bindingOf(agent)).map((a) => a.pubkey), + ); for (const runner of this.runners.values()) if ( runner.queue.length || - shown.has(runner.pubkey) !== wasShown.has(runner.pubkey) + ready.has(runner.pubkey) !== this.connected.has(runner.pubkey) ) void this.drain(runner); + this.connected = ready; } private notify() { for (const listener of this.listeners) listener(); @@ -711,10 +842,12 @@ export class Agents2Service extends Service implements Agents2 { return watches; } - private dispatch(batch: LiveBatch) { + /** Gives a community's live events to the agents in it. */ + private dispatch(batch: LiveBatch, origin: string) { for (const runner of this.runners.values()) { - const agent = this.find(runner.pubkey); - if (!agent || !runner.type?.run) continue; + const agent = this.ready(runner.pubkey); + if (!agent || originOf(agent.relay) !== origin || !runner.type?.run) + continue; for (const event of batch.events) { if (runner.seen.has(event.id)) continue; bounded(runner.seen, event.id); @@ -818,7 +951,7 @@ export class Agents2Service extends Service implements Agents2 { runner.admitted = 0; } if (runner.queue.length >= QUEUE_LIMIT || runner.admitted >= RATE_LIMIT) { - console.warn(`Agent ${this.find(runner.pubkey)?.name} dropped an event`); + console.warn(`Agent ${this.agent(runner.pubkey)?.name} dropped an event`); return; } runner.admitted++; @@ -827,16 +960,15 @@ export class Agents2Service extends Service implements Agents2 { } // One run at a time per agent: events in arrival order, then a due timer. - // Each job runs the agent as it is when the job starts. While the agent is out - // of view (disconnected, another community) the queue pauses and update() - // resumes it; a type that cannot run drops what is queued. A run that ignores + // Each job runs the agent as it is when the job starts. While its community's + // session is not live the queue pauses and update() resumes it; a type that cannot run drops what is queued. A run that ignores // its deadline stops holding the queue when the deadline passes; it is not // otherwise fenced. private async drain(runner: Runner) { if (runner.running) return; runner.running = true; while (this.runners.get(runner.pubkey) === runner) { - const agent = this.find(runner.pubkey); + const agent = this.ready(runner.pubkey); const type = runner.type; const run = type?.run; // An agent this copy cannot run is left to another copy of the app. @@ -844,7 +976,7 @@ export class Agents2Service extends Service implements Agents2 { if (agent) runner.queue.length = 0; await this.release(runner); // It may have come back into view, or gained its type, meanwhile. - if (this.find(runner.pubkey) !== agent || runner.type !== type) + if (this.ready(runner.pubkey) !== agent || runner.type !== type) continue; break; } @@ -856,7 +988,7 @@ export class Agents2Service extends Service implements Agents2 { break; } // The agent or its type may have changed meanwhile. - if (this.find(runner.pubkey) !== agent || runner.type !== type) continue; + if (this.ready(runner.pubkey) !== agent || runner.type !== type) continue; const job = runner.queue.shift() ?? this.dueTimer(agent); if (!job) break; const lifetime = runner.controller.signal; @@ -912,7 +1044,7 @@ export class Agents2Service extends Service implements Agents2 { runner.mayHold = true; // Answering twice beats not answering at all. console.warn( - `Agent ${this.find(runner.pubkey)?.name} was not claimed`, + `Agent ${this.agent(runner.pubkey)?.name} was not claimed`, error, ); return true; diff --git a/src/features/communities/service.test.ts b/src/features/communities/service.test.ts index 4d952c1b5..c0a82f3d7 100644 --- a/src/features/communities/service.test.ts +++ b/src/features/communities/service.test.ts @@ -133,6 +133,28 @@ it("restores only the selected membership on launch", async () => { ]); }); +it("opens a joined community without selecting it, and never one that is not joined", async () => { + const client = setup({ + profile: { name: "Local", picture: "" }, + memberships: [ + { id: "primary", name: "Primary" }, + { id: "secondary", name: "Secondary" }, + ], + selected: "secondary", + }); + await flush(); + await flush(); + const primary = client.open("primary"); + await flush(); + expect(client.snapshot().selected).toBe("secondary"); + expect(primary?.snapshot().status).toBe("ready"); + expect(client.open("primary")).toBe(primary); + expect(client.open("https://unjoined.example.test")).toBeUndefined(); + expect(requests.filter((url) => url.endsWith("/session"))).toEqual([ + "/api/relay/secondary/session", + "/api/relay/primary/session", + ]); +}); it("a failing community does not replace its healthy sibling or the local profile", async () => { const client = setup(); await flush(); diff --git a/src/features/communities/service.ts b/src/features/communities/service.ts index 7345bad55..c15e88918 100644 --- a/src/features/communities/service.ts +++ b/src/features/communities/service.ts @@ -291,6 +291,14 @@ export function createCommunities( if (id) acquire(id); update({ selected: id }); }, + /** A joined community's session, connected without selecting it, for + * background work such as agents; it lives until the community is left. */ + open(id: string): RelayData | undefined { + id = communityDestination(id).id; + return state.memberships.some((m) => m.id === id) + ? acquire(id) + : undefined; + }, saveProfile(profile: PersonalProfile) { update({ profile }); }, diff --git a/tests/integration/session-debug.test.mjs b/tests/integration/session-debug.test.mjs new file mode 100644 index 000000000..7cf3a0c47 --- /dev/null +++ b/tests/integration/session-debug.test.mjs @@ -0,0 +1,401 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { mkdtemp, writeFile, appendFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + createInspector, + ownsThread, + threadId, +} from "../../scripts/session-debug.mjs"; +import { parseLines, sessionModel } from "../../scripts/session-model.mjs"; +const stamp = (n) => new Date(1700000000000 + n * 1000).toISOString(); +const event = (n, type, p = {}) => ({ + type: "event_msg", + timestamp: stamp(n), + payload: { type, ...p }, +}); +const id = "a".repeat(64); +const user = (n, text) => ({ + type: "response_item", + timestamp: stamp(n), + payload: { + type: "message", + role: "user", + content: [{ type: "input_text", text }], + }, +}); +const codex = [ + { + type: "session_meta", + payload: { + id: "test", + base_instructions: { text: "base " }, + dynamic_tools: [{ name: "tool", inputSchema: { type: "object" } }], + }, + }, + event(0, "task_started", { turn_id: "t1" }), + { + type: "turn_context", + timestamp: stamp(0), + payload: { model: "test-model" }, + }, + user( + 1, + `\n${JSON.stringify({ session_thread: id })}\n\nRequest: "hello"`, + ), + { + type: "token_usage_record", + timestamp: stamp(2), + payload: { + turn_id: "t1", + response_id: "r1", + usage: { input_tokens: 100, cached_input_tokens: 50, output_tokens: 10 }, + }, + }, + event(2, "token_count", { + info: { + total_token_usage: { + input_tokens: 100, + cached_input_tokens: 50, + output_tokens: 10, + }, + }, + }), + event(3, "task_complete", { turn_id: "t1", duration_ms: 2900 }), +]; + +test("accepts thread links, not arbitrary URLs", () => { + assert.equal(threadId(`buzz://message?id=${id}`), id); + assert.throws(() => threadId(`https://example.com/?id=${id}`)); + assert.throws(() => threadId("bad")); +}); +test("matches user delivery context, not tool outputs or quoted history", () => { + assert.equal(ownsThread(codex, id), true); + assert.equal( + ownsThread( + [ + { + type: "response_item", + payload: { + type: "custom_tool_call_output", + output: JSON.stringify(codex), + }, + }, + ], + id, + ), + false, + ); + assert.equal( + ownsThread( + [user(0, `${JSON.stringify(codex)}`)], + id, + ), + false, + ); +}); +test("only tolerates a partial final line", () => { + assert.equal(parseLines('{"a":1}\n{"a":', true).length, 1); + assert.throws(() => parseLines('{bad}\n{"a":1}\n', true)); +}); +test("Codex exposes recorded context and counts usage only once", () => { + const result = sessionModel([...codex, codex[4]], { + rates: { "test-model": { input: 2, output: 10, cached: 1, written: 2 } }, + }); + assert.equal(result.turns.length, 1); + assert.equal(result.turns[0].durationMs, 2900); + assert.equal(result.turns[0].usage.input, 100); + assert.equal(result.turns[0].requests.length, 1); + assert.equal(result.turns[0].estimate, 0.00025); + assert.equal(result.turns[0].cost, null); + const { instructions, tools } = result.setup; + assert.deepEqual( + instructions.map((e) => [e.label, e.scope]), + [ + ["Base instructions", "developer"], + ["Run settings", "settings"], + ], + ); + assert.equal(instructions[0].body, "base "); + assert.equal(tools[0].label, "Dynamic tools · 1 (native tools not recorded)"); + assert.match(tools[0].note, /native tools/); + assert.deepEqual(tools[0].tools, [ + { name: "tool", description: undefined, schema: { type: "object" } }, + ]); + assert.ok(result.entries.every((e) => e.kind !== "instructions")); +}); +test("Codex keeps prompt setup apart and shows later setting changes inline", () => { + const context = (n, p) => ({ + type: "turn_context", + timestamp: stamp(n), + payload: { turn_id: `t${n}`, model: "m", ...p }, + }); + const message = (n, role, text) => ({ + type: "response_item", + timestamp: stamp(n), + payload: { + type: "message", + role, + content: [ + { type: role === "assistant" ? "output_text" : "input_text", text }, + ], + }, + }); + const result = sessionModel([ + { type: "session_meta", payload: { id: "s", model_provider: "openai" } }, + context(0, { approval_policy: "never" }), + message(1, "developer", "lead"), + message(1, "user", "# AGENTS.md instructions for /x\n\nbe kind"), + message(2, "assistant", "ok"), + context(3, { approval_policy: "never" }), + context(4, { approval_policy: "on-request" }), + ]); + assert.deepEqual( + result.setup.instructions.map((e) => [e.label, e.scope]), + [ + ["Run settings", "settings"], + ["developer · multi agent role", "developer"], + ["user · AGENTS.md", "user"], + ], + ); + const settings = result.setup.instructions[0].body; + assert.match(settings, /"model_provider": "openai"/); + assert.doesNotMatch(settings, /turn_id/); + const inline = result.entries.filter((e) => e.kind === "instructions"); + assert.deepEqual( + inline.map((e) => [e.label, e.scope, e.timestamp]), + [["Run settings", "settings", stamp(4)]], + ); +}); +test("Claude puts the recorded system prompt and tools in the prompt setup", () => { + const attachment = (n, a, extra = {}) => ({ + type: "attachment", + timestamp: stamp(n), + attachment: a, + ...extra, + }); + const reminder = (n, type, text) => + attachment( + n, + { type }, + { rendered: [{ content: text }], renderedRole: "system" }, + ); + const systemPrompt = ["base ", "__SYSTEM_PROMPT_DYNAMIC_BOUNDARY__"]; + const wire = { name: "Bash", description: "Run", input_schema: {} }; + const tools = [{ name: "Bash", description: "Run", schema: wire }]; + const result = sessionModel([ + { + type: "user", + uuid: "u", + timestamp: stamp(0), + message: { content: "hi" }, + }, + reminder(0, "date", "Today is 2026-10-10."), + attachment(0, { type: "credential_org", organizationUuid: "x" }), + attachment(0, { type: "prompt_snapshot", systemPrompt }), + { + type: "assistant", + timestamp: stamp(1), + message: { + id: "m", + model: "m1", + content: [{ type: "text", text: "ok" }], + }, + }, + attachment(1, { + type: "prompt_snapshot", + systemPrompt, + cliPrefix: "You are Claude Code.", + tools, + }), + reminder(2, "total_tokens_reminder", "9 left"), + attachment(2, { type: "prompt_snapshot", systemPrompt, tools }), + ]); + const { instructions, tools: groups } = result.setup; + assert.deepEqual( + instructions.map((e) => [e.label, e.scope]), + [ + ["System prompt", "system"], + ["system · date", "system"], + ], + ); + assert.equal(instructions[0].timestamp, stamp(0)); + assert.deepEqual(instructions[0].blocks, [ + "You are Claude Code.", + ...systemPrompt, + ]); + assert.deepEqual( + groups.map((g) => [g.label, g.tools]), + [ + [ + "Tools · 1", + [{ name: "Bash", description: "Run", schema: { input_schema: {} } }], + ], + ], + ); + // Later reminders stay inline, after the reply they followed. + const order = result.entries.map((e) => e.label); + assert.deepEqual(order, [ + "user", + "assistant", + "system · total tokens reminder", + ]); + assert.equal(result.entries[2].scope, "system"); + assert.match(result.entries[2].html, /<b>9 left<\/b>/); +}); +test("Claude deduplicates streamed blocks and ignores tool-result turns", () => { + const records = [ + { + type: "user", + uuid: "u", + timestamp: stamp(0), + message: { content: "hi" }, + }, + ...[1, 2].map((n) => ({ + type: "assistant", + timestamp: stamp(n), + message: { + id: "m", + model: "m1", + content: [{ type: "text", text: "ok" }], + usage: { + input_tokens: 5, + cache_read_input_tokens: 20, + output_tokens: 2, + }, + }, + })), + { + type: "user", + timestamp: stamp(3), + message: { + content: [{ type: "tool_result", tool_use_id: "x", content: "ok" }], + }, + }, + ]; + const result = sessionModel(records); + assert.equal(result.turns.length, 1); + assert.equal(result.turns[0].usage.input, 5); + assert.equal(result.turns[0].durationMs, null); + assert.equal(result.turns[0].requests.length, 1); +}); +test("captured costs are differenced within a launch and reset on restart", () => { + const records = [0, 10, 20].map((n) => ({ + type: "user", + uuid: String(n), + timestamp: stamp(n), + message: { content: "hi" }, + })); + const trace = [ + { type: "context", timestamp: stamp(0), payload: { systemPrompt: "base" } }, + { + type: "result", + timestamp: stamp(5), + payload: { total_cost_usd: 1, duration_ms: 5000 }, + }, + { + type: "result", + timestamp: stamp(15), + payload: { total_cost_usd: 3, duration_ms: 5000 }, + }, + { type: "context", timestamp: stamp(20), payload: { systemPrompt: "new" } }, + { + type: "result", + timestamp: stamp(25), + payload: { total_cost_usd: 0.5, duration_ms: 5000 }, + }, + ]; + assert.deepEqual( + sessionModel(records, { trace }).turns.map((t) => t.cost), + [1, 2, 0.5], + ); +}); +test("live server follows appended records and rejects cross-origin reads", async (t) => { + const dir = await mkdtemp(join(tmpdir(), "session-debug-")); + t.after(() => rm(dir, { recursive: true, force: true })); + const file = join(dir, "session.jsonl"); + await writeFile(file, `${codex.map((r) => JSON.stringify(r)).join("\n")}\n`); + const { server, url } = await createInspector({ file, traceRoot: dir }); + t.after(() => new Promise((resolve) => server.close(resolve))); + let response = await fetch(`${url}state`); + assert.equal(response.status, 200); + assert.equal((await response.json()).turns.length, 1); + await appendFile( + file, + `${JSON.stringify(event(10, "task_started", { turn_id: "t2" }))}\n`, + ); + response = await fetch(`${url}state`); + assert.equal((await response.json()).turns.length, 2); + assert.equal( + ( + await fetch(`${url}state`, { + headers: { Origin: "https://evil.example" }, + }) + ).status, + 403, + ); + assert.equal((await fetch(`${url}../../other`)).status, 404); +}); + +test("historical response views never pair a tool with its later result", async (t) => { + const dir = await mkdtemp(join(tmpdir(), "session-boundary-")); + t.after(() => rm(dir, { recursive: true, force: true })); + const file = join(dir, "session.jsonl"); + const records = [ + ...codex.slice(0, 4), + { + type: "response_item", + timestamp: stamp(1), + payload: { + type: "function_call", + call_id: "call", + name: "tool", + arguments: "{}", + }, + }, + codex[4], + { + type: "response_item", + timestamp: stamp(3), + payload: { + type: "function_call_output", + call_id: "call", + output: "FUTURE_RESULT", + }, + }, + codex[6], + ]; + await writeFile(file, records.map((r) => JSON.stringify(r)).join("\n")); + const { server, url } = await createInspector({ file, traceRoot: dir }); + t.after(() => new Promise((resolve) => server.close(resolve))); + const snapshot = await (await fetch(`${url}state?through=r1`)).json(); + assert.equal(snapshot.snapshotAt, stamp(2)); + assert.equal( + JSON.stringify(snapshot.entries).includes("FUTURE_RESULT"), + false, + ); + assert.equal(snapshot.turns.length, 1); + const full = await (await fetch(`${url}state`)).json(); + assert.equal(JSON.stringify(full.entries).includes("FUTURE_RESULT"), true); +}); + +test("Claude background notifications remain inside their originating turn", () => { + const records = [ + { type: "user", timestamp: stamp(0), message: { content: "hello" } }, + { + type: "user", + timestamp: stamp(1), + message: { content: "done" }, + }, + { + type: "user", + timestamp: stamp(2), + message: { + content: + "also this", + }, + }, + ]; + assert.equal(sessionModel(records).turns.length, 1); +}); diff --git a/tests/integration/session-timing.test.mjs b/tests/integration/session-timing.test.mjs new file mode 100644 index 000000000..71a15b767 --- /dev/null +++ b/tests/integration/session-timing.test.mjs @@ -0,0 +1,227 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { + partitionTime, + tokenBreakdown, + wallTiming, +} from "../../scripts/session-timing.mjs"; +import { sessionModel } from "../../scripts/session-model.mjs"; +const codexModel = (records) => sessionModel(records, { provider: "codex" }); +const stamp = (s) => new Date(1700000000000 + s * 1000).toISOString(); +const event = (s, type, p = {}) => ({ + timestamp: stamp(s), + type: "event_msg", + payload: { type, ...p }, +}); +const item = (s, p) => ({ + timestamp: stamp(s), + type: "response_item", + payload: p, +}); +const usage = (s, id) => ({ + timestamp: stamp(s), + type: "token_usage_record", + payload: { + turn_id: "t", + response_id: id, + usage: { input_tokens: 100, cached_input_tokens: 80, output_tokens: 10 }, + }, +}); +const records = [ + event(0, "task_started", { turn_id: "t" }), + item(0.5, { type: "message", role: "user", content: [] }), + item(2, { + type: "function_call", + call_id: "c", + name: "read", + arguments: "{}", + }), + usage(2.1, "r1"), + item(5, { type: "function_call_output", call_id: "c", output: "ok" }), + item(6, { + type: "message", + role: "assistant", + content: [{ type: "output_text", text: "done" }], + }), + usage(6.1, "r2"), + event(7, "task_complete", { turn_id: "t", duration_ms: 7000 }), +]; + +test("wall time includes inferred inference, tool wait, and every uncovered gap", () => { + const model = codexModel(records); + assert.deepEqual(model.timing.totals, { + inference: 2500, + tool: 3000, + unknown: 1500, + idle: 0, + overlap: 0, + }); + assert.equal( + Object.values(model.timing.totals).reduce((a, b) => a + b, 0), + 7000, + ); + const inference = model.timing.runs[0].items.filter( + (i) => i.kind === "inference", + ); + assert.equal(inference.length, 2); + assert.equal(inference[0].estimated, true); + assert.deepEqual(inference[0].tokens, { + fresh: 20, + cached: 80, + written: 0, + output: 10, + total: 110, + }); +}); +test("parallel tools and overlapping inference never inflate the wall-time total", () => { + const parts = partitionTime(0, 100, [ + { kind: "tool", start: 10, end: 70 }, + { kind: "tool", start: 30, end: 80 }, + { kind: "inference", start: 60, end: 90 }, + ]); + assert.deepEqual(parts, [ + { kind: "unknown", start: 0, end: 10 }, + { kind: "tool", start: 10, end: 60 }, + { kind: "overlap", start: 60, end: 80 }, + { kind: "inference", start: 80, end: 90 }, + { kind: "unknown", start: 90, end: 100 }, + ]); + assert.equal( + parts.reduce((n, s) => n + s.end - s.start, 0), + 100, + ); +}); +test("idle waits between runs stay in the session denominator", () => { + const model = codexModel([ + ...records, + event(12, "task_started", { turn_id: "t2" }), + event(14, "task_complete", { turn_id: "t2" }), + ]); + assert.equal(model.timing.totals.idle, 5000); + assert.equal( + Object.values(model.timing.totals).reduce((a, b) => a + b, 0), + 14000, + ); +}); +test("missing model output timing stays unknown instead of inventing a duration", () => { + const model = codexModel([ + event(0, "task_started", { turn_id: "t" }), + usage(4, "r1"), + event(5, "task_complete", { turn_id: "t" }), + ]); + assert.equal(model.timing.totals.inference, 0); + assert.equal(model.timing.totals.unknown, 5000); + assert.equal( + model.timing.runs[0].items.find((i) => i.kind === "inference").estimated, + false, + ); +}); +test("Claude blocks count one inference with tools paired across user results", () => { + const model = sessionModel([ + { type: "user", timestamp: stamp(0), message: { content: "hi" } }, + { + type: "assistant", + timestamp: stamp(1), + message: { + id: "m", + usage: { + input_tokens: 2, + cache_read_input_tokens: 80, + cache_creation_input_tokens: 20, + output_tokens: 3, + }, + content: [{ type: "text", text: "hello" }], + }, + }, + { + type: "assistant", + timestamp: stamp(2), + message: { + id: "m", + usage: { + input_tokens: 2, + cache_read_input_tokens: 80, + cache_creation_input_tokens: 20, + output_tokens: 3, + }, + content: [{ type: "tool_use", id: "c", name: "read", input: {} }], + }, + }, + { + type: "user", + timestamp: stamp(5), + message: { + content: [{ type: "tool_result", tool_use_id: "c", content: "ok" }], + }, + }, + ]); + const run = model.timing.runs[0]; + assert.equal(run.items.filter((i) => i.kind === "inference").length, 1); + assert.equal(run.end - run.start, 5000); + assert.deepEqual(run.items[0].tokens, { + fresh: 2, + cached: 80, + written: 20, + output: 3, + total: 105, + }); +}); +test("only explicit running turns extend to now", () => { + const model = codexModel(records.slice(0, -1)); + const timing = wallTiming( + records.slice(0, -1), + model.turns, + "codex", + Date.parse(stamp(10)), + ); + assert.equal(timing.end - timing.start, 10000); + assert.equal( + tokenBreakdown({ input: 10, cached: 8, written: 1, output: 2 }, "codex") + .fresh, + 1, + ); +}); +test("Claude resume bookkeeping after a finished run counts as idle, not inference", () => { + const assistant = (s, id, content, model = "claude-opus") => ({ + type: "assistant", + timestamp: stamp(s), + message: { + id, + model, + usage: { input_tokens: 1, output_tokens: 1 }, + content, + }, + }); + const model = sessionModel([ + { type: "user", timestamp: stamp(0), message: { content: "hi" } }, + assistant(2, "m1", [ + { type: "tool_use", id: "c", name: "send", input: {} }, + ]), + { + type: "user", + timestamp: stamp(3), + message: { + content: [{ type: "tool_result", tool_use_id: "c", content: "sent" }], + }, + }, + { type: "attachment", timestamp: stamp(3) }, + { type: "queue-operation", timestamp: stamp(1500) }, + assistant( + 1500, + "synthetic-id", + [{ type: "text", text: "No response requested." }], + "", + ), + { type: "user", timestamp: stamp(1501), message: { content: "next" } }, + assistant(1504, "m2", [{ type: "text", text: "ok" }]), + ]); + const { totals, runs } = model.timing; + assert.equal(runs.length, 2); + assert.equal(runs[0].end - runs[0].start, 3000); + assert.equal(totals.idle, 1498000); + assert.equal(totals.inference, 5000); + assert.equal( + model.turns.flatMap((t) => t.requests).some((r) => r.id === "synthetic-id"), + false, + ); +});