Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/better-harness-doc-links.mmd
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ flowchart LR
n_skills_better_harness_references_asset_demand_reconciliation_md["asset-demand-reconciliation.md"]
n_skills_better_harness_references_finding_bound_fix_md["finding-bound-fix.md"]
n_skills_better_harness_references_findings_review_md["findings-review.md"]
n_skills_better_harness_references_manual_direct_fix_md["manual-direct-fix.md"]
n_skills_better_harness_references_project_harness_md["project-harness.md"]
n_skills_better_harness_references_session_evidence_md["session-evidence.md"]
n_skills_better_harness_references_session_repeated_workflows_md["session-repeated-workflows.md"]
Expand All @@ -69,6 +70,7 @@ flowchart LR
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_asset_demand_reconciliation_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_finding_bound_fix_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_findings_review_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_manual_direct_fix_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_project_harness_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_session_evidence_md
n_skills_better_harness_SKILL_md --> n_skills_better_harness_references_session_repeated_workflows_md
Expand Down
167 changes: 167 additions & 0 deletions docs/specs/2026-07-30-html-finding-fix-callback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# Keep copied finding fixes bound without exposing local paths

## Traceability

- Spec ID: html-finding-fix-callback
- Review: QoderAI/better-harness#39
- Status: Implemented

## Intent

Make `Copy AI Fix` from a durable Codex HTML report executable by the
activated Better Harness Skill when the report remains in its generated
workspace-relative location. Preserve callback-free report compatibility and
the report's portable reader surface without embedding renderer-added absolute
workspace or artifact paths.

The callback is local action transport, not an analysis conclusion or
AI-authored reader field. Rendering must leave the persisted `findings.json`,
paired Markdown report, finding scores, and reviewed `aiFixPrompt` unchanged.

## Acceptance Scenarios

- AC-1: A report renders two Copy controls only for each finding whose reviewed
`aiFixPrompt` is a non-empty, non-whitespace string. A finding without a
usable prompt retains its details control but has no action row and no
callback metadata.
- AC-2: A bound report embeds only a normalized workspace-relative POSIX route
to the final `report.html`, plus finding ids and current revisions. It does
not embed renderer-added absolute `workspacePath`, `findingsPath`,
`dataPath`, or `target.path` values in either interaction payload.
- AC-3: When the current document is a `file:` URL whose decoded path ends in
the exact embedded report route, Copy derives the workspace root and sibling
final `findings.json` path locally. It appends exactly one
`<better-harness-fix-output>` block with contract
`better-harness-fix-output/v1`, the derived exact paths, exact finding id,
and the finding's current `actualOutputRevision` or `0`.
- AC-4: When the renderer has no safe action context, or the report is opened
through HTTP or from a file path that does not match the expected route, all
clipboard and manual-copy paths transport the unchanged reviewed
`aiFixPrompt` without a callback or renderer-added local paths.
- AC-5: An intentionally empty action binding enables only the compatibility
fallback. A present but malformed, incomplete, stale, cross-bound, absolute,
or escaping binding fails deterministic HTML validation and never becomes a
compatibility fallback.
- AC-6: Action transport remains finding-scoped across multiple rows and
produces parseable callback JSON for Windows drive paths, Windows UNC paths,
macOS paths, Linux paths, spaces, Unicode, and punctuation.
- AC-7: The final route is derived from the published run directory, never the
temporary staging directory or source findings path. Existing output-location
allocation and replacement behavior remains unchanged.
- AC-8: The rendered `findings.json`, paired `report.md`, source
`aiFixPrompt`, Qoder Canvas behavior, report scoring, finding eligibility,
and visible details content remain unchanged.

## Non-goals

- Automatically submit, approve, or execute the copied fix.
- Add a ChatGPT Desktop host bridge, deep link, network request, sidecar action
file, new persisted finding field, or runtime file read.
- Guarantee a finding-bound callback after the HTML file is moved, downloaded
outside its generated route, shared to another machine, or served over HTTP.
Those states retain the reviewed manual prompt only.
- Change Qoder Canvas handoffs or broaden one finding-bound repair into a new
Harness review.
- Repair previously generated HTML files in place.

## Plan and Tasks

1. Project the full report into a minimal reader-safe interaction payload
containing only actionable finding ids and unchanged reviewed prompts.
Remove the full `reportData` object from embedded interaction JSON.
(AC-1, AC-2, AC-8)
2. Build deterministic machine action metadata from the target workspace and
final report location. Emit only a safe workspace-relative report route and
per-finding revision metadata; emit an intentionally empty binding when the
report is not safely beneath the workspace. (AC-1, AC-2, AC-5, AC-7)
3. Resolve file-local bindings inside the self-contained interaction
controller. Require exact route suffix matching, derive platform-native
workspace and sibling findings paths, and append the v1 callback only at
copy time. (AC-3, AC-6)
4. Preserve the raw-prompt compatibility route for intentionally empty
bindings, HTTP previews, and moved file reports. Keep malformed declared
bindings fail-closed. (AC-4, AC-5)
5. Render and validate Copy controls only for usable prompts. Validate exact
interaction and action payload projections plus per-finding action counts.
(AC-1, AC-5)
6. Add unit, renderer, CLI, and browser regression coverage for callback
generation, fallback states, empty prompts, binding tampering, final routes,
and cross-platform URL/path conversion. (AC-1..AC-8)

## Test and Review Evidence

- Interaction coverage:
`node --test test/html-report-interactions.test.mjs` must verify original
file binding, callback JSON parsing, Clipboard API/legacy/manual parity,
intentional-empty fallback, HTTP fallback, moved-file fallback, malformed
binding failure, and Windows/macOS/Linux paths.
- Renderer and CLI coverage:
`node --test test/harness-report-render-cli.test.mjs` must verify minimal
embedded interaction data, relative final report routes, actionable finding
counts, no renderer-added absolute paths, no staging routes, exact revisions,
and validator rejection of tampering.
- AC-8: compare rendered `findings.json` and Markdown semantics with the input,
and run the existing Canvas and report-render suites.
- Documentation integrity:
`node scripts/doc-link-graph/cli.mjs skills/better-harness`, then
`node --test test/doc-link-graph.test.mjs`.
- Complete validation: `npm test`, `npm run pack:verify`, and
`git diff --check`.
- Visual validation: serve a generated HTML report through the local HTML
preview, use Playwright to inspect console/page errors, exercise raw-prompt
HTTP fallback and the details dialog, and retain a screenshot.
- Repository preview smoke: run `npm run preview`, then request
`http://localhost:58575/health` and
`http://localhost:58575/canvas-module.js`. If the external Canvas SDK
runtime is unavailable, record the limitation separately rather than
claiming success.
- Risk: route suffix matching could bind a moved report to the wrong workspace.
Accept only safe relative segments and an exact decoded `file:` suffix;
otherwise transport only the reviewed prompt.
- Risk: Windows URL decoding and separator conversion could create invalid JSON
or paths. Generate the callback with `JSON.stringify` and parse it in
cross-platform fixtures.
- Risk: compatibility fallback could hide a corrupted binding. Distinguish an
explicit empty binding from any declared malformed or mismatched binding and
keep the latter fail-closed.

## Revision Baseline

- PR #39 review confirmed that callback-free renders expose Copy buttons whose
interaction controller can no longer find the original prompt.
- The same review confirmed that empty prompts currently receive callback-only
actions and that the hand-written Windows callback fixture is invalid JSON.
- The pre-change HTML already embeds absolute `dataPath` and `target.path`;
the first callback implementation additionally embeds `workspacePath` and
`findingsPath`. This revision removes renderer-added absolute paths from both
interaction payloads rather than documenting a local-path disclosure.

## Implementation Evidence

- The renderer now embeds only actionable finding ids/prompts and a separate
final workspace-relative report route with ids/revisions. Context-free or
unsafe output locations emit `{ reportRoute: null, findings: [] }`.
- The browser controller validates both minimal payloads, derives native
workspace and sibling `findings.json` paths only for an exact matching
`file:` route, and uses the same computed text for Clipboard API, legacy, and
manual-copy paths. HTTP and moved-file routes preserve the raw reviewed
prompt; malformed declared bindings fail closed.
- Copy controls and binding rows are omitted for empty or whitespace-only
prompts while details remain visible. The HTML validator enforces the minimal
payloads, relative route, revision, finding binding, and actionable counts.
- `node --test test/html-report-interactions.test.mjs` passed 9/9, including
Windows drive, Windows UNC, macOS, Linux Unicode, local-file, HTTP, moved-file,
empty-binding, and malformed-binding cases. All HTML-focused render tests
passed; the enclosing file has one unrelated local symlink-permission case.
- Playwright verified the original local file callback fields and the raw-prompt
HTTP fallback, opened the details dialog, retained a screenshot, and found no
report-script warning or error. The temporary HTTP server produced only a
missing `favicon.ico` 404.
- `npm run pack:verify` passed with 365 npm entries and 388 runtime ZIP entries.
The full `npm test` run passed 1036/1046 with 6 platform skips; the remaining
4 failures are local Windows `EPERM` errors from tests that require symlink
creation and do not touch this change.
- `npm run preview` could not start because no Canvas SDK runtime or
`CANVAS_SDK_MEDIA_DIR` / `CANVAS_SDK_ROOT` is available, so `/health` and
`/canvas-module.js` were unavailable. The self-contained HTML browser route
has no Canvas runtime dependency.
125 changes: 125 additions & 0 deletions docs/specs/2026-07-30-manual-direct-fix-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Route explicit manual fixes without requiring a report callback

## Traceability

- Spec ID: manual-direct-fix-routing
- Review: QoderAI/better-harness#39
- Status: Implemented

## Intent

Let ChatGPT Desktop users invoke an explicit command such as
`/better-harness fix this issue` with a concrete problem, bounded change
request, and validation instructions without first generating a Better Harness
report. Preserve the Skill's existing review trigger surface and keep the
manual route separate from report-bound repair and ordinary Harness reviews.

The manual route may change only the task-local workspace authorized by the
prompt. It must not discover, update, or claim repair progress for a
`findings.json`.

## Acceptance Scenarios

- AC-1: A callback-free slash-command request selects Manual Direct Fix only
when its first instruction is an explicit `fix`, `repair`, or `修复`
directive and the prompt identifies a concrete problem or requested outcome.
The agent inspects the smallest relevant owner, applies the bounded change,
and runs the smallest relevant validation without requesting report callback
fields or starting a new Harness review.
- AC-2: A request containing a machine-owned
`<better-harness-fix-output>` callback takes precedence and continues to use
the existing Finding-bound Fix protocol, including exact path, finding, and
revision validation plus `record-fix-output`.
- AC-3: Requests whose first instruction is review, evaluation, reporting, or
another Harness-review intent continue to use the evidence-bundle,
reconciliation, and durable-report workflow. A mixed request such as
`review my harness and fix issues` does not grant direct mutation authority.
- AC-4: Manual direct fixes never search for a recent report, infer a Finding,
read or update `findings.json`, run `record-fix-output`, or claim Assignment
Summary, Repair Progress, revision, or score changes.
- AC-5: If an explicit manual repair request does not identify a concrete
problem, requested outcome, or target-local scope, the agent asks for the
missing repair brief; it does not ask the user to hand-author machine callback
fields.
- AC-6: The root Skill retains its established review-trigger keywords and
appends only `manual direct fixes` for discoverability. Step 5 continues to
route Finding-bound repair through its reference and preserves the separate
independent post-fix reassessment constraint.

## Non-goals

- Treat fuzzy repair intent or a repair clause inside a review request as direct
mutation authority.
- Weaken callback validation or add fallback inference to report-bound repair.
- Convert ordinary `/better-harness` reviews into code-changing tasks.
- Backfill or migrate existing reports.
- Change Qoder Canvas, HTML action transport, report schemas, or renderer
artifacts.

## Plan and Tasks

1. Restore the root Skill's original description and append
`manual direct fixes`; restore the Step 5 Finding-bound route and independent
reassessment language without exceeding the root prompt budget. (AC-6)
2. Add a deterministic entry router before the evidence workflow: a valid
callback selects Finding-bound Fix; an explicit leading manual-fix directive
selects Manual Direct Fix; all other invocations retain the review/report
route. (AC-1..AC-3)
3. Tighten the Manual Direct Fix reference around explicit instruction,
task-local authority, minimal owner inspection, validation evidence, and
strict separation from report state. (AC-1, AC-4, AC-5)
4. Add positive and negative routing coverage for English and Chinese explicit
fixes, callback-bound repair, vague fixes, ordinary reviews, and mixed
review-and-fix requests. (AC-1..AC-6)
5. Regenerate the documentation routing graph and verify packaged artifacts.
(AC-1..AC-6)

## Test and Review Evidence

- Root contract and route coverage:
`node --test test/better-harness-skill.test.mjs`.
- Fresh-context behavior:
- explicit English `fix` and `repair`, plus Chinese `修复`, select Manual
Direct Fix without callback fields;
- callback-bearing input selects Finding-bound Fix;
- ordinary review and `review my harness and fix issues` retain the
evidence/report route;
- vague explicit fixes request only the missing task-local brief.
- Documentation integrity:
`node scripts/doc-link-graph/cli.mjs skills/better-harness`, then
`node --test test/doc-link-graph.test.mjs`.
- Complete validation: `npm test`, `npm run pack:verify`, and
`git diff --check`.
- Risk: fuzzy routing could broaden mutation authority. The explicit leading
directive and negative mixed-intent AC keep review requests on the read-only
evidence route.
- Risk: root guidance could lose trigger recall or independent repair review.
Preserve the original keywords and reassessment language and enforce both in
the focused Skill contract test.

## Revision Baseline

- PR #39 review confirmed that the previous description rewrite removed
established trigger keywords and that the `repair intent` predicate was too
broad.
- The same review confirmed that Step 5 dropped an independent post-fix
reassessment guarantee without matching acceptance coverage.
- Earlier environment-specific backup movement is not acceptance evidence for
this revision. Validation must come from the current worktree and CI.

## Implementation Evidence

- The root description retains all established review keywords and appends only
`manual direct fixes`. The root stays within its contract budget at 217 lines
and 11,997 UTF-8 bytes.
- Callback-bearing input routes first to Finding-bound Fix. Without a callback,
only an explicit leading `fix`, `repair`, or Unicode-literal Chinese `fix`
directive routes to Manual Direct Fix; review/evaluation/reporting and mixed
review-and-fix requests remain on Step 1.
- Step 5 again requires the Finding-bound reference and a separate independent
post-fix agent before verified finding state or Repair Progress can change.
- `node --test test/better-harness-skill.test.mjs` passed 13/13 and enforces the
original trigger keywords, explicit positive route, mixed-intent negative
route, report-state boundary, independent reassessment, and prompt budget.
- The regenerated documentation graph passed 6/6 integrity tests. Package and
full-suite results are recorded in the paired HTML callback spec.
Loading