Skillwalker

Skillwalker evaluates Claude Code skills and agents. It measures two things you can improve independently:

• Trigger accuracy — does Claude call your skill (or delegate to your agent) when it should, and stay quiet when it shouldn't?
• Effectiveness — once invoked, does the output meet quality criteria you define? An LLM judge scores it against a rubric.

You write evals, run them inside a Test Sandbox, and track trigger accuracy, output quality, and cost over time through a dashboard and analytics. Skills and agents each have their own trigger-accuracy and effectiveness paths — pick the one that matches what you're improving.

Prerequisites

• Docker Sandboxes (sbx) — Skillwalker runs Claude Code inside Docker Sandboxes via the standalone sbx CLI. Install sbx using Docker's instructions, then run sbx login before creating the Skillwalker sandbox.
• Bun 1.4.2 — the CLI and web app are built with Bun; the root package.json pins "packageManager": "bun@1.4.2" and CI installs that version. Install from bun.sh.
• Node.js 22.12 or newer on PATH — Vitest runs its test workers under node; without it the tests that stub Bun globals fail with "Attempting to change configurable attribute of unconfigurable property". Needed only to run the test suite.

Setup

All commands run from the repository root.

1. Build Skillwalker. This installs dependencies and compiles the skillwalker and skillwalker-web binaries into build/:

   make build

2. Create the Test Sandbox and log in. Authenticate the Sandbox CLI, then create a persistent sandbox and open Claude Code so you can authenticate:

   sbx login
   ./build/skillwalker sandbox create

   Complete the login in the Claude TUI. If you aren't prompted, run /login. When setup finishes, exit with /exit.

   To move to a newer Claude Code release later, run ./build/skillwalker sandbox update. It deletes the sandbox and recreates it from the latest template, so you will log in again.

You're ready. Now choose what you want to measure.

What do you want to measure?

Pick the path that matches your goal — each is a short, end-to-end guide.

Improving a custom skill

1. Know when Claude should call my skill — trigger accuracy
2. Make my skill better at its job — effectiveness

Improving a custom agent

1. Know when Claude should delegate to my agent — trigger accuracy
2. Make my agent better at its job — effectiveness

After your first run, view results in the dashboard and query trends with analytics.

---

Reference

Everything above is all you need for first-time use. The links below are reference material — reach for them when a guide points you here.

Getting Started Guides

• Skill Trigger Accuracy — measure and improve when Claude calls your skill
• Skill Effectiveness — measure and improve skill output quality with LLM-judge rubrics
• Agent Trigger Accuracy — measure and improve when Claude delegates to your agent
• Agent Effectiveness — measure and improve agent output quality with LLM-judge rubrics
• Viewing Results — using the skillwalker-web dashboard
• Analytics — importing data and CLI queries

Full Workflow Guides

• Building SCIL Evals — manual test authoring and the Skill Call Improvement Loop
• Building Rubric Evals — manual rubric authoring and iterating on LLM-judge quality criteria

Configuration Reference

• Evals Reference — full tests.json field reference: test types, expectation types, validation
• Test Scaffolding — how scaffolds provide project context inside the Test Sandbox

Eval Authoring Skills

Claude Code skills that generate evals for you. They ship as the eval-authoring plugin in this repo:

/plugin marketplace add testdouble/skills-test-harness
/plugin install eval-authoring@skillwalker

• Building Skill Eval Scaffolds — /build-skill-eval-scaffold: analysis, signal planning, scaffold generation; --for trigger builds context scaffolds for skill-call tests
• Building Agent Eval Scaffolds — /build-agent-eval-scaffold: the agent equivalent
• Writing Skill-Call Evals — /write-scil-evals: prompt categories, output format
• Writing Agent-Call Evals — /write-acil-evals: the agent equivalent
• Writing Skill Eval Rubrics — /write-skill-eval-rubric: criteria categories, output format
• Writing Agent Eval Rubrics — /write-agent-eval-rubric: the agent equivalent
• Auditing Evals — /audit-evals: find drift between an eval and the skill or agent it tests, and which skill repairs it
• Script Extraction — /script-extraction: hardening skills by extracting mechanical steps into shell scripts

Deep Dives

• Skill Call Improvement Loop — SCIL mechanics: holdout splits, scoring, improvement prompt, CLI flags
• Agent Call Improvement Loop — ACIL mechanics: agent detection, temp plugin isolation, holdout splits, scoring
• LLM Judge Evaluation — judge mechanics: prompt construction, scoring, output format, error handling
• Parquet Schema — field reference for analytics Parquet files

Architecture

• Skillwalker Architecture — system architecture, package boundaries, data flow, and dependency graph
• Sandbox Integration — Test Sandbox architecture, API, lifecycle, and consumer patterns
• Project Discovery — generated project attributes: languages, frameworks, tooling, commands

Package Documentation (contributor)

• CLI — Yargs command layer and path resolution
• Execution — test-run pipeline, test-eval, SCIL/ACIL orchestration, error hierarchy
• Data — shared data layer: types, config parsing, JSONL I/O, DuckDB analytics, SCIL utilities
• Evals — evaluation engine: boolean evals, LLM judge scoring, rubric parsing, orchestrator
• Claude Integration — Claude CLI wrapper API, argument construction, sandbox delegation
• Sandbox Integration Package — Test Sandbox API: full public interface, error handling, testing patterns
• Web — Hono API server, React SPA, test run and SCIL views, per-test analytics
• Bun Helpers — cross-runtime path resolution utilities
• Test Fixtures — shared fixture data, loadFixtures utility, analytics JSONL scenarios

Maintenance and Support

• Maintenance horizon: Indefinitely maintained, best-effort. No SLA.
• Project type: Personal project, with some Test Double support.
• How to report issues: GitHub Issues, with best-effort response within 2 weeks.

Han is an open source product of Test Double, and maintained by the following people:

• River Lynn Bailey: Creator, and primary maintainer

LEGAL NOTICES

Copyright 2026 Test Double, Inc. Distributed under the MIT license.