Benchmark Stirrup runs currently create code-execution workspaces during each scenario, but those files are cleaned up when the Stirrup session exits. This makes it difficult to inspect scripts, intermediate outputs, and other artifacts produced by code_exec after a scenario finishes.
Expected behavior:
- Scenario-suite Stirrup runs can write each scenario’s code workspace to a stable host directory.
- Workspaces are nested by agent/model/run id.
- Users can opt into preserving workspaces for later verification.
- Existing benchmark options such as scenario ids, trajectory root, reports root, model id, and continue-on-error continue to work.
Benchmark Stirrup runs currently create code-execution workspaces during each scenario, but those files are cleaned up when the Stirrup session exits. This makes it difficult to inspect scripts, intermediate outputs, and other artifacts produced by
code_execafter a scenario finishes.Expected behavior: