Skip to content

Refine Scenario configuration and sizing - #2377

Draft
Roman Lutz (romanlutz) wants to merge 1 commit into
copilot/romanlutz-scenario-pr-05-queue-upstreamfrom
copilot/romanlutz-scenario-pr-06-sizing-upstream
Draft

Refine Scenario configuration and sizing#2377
Roman Lutz (romanlutz) wants to merge 1 commit into
copilot/romanlutz-scenario-pr-05-queue-upstreamfrom
copilot/romanlutz-scenario-pr-06-sizing-upstream

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds named technique sets, Adaptive attempt caps, searchable datasets, per-dataset limits, compatibility-aware identities, and exact/ranged backend-owned run arithmetic. Default estimates use bounded concurrency with execution-only timeouts.

What this layer adds

  • Named technique sets plus explicit Custom selection.
  • Adaptive per-objective attempt limits and baseline comparison controls.
  • Searchable dataset selection with bounded per-dataset caps and provenance.
  • Compatibility-aware identities and unsupported Jailbreak-combination filtering.
  • Backend-owned exact/ranged run-size arithmetic shared by preview and launch.
  • Bounded concurrent default estimation with queue time excluded from execution timeout.

Preview

Authoritative Scenario configuration and run-size preview

Stack

This is 6 of 7 in native GitHub Stack #2390 and depends on #2376.

Layer Scope Pull request
1 Progress foundation #2372
2 Catalog and launch #2373
3 Live progress #2374
4 Run history #2375
5 FIFO scheduling #2376
6 Configuration sizing (this PR) #2377
7 Result details #2378

See the full implementation, screenshots, walkthrough, validation, and operational notes.

Validation

  • Backend: 15,212 passed, 120 skipped.
  • Frontend: 68/68 suites, 1,286/1,286 tests; lint, TypeScript, and production build passed.
  • Mock E2E, retries disabled: 91 passed, 0 failed, 0 skipped.
  • Seeded E2E, retries disabled: 62 passed, 0 failed, 0 skipped.
  • All configured pre-commit hooks passed; distinguished-engineer review approved the final stack.

Compatibility notes

Unsupported Jailbreak v4 technique combinations are rejected, v3 persisted Jailbreak runs intentionally do not resume as v4, and configuration-aware Adaptive identities may regroup historical analytics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5d02c2d5-b499-4f78-a04d-03bffa750817
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants