Skip to content

feat(engine): bandit selector optional - #56

Merged
Mattral merged 1 commit into
mainfrom
feat/optional-selector
Oct 26, 2026
Merged

Mattral merged 1 commit into
mainfrom
feat/optional-selector

Conversation

@Mattral

@Mattral Mattral commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

feat(engine): bandit selector optional

Stacked on #55 (base docs/simpler-quickstart): this PR shortens the quickstart #55 rewrote, so it builds on it to avoid a
conflict in the same README lines. After #55 merges, retarget this PR to main. Do not merge until reviewed.

Change

SpeculativeEngine(config, drafters, target, bandit=None, arms=None). bandit keeps its position and gets the default None.

  • Omitted, and the resolved strategy has one arm: the engine uses a one-arm UCB1Selector
    (window_size=config.bandit.window_size), which always selects arm 0. The engine already ran with a one-arm selector
    in every existing one-drafter call, so this changes nothing at run time. Cases:
    • the default drafter_strategy="auto" with one model drafter (composite-g6-f2), including its per-call fallback at
      batch > 1;
    • an explicit "bandit" strategy with one drafter;
    • one explicit arm.
  • Omitted, with several arms: a ValueError, for example: "2 arms need a bandit selector to choose between them (drafter
    strategy 'bandit'). Pass one, for example flashspec.bandit.UCB1Selector(n_arms=2) or
    flashspec.bandit.ThompsonSelector(n_arms=2). It is optional only when there is one arm."
  • Backward compatible:
    • every existing call (selector positional or bandit=, with or without arms=) is unchanged, and no existing test was
      edited;
    • the whole suite passes;
    • the serve CLI, harness and runners still pass their selectors.

Tests (tests/unit/test_optional_selector.py)

  • One arm, no selector:
    • auto composite: output equals both the engine built with UCB1Selector(n_arms=1) and HF generate(do_sample=False), on 3
      prompts;
    • explicit "bandit" with one drafter: equals HF greedy;
    • one explicit arm;
    • the batch-2 fallback.
  • Several arms, no selector: two model drafters → ValueError naming UCB1Selector(n_arms=2) and
    ThompsonSelector(n_arms=2); three explicit arms → ValueError.
  • Backward compatible: UCB1 and Thompson passed positionally and as bandit=; arms= still by keyword after a positional
    selector.
  • Without the engine change, the 6 new-behaviour tests fail and the 3 compatibility tests pass.

Docs

  • README:
    • quickstart drops from flashspec.bandit import UCB1Selector and the argument: SpeculativeEngine(config, [drafter], target).
      It is now 19 lines;
    • all five one-arm recipe calls drop it too;
    • the "Choose a drafter" paragraph says when a selector is needed;
    • the API table is regenerated (bandit: DraftSelector | None = None).
  • docs/examples.md: SpeculativeEngine(config, drafters, target).
  • llms.txt says when the selector is optional.
  • CHANGELOG [Unreleased]: Added (optional selector) and Changed (simpler quickstart). The 0.3.1 section is written in the
    version-bump PR, stacked on this one.
  • The README quickstart test also asserts there's no selector or flashspec.bandit import in the quickstart.

Gate

Local, on this branch:

  • make lint (ruff, format, import-linter: engine → bandit is an allowed layer import), mypy --strict flashspec: clean
  • pytest tests/unit tests/integration -m "not gpu": 568 passed, 5 skipped; pytest tests/chaos: 5 passed
  • python scripts/readme_api.py --check: current
  • CI on this PR: docs-build pass, lint pass, onnx-parity pass, paper pass, test-chaos pass, test-cpu pass

SpeculativeEngine(config, drafters, target, bandit=None, arms=None): when the
selector is omitted and the resolved strategy has one arm, the engine uses a
one-arm UCB1Selector; with several arms it raises a ValueError naming
UCB1Selector/ThompsonSelector and n_arms. Existing calls are unchanged.

The README quickstart and one-arm recipes, the examples page and llms.txt drop
the selector; the API table is regenerated; CHANGELOG [Unreleased] updated.
@Mattral Mattral mentioned this pull request Oct 26, 2026
6 tasks done
@Mattral
Mattral changed the base branch from docs/simpler-quickstart to main October 26, 2026 08:15
@Mattral
Mattral merged commit fe252c4 into main Oct 26, 2026
6 checks passed
@Mattral
Mattral deleted the feat/optional-selector branch October 26, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant