Skip to content

feat: [routing] send altimate routing hint and register altimate-auto (Phase 0a client) - #1356

Draft
anandgupta42 wants to merge 2 commits into
mainfrom
feat/altimate-auto-hint
Draft

anandgupta42 wants to merge 2 commits into
mainfrom
feat/altimate-auto-hint

Conversation

@anandgupta42

@anandgupta42 anandgupta42 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Issue for this PR

Closes # (no issue; first slice of the gateway model-routing plan, docs/internal/2026-09-22-gateway-model-routing-research.md, Phase 0a)

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

The gateway cannot tell what kind of LLM call it is serving: every request from the CLI carries the same model id and no hint about whether it is a main agent turn, a subagent, a title, a compaction or a skill-selection call. That blocks any routing decision on the gateway side and it also blocks joining gateway traces to CLI outcomes. This PR adds the client half of the plumbing.

  • Sends metadata.altimate = { task_kind, agent, tools, session_pos, message_id } in the request body for the altimate-free and altimate-backend providers only. It rides @ai-sdk/openai-compatible 2.0.41's providerOptions passthrough (verified in the SDK source: getArgs spreads providerOptions[providerId] into the body, filtering only four known keys). It is injected in LLM.stream rather than ProviderTransform.options because small: true calls (title, enhance, project-copy) use smallOptions and never reach options.
  • Stamps task_kind at each call site: main / subagent / summary (shared processor call, branched on parentID and agent name), title, compaction, skill_select, enhance, review, project_copy; everything else is other. message_id is the assistant message id, the same id the generation telemetry event records, so gateway traces and CLI telemetry join on it; processor-less call sites omit it rather than send a throwaway id. tools is clamped to 512 and agent is validated before sending because the gateway drops invalid keys silently.
  • Registers altimate-auto beside altimate-base under the free provider, behind ALTIMATE_AUTO_MODEL=1 (default off). The gateway on main still returns 403 for any model other than altimate-base, so the flag stays off until the gateway's Phase 0a alias lands. When the flag is off the model is not registered at all. Default model selection is unchanged. The three id.includes("altimate-base") checks in transform.ts become an exact-match isAltimateManagedModel helper.
  • Skill selection now uses the session's model when that model is Altimate-managed instead of Provider.defaultModel(). Enhance-prompt and the review AI lane are unchanged because their call chains carry no session; documented in code.

Sending the hint before the gateway change is deployed is safe: today's gateway _strip_client_params removes client metadata without rejecting the request.

How did you verify your code works?

  • New tests in test/session/llm.test.ts drive the real LLM.streamProviderTransform@ai-sdk/openai-compatiblefetch path against a local Bun.serve server and assert the outgoing /chat/completions body: hint present with the right fields for the backend provider, the small: true free-provider path, other default, and no metadata key for a non-Altimate provider.
  • test/session/processor-effect.test.ts: a real main turn asserts the hint's message_id equals the assistant message id used by the generation telemetry event.
  • test/provider/provider.test.ts: altimate-auto absent by default and present with the flag (subprocess env test plus in-process registration test).
  • test/altimate/skill-filtering.test.ts: Altimate session model bypasses defaultModel(); non-Altimate still falls back.
  • From packages/opencode: bun run typecheck clean; bun test on the 17 touched/related files: 932 pass, 17 skip, 43 todo, 0 fail. bun run script/upstream/analyze.ts --markers --base origin/main --strict: all custom code in upstream-shared files properly marked.
  • Not verified: no live run against the gateway (the gateway change is a separate PR in altimate-gateway); e2e suite not run. One pre-existing, order-dependent Effect fiber-cleanup flake in processor-effect.test.ts was observed once in a large combined run and reproduces on the unmodified test when isolated; it is not caused by this change.
  • Reviewed by Codex (two passes); the second pass's blocker (Auto visible before the gateway accepts it), the telemetry join key and the marker gaps are fixed in the second commit.

Screenshots / recordings

Not a UI change. The model picker shows "Altimate Auto" only when ALTIMATE_AUTO_MODEL=1.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code


Summary by cubic

Adds the client half of the Altimate gateway model-routing work: requests to altimate-free and altimate-backend now carry a metadata.altimate hint describing the call, and a new altimate-auto model becomes available behind a default-off flag.

What changed

  • Sends metadata.altimate with task_kind, agent, tools, session_pos, and message_id (matching the generation telemetry event id) for Altimate-managed requests only; other providers are untouched.
  • Tags each call site (main, subagent, title, summary, compaction, skill_select, enhance, review, project_copy), defaulting to other when unstamped.
  • Registers altimate-auto as a pickable model under altimate-free only when ALTIMATE_AUTO_MODEL=1 (default off).
  • Skill selection now reuses the session's Altimate-managed model instead of resolving Provider.defaultModel().

Rollout

  • Sending the hint is safe against the current gateway, which strips client metadata without rejecting the request.
  • Enable ALTIMATE_AUTO_MODEL only after the gateway’s Phase 0a alias change ships; until then, altimate-auto requests return 403.

Written for commit b041cd6. Summary will update on new commits.

Review in cubic

anandgupta42 and others added 2 commits September 22, 2026 14:46
Phase 0 client instrumentation for the gateway's model-routing work (see
docs/internal/2026-09-22-gateway-model-routing-research.md, client section).
No routing decision changes client-side; this only adds the observability
hint and a new selectable model alias.

- Verified `@ai-sdk/openai-compatible@2.0.41`'s chat language model spreads
  unrecognized `providerOptions[<providerID>]` keys directly into the request
  body (only `user`/`reasoningEffort`/`textVerbosity`/`strictJsonSchema` are
  filtered out), so `metadata` rides through the existing providerOptions
  plumbing with no fetch-wrapper hack needed.
- `session/llm.ts`: attach `metadata.altimate` (`task_kind`, `agent`, `tools`,
  `session_pos`, `message_id`, optional `min_tier`) to the outgoing body for
  the `altimate-free`/`altimate-backend` providers only. Injected in
  `stream()` itself (not inside `ProviderTransform.options()`) because
  small-model calls use `ProviderTransform.smallOptions()` and never reach
  `options()`.
- Stamp `task_kind` explicitly at every call site: main loop / subagent /
  summary (`session/prompt.ts`, branching on `session.parentID` and agent
  name), title (`ensureTitle`), compaction (`session/compaction.ts`),
  skill-selector, enhance-prompt, ai-review, project-copy. Anything else
  reports `"other"`.
- `provider/transform.ts`: new `AltimateTaskKind` enum,
  `isAltimateManagedModel()` (covers `altimate-base` and `altimate-auto`,
  replacing three `id.includes("altimate-base")` checks), and
  `isAltimateManagedProviderID()`.
- `provider/provider.ts` + `altimate/free/client.ts`: register a second
  hand-registered model, `altimate-auto` ("Altimate Auto"), under the
  `altimate-free` provider with the same shape/limits as Altimate Base.
  Default-model selection is unchanged — the whole `altimate-free` provider
  is already excluded from `Provider.defaultModel()`'s ordinary scan.
- `skill-selector.ts`/`system.ts`/`prompt.ts`: when the session's model is
  already Altimate-managed, skill selection reuses it instead of always
  resolving `Provider.defaultModel()`. Not applied to `enhance-prompt.ts`,
  `ai-review.ts`, or the `skill` tool's own init-time description builder —
  none of those call sites currently have a session/model in scope without a
  larger refactor (documented in code comments).
- Tests: `test/session/llm.test.ts` adds an end-to-end request-capture test
  (real `@ai-sdk/openai-compatible` serialization against a local HTTP
  server) asserting the outgoing body carries `metadata.altimate` with the
  expected fields, the `"other"` default, and that non-Altimate providers
  never get the field.

Verified: marker guard clean (no upstream-shared files touched), `tsgo
--noEmit` clean, and `bun test` green for every touched file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t gate, id join, markers)

Fixes for f503676's Codex review ("needs changes"):

1. BLOCKER — rollout gate. `altimate-auto` was registered as an active,
   pickable model unconditionally, but the gateway 403s the alias until its
   own Phase 0a rollout ships. Gated registration behind a new
   `Flag.ALTIMATE_AUTO_MODEL` (env `ALTIMATE_AUTO_MODEL`, default off, read
   once at `flag.ts` import time like every other Flag.* gate). When off,
   `provider.ts`'s `baseModels` does not include the `altimate-auto` entry at
   all — the routing-hint metadata plumbing in session/llm.ts is unaffected
   either way. Flip the default only after the gateway's Phase 0a
   allowlist/rewrite deploys. Tests: `provider.test.ts` covers both states —
   the flag itself (subprocess, matching `external-skills-flag.test.ts`'s
   established pattern for import-time env flags) and the actual
   registration behavior in-process (off by default; on via a direct
   Flag-object mutation, restored after).

2. MAJOR — message_id join with the generation telemetry event.
   session/llm.ts previously always used `input.user.id` for the hint's
   `message_id`, but processor.ts's `generation` telemetry event
   (Telemetry.track({ type: "generation", message_id: input.assistantMessage.id
   })) uses the assistant message's id — a different id for every call that
   goes through a processor turn. Added `StreamInput.messageId`, set by
   processor.ts to `assistantMessage.id` right before calling LLM.stream
   (covers main, subagent, summary, AND compaction uniformly, since
   compaction also goes through SessionProcessor.create()/process() and
   creates its own summary assistant message — note this differs slightly
   from the review's parenthetical list, which named compaction among "calls
   with no assistant message"; compaction does have one, and joining it to
   the same id as its own generation event is the technically correct
   behavior). Call sites with no processor turn at all set `messageId`
   explicitly to the message they're about when one exists (title: the
   first real user message) or leave it unset — `message_id` is then omitted
   from the hint entirely (skill-selector, enhance-prompt, ai-review,
   project-copy all construct synthetic per-call messages with no real
   "message this is about"). Tests: llm.test.ts's omission/explicit-id cases,
   plus a new processor-effect.test.ts end-to-end test that drives a real
   main-turn against the real "altimate-backend" provider and asserts the
   captured request's `metadata.altimate.message_id` equals the assistant
   message's own id (the same id `generation` telemetry reads).

3. MAJOR — marker gate. Two real gaps: server/routes/.../project-copy.ts:52
   and session/prompt.ts:1446 had single-line `// altimate_change — ...`
   comments instead of a `start`/`end` wrap. Fixed both. Also fixed a
   pre-existing imbalance in provider/transform.ts from f503676 itself: an
   Edit there had unintentionally split one existing marked block into two,
   leaving a duplicate `// altimate_change end` orphaned after
   `sanitizeSurrogates()` — restructured so the original block (const +
   sanitizeSurrogates) closes exactly where it did before, and the new
   routing-hint block (AltimateTaskKind, isAltimateManagedModel,
   isAltimateManagedProviderID) is its own separate, correctly closed block
   after it. `bun run script/upstream/analyze.ts --markers --base origin/main
   --strict` now exits 0.

4. MINOR — hardening + tests.
   - `isAltimateManagedModel` is now an exact Set membership check
     ("altimate-base" / "altimate-auto") instead of substring `.includes()`.
   - The hint's `tools` count is clamped to 512 (the gateway's cap) instead
     of sent uncapped.
   - The hint's `agent` field is validated against the gateway's own
     ^[a-z][a-z0-9_-]{0,31}$ allowlist and dropped (not sent) on a mismatch,
     rather than relying on the gateway to silently drop an invalid key.
   - New tests in llm.test.ts: a `small: true` + real "altimate-free"
     provider path (mirrors the actual title/enhance-prompt/project-copy
     shape — small-model calls use ProviderTransform.smallOptions(), not
     .options(), which is exactly why the hint is injected in stream()
     itself rather than inside options()), and a tools-clamp +
     invalid-agent-name-dropped case.
   - New tests in skill-filtering.test.ts covering skill-selector's
     session-model reuse: an Altimate-managed session model skips
     Provider.defaultModel() entirely; a non-Altimate session model (or no
     session model at all) still falls back to it.

Verified: marker guard clean, `tsgo --noEmit` clean, and `bun test` green
for every touched file. One pre-existing, order-dependent flake was
observed once in processor-effect.test.ts (`it.live`-based Effect fiber
cleanup racing across combined test files — the same "Unhandled error
between tests: All fibers interrupted without error" pattern reproduces on
the unmodified "capture llm input cleanly" test when run filtered/isolated)
and did not reproduce on three immediate retries of the identical combined
file set; not caused by this change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
             1 session behind this PR             

claude-sonnet-5.........................≥ $35.0021
  session slice: turns 1–348 of 355
--------------------------------------------------
TOTAL priced............................≥ $35.0021
  standard API-equivalent floor; not an invoice
  counted: 1 session
  cache served >99% of input tokens

1 candidate session not attributed
(in repo + branch window, no branch commit)

1 session made git writes that could not be anchored
(see docs/trust.md)
  full receipts + session ids: section below
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
full receipts (1 session)
session id scope turns time tokens in / out cached
builder fd8fad7c turns 1–348 of 355 348 1h 05m 696 / 185k >99%

builder · fd8fad7c

- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
   Claude Code · Sep 22 2026 21:16 UTC · 1h 05m   
               claude-sonnet-5 100%               
        cache served >99% of input tokens         

pre-edit: 10% of priced floor (62/348 turns)
  (share before the first named edit tool)

Bash.......................≥ $24.8797  (253 calls)
Edit.........................≥ $5.2350  (48 calls)
Read.........................≥ $4.1190  (46 calls)
SendMessage...................≥ $0.2662  (2 calls)
(thinking/reply)..............≥ $0.2182  (2 turns)
Write..........................≥ $0.1835  (1 call)
ToolSearch.....................≥ $0.1002  (1 call)
--------------------------------------------------
TOTAL...................................≥ $35.0018
standard API-equivalent floor; not an invoice
same tokens on claude-haiku-4-5.........≥ $11.6673
  (67% lower observable floor)
  (arithmetic, not a prediction)
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -

Generated by aireceipts

@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant