fix(pulse): map adapter models to inference levels that exist - #1648
Open
elhoim wants to merge 1 commit into
Open
fix(pulse): map adapter models to inference levels that exist#1648elhoim wants to merge 1 commit into
elhoim wants to merge 1 commit into
Conversation
AdapterRunner.modelToLevel returned "fast" / "standard" / "smart" — names Inference.ts removed on 2026-06-10. normalizeLevel() rejects all three, so every level the mapping could produce was refused and 100% of adapter builds that reached inference threw: AdapterCli died on its top-level await, and RebuildAll's Promise.allSettled printed an unlabelled "unhandled" row. - Derive the mapping from models.ts EFFORT_MODEL instead of restating level names (new adapters/model-level.ts): haiku->low, sonnet->medium, opus->high, fable->max. A tie resolves to the cheaper rung and a non-Claude model falls back to medium, so a manifest never silently buys a costlier rung than the model it names. Previously fable fell through to the sonnet rung. - Bind InferenceLevel to models.ts EffortLevel and derive VALID_LEVELS from EFFORT_MODEL's keys, so the type, the validator, and the CLI all follow one source of truth rather than three hand-kept copies. - Catch a rejecting inference() in callInference: a bad request now yields a structured inference-failed result, written to the data plane like any other failure, instead of escaping runAdapter. - Report a rejected page by id and reason in RebuildAll and AdapterCli. - Add test/pulse/adapters/model-level.test.ts (8 tests, no API calls) covering every model family plus the mapping/inference drift guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
AdapterRunner.ts'smodelToLevel()could only return"fast","smart"or"standard". Those names were removed from the inference layer on 2026-06-10.Inference.tsnow validates againstlow | medium | high | maxand throws on anything else:So every level the adapter could produce was rejected. Nothing caught it:
callInferenceandrunAdapterhave no catch,AdapterCli.tstop-level-awaitsrunAdapterso the process dies, andRebuildAll.tsusesPromise.allSettled, printing each rejection as an unlabelled "unhandled" row. Only the cache-hit, no-sources and missing-prompt early returns avoid inference, so 100% of actual adapter builds failed.Why it compiled
This is the part worth fixing properly.
Inference.tsdeclared its own copy of the level names:models.tsalready hadEffortLevelas the real source of truth. Two hand-maintained lists of the same names means a rename in one leaves the other stale and still type-checking — which is exactly what happened. The type mismatch didn't compile-error because the adapter was typed against a copy that agreed with itself.The fix
Bind both ends to one source.
The type now IS
models.ts's level type, and the runtime validator derives fromEFFORT_MODEL's keys. Rename a level inmodels.tsand every stale call site fails at the type level rather than at runtime. The error message and the CLI's own validation also read fromVALID_LEVELSinstead of repeating the literals a third and fourth time.Mapping, extracted to
adapters/model-level.tsand typedInferenceLevel:lowmediumhighmaxmediumCheap models map to cheap tiers, and an unrecognised model gets a sane middle default rather than the most expensive one.
Failures stop being unlabelled.
callInferencecatches a rejection and returns the same structured failure shape as every other error path, so one bad page reports as a named page with a real message and the batch keeps going.Tests
test/pulse/adapters/model-level.test.ts, 8 cases:Separately verified that every level the mapping can emit is one the inference layer actually accepts, checked against
EFFORT_MODEL's real keys rather than a hardcoded list:No API calls are made by the tests or the check.