Repository navigation
Describe video frames with OpenAI reasoning models - #456
Merged
Merged
Conversation
The frame description call always sent max_tokens = 300, which gpt-5 and later OpenAI models reject. It now sets the output limit the way image OCR does: max_completion_tokens for OpenAI, max_tokens elsewhere, allowing reasoning_effort through LiteLLM. The shared logic moves to core/parser/model_params.py. The budget is 4096 so a reasoning model has room to think; a description that runs out of output fails instead of indexing blank text, and rate limits and outages are retried. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Video frame descriptions failed on OpenAI reasoning models.
VisionModelClient.get_frame_descriptionalways sentmax_tokens = 300, and gpt-5 and later reject it:Image OCR already handled this (#454). This moves that logic into
core/parser/model_params.pyand uses it for both.Changes
add_output_limitsetsmax_completion_tokensfor OpenAI models andmax_tokensfor other providers, keeps a limit set on the registered model, and allowsreasoning_effortthrough LiteLLM. Image OCR calls it instead of its inline copy, so OCR behavior is unchanged.ValueErrorthat suggestsreasoning_effort = "low", instead of indexing a blank frame.ConnectionError, so the ingestion worker retries, as for OCR.reasoning_effort = "low".Testing
test_video_transcription.pyadds 4 tests: OpenAI reasoning model parameters, other providers and registered limits, running out of output, and transient errors. Withtest_video_parserandtest_image_ocr_model: 37 passed.get_frame_descriptionon a generated frame with a red "Error 4012: Invoice total does not match" box.openai/gpt-6.1-solreturned the error above before this change. After it,openai/gpt-6.1-sol(with and withoutreasoning_effort = "low") andgpt-4.1-miniall described the box and quoted its text.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.