Align OpenAI fallback tools with prompt family - #333938
Draft
Dileep Yavanmandha (dileepyavan) wants to merge 3 commits into
Draft
Align OpenAI fallback tools with prompt family#333938Dileep Yavanmandha (dileepyavan) wants to merge 3 commits into
Dileep Yavanmandha (dileepyavan) wants to merge 3 commits into
Conversation
Track the model family selected by provider prompt fallbacks and use it when choosing agent edit tools. Limit versioned GPT fallback routing to families newer than GPT-5.6 while retaining opaque OpenAI provider models. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot started reviewing on behalf of
Dileep Yavanmandha (dileepyavan)
September 2, 2026 06:12
View session
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Opaque fallback models still receive non-GPT-5 system-prompt patch instructions, and promotion metadata can drift.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
 |
extensions/copilot/src/extension/intents/node/agentIntent.ts — The resolved family is only threaded through tool selection. AgentPrompt.getSystemPrompt still… |
|
extensions/copilot/src/extension/intents/node/agentIntent.ts — The new fallback branch changes the actual getAgentTools result and bypasses all BYOK… |
 |
extensions/copilot/src/extension/prompts/node/agent/openai/latestOpenAIPrompt.ts — This introduces two additional promotion points beyond the resolver import: the 5/6 comparison… |
What changed in this PR
Aligns OpenAI fallback prompts with their expected edit-tool capabilities.
Changes:
- Records and resolves fallback model families.
- Restricts fallback routing to newer or opaque OpenAI models.
- Uses fallback families for edit-tool selection and adds coverage.
| File | Description |
|---|---|
model-prompts.instructions.md |
Documents fallback routing. |
agentIntent.ts |
Applies fallback tool capabilities. |
openAIPrompts.spec.ts |
Tests prompt routing. |
latestOpenAIPrompt.ts |
Filters and registers the fallback. |
promptRegistry.ts |
Tracks fallback families. |
chatModelCapabilities.ts |
Accepts fallback capability families. |
chatModelCapabilities.spec.ts |
Tests fallback capabilities. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const logService = accessor.get<ILogService>(ILogService); | ||
|
|
||
| model ??= await endpointProvider.getChatEndpoint(request); | ||
| const fallbackModelFamily = await PromptRegistry.resolveFallbackModelFamily(model); |
| const allowTools: Record<string, boolean> = {}; | ||
|
|
||
| const learned = editToolLearningService.getPreferredEndpointEditTool(model); | ||
| const learned = fallbackModelFamily ? undefined : editToolLearningService.getPreferredEndpointEditTool(model); |
|
|
||
| // Promote the entire prompt bundle by changing the resolver imported above. | ||
| PromptRegistry.registerFallbackPrompt(LatestOpenAIPromptResolver, isOpenAIModel); | ||
| PromptRegistry.registerFallbackPrompt(LatestOpenAIPromptResolver, usesLatestOpenAIPrompt, 'gpt-5.6'); |
Carry the fallback model family in resolved prompt customizations and use it when rendering model-specific system instructions without changing the endpoint family. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover the final getAgentTools result for an extension-contributed opaque OpenAI endpoint whose declared edit-tool preference conflicts with the GPT-5.6 prompt fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
apply_patchand its GPT-5-style tool descriptionAgentPromptsystem-prompt rendering without mutating the endpointgpt-*families to versions newer than GPT-5.6, while preserving opaque models identified through OpenAI provider metadataBehavior
Known GPT versions through GPT-5.6 continue using their explicit prompt resolvers. Unknown versioned families such as GPT-5.7 and GPT-6 inherit the GPT-5.6 prompt and tool capabilities, while older unknown families such as GPT-4.2 do not.
Opaque OpenAI fallback endpoints now receive a consistent GPT-5.6 experience across tool selection, tool descriptions, and rendered system instructions. The endpoint's original model and family remain unchanged.
Testing
git diff --checkgetAgentToolsintegration test using an extension-contributed opaque OpenAI endpoint with a conflicting declaredfind-replacepreference; verifies onlyapply_patchis enabled with the GPT-5 description