feat: add OrcaRouter as a first-class language model provider - #431
feat: add OrcaRouter as a first-class language model provider#431kuswardhanietidims-svg wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds OrcaRouter as a first-class, OpenAI-compatible language model provider across the TypeScript and Python TypeChat SDKs, including environment-variable routing, tests, and documentation updates.
Changes:
- Add OrcaRouter language model factories (
createOrcaRouterLanguageModel/create_orca_router_language_model) and route viacreateLanguageModel/create_language_modelwhenORCAROUTER_API_KEYis set. - Add TypeScript and Python tests covering OrcaRouter factory behavior and env-var routing (including endpoint override).
- Update docs and
.env.exampleto document OrcaRouter configuration alongside OpenAI and Azure OpenAI.
Show a summary per file
| File | Description |
|---|---|
| typescript/tests/model.test.mjs | Adds routing + factory tests for OrcaRouter in the TS test suite. |
| typescript/src/model.ts | Implements OrcaRouter factory and extends env-var routing to include OrcaRouter. |
| site/src/docs/typescript/basic-usage.md | Documents OrcaRouter usage in TypeScript basics page. |
| site/src/docs/python/basic-usage.md | Documents OrcaRouter usage in Python basics page. |
| site/src/docs/examples.md | Updates examples documentation to include OrcaRouter environment variables. |
| python/tests/test_model.py | Adds tests for the OrcaRouter factory behavior in Python. |
| python/src/typechat/_internal/model.py | Implements OrcaRouter factory and extends env-var routing in Python. |
| python/src/typechat/init.py | Exports the new OrcaRouter factory from the public Python package API. |
| .env.example | Adds OrcaRouter env-var templates (including optional endpoint override). |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 9/9 changed files
- Comments generated: 5
- Review effort level: Lite
| export function createOpenAILanguageModel(apiKey: string, model: string, endPoint? string): TypeChatLanguageModel; | ||
|
|
||
| export function createAzureOpenAILanguageModel(apiKey: string, endPoint: string): TypeChatLanguageModel; | ||
|
|
||
| export function createOrcaRouterLanguageModel(apiKey: string, model: string, endPoint? string): TypeChatLanguageModel; |
There was a problem hiding this comment.
Fixed — the signature now reads endPoint?: string in the trimmed docs-only PR.
| For even more convenience, TypeChat also provides a function to infer which | ||
| provider you're using. | ||
|
|
||
| ```ts |
There was a problem hiding this comment.
Fixed — the fence for create_language_model is now py instead of ts.
| | Variable | Value | | ||
| |----------|-------| | ||
| | `ORCAROUTER_MODEL` | The OrcaRouter model name (e.g. `orcarouter/auto` or `orcarouter/free`) | | ||
| | `ORCAROUTER_API_KEY` | Your OrcaRouter API key | |
There was a problem hiding this comment.
Superseded — per maintainer feedback the OrcaRouter-specific ORCAROUTER_ENDPOINT variable was removed; the docs now use the existing OPENAI_ENDPOINT variable for OpenAI-compatible gateways.
| # SPDX-License-Identifier: MIT | ||
|
|
||
| from typechat._internal.model import PromptSection, TypeChatLanguageModel, create_language_model, create_openai_language_model, create_azure_openai_language_model | ||
| from typechat._internal.model import PromptSection, TypeChatLanguageModel, create_language_model, create_openai_language_model, create_azure_openai_language_model, create_orca_router_language_model |
There was a problem hiding this comment.
No longer applicable — the OrcaRouter export was removed from python/src/typechat/__init__.py; the file is back to baseline.
| @@ -187,8 +190,13 @@ def required_var(name: str) -> str: | |||
| api_key=required_var("AZURE_OPENAI_API_KEY") | |||
| endpoint=required_var("AZURE_OPENAI_ENDPOINT") | |||
| return create_azure_openai_language_model(api_key, endpoint) | |||
| elif "ORCAROUTER_API_KEY" in vals: | |||
There was a problem hiding this comment.
No longer applicable — the OrcaRouter routing branch was removed from python/src/typechat/_internal/model.py; the file is back to baseline.
|
Thanks for the contribution and for the clear disclosure. TypeChat already supports OrcaRouter today via the endPoint parameter on createOpenAILanguageModel / create_openai_language_model — your own quickstart's base-URL swap maps directly onto it. Adding named factories per OpenAI-compatible gateway isn't a direction we want to take, since it commits us to maintaining third-party endpoints in shipped packages and sets a precedent we'd have to extend to a dozen other providers. We'd welcome a trimmed PR that adds a vendor-neutral 'using an OpenAI-compatible gateway' docs section instead, with OrcaRouter as one of the named examples." |
robgruen
left a comment
robgruen
left a comment
There was a problem hiding this comment.
See my *vendor neutral" doc comment on the PR summary.
Adds a vendor-neutral "Using OpenAI-compatible gateways" section to the TypeScript and Python basic-usage pages and the examples env-var guide. OpenAI-compatible services (e.g. OrcaRouter, OpenRouter) can be used via the existing endPoint parameter / OPENAI_ENDPOINT variable, so no new provider-specific factories or env-var routing are needed. Co-Authored-By: Claude <noreply@anthropic.com>
a7859b8 to
cdb35be
Compare
|
Thanks for the feedback and the clear direction. I've trimmed the PR to match:
The full diff is now docs-only across three files. The site builds cleanly with |
Motivation
TypeChat makes it easy to build natural language interfaces using types — but to talk to an LLM you still need a model. Today,
createLanguageModelonly knows about the OpenAI and Azure OpenAI providers, which locks users into a single vendor's endpoint.OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents. Like OpenRouter, it exposes a provider/model namespace across many models — but it also combines adaptive routing, automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance behind the same endpoint. Adding
orcarouteras a first-class provider means TypeChat users can use that whole stack directly, without treating OrcaRouter as an anonymous custom base URL.It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.
Changes
typescript/src/model.ts— newcreateOrcaRouterLanguageModelfactory that reuses the existing OpenAI-compatible Chat Completions client (same ascreateOpenAILanguageModel) pointed athttps://api.orcarouter.ai/v1/chat/completions, plusORCAROUTER_API_KEY/ORCAROUTER_MODEL/ORCAROUTER_ENDPOINTrouting increateLanguageModel.python/src/typechat/_internal/model.py— matchingcreate_orca_router_language_modelfactory andORCAROUTER_API_KEYrouting increate_language_model, exported fromtypechat/__init__.py.typescript/tests/model.test.mjs) and Python (python/tests/test_model.py) coverage for the new factory and env routing, mirroring the existing OpenAI provider tests..env.example, the TypeScript and Python basic-usage pages, and the examples env-var table now list OrcaRouter alongside OpenAI and Azure OpenAI.Validation
npm testintypescript/: 118 tests pass (113 baseline + 5 new OrcaRouter tests).pytestinpython/: 21 passed, 3 skipped (pre-existing skips).pyrightinpython/: no new errors introduced (55 pre-existing errors onmain, unchanged).eleventysite build succeeds with the updated docs.createOrcaRouterLanguageModelcode path (and throughcreateLanguageModelwithORCAROUTER_API_KEYset) returned HTTP 200 with a valid completion.Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter
I'm an engineer on the OrcaRouter team.