feat(llm): add OrcaRouter as a named gateway provider - #3584
feat(llm): add OrcaRouter as a named gateway provider#3584jinhaosong-source wants to merge 3 commits into
Conversation
OrcaRouter is an OpenAI-compatible LLM gateway. Register it alongside the existing openrouter / litellm-proxy / custom-gateway entries so llm="orcarouter/openai/gpt-5.5" works without routing users through a hand-configured custom gateway. - llm/gateways.py: OrcaRouterProvider (ORCAROUTER_API_KEY, base URL https://api.orcarouter.ai/v1), registered as "orcarouter" with an "orca" alias; llm/__init__.py exports it lazily - inc/models.py: a branch in the PraisonAIModel ladder plus "orcarouter" in _BUILTIN_MODEL_PREFIXES, and the mirrored copy of that set in auto.py - praisonai_code llm/env.py, llm/credentials.py, llm/catalogue.py: provider map, credential lookup, key-creation URL, and catalogue entries so the catalogue-driven picker in `praisonai setup` can offer it - praisonai_code cli/commands/auth.py, cli/main.py: auth listing and help text - README (both) and examples/python/providers/orcarouter/ Unlike OpenRouterProvider this does not rewrite the model id. OrcaRouter model ids are themselves namespaced (openai/gpt-5.5, orcarouter/auto) and the gateway rejects bare names, so the provider prefixes "openai/" unconditionally and LiteLLM strips only that segment, leaving the id intact. Using custom_llm_provider instead makes LiteLLM consume the id's own namespace and the gateway returns 503 No available channel.
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
MervinPraison
added
pipeline/blocked:ci
📝 WalkthroughWalkthroughOrcaRouter is added as an OpenAI-compatible provider. The change includes gateway registration, model and credential resolution, catalogue metadata, CLI support, tests, documentation, and a usage example. ChangesOrcaRouter support
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Application
participant PraisonAIModel
participant OrcaRouterProvider
participant LiteLLM
Application->>PraisonAIModel: select orcarouter model
PraisonAIModel->>OrcaRouterProvider: resolve model and credentials
OrcaRouterProvider->>LiteLLM: send completion request
LiteLLM-->>OrcaRouterProvider: return response
OrcaRouterProvider-->>Application: return generated output
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThe PR adds OrcaRouter as a named OpenAI-compatible gateway and integrates it across provider registration, model resolution, credentials, onboarding, documentation, and tests.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; the previously reported adaptive-router namespace issue is addressed across catalogue storage and the relevant execution normalizers.
|
| Filename | Overview |
|---|---|
| src/praisonai/praisonai/llm/gateways.py | Adds the OrcaRouter gateway provider, alias registration, endpoint defaults, and LiteLLM-compatible model rewriting. |
| src/praisonai/praisonai/auto.py | Normalizes OrcaRouter model IDs separately for LiteLLM and direct OpenAI SDK structured-completion paths. |
| src/praisonai/praisonai/inc/models.py | Adds OrcaRouter to built-in model handling while stripping only its outer routing prefix. |
| src/praisonai-code/praisonai_code/llm/catalogue.py | Adds selectable OrcaRouter fallback models, including the fully qualified adaptive-router ID, and its credential URL. |
| src/praisonai-code/praisonai_code/llm/env.py | Adds OrcaRouter endpoint, credential, provider detection, and default-model resolution. |
| src/praisonai-code/praisonai_code/llm/credentials.py | Integrates OrcaRouter with credential injection, model-specific key lookup, and configuration detection. |
| src/praisonai/tests/unit/llm/test_registered_provider_hot_paths.py | Verifies the per-runtime model normalization that preserves OrcaRouter namespaces. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
User["orcarouter/vendor/model"] --> Resolve["Provider and credential resolution"]
Resolve --> Catalogue["Stored model remains fully qualified"]
Catalogue --> Runtime{"Execution path"}
Runtime --> LiteLLM["LiteLLM: openai/vendor/model"]
Runtime --> OpenAI["OpenAI SDK: vendor/model"]
LiteLLM --> Gateway["OrcaRouter API"]
OpenAI --> Gateway
Reviews (3): Last reviewed commit: "fix(llm): route orcarouter through struc..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
@claude You are the FINAL architecture reviewer. If the branch is under MervinPraison/PraisonAI (not a fork), you are able to make modifications to this branch and push directly. SCOPE: Focus ONLY on Python packages (praisonaiagents, praisonai). Do NOT modify praisonai-rust or praisonai-ts. Read ALL comments above from Gemini, Qodo, CodeRabbit, and Copilot carefully before responding. Phase 1: Review per AGENTS.md
Phase 2: FIX Valid Issues Phase 3: Final Verdict |
MervinPraison
added
pipeline/blocked:cooldown
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Line 172: Update the provider-count summary from 24 to 25 in both README.md
(lines 172-172) and src/praisonai/README.md (lines 172-172), leaving the
provider tables unchanged.
In `@src/praisonai/praisonai/auto.py`:
- Line 472: Remove "orcarouter" from the built-in structured-completion prefix
collection near the completion configuration, leaving the other provider
prefixes unchanged so orcarouter models follow the normal Litellm model
normalization path.
In `@src/praisonai/README.md`:
- Line 172: Update the OrcaRouter link in the provider examples table of the
README to use the repository-relative path
../../examples/python/providers/orcarouter/orcarouter_example.py, leaving the
link text unchanged.
In `@src/praisonai/tests/unit/llm/test_env_resolver.py`:
- Line 283: In the loop over _PROVIDER_MAP, rename the unused key_var binding to
_key_var while preserving the existing prefix and base_url behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 6a489d4f-a225-47fa-987f-3a4a7664e9a6
📒 Files selected for processing (14)
README.mdexamples/python/providers/orcarouter/orcarouter_example.pysrc/praisonai-code/praisonai_code/cli/commands/auth.pysrc/praisonai-code/praisonai_code/cli/main.pysrc/praisonai-code/praisonai_code/llm/catalogue.pysrc/praisonai-code/praisonai_code/llm/credentials.pysrc/praisonai-code/praisonai_code/llm/env.pysrc/praisonai/README.mdsrc/praisonai/praisonai/auto.pysrc/praisonai/praisonai/inc/models.pysrc/praisonai/praisonai/llm/__init__.pysrc/praisonai/praisonai/llm/gateways.pysrc/praisonai/tests/unit/llm/test_env_resolver.pysrc/praisonai/tests/unit/llm/test_gateway_providers.py
| | Fireworks | [Example](examples/python/providers/fireworks/fireworks_example.py) | | ||
| | Together AI | [Example](examples/python/providers/together/together_ai_example.py) | | ||
| | OpenRouter | [Example](examples/python/providers/openrouter/openrouter_example.py) | | ||
| | OrcaRouter | [Example](examples/python/providers/orcarouter/orcarouter_example.py) | |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== repo files of interest =="
git ls-files | rg '(^|/)README\.md$|src/praisonai/README.md|examples/python/providers/orcarouter/orcarouter_example.py|mkdocs|docs|README' | sed -n '1,200p'
echo
echo "== target file exists =="
test -f examples/python/providers/orcarouter/orcarouter_example.py && echo "exists: examples/python/providers/orcarouter/orcarouter_example.py" || echo "missing target"
echo
echo "== link line in src/praisonai/README.md =="
sed -n '165,178p' src/praisonai/README.md
echo
echo "== doc build/config references =="
for f in $(git ls-files | rg '(^|/)(mkdocs\.ya?ml|docs|README\.md)$'); do
case "$f" in
*README.md)
if rg -n '\[\|.*OrcaRouter|OrcaRouter|\[\*\].*OrcaRouter|src/praisonai/README|README.md' "$f" >/dev/null; then
echo "--- $f"
rg -n '\[\|.*OrcaRouter|OrcaRouter|src/praisonai/README|README|link' "$f" | sed -n '1,80p'
fi
;;
mkdocs.yml|mkdocs.yaml)
echo "--- $f"
cat "$f"
;;
esac
done
echo
echo "== relative link resolution check =="
python3 - <<'PY'
from pathlib import Path
readme = Path("src/praisonai/README.md")
link = Path("examples/python/providers/orcarouter/orcarouter_example.py")
absolute = (readme.parent / link).resolve()
print("readme path:", readme)
print("current link target:", link)
print("readme-relative resolution:", absolute)
print("target exists:", Path(link).exists())
print("current link resolves to existing file:", absolute.exists())
print("recommended sibling-relative link exists:", (Path("src/praisonai") / "../../examples/python/providers/orcarouter/orcarouter_example.py").resolve().exists())
PYRepository: MervinPraison/PraisonAI
Length of output: 7708
Fix the OrcaRouter example link.
When src/praisonai/README.md is rendered from its repository path, this link resolves at src/praisonai/examples/..., but the file is at examples/python/providers/orcarouter/orcarouter_example.py. Use ../../examples/python/providers/orcarouter/orcarouter_example.py.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/praisonai/README.md` at line 172, Update the OrcaRouter link in the
provider examples table of the README to use the repository-relative path
../../examples/python/providers/orcarouter/orcarouter_example.py, leaving the
link text unchanged.
|
Claude finished @MervinPraison's task —— View job Final Architecture Review — PR #3584 (OrcaRouter provider) ✅Pushed Architecture (AGENTS.md): Additive gateway wiring — no new Agent params, no new deps, protocol-driven (provider registered via the existing registry seam), backward-compatible, no hot-path regression. The provider adds real SDK value (a working named gateway + fast path). No scope creep. ✅ Issues verified & fixed
#1 — catalogue (broader than reported). #2 — structured completion. Confirmed live that #5 — skipped (with reason). All 24 existing rows in Files modified
Tests
VerdictAll blocking issues resolved. Approve — the two functional bugs (setup default + structured completion) are fixed with regression coverage, and the change is a clean, additive, backward-compatible gateway integration. (Note: I can't submit a formal GitHub PR approval for security reasons — this comment is the review.) Job run · branch |
The providers table now has 25 rows after adding OrcaRouter.
MervinPraison
added
the
pipeline/blocked:stale-final
|
Thanks for the review — went through all four. 1. Provider count 24 → 25 — fixed (d8d6bfc). Confirmed by counting the table: it really is 25 rows now. Updated in both 2. Remove That literal is the
Dropping It also can't be dropped from the primary set in One real limitation your comment does point at, which I'd rather state than paper over: 3. Use 4. Rename |
… catalogue defaults Fixes two OrcaRouter gaps found in review: - catalogue: the setup wizard stores a catalogue id verbatim and only an outer "orcarouter/" prefix routes a model to the gateway (resolution strips it, preserving the vendor namespace the gateway requires). The four OrcaRouter catalogue ids lacked it, so the picker default "auto" reached the gateway bare (503) and "openai/gpt-5.5" would even route to OpenAI. Give all four the full "orcarouter/<vendor>/<model>" shape. - auto.py structured completion: orcarouter is a built-in prefix (so it never delegates to the registry) but LiteLLM has no native "orcarouter/" route and the gateway provider has no generate_structured, so both ladder legs sent the raw id and failed. Normalize per leg (LiteLLM -> openai/<gateway-id>, OpenAI SDK -> <gateway-id>) mirroring OrcaRouterProvider/PraisonAIModel; other providers pass through unchanged. Also: README provider count 24->25, and B007 unused loop var in test_env_resolver. Adds regression tests for _normalize_gateway_model and the LiteLLM-leg id. Co-authored-by: Mervin Praison <MervinPraison@users.noreply.github.com>
MervinPraison
removed
the
pipeline/blocked:stale-final
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/praisonai/praisonai/auto.py`:
- Around line 527-531: Update _normalize_gateway_model to recognize the
registered “orca/” alias as well as “orcarouter/”, rewriting either prefix to
the same openai gateway model and gateway ID while preserving unrelated model
names. Add coverage verifying that an alias such as “orca/openai/gpt-5.5” is
normalized before structured completion/config_list construction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 2f45b9d4-eead-44bd-ac8b-1d2fd05de760
📒 Files selected for processing (4)
src/praisonai-code/praisonai_code/llm/catalogue.pysrc/praisonai/praisonai/auto.pysrc/praisonai/tests/unit/llm/test_env_resolver.pysrc/praisonai/tests/unit/llm/test_registered_provider_hot_paths.py
🚧 Files skipped from review as they are similar to previous changes (2)
- src/praisonai-code/praisonai_code/llm/catalogue.py
- src/praisonai/tests/unit/llm/test_env_resolver.py
| prefix = "orcarouter/" | ||
| if not model_name.lower().startswith(prefix): | ||
| return model_name, model_name | ||
| gateway_id = model_name[len(prefix):] | ||
| return f"openai/{gateway_id}", gateway_id |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect provider alias registration and built-in-prefix handling.
rg -n -C 5 --glob '*.py' \
'OrcaRouterProvider|register_llm_provider|_BUILTIN_MODEL_PREFIXES|orcarouter|["'\'']orca["'\'']' \
src/praisonai
# Inspect all structured-completion normalization call sites and tests.
rg -n -C 5 --glob '*.py' \
'_normalize_gateway_model|_structured_via_registered_provider|orca/' \
src/praisonaiRepository: MervinPraison/PraisonAI
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== relevant auto.py =="
sed -n '440,585p' src/praisonai/praisonai/auto.py
echo "== relevant registry resolution =="
sed -n '240,450p' src/praisonai/praisonai/llm/registry.py
echo "== relevant models.py =="
sed -n '86,170p' src/praisonai/praisonai/inc/models.py
echo "== model string docs/usages =="
rg -n -C 3 --glob '*.py' 'parse_model_string|model_path|model\s*=|model:"|model=' src/praisonai/tests src/praisonai/praisonai | head -n 200
echo "== tests for orca/ prefix =="
rg -n 'orca/' src/praisonai/tests src/praisonai/praisonai || trueRepository: MervinPraison/PraisonAI
Length of output: 34731
Normalize the orca/ model-ID alias.
orcarouter is registered with the orca alias, but _normalize_gateway_model only rewrites orcarouter/..., leaving orca/openai/gpt-5.5 unchanged for structured completion. Treat orca/... as a built-in and rewrite it the same way, or canonicalize aliases before building config_list; add coverage for the alias form.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/praisonai/praisonai/auto.py` around lines 527 - 531, Update
_normalize_gateway_model to recognize the registered “orca/” alias as well as
“orcarouter/”, rewriting either prefix to the same openai gateway model and
gateway ID while preserving unrelated model names. Add coverage verifying that
an alias such as “orca/openai/gpt-5.5” is normalized before structured
completion/config_list construction.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3584 +/- ##
==========================================
- Coverage 32.73% 32.35% -0.39%
==========================================
Files 542 531 -11
Lines 57463 56207 -1256
==========================================
- Hits 18813 18187 -626
+ Misses 38650 38020 -630
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2 participants
Summary
Adds OrcaRouter as a named gateway provider, alongside the existing
openrouter/litellm-proxy/custom-gatewayentries inpraisonai/llm/gateways.py. OrcaRouter is an OpenAI-compatible LLM gateway that fronts ~190 models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and others behind one key and one base URL.Users can reach it today via
custom-gatewayplus a hand-typed base URL, but that path gives no credential discovery, no key-creation hint inpraisonai setup/praisonai auth login, and no entry in the built-in model ladder. This wires it up the same way OpenRouter already is, sollm="orcarouter/openai/gpt-5.5"just works.It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.
I'm an engineer on the OrcaRouter team.
Changes
src/praisonai(gateway provider + built-in ladder)praisonai/llm/gateways.py—OrcaRouterProvider(base URLhttps://api.orcarouter.ai/v1,ORCAROUTER_API_KEY), registered asorcarouterwith anorcaalias.praisonai/llm/__init__.py— lazy export +__all__.praisonai/inc/models.py— a branch in thePraisonAIModelladder andorcarouterin_BUILTIN_MODEL_PREFIXES, so every framework adapter that goes throughPraisonAIModelresolves it on the fast path.praisonai/auto.py— the mirrored fallback copy of that prefix set.src/praisonai-code(CLI resolution + onboarding)llm/env.py—orcarouter/in_PROVIDER_MAP,ORCAROUTER_API_KEYin the key-var → stored-provider map and_ALL_FALLBACK_PROVIDERS, and a_PROVIDER_DEFAULTSrow so a bareORCAROUTER_API_KEYpicks an OrcaRouter default model.llm/credentials.py— env mapping, model → key-var, key-var → stored provider, and theis_configuredvar list.llm/catalogue.py—PROVIDER_KEY_URLSentry, plus fourFALLBACK_MODELSrows. The rows are the reason the provider is selectable: the onboarding picker is catalogue-driven vialist_providers(), and unlike OpenRouter, litellm ships no catalogue for this gateway, so without thempraisonai setupcould never offer it.cli/commands/auth.py—_PROVIDER_ENV_KEYSentry soauth list/auth statusshow it.cli/main.py— the provider list in the "no LLM provider configured" help.Docs / examples
README.md+src/praisonai/README.md— provider badge and a row in the providers table.examples/python/providers/orcarouter/orcarouter_example.py— mirrors the OpenRouter example.One thing worth reviewing closely
OrcaRouterProviderdoes not rewrite the model id the wayOpenRouterProviderdoes. OrcaRouter's own model ids are namespaced (openai/gpt-5.5,anthropic/claude-sonnet-5,orcarouter/auto) and the gateway rejects bare names, so the id has to survive intact. LiteLLM strips only the leadingopenai/and forwards the rest, so the provider prefixesopenai/unconditionally — including for ids that already start withopenai/:llm=orcarouter/openai/gpt-5.5openai/openai/gpt-5.5openai/gpt-5.5orcarouter/anthropic/claude-sonnet-5openai/anthropic/claude-sonnet-5anthropic/claude-sonnet-5orcarouter/orcarouter/autoopenai/orcarouter/autoorcarouter/autoI tried the tidier-looking
custom_llm_provider="openai"first and it is wrong: withopenai/gpt-5.5LiteLLM consumes the id's own namespace and the gateway returns503 No available channel for model gpt. Verified live against the real API; there's a test pinning this.Also note
orcarouter/auto(the adaptive router) picks an upstream per request, so its structured-output behaviour varies. The example file and docstring point at pinned models for anything schema-dependent.Testing
New tests — added to the existing files rather than new ones:
tests/unit/llm/test_gateway_providers.py— 5 OrcaRouter tests (init, env var, the namespace-preservation table above, base-URL override, and alitellm.completioncall assertion), plus OrcaRouter added to the registration andcreate_llm_providertests.tests/unit/llm/test_env_resolver.py—orcarouter/added to the exhaustive_PROVIDER_MAPURL assertion (which would otherwiseKeyErroron the new entry), plus 2 resolution tests.Results — run with the same env and marker exclusions as
test-core.yml:tests/unit/ --collect-only, CI's own gate)subdirsshard subset —tests/unit/llm/ tests/unit/code/ tests/unit/integrations/tests/unit/llm/alonetest_model_routing,test_auto_generator,test_auto_lazy_loading,test_enhanced_auto,test_framework_adapter_simple,test_hybrid_workflow,test_agents_schema_publish)I could not get the full root shard (
tests/unit/*.py) to finish locally — it stalls partway ontest_logging_regression.pyon this Windows box. That file passes in isolation on both this branch and cleanmain(7 passed each), so it's a cross-test interaction in my environment rather than something this change introduced. CI will cover the rest.Note for reviewers: the
_PROVIDER_MAPassertion intest_env_resolver.pyenumerates every entry against anexpected_urlsdict, so adding a provider without updating it raisesKeyError. That's why the test file is touched.Live verification against the real gateway — 17/17, driven through PraisonAI's own entry points (
create_llm_provider,PraisonAIModel,praisonai_code.llm.env.resolve_llm_endpoint) rather than a hand-built client:orcarouterand theorcaalias and lists it besideopenrouteropenai/gpt-5.5,anthropic/claude-sonnet-5,google/gemini-3.5-flash, andorcarouter/autoPraisonAIModelresolvesORCAROUTER_API_KEY+ base URL + stripped model name, its OpenAI client completes, andorcarouter/autokeeps its namespace through the ladderresolve_llm_endpointroutes anorcarouter/model to the gateway with the right keykey_url_for_provider("orcarouter")returns the console URL andModelCatalogue().list_providers()includesorcarouter(the onboarding picker)AuthenticationErrorHow a new user gets started
sk-orca-).export ORCAROUTER_API_KEY=sk-orca-...— or runpraisonai setupand pick orcarouter, which prints the key page and stores the credential.Agent(instructions="...", llm="orcarouter/openai/gpt-5.5"), orpraisonai --model orcarouter/openai/gpt-5.5.orcarouter/orcarouter/autouses the adaptive router.Summary by CodeRabbit
New Features
Documentation