Skip to content

feat(llm): add OCI Generative AI provider - #5186

Open
Xiaowen-Yang wants to merge 1 commit into
OpenHands:mainfrom
Xiaowen-Yang:feat/17512-oci-generative-ai-provider
Open

Xiaowen-Yang wants to merge 1 commit into
OpenHands:mainfrom
Xiaowen-Yang:feat/17512-oci-generative-ai-provider

Conversation

@Xiaowen-Yang

@Xiaowen-Yang Xiaowen-Yang commented Sep 19, 2026

Copy link
Copy Markdown

HUMAN:
I reviewed the OCI Generative AI provider implementation and the automated test results. I did not run a live OCI request because I do not have the required OCI credentials.


AGENT:

Why

OCI Generative AI provides an OpenAI-compatible Responses API, but OpenHands
currently has no first-class configuration path for its regional endpoint,
Generative AI project OCID, or API-key authentication.

Users must manually construct the endpoint, add the openai-project header,
and configure Responses API capabilities. This is error-prone and makes OCI
deployments difficult to configure reliably.

This PR adds the SDK and Agent Server support needed for a guided OCI Generative
AI profile. The dependent Agent Canvas UI is implemented in
OpenHands/OpenHands#17543, and the user-facing setup documentation is provided
in OpenHands/docs#813.

Summary

  • Add oci_genai/<model-id> as a first-class LLM provider route with validated
    OCI region and Generative AI project OCID fields.
  • Derive the regional /openai/v1 endpoint, select the Responses API, and add
    the required openai-project header automatically.
  • Add provider discovery, request-construction, normal-response, tool-call,
    validation, serialization, and encrypted-secret tests.

Issue Number

Fixes: #5188
Related to OpenHands/OpenHands#17512.

How to Test

Automated Verification Performed

Set up the development environment:

make build

Result: completed successfully.

Run the targeted OCI and provider-discovery tests:

uv run pytest -q \
  tests/sdk/llm/test_oci_genai_provider.py \
  tests/agent_server/test_llm_router.py

Result:

28 passed, 3 warnings

Run the broader LLM profile and Agent Server regression tests:

uv run pytest -q --disable-warnings \
  tests/sdk/llm/test_llm.py \
  tests/sdk/llm/test_llm_profile_store.py \
  tests/agent_server/test_profiles_router.py

Result:

246 passed, 4 warnings

Run the repository checks on all changed files:

uv run pre-commit run --files \
  openhands-sdk/openhands/sdk/llm/llm.py \
  openhands-sdk/openhands/sdk/llm/utils/oci_genai_provider.py \
  openhands-agent-server/openhands/agent_server/llm_router.py \
  tests/sdk/llm/test_oci_genai_provider.py \
  tests/agent_server/test_llm_router.py

Result:

Ruff format........................................................Passed
Ruff lint..........................................................Passed
PEP8 style check (pycodestyle).....................................Passed
Type check with pyright............................................Passed
Forbid dynamic attribute access in SDK.............................Passed
Check import dependency rules......................................Passed
Check Tool subclass registration...................................Passed

Validate the generated Agent Server OpenAPI schema:

make test-server-schema

Result:

Agent Server OpenAPI type-quality check passed.
openapi.json is valid

Live OCI Verification

A credentialed live request was not run because no OCI Generative AI API key,
project OCID, and region-enabled model were available in the development
environment.

The automated tests exercise both normal Responses API output and function-tool
output through mocked provider responses, but they do not make a billable OCI
request. Credentialed validation by a maintainer with OCI access would be
appreciated. The live verification procedure is:

  1. Create or select an OCI Generative AI project.
  2. Create a Generative AI API key.
  3. Select a model available in the same OCI region.
  4. Configure an LLM with:
    • model="oci_genai/<exact-model-id>"
    • oci_region="<region>"
    • oci_project_id="<project-ocid>"
    • the OCI Generative AI API key
  5. Send a normal Responses API request and confirm that text is returned.
  6. Start an agent conversation that requests a terminal tool call.
  7. Confirm that the model invokes the tool and continues after consuming its
    result.

OpenHands/docs#813 includes the same credential-gated live test plan for users.

Video/Screenshots

Not applicable to this SDK and Agent Server PR because it does not change a GUI.

OpenHands/OpenHands#17543 contains the user-facing provider selector, OCI
configuration fields, and a screenshot of the rendered form. CLI test results
and reproduction steps are included above.

Design Doc

Not included. The implementation is limited to provider-specific configuration
normalization and transport routing:

oci_genai/<model-id>
        |
        v
openai/<model-id> through LiteLLM
        |
        v
https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1
        |
        +-- openai-project: <Generative AI project OCID>

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • oci_genai/ is intentionally separate from LiteLLM's native oci/ provider.
    The native route targets OCI IAM-style authentication, while this initial
    integration uses OCI Generative AI API keys and the OpenAI-compatible
    Responses API.
  • Model IDs are not hardcoded because OCI model availability differs by region.
  • OCI IAM authentication through user, session, instance, or resource
    principals is outside the scope of this PR and can be added in a follow-up.
  • The provider-required openai-project header takes precedence over conflicting
    caller-supplied headers.
  • Agent Canvas UI: feat(llm): add OCI Generative AI setup OpenHands#17543.
  • User-facing setup documentation: docs(llm): document OCI Generative AI setup docs#813.

Jev-Fast-Audit

Jev fast audit · estimates · 0.52s · commit 1f8bab5
Strongest signal: No primary concern selected.
Evidence: No primary concern to locate.
Coverage: complete supplied coverage; 14/14 hunks, 5/5 files.

All estimates and evidence
Estimate Likelihood / value Direct evidence
SQL injection 3.0% No direct hunk selected
Command injection 5.0% No direct hunk selected
Weakened authentication 6.0% No direct hunk selected
Weakened authorization 9.0% No direct hunk selected
Contract regression 17.0% No direct hunk selected
Data loss 3.0% No direct hunk selected
Sensitive data disclosure 10.0% No direct hunk selected
Unexpected data transfer 4.0% No direct hunk selected
Credential misuse 10.0% No direct hunk selected
Untrusted instruction authority 2.0% No direct hunk selected
Package source redirection 3.0% No direct hunk selected
Unverified remote execution 2.0% No direct hunk selected
Privileged environment access 3.0% No direct hunk selected
Security assessment bypass 6.0% No direct hunk selected
Prohibited workload 2.0% No direct hunk selected
Primary concern None selected; confidence 83.0% No primary concern to locate

Co-authored-by: openhands <openhands@all-hands.dev>
@all-hands-bot

Copy link
Copy Markdown
Collaborator

🚦 CI is currently failing on this PR's latest commit.

Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request @all-hands-bot as a reviewer to have it reviewed regardless of CI status.)

This is an automated check - no AI was used to generate this comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add OCI Generative AI provider support

2 participants