feat(sandbox,providers): add aws-bedrock as a recognized inference provider#1704
Open
st-gr wants to merge 2 commits into
Open
feat(sandbox,providers): add aws-bedrock as a recognized inference provider#1704st-gr wants to merge 2 commits into
st-gr wants to merge 2 commits into
Conversation
Adds two patterns to `default_patterns()` so the supervisor's L7
inference router recognizes the Bedrock InvokeModel URL shape and
forwards matched requests to the registered upstream:
- `POST /model/{modelId}/invoke` → aws_bedrock_invoke
- `POST /model/{modelId}/invoke-with-response-stream` → aws_bedrock_invoke_stream
The `{modelId}` segment is wildcarded by extending `detect_inference_pattern`
to handle one middle `/*/` segment in addition to the existing trailing
`/*`. The wildcard is constrained to a single non-empty path segment to
avoid path-traversal liabilities — `/model//invoke` and `/model/a/b/invoke`
both no-match.
Without this, sandboxes running Claude Code in its native Bedrock mode
(`CLAUDE_CODE_USE_BEDROCK=1`, `ANTHROPIC_BEDROCK_BASE_URL`, AWS-style
auth) hit the supervisor with `403 connection not allowed by policy`
because their URL doesn't match `/v1/*` shapes. The fix unblocks
operators wanting to register direct AWS Bedrock, an in-cluster
Bedrock-compatible bridge, or a Bedrock-emulating LiteLLM as
`--type aws-bedrock` providers.
Tests cover: positive matches for invoke + invoke-with-response-stream,
query-string handling, GET rejection, empty-segment rejection,
multi-segment rejection, and unknown-action rejection.
Companion changes (provider discovery spec + YAML profile) follow in
the next commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Adds `aws-bedrock` to the built-in provider catalog so operators can run `openshell provider create --type aws-bedrock --credential ...` and have the gateway treat it as a first-class inference provider alongside `anthropic`, `openai`, etc. - `providers/aws-bedrock.yaml`: YAML profile declaring four credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_REGION). Default endpoint is `bedrock-runtime.us-east-1.amazonaws.com:443`; operators in other regions or running against a Bedrock-compatible proxy override via the operator-supplied `BEDROCK_BASE_URL` config-key (mirrors `ANTHROPIC_BASE_URL` for the `anthropic` provider). - `crates/openshell-providers/src/providers/aws_bedrock.rs`: the `ProviderDiscoverySpec` so `openshell provider create --auto-providers` picks up AWS_* env vars from local credentials. - `crates/openshell-providers/src/providers/mod.rs`: register the module. - `crates/openshell-providers/src/lib.rs`: register the SPEC in the default registry alongside the other providers. - `crates/openshell-providers/src/profiles.rs`: include the new YAML in `BUILT_IN_PROFILE_YAMLS`. What this PR explicitly does NOT add (intentionally separated for review-size reasons; will follow up): - A SigV4 signer in `openshell-router`. The current change simply declares the protocol; a follow-up PR adds outbound SigV4 signing using the `aws-sigv4` crate and a new `auth_style: sigv4` validator branch in profiles.rs. Operators who don't need SigV4 (e.g. an in-cluster bridge that ignores it and authenticates separately to the upstream) can use this PR today. - Body translation between Bedrock InvokeModel shape and other inference shapes. The router treats Bedrock requests as opaque pass-through; if the operator's upstream is real AWS Bedrock it speaks Bedrock natively, if it's a translating bridge the bridge does any conversion server-side. - `BEDROCK_BASE_URL` placeholder substitution in the YAML loader. Today the YAML's `host` is a literal default; operators override with the config-key the same way `ANTHROPIC_BASE_URL` works. Tested: `cargo test -p openshell-providers` (35 tests green) and `cargo test -p openshell-sandbox --lib l7::inference` (40 tests green including the seven new aws_bedrock cases from the previous commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
st-gr
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
|
All contributors have signed the DCO ✍️ ✅ |
Author
|
I have read the DCO document and I hereby sign the DCO. |
Author
|
recheck |
johntmyers
reviewed
Jun 3, 2026
Comment on lines
+1
to
+20
| // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| use crate::ProviderDiscoverySpec; | ||
|
|
||
| pub const SPEC: ProviderDiscoverySpec = ProviderDiscoverySpec { | ||
| id: "aws-bedrock", | ||
| credential_env_vars: &[ | ||
| "AWS_ACCESS_KEY_ID", | ||
| "AWS_SECRET_ACCESS_KEY", | ||
| "AWS_SESSION_TOKEN", | ||
| "AWS_REGION", | ||
| ], | ||
| }; | ||
|
|
||
| test_discovers_env_credential!( | ||
| discovers_aws_bedrock_env_credentials, | ||
| "AWS_ACCESS_KEY_ID", | ||
| "AKIA-test-key" | ||
| ); |
Collaborator
There was a problem hiding this comment.
Please remove legacy provider updates, we should only be relying on providers v2 changes which should NOT require any changes to the discovery specs.
johntmyers
reviewed
Jun 3, 2026
| # substitute the `{region}` placeholder. Operators in other regions | ||
| # override via the `BEDROCK_BASE_URL` config-key the same way the | ||
| # `anthropic` provider accepts `ANTHROPIC_BASE_URL`. | ||
| - host: bedrock-runtime.us-east-1.amazonaws.com |
Collaborator
There was a problem hiding this comment.
You can also use wildcards/glob patterns here unless you feel that is too broad of access
Collaborator
PR Review StatusValidation: this PR is project-valid as a concentrated Bedrock-compatible inference provider/profile and sandbox L7 routing update with a clear operator UX path and credential-boundary discussion. Provider additions must stay on providers v2 through provider profiles unless a maintainer explicitly requests legacy discovery. Review findings:
Checks: Branch Checks and Helm Lint are currently waiting for Next state: |
johntmyers
added
the
gator:in-review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
aws-bedrockas a recognized inference protocol in the supervisor's L7 router and the providers catalog so operators can register a Bedrock-shaped upstream as--type aws-bedrockand route Claude Code Bedrock-mode traffic (POST /model/{id}/invoke[-with-response-stream]) throughinference.local. Without this, sandboxes hit403 "connection not allowed by policy"because no L7 pattern matches Bedrock URLs. The canonical no-SigV4 use case is SAP AI Core deployed Bedrock models (Anthropic models behind a Bedrock-shape API with XSUAA bearer auth instead of SigV4); operators wanting real AWS Bedrock additionally need #1630's proxy-side SigV4 signing.Related Issue
Complementary to #1630 ("Sigv4 credential signing"). #1630 adds proxy-side SigV4 re-signing as a
credential_signing: sigv4policy field. This PR is the URL-pattern half: the supervisor's L7 router needs to recognize Bedrock InvokeModel paths before anything can be signed, regardless of whether the upstream needs SigV4. The two patches don't touch the same files; they're complementary, not overlapping.No upstream tracking issue filed — happy to file one if reviewers prefer.
Changes
crates/openshell-sandbox/src/l7/inference.rs:default_patterns():POST /model/*/invoke(aws_bedrock_invoke) andPOST /model/*/invoke-with-response-stream(aws_bedrock_invoke_stream).detect_inference_patternto support a single middle/*/glob in addition to the existing trailing/*. The middle wildcard matches exactly one non-empty path segment containing no/—/model//invokeand/model/a/b/invokeboth no-match.providers/aws-bedrock.yaml: new YAML profile declaring four credentials (AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_SESSION_TOKEN,AWS_REGION) and a default endpoint ofbedrock-runtime.us-east-1.amazonaws.com:443. Operators in other regions or pointing at non-AWS Bedrock-compatible upstreams override per-deployment via the operator-suppliedBEDROCK_BASE_URLconfig-key (mirroring how theanthropicprovider acceptsANTHROPIC_BASE_URL).crates/openshell-providers/src/providers/aws_bedrock.rs: theProviderDiscoverySpecso--auto-providerspicks upAWS_*env vars from local credentials.crates/openshell-providers/src/{providers/mod.rs,lib.rs,profiles.rs}: register the new module + SPEC + YAML.Testing
mise run pre-commitpasses — not run end-to-end (misenot on author's dev environment), but the equivalent rust pieces verified independently:cargo fmt --all -- --checkclean;cargo clippy --no-deps -p openshell-providers -p openshell-sandbox --all-targets -- -D warningsclean.crates/openshell-sandbox/src/l7/inference.rs::testscover positive path, query-string handling, GET rejection, empty-segment rejection, multi-segment rejection, unknown-action rejection. Provider-discovery test follows the existingtest_discovers_env_credential!macro convention.cargo test -p openshell-sandbox --lib l7::inference: 40 passed;cargo test -p openshell-providers: 35 passed.aws-bedrockprovider end-to-end requires either real AWS Bedrock with SigV4 (covered by Sigv4 credential signing #1630) or a Bedrock-compatible stub backend. Deferring the E2E test to whichever PR lands second so it can exercise the full URL-pattern + auth path together.Checklist
feat(sandbox):,feat(providers):)default_patterns(); the new YAML profile follows the same shape asclaude-code.yaml/nvidia.yaml. Happy to add a paragraph todocs/sandboxes/manage-providers.mdx(or another spot reviewers prefer) in this PR rather than a doc-only follow-up.Notes for reviewers
Use cases (which PRs you need for which upstream):
credential_signing: sigv4policy field. The two are complementary; this PR is the prerequisite that makes #1630's signing applicable to Bedrock paths.In all three cases,
provider create --type aws-bedrockrequires--no-verifyuntil a Bedrock-aware arm is added tovalidation_probe()incrates/openshell-router/src/backend.rs. That extension is left for a follow-up PR to keep this one focused on the URL-pattern + provider-registration changes.Out of scope (intentional):
{region}placeholder substitution in the YAML loader. Operators override per-deployment viaBEDROCK_BASE_URLconfig-key the same wayANTHROPIC_BASE_URLworks for theanthropicprovider.openshell provider create --type aws-bedrockbecause the registry recognizes the new id; surfacing it in the TUI's provider-type picker is a follow-up.Operator context:
The
st-gr/openshell-driver-kymaHelm chart currently registers its SAP AI Core ↔ Bedrock translation bridge as--type anthropicwith/v1/messageson the inside, becauseaws-bedrockisn't a recognized provider type. The chart therefore carries a server-side Anthropic→Bedrock body translator and a denylist for Anthropic-API-only fields the SAP gateway rejects. After this PR, the bridge becomes a path-translating + auth-substituting pass-through with no body work — the chart's translator code goes away.