From 11d471825415cc4d1c7dc2741e5e5b3c5f616383 Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 16:40:55 +0800 Subject: [PATCH 1/3] docs(skills): restore APISIX 3.17 Bedrock guidance --- skills/a6-plugin-ai-proxy/SKILL.md | 79 +++++++++++++++++++++++------- 1 file changed, 60 insertions(+), 19 deletions(-) diff --git a/skills/a6-plugin-ai-proxy/SKILL.md b/skills/a6-plugin-ai-proxy/SKILL.md index 89b9dda..1ff9d7e 100644 --- a/skills/a6-plugin-ai-proxy/SKILL.md +++ b/skills/a6-plugin-ai-proxy/SKILL.md @@ -3,8 +3,8 @@ name: a6-plugin-ai-proxy description: >- Skill for configuring the Apache APISIX ai-proxy plugin via the a6 CLI. Covers proxying requests to LLM providers (OpenAI, Azure OpenAI, DeepSeek, - Anthropic, Gemini, Vertex AI, and more), authentication per provider, - model configuration, streaming, logging, and load balancing with + Anthropic, Gemini, Vertex AI, Amazon Bedrock, and more), authentication per + provider, model configuration, streaming, logging, and load balancing with ai-proxy-multi. version: "1.0.0" author: Apache APISIX Contributors @@ -23,14 +23,17 @@ metadata: ## Overview -The `ai-proxy` plugin turns APISIX into an AI gateway. It proxies requests in -OpenAI-compatible format to LLM providers, handling authentication, endpoint -routing, and response streaming. Clients send a standard chat-completion -request; the plugin translates and forwards it to the configured provider. +The `ai-proxy` plugin turns APISIX into an AI gateway. Clients can send +requests in supported protocols to APISIX instead of handling provider +authentication and endpoint selection themselves. The plugin detects the +client protocol, selects a compatible provider endpoint, forwards the native +format or converts it when an adapter is available, and handles response +streaming. ## When to Use -- Proxy chat-completion or embedding requests to any supported LLM provider +- Proxy Chat Completions, Responses API, Embeddings, Anthropic Messages, or + Bedrock Converse requests to a compatible provider - Centralize API keys at the gateway instead of distributing to clients - Add observability (token counts, latency) to LLM calls - Combine with `ai-prompt-template`, `ai-prompt-decorator`, or content @@ -38,35 +41,36 @@ request; the plugin translates and forwards it to the configured provider. ## Supported Providers -| Provider | Value | Default Endpoint | -|----------|-------|------------------| -| OpenAI | `openai` | `https://api.openai.com/v1/chat/completions` | +| Provider | Value | Endpoint Behavior | +|----------|-------|-------------------| +| OpenAI | `openai` | Automatically selects `/v1/chat/completions`, `/v1/responses`, or `/v1/embeddings` on `https://api.openai.com` | | DeepSeek | `deepseek` | `https://api.deepseek.com/chat/completions` | | Azure OpenAI | `azure-openai` | Custom via `override.endpoint` | -| Anthropic | `anthropic` | `https://api.anthropic.com/v1/chat/completions` | +| Anthropic | `anthropic` | Automatically selects `/v1/chat/completions` or `/v1/messages` on `https://api.anthropic.com` | | AIMLAPI | `aimlapi` | `https://api.aimlapi.com/v1/chat/completions` | | OpenRouter | `openrouter` | `https://openrouter.ai/api/v1/chat/completions` | | Gemini | `gemini` | `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` | | Vertex AI | `vertex-ai` | `https://aiplatform.googleapis.com` | +| Amazon Bedrock | `bedrock` | Region- and model-specific Bedrock Runtime endpoint; available from APISIX 3.17.0 | | OpenAI-Compatible | `openai-compatible` | Custom via `override.endpoint` | ## Plugin Configuration Reference | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| -| `provider` | string | **Yes** | — | One of the 9 supported providers | +| `provider` | string | **Yes** | — | One of the 10 supported providers | | `auth` | object | **Yes** | — | Authentication config (see below) | | `options` | object | No | — | Model and generation parameters | | `options.model` | string | No | — | Model name (provider-specific) | | `options.temperature` | number | No | — | Sampling temperature | | `options.top_p` | number | No | — | Nucleus sampling | | `options.max_tokens` | integer | No | — | Maximum tokens to generate | -| `options.stream` | boolean | No | `false` | Enable SSE streaming | -| `override` | object | No | — | Override default endpoint | -| `override.endpoint` | string | No | — | Full URL for the provider API | -| `provider_conf` | object | No | — | Provider-specific config (Vertex AI) | -| `provider_conf.project_id` | string | No | — | GCP project ID (Vertex AI) | -| `provider_conf.region` | string | No | — | GCP region (Vertex AI) | +| `options.stream` | boolean | No | — | Override the outgoing `stream` field; for Bedrock Converse, set `stream: true` in the client request instead | +| `override` | object | No | — | Provider endpoint and request-body override settings | +| `override.endpoint` | string | No | — | Provider scheme and host, or a full URL including the path and query | +| `provider_conf` | object | No | — | Provider-specific config for Vertex AI or Amazon Bedrock | +| `provider_conf.project_id` | string | No | — | GCP project ID for Vertex AI; required with `region` unless `override.endpoint` is configured | +| `provider_conf.region` | string | No | — | GCP region for Vertex AI; required AWS region for Amazon Bedrock | | `logging` | object | No | — | Logging options | | `logging.summaries` | boolean | No | `false` | Log model, duration, tokens | | `logging.payloads` | boolean | No | `false` | Log request/response bodies | @@ -78,7 +82,7 @@ request; the plugin translates and forwards it to the configured provider. ## Authentication by Provider -### OpenAI / DeepSeek / Anthropic / AIMLAPI / OpenRouter +### OpenAI / DeepSeek / AIMLAPI / OpenRouter ```json { @@ -90,6 +94,21 @@ request; the plugin translates and forwards it to the configured provider. } ``` +### Anthropic + +```json +{ + "auth": { + "header": { + "x-api-key": "your-anthropic-api-key" + } + } +} +``` + +Native Anthropic Messages requests also require an `anthropic-version` header. +Configure it in `auth.header` or require clients to send it. + ### Azure OpenAI ```json @@ -138,6 +157,28 @@ request; the plugin translates and forwards it to the configured provider. The `service_account_json` can also be set via the `GCP_SERVICE_ACCOUNT` environment variable. +### Amazon Bedrock + +```json +{ + "auth": { + "aws": { + "access_key_id": "your-access-key-id", + "secret_access_key": "your-secret-access-key", + "session_token": "your-session-token" + } + }, + "provider_conf": { + "region": "us-east-1" + }, + "options": { + "model": "your-model-id" + } +} +``` + +The session token is required when you use temporary AWS credentials. + ### Custom OpenAI-Compatible API ```json From bb66c09fad219046d603d011fe7a30bb1c637477 Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 17:31:48 +0800 Subject: [PATCH 2/3] docs(skills): clarify AI protocol detection --- skills/a6-plugin-ai-proxy/SKILL.md | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/skills/a6-plugin-ai-proxy/SKILL.md b/skills/a6-plugin-ai-proxy/SKILL.md index 1ff9d7e..886e745 100644 --- a/skills/a6-plugin-ai-proxy/SKILL.md +++ b/skills/a6-plugin-ai-proxy/SKILL.md @@ -39,6 +39,17 @@ streaming. - Combine with `ai-prompt-template`, `ai-prompt-decorator`, or content moderation plugins for a full AI gateway pipeline +## Protocol Detection + +APISIX uses the request URI as part of protocol detection. Anthropic Messages +requests must use a URI ending in `/v1/messages`, and Bedrock Converse requests +must use a URI ending in `/converse`. Without these suffixes, a request body can +match another protocol, such as OpenAI Chat. + +For Bedrock streaming, keep the client-facing URI ending in `/converse` and set +`stream: true` in the request body. APISIX then selects the upstream +`/model/{modelId}/converse-stream` endpoint. + ## Supported Providers | Provider | Value | Endpoint Behavior | @@ -65,7 +76,7 @@ streaming. | `options.temperature` | number | No | — | Sampling temperature | | `options.top_p` | number | No | — | Nucleus sampling | | `options.max_tokens` | integer | No | — | Maximum tokens to generate | -| `options.stream` | boolean | No | — | Override the outgoing `stream` field; for Bedrock Converse, set `stream: true` in the client request instead | +| `options.stream` | boolean | No | — | Override the outgoing `stream` field. For Bedrock Converse, `stream: true` on a `/converse` request selects `/model/{modelId}/converse-stream` and returns unmodified AWS EventStream binary frames with `Content-Type: application/vnd.amazon.eventstream`, not SSE; clients must parse EventStream responses. | | `override` | object | No | — | Provider endpoint and request-body override settings | | `override.endpoint` | string | No | — | Provider scheme and host, or a full URL including the path and query | | `provider_conf` | object | No | — | Provider-specific config for Vertex AI or Amazon Bedrock | @@ -100,14 +111,15 @@ streaming. { "auth": { "header": { - "x-api-key": "your-anthropic-api-key" + "x-api-key": "your-anthropic-api-key", + "anthropic-version": "2023-06-01" } } } ``` -Native Anthropic Messages requests also require an `anthropic-version` header. -Configure it in `auth.header` or require clients to send it. +Native Anthropic Messages requests require an `anthropic-version` header. +Configure it in `auth.header`, as shown, or require clients to send it. ### Azure OpenAI From 1694739a0076fa3b19d4a04b03cd1637c1e0d88f Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 17:47:36 +0800 Subject: [PATCH 3/3] docs(skills): document Responses protocol detection --- skills/a6-plugin-ai-proxy/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/a6-plugin-ai-proxy/SKILL.md b/skills/a6-plugin-ai-proxy/SKILL.md index 886e745..7fad8bd 100644 --- a/skills/a6-plugin-ai-proxy/SKILL.md +++ b/skills/a6-plugin-ai-proxy/SKILL.md @@ -46,6 +46,10 @@ requests must use a URI ending in `/v1/messages`, and Bedrock Converse requests must use a URI ending in `/converse`. Without these suffixes, a request body can match another protocol, such as OpenAI Chat. +OpenAI Responses requests with an `input` field must use a URI ending in +`/v1/responses`. Otherwise, APISIX detects the body as OpenAI Embeddings; use a +URI ending in `/v1/embeddings` for embedding routes. + For Bedrock streaming, keep the client-facing URI ending in `/converse` and set `stream: true` in the request body. APISIX then selects the upstream `/model/{modelId}/converse-stream` endpoint.