Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,15 @@ Covered by 183 end-to-end scenario files (496 cases) that run against real gatew
Moderation, AWS Bedrock Guardrails, Azure AI Content Safety (Prompt Shield + text
moderation), and two Alibaba Cloud services. A block returns `422 content_filter`;
monitor mode records what would have happened without blocking.
- **Caching** — exact-match response cache with per-policy TTL and model/key scope matchers;
memory and Redis backends; cost-saved telemetry on every hit. Separately, **automatic
prompt caching** can be enabled per direct Anthropic model to inject cache breakpoints, so
callers get provider-side prompt discounts without changing their requests.
- **Caching** — exact-match and **semantic** response caching with per-policy TTL and
model/key match rules; memory and Redis backends; cost-saved telemetry on every hit.
A cache policy carrying a `semantic` block serves a cached answer to a differently-worded
question at or above its cosine similarity threshold — in-process, or shared across
gateway replicas on Redis vector search. Only fully textual requests match semantically,
and entries stay private to the caller's API key unless the policy sets `scope: env`.
Separately, **automatic prompt caching** can be enabled per direct Anthropic model to
inject cache breakpoints, so callers get provider-side prompt discounts without changing
their requests.
- **MCP gateway** — front registered upstream MCP servers at `/mcp` with gateway-held
credentials, per-server tool namespaces, and per-caller access. It serves every
Streamable HTTP revision from `2025-03-26` through stateless `2026-07-28` without
Expand All @@ -193,9 +198,10 @@ Covered by 183 end-to-end scenario files (496 cases) that run against real gatew
events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver),
plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob)
telemetry.
- **Declarative configuration** — one `resources.yaml` carries all ten resource collections
(provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies,
observability exporters, rate-limit policies, OIDC providers), validated against the same
- **Declarative configuration** — one `resources.yaml` carries all thirteen resource
collections (provider keys, models, caller keys, guardrails, MCP servers, MCP auth
settings, A2A agents, cache policies, observability exporters, rate-limit policies,
OIDC providers, JWT claim mappings, passthrough routes), validated against the same
JSON Schemas the gateway uses at runtime. `aisix validate` checks a file offline; `SIGHUP`
reloads it atomically.
- **Operational endpoints** — `/livez` and `/readyz` on the proxy listener; `/status/config`,
Expand Down Expand Up @@ -305,14 +311,14 @@ crates/
Highlights on the [roadmap](ROADMAP.md); tracked live in
[issues](https://github.com/api7/aisix/issues):

- Semantic (embedding-similarity) response caching
- More observability sinks — Langsmith, Helicone, Slack alerts
- Prompt templates managed as gateway resources
- Llama-Guard as a guardrail provider

Shipped since this list was last written: the MCP gateway, the A2A agent gateway,
OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio,
PII, and OpenAI Moderation guardrails — see **Features** above.
Shipped since this list was last written: semantic (embedding-similarity) response caching,
the MCP gateway, the A2A agent gateway, OIDC/JWT inbound auth, Redis-backed distributed
rate limiting, and the Lakera, Presidio, PII, and OpenAI Moderation guardrails — see
**Features** above.

## 🛠️ Development

Expand Down
3 changes: 1 addition & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,13 @@ The **Surface** column shows where a capability lands: **Gateway** is the AISIX
| Enterprise SSO | Single sign-on through SAML and generic OIDC, beyond today's social logins. | Cloud |
| Directory sync (SCIM) | Provision and deprovision users and groups from your identity provider. | Cloud |
| Service accounts | Login-less, first-class principals for automated callers. | Cloud |
| Semantic caching | Serve responses for prompts close in meaning, on top of today's exact-match cache. | Gateway |

## Next

| Capability | What's planned | Surface |
| --- | --- | --- |
| Fine-grained authorization | Custom roles with per-resource and per-action permissions, beyond today's fixed roles and read/write scopes. | Cloud |
| Conditional and wildcard routing | Route on request metadata, headers, and tags, and match upstreams by wildcard names such as `provider/*`. | Gateway |
| Conditional routing | Route on request metadata and arbitrary headers, beyond today's tag-conditional targets and wildcard model names such as `provider/*`. | Gateway |
| Prompt management | Store, version, and reuse prompt templates with variables, resolved at the gateway. | Gateway · Cloud |
| Caller key rotation experience | Self-service key rotation in the dashboard, plus scheduled auto-rotation with a grace overlap. | Cloud |
| Production-path playground | Run the Cloud playground through a connected AISIX gateway so it reflects real routing, caching, guardrails, and rate limiting. | Cloud |
Expand Down