diff --git a/CLAUDE.md b/CLAUDE.md index 76a2579afcdb..76f942cd1045 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,6 +120,7 @@ How it's built + upgrade re-apply notes: [.specify/CUSTOMIZATIONS.md](.specify/C - [Java Standards](docs/backend/JAVA_STANDARDS.md) — Coding patterns, immutables, exceptions, utilities - [When to Use Virtual Threads](docs/backend/VIRTUAL_THREADS.md) — Socket I/O yes, file I/O no; carrier pinning - [REST API Patterns](docs/backend/REST_API_PATTERNS.md) — JAX-RS, Swagger, @Schema rules +- [Inference API](docs/backend/INFERENCE_API.md) — OpenAI-compatible `/api/inference/v1` family: chat completions, streaming, models, embeddings, images; bearer-only auth, per-site resolution, limits - [Maven Build System](docs/backend/MAVEN_BUILD_SYSTEM.md) — Dependency management - [Configuration Patterns](docs/backend/CONFIGURATION_PATTERNS.md) — Config.getProperty() usage - [Database Patterns](docs/backend/DATABASE_PATTERNS.md) — DotConnect, transactions diff --git a/docs/README.md b/docs/README.md index d6ca3f4edf1a..1e61cfdd7019 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,6 +16,7 @@ Standards that apply to all development: Backend-specific patterns and standards: - **JAVA_STANDARDS.md** - Java development patterns and API usage - **MAVEN_BUILD_SYSTEM.md** - Dependency and plugin management +- **[INFERENCE_API.md](backend/INFERENCE_API.md)** - OpenAI-compatible `/api/inference/v1` endpoint family: operations, bearer-only auth, site resolution, limits, errors ### `/docs/frontend/` - Angular/TypeScript Development Frontend-specific patterns and standards. **Start at the index: [frontend/README.md](frontend/README.md)** — it lists every doc and when to load it. diff --git a/docs/backend/INFERENCE_API.md b/docs/backend/INFERENCE_API.md new file mode 100644 index 000000000000..83202949449d --- /dev/null +++ b/docs/backend/INFERENCE_API.md @@ -0,0 +1,317 @@ +# OpenAI-Compatible Inference API (`/api/inference/v1`) + +`/api/inference/v1` is a REST endpoint family that speaks the OpenAI chat-completions wire format — the request and response JSON shapes published by OpenAI and served by most AI gateways (OpenRouter, LiteLLM, Vercel AI Gateway, Ollama, vLLM). Because the format is the one standard client libraries and agent frameworks already emit, pointing such a client at dotCMS takes exactly two settings: a base URL and a dotCMS API token. No dotCMS-specific client code is written. + +The endpoints sit in front of dotAI, which is already a multi-provider gateway with per-site credentials held in App secrets, model fallback chains, and provider selection. What the family adds is the door: an application that previously had to embed a provider key of its own can route through dotCMS instead, so the site's credentials stay server-side, the provider can be swapped centrally, and every call can be attributed to a site. + +**This is not a replacement for `/api/v1/ai/*`.** Those endpoints stay and keep their behavior. Retrieval-augmented completions, semantic search, the vector-corpus operations and provider administration have no equivalent verb in the OpenAI format and remain first-class there. Three `/api/v1/ai/*` operations are superseded by this family and are now `@Deprecated` — see [Superseded endpoints](#superseded-endpoints). + +Source: `dotCMS/src/main/java/com/dotcms/inference/`. Design intent lives in `specs/37431-openai-compatible-inference/`; the published contract is the generated `openapi.yaml`. + +--- + +## Operations + +| Operation | Path | Notes | +|---|---|---| +| Chat completion | `POST /api/inference/v1/chat/completions` | Multi-turn messages, tool calling, optional SSE streaming | +| Model listing | `GET /api/inference/v1/models` | Every model the resolved site configured — chat, embeddings and images — chat first | +| Embeddings | `POST /api/inference/v1/embeddings` | One string or an array of strings | +| Image generation | `POST /api/inference/v1/images/generations` | Always base64, never a URL | + +The trailing `v1` is the **wire-protocol** version, not dotCMS's own `/api/v1` resource versioning. A second wire format would be added as a sibling under the same version (`/api/inference/v1/responses`), not as `/api/inference/v2`. OpenAI's legacy non-chat `POST /completions` is deliberately absent. + +--- + +## Authentication + +**Bearer token only.** Every endpoint requires `Authorization: Bearer `. Session cookies and basic authentication are refused with a `401`, even though the surrounding dotCMS filters would otherwise accept them — `BearerOnlyAuthFilter` enforces this for the whole family, and each resource calls the same check directly so it also holds when a resource method is invoked outside the JAX-RS chain. + +Any authenticated user is accepted, backend or frontend, matching the existing dotAI endpoints. + +**These endpoints are server-side only. Never put the token in browser JavaScript.** The token carries the full authority of the user it was issued for, so a token that reaches a browser is a token that can be read out of it. This is why **no CORS headers are emitted**: each resource carries the `@NoCors` marker (`com.dotcms.rest.annotation.NoCors`), which `com.dotcms.rest.api.CorsFilter` honors by skipping cross-origin headers entirely, so a browser blocks a cross-origin call before it reaches the endpoint. Refusing session and basic credentials is what keeps that a real control rather than a formality — if an ambient browser credential authenticated here, any page the user had open would be one fetch away from spending the site's AI budget. + +--- + +## Model selection + +`model` is **required on every request**. There is no implicit default, no reserved alias and no sentinel name — a request that omits it is refused with a `400` naming the field. + +The value is validated against what the resolved site has configured **for that capability**, for every caller including administrators: + +| Endpoint | Validated against the site's | +|---|---| +| `chat/completions` | `chat` section of `providerConfig` (including every fallback-chain entry) | +| `embeddings` | `embeddings` section | +| `images/generations` | `image` section | + +A model the site has not configured for that capability is refused with a `404`, so a chat model sent to the images endpoint is refused even though the same site configured it perfectly well for chat. Nothing is passed through to the provider. + +`GET /api/inference/v1/models` is the discovery mechanism: it lists every model the resolved site has configured, in configured order, chat section first — so a caller who wants "whatever this site runs" reads the list and takes the first entry. Embeddings and image models are listed alongside chat models because every operation here requires an exact model name and this is the only place to learn one; listing chat alone would leave two of the four operations undiscoverable. The format has nowhere to record what a model is for — its model object carries no type, mode or modality field, which is why OpenAI's own listing mixes chat, embedding and image models the same way — so picking an entry the operation does not serve is refused by that operation with a `404` naming `model`. Nothing is added to the entries to signal capability: adding a field the format does not define is what the no-adapter promise of this family exists to avoid. + +Because the model is required and validated, swapping a site's provider is **not** invisible to callers: a caller pinning the old vendor's model gets an explicit `404` and has to re-read the model list. That is deliberate. + +Within the chat capability, the site's own configuration and fallback chain decide which model actually runs. The `model` field of the response names the model that served the request, which may be a later entry in the chain than the one requested. + +--- + +## Site resolution + +Site resolution is **standard dotCMS resolution** — this family invents no semantics of its own: + +1. An explicit override, given as the `X-dotCMS-Site` request header or the `siteId` query parameter (a site identifier or a host name). The header wins when both are present. An override is resolved **as the caller**, so a site the caller cannot read is refused with a `403`. +2. Otherwise the request's host name. +3. Otherwise the default site, exactly as everywhere else in dotCMS. Server-side callers routinely present a host name that is no site alias — internal DNS, a container service name, `localhost` — so refusing those would break the deployments this family exists for. + +A default-site fallback is **logged** with the unmatched host name, at warn level, stating that the default site's credentials will be spent. + +The legacy `host` and `host_id` request parameters, which act as a de-facto site override elsewhere in dotCMS, are **ignored** here. + +Every response — success, refusal, streamed or not — carries the serving site in the **`X-dotCMS-Resolved-Site`** response header, holding the site identifier. It is a header rather than a body field so that payloads still deserialize into a standard client's own types, and so that it survives on errors and on streams. `ResolvedSiteHeaderFilter` writes it from an attribute the resource publishes the instant the site is known, before anything that can fail. + +A site with no dotAI configuration of its own inherits the system-level configuration, as every other dotAI caller does. Only when neither the site nor the system level has any configuration is a completion refused; the model listing returns an empty array in that case, never another site's models. + +--- + +## Configuration + +All four keys are read through `Config` at the point of use (`com.dotcms.inference.model.InferenceLimits`), so changing one takes effect without restarting the node. + +| Key | Default | What it bounds | +|---|---|---| +| `DOT_INFERENCE_MAX_CONCURRENT_STREAMS` | `50` | Streamed completions in flight on this node. A streamed completion parks a request thread for the whole generation, so concurrency — not request rate — is the scarce resource here. Over the ceiling, a streaming request is refused with `429`. | +| `DOT_INFERENCE_COMPLETION_TIMEOUT_SECONDS` | `300` | Hard ceiling on one streamed completion. A provider that stops producing events ends the stream with an error event instead of parking the thread indefinitely. | +| `DOT_INFERENCE_MAX_REQUEST_BYTES` | `1048576` (1 MiB) | Largest request body accepted. A declared `Content-Length` over the ceiling is refused with `413` before the body is read. | +| `DOT_INFERENCE_MAX_IMAGES_PER_REQUEST` | `10` | Most images one generation request may ask for. Ten is the ceiling the OpenAI images API itself documents for `n`. Over it, a `400` naming `n`. | + +> Earlier task notes refer to "the three config keys". There are four: `DOT_INFERENCE_MAX_IMAGES_PER_REQUEST` was added later, because image generation is priced per image and per-site spend quotas are out of scope — without a ceiling, one accepted request is an unbounded bill. + +Each endpoint also registers `@RequestCost(Price.HTTP_FETCH)`, the band used for operations that make one remote round trip, so the instance-wide rate-limit backstop prices them. That price is known to under-count a stream; it is not a per-site AI spend quota, which is out of scope. + +--- + +## `POST /chat/completions` + +Non-streaming, with tool declarations: + +```bash +curl -X POST "https://demo.dotcms.com/api/inference/v1/chat/completions" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o", + "messages": [ + { "role": "system", "content": "You are a helpful assistant." }, + { "role": "user", "content": "What is the weather in Bogota?" } + ], + "tools": [ + { "type": "function", "function": { + "name": "get_weather", + "description": "Current weather for a city", + "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } } + ], + "tool_choice": "auto", + "temperature": 0.7, + "max_tokens": 1024 + }' +``` + +```json +{ + "id": "chatcmpl-…", "object": "chat.completion", "created": 1789000000, "model": "gpt-4o", + "choices": [ { "index": 0, "finish_reason": "tool_calls", + "message": { "role": "assistant", "content": null, + "tool_calls": [ { "id": "call_1", "type": "function", + "function": { "name": "get_weather", "arguments": "{\"city\":\"Bogota\"}" } } ] } } ], + "usage": { "prompt_tokens": 82, "completion_tokens": 17, "total_tokens": 99 } +} +``` + +The next turn appends the assistant's tool-call message and a `tool` message carrying the result, correlated by `tool_call_id`: + +```bash +curl -X POST "https://demo.dotcms.com/api/inference/v1/chat/completions" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o", + "messages": [ + { "role": "system", "content": "You are a helpful assistant." }, + { "role": "user", "content": "What is the weather in Bogota?" }, + { "role": "assistant", "content": null, + "tool_calls": [ { "id": "call_1", "type": "function", + "function": { "name": "get_weather", "arguments": "{\"city\":\"Bogota\"}" } } ] }, + { "role": "tool", "tool_call_id": "call_1", "content": "{\"tempC\":19}" } + ] + }' +``` + +`temperature`, `max_tokens`, `top_p` and `stop` are passed through to the provider, because they change what the caller gets and what the site pays. Incidental fields a standard client sends by default are ignored. `n` is the exception among the fields this family does not honor: asking for more than one choice is refused with a `400` rather than silently ignored. `response_format` is accepted. + +### Streaming + +Set `"stream": true` and the answer is served as `text/event-stream`: + +```bash +curl -N -X POST "https://demo.dotcms.com/api/inference/v1/chat/completions" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o", + "messages": [ { "role": "user", "content": "Write a haiku about caching." } ], + "stream": true, + "stream_options": { "include_usage": true } + }' +``` + +``` +data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1789000000,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]} + +data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1789000000,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"Cold bytes"}}]} + +data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1789000000,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} + +data: [DONE] +``` + +Every chunk carries `id`, `object`, `created` and `model`, fixed for the life of the stream so a client can correlate the fragments it stitches together. Tool calls stream incrementally: each fragment carries the call's `index`, and `id`, `type` and `function.name` arrive on the first fragment, so a standard reader reassembles the arguments as they land. + +Token usage on a stream is emitted **only** when the request asks for it through `stream_options.include_usage`. It arrives as a final chunk with an empty `choices` array, immediately before `[DONE]`. dotCMS builds that chunk itself from the counts the provider abstraction reports and does not forward the option upstream; a usage chunk a provider volunteers unasked is suppressed, because the empty `choices` array is exactly what breaks readers that assume every chunk carries a choice. + +**A stream that fails after it has begun** — a provider error mid-generation, or `DOT_INFERENCE_COMPLETION_TIMEOUT_SECONDS` firing — emits the standard error object as its final event and closes **without** `[DONE]`: + +``` +data: {"error":{"message":"The model provider failed to complete the request","type":"api_error","param":null,"code":null}} +``` + +Once the first chunk is on the wire the HTTP status can no longer carry a failure, so withholding the done marker is what stops a client that does not parse the error event from reading a truncated answer as a finished one. + +--- + +## `GET /models` + +```bash +curl "https://demo.dotcms.com/api/inference/v1/models" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" +``` + +```json +{ "object": "list", "data": [ + { "id": "gpt-4o", "object": "model", "created": 1789000000, "owned_by": "dotcms" }, + { "id": "gpt-4o-mini", "object": "model", "created": 1789000000, "owned_by": "dotcms" } ] } +``` + +`created` is the time the listing was built, the same value on every entry: dotCMS serves a configuration, not a catalogue, and does not know when a vendor published a model. + +--- + +## `POST /embeddings` + +`input` accepts a single string or an array of strings; batching is how content is ordinarily embedded. + +```bash +curl -X POST "https://demo.dotcms.com/api/inference/v1/embeddings" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "text-embedding-3-small", + "input": ["The quick brown fox", "jumps over the lazy dog"] + }' +``` + +```json +{ "object": "list", "model": "text-embedding-3-small", + "data": [ { "object": "embedding", "index": 0, "embedding": [0.0023, -0.0091] }, + { "object": "embedding", "index": 1, "embedding": [0.0512, 0.0034] } ], + "usage": { "prompt_tokens": 12, "total_tokens": 12 } } +``` + +The response is always a list with one entry per input, each carrying the `index` of the input it embeds — the caller's only way to correlate a vector back to the text they sent. A batch with one bad element is refused whole rather than cleaned, because dropping an element would shift every index after it. + +Refused with a `400` naming `input`: an absent, null or empty `input`; an empty array; an element that is not a string (arrays of token ids are not supported); and a blank or whitespace-only string wherever it appears. Where one element of an array is at fault, the message says which index. There is no separate cap on element count — `DOT_INFERENCE_MAX_REQUEST_BYTES` already bounds what can arrive. + +Input text is never logged and never echoed back into an error message. + +--- + +## `POST /images/generations` + +```bash +curl -X POST "https://demo.dotcms.com/api/inference/v1/images/generations" \ + -H "Authorization: Bearer $DOTCMS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "dall-e-3", + "prompt": "A cat in a hammock", + "n": 2, + "size": "1024x1024" + }' +``` + +```json +{ "created": 1789000000, "data": [ { "b64_json": "iVBORw0KGgo…" }, { "b64_json": "R0lGODlhAQAB…" } ] } +``` + +**Results are always base64 (`b64_json`), never a hosted URL.** The response shape has no `url` component at all. A URL would mean deciding storage, authentication and lifetime for an artifact generated from a prompt that may carry customer data, so this family declines to create a separately-addressable artifact. dotCMS does not send an output-format parameter: the current image models reject it — they only ever return base64, so there is nothing to ask for — and sending it broke every request to them. Where a model answers with a link instead, dotCMS fetches and re-encodes it before the caller sees anything. On those providers an artifact does exist upstream — the guarantee is that a caller never receives one. + +`n` is honored; omitting it means one image. It is refused with a `400` naming `n` — never clamped, and never surfaced as a retryable upstream error — when it is: + +- below `1`; +- above `DOT_INFERENCE_MAX_IMAGES_PER_REQUEST`; +- above `1` on a site whose configured image model can only produce one. Support is probed from the model implementation rather than a list of provider names, so it cannot rot on a library upgrade. In the current provider set, `GoogleAiGeminiImageModel` is the one that does not support it, while the OpenAI image models do. + +A model limitation that can never succeed is a `400` precisely because a `502` would tell a standard client's back-off to keep retrying a request that cannot succeed however long it waits. + +`prompt` is required, and is refused with a `400` naming `prompt` when blank. `size` is passed through to the provider as a `WIDTHxHEIGHT` string; where the site carries an `imageSize` setting, the caller's value wins and the site's is the default for a request that omits it, which is what the existing dotAI image endpoint already does. + +Prompts are never logged and never echoed back into an error message. + +--- + +## Errors + +Every refusal uses the standard OpenAI error envelope, so a client library deserializes a failure into its own types with no adapter. Nothing is wrapped in dotCMS's `ResponseEntityView`. + +```json +{ "error": { "message": "The model field is required; there is no implicit default model", "type": "invalid_request_error", "param": "model", "code": null } } +``` + +**Retryability is carried by the HTTP status, not by a body field** — that is what a standard client's back-off keys off, and the standard error shape has no retryable field. + +| Status | `type` | When | +|---|---|---| +| `400` | `invalid_request_error` | Two different things. **Caller mistakes**: missing `model`, empty `messages`, a malformed or uncorrelated tool message, `n > 1` on chat, a bad `input`, a bad `prompt`, an `n` the site's image model or the configured ceiling will not allow, an unresolvable site override. **And a provider refusal a retry cannot fix**: an exhausted provider account, a rejected credential, a model the provider does not serve. The second kind is not the caller's fault and says so — it points at the site's provider account or configuration — but it is a `4xx` because that is the only way to tell a client's back-off to stop | +| `401` | `invalid_request_error` | Anonymous, or a credential that is not a bearer token | +| `403` | `invalid_request_error` | An explicit site override naming a site the caller cannot read | +| `404` | `invalid_request_error` | A model the resolved site has not configured for that capability | +| `413` | `invalid_request_error` | Request body over `DOT_INFERENCE_MAX_REQUEST_BYTES` | +| `429` | `rate_limit_error` | The node is already serving its ceiling of concurrent streamed completions (carries `Retry-After`), **or** the model provider is rate limiting this site (no `Retry-After` — see below) | +| `502` | `api_error` | A genuine upstream fault — the provider had a bad minute. Retryable, and the only provider failure that is | + +A `502` carries a fixed, safe message; the provider's own message is logged and never returned, because it can carry the provider's endpoint, account identifiers, or a fragment of the prompt. Retryability is carried by the status alone — the standard error shape has no retryable field, and adding one would break the no-adapter promise these endpoints exist to keep — so a provider rate limit is reported as `429` rather than folded into `502`: the two call for different back-off, and a client that reads `502` treats a busy provider as a broken one. **Which failures are worth retrying is carried entirely by the status.** The standard error shape has no retryable field and adding one would break the no-adapter promise, so a client reads the number: `429` and `502` mean come back, every other `4xx` means do not. That is why a provider account out of credit answers `400` rather than `502` — it will refuse the next attempt identically, and a client that retried it would burn a provider round trip each time for as long as the account stayed empty. Whether a refusal is permanent is read from the provider library's own classification rather than a list of status codes dotCMS maintains, so a new failure kind in a later release is classified without a code change here. + +The two `429`s differ in one way that matters to a client. When dotCMS refuses because **this node** is at its streaming ceiling, the response carries `Retry-After: 5` — that is a wait dotCMS knows, so it states it in the form every standard client already honours rather than only in prose. When the refusal comes from the **provider**, no `Retry-After` is sent: the provider knows how long it wants to be left alone and says so in a header the client library discards on its error path, so the value never reaches dotCMS, and inventing one would present a guess to the client as an instruction. Relaying the provider's value needs a custom HTTP client under the provider abstraction and is tracked as follow-up work; until then a client backs off on the status alone for upstream throttling. A site with no usable configuration for the requested capability lands on the same `404` as an unknown model, so a caller cannot use the distinction to learn which sites have dotAI set up. + +Only exchange **metadata** is recorded — resolved site, model, token counts, status, duration. Request and response bodies are never written to a log or a durable store, and no conversation is retained after the response is delivered; the family is stateless. + +--- + +## Superseded endpoints + +These three `/api/v1/ai/*` operations are now `@Deprecated`. They remain fully functional and no removal date is committed. + +| Deprecated | Replaced by | +|---|---| +| `POST /api/v1/ai/text/generate` | `POST /api/inference/v1/chat/completions` | +| `POST /api/v1/ai/image/generate` | `POST /api/inference/v1/images/generations` | +| `POST /api/v1/ai/completions/rawPrompt` | `POST /api/inference/v1/chat/completions` | + +Every other `/api/v1/ai/*` endpoint is unaffected and is not superseded. + +--- + +## Related + +- [REST API Patterns](REST_API_PATTERNS.md) — JAX-RS, Swagger, `@Schema` rules +- [Configuration Patterns](CONFIGURATION_PATTERNS.md) — `Config.getProperty()` usage +- [Security Patterns](SECURITY_BACKEND.md) — auth, input validation, secure logging diff --git a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/InferenceAIClient.java b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/InferenceAIClient.java index 1204d582c24e..7a8796fecf2b 100644 --- a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/InferenceAIClient.java +++ b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/InferenceAIClient.java @@ -1,6 +1,7 @@ package com.dotcms.ai.client.langchain4j; import com.dotcms.ai.app.AppConfig; +import com.dotcms.inference.model.CallerSafeException; import com.dotcms.inference.model.InferenceError; import com.dotcms.inference.model.InferenceLimits; import com.dotcms.inference.model.MultipleImagesUnsupportedException; @@ -180,8 +181,12 @@ public void stream(final AppConfig appConfig, } catch (final RuntimeException e) { // Nothing reached the sink — every model in the chain failed to start — so the sink // still has to learn that this stream will not finish. - Logger.warn(InferenceAIClient.class, - "Inference stream could not be started: " + e.getClass().getSimpleName()); + // Logged with the exception rather than just its type. What the caller receives is a + // safe sentence that says the detail is in the log, so the detail has to actually be + // here — a bare class name leaves an operator with "InvalidRequestException" and no + // way to tell an exhausted account from a rejected key from a malformed request. The + // buffered path has always logged the full exception; this matches it. + Logger.warn(InferenceAIClient.class, "Inference stream could not be started", e); state.fail(toInferenceError(e)); } } @@ -343,8 +348,7 @@ public void onCompleteResponse(final ChatResponse response) { @Override public void onError(final Throwable throwable) { - Logger.warn(InferenceAIClient.class, - "Inference stream failed: " + throwable.getClass().getSimpleName()); + Logger.warn(InferenceAIClient.class, "Inference stream failed", throwable); state.fail(toInferenceError(throwable)); } }); @@ -695,13 +699,24 @@ private static InferenceUsage toInferenceUsage(final TokenUsage tokenUsage) { * @param throwable what went wrong * @return the error to report */ - private static InferenceError toInferenceError(final Throwable throwable) { - if (throwable instanceof IllegalArgumentException) { - return InferenceError.invalidRequest(nullToEmpty(throwable.getMessage()).isBlank() - ? "The request could not be served with this site's configuration" - : throwable.getMessage(), null); + static InferenceError toInferenceError(final Throwable throwable) { + if (throwable instanceof CallerSafeException + && !nullToEmpty(throwable.getMessage()).isBlank()) { + // dotCMS wrote this sentence, so it is returned: it names something about the site's + // configuration that the caller or an operator can actually act on. + return InferenceError.invalidRequest(throwable.getMessage(), null); } - return InferenceError.upstream(UPSTREAM_FAILURE_MESSAGE); + if (throwable instanceof IllegalArgumentException) { + // dotCMS did not write this one. The fallback chain reports a failed model + // initialisation as an IllegalArgumentException with the provider client's own message + // appended, and that message can carry the provider's endpoint, its account + // identifiers, or a fragment of the prompt. Logged in full, never returned. + return InferenceError.invalidRequest( + "The request could not be served with this site's configuration", null); + } + // A streamed exchange is rate limited exactly as often as a buffered one, and a client + // reading an error event backs off on the same status, so the translation belongs here too. + return InferenceError.fromProviderFailure(throwable, UPSTREAM_FAILURE_MESSAGE); } /** diff --git a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/LangChain4jAIClient.java b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/LangChain4jAIClient.java index 6b0d33b46ed8..b85cf59303b0 100644 --- a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/LangChain4jAIClient.java +++ b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/LangChain4jAIClient.java @@ -4,6 +4,7 @@ import com.dotcms.ai.AiKeys; import com.dotcms.ai.app.AIModelType; import com.dotcms.ai.app.AppConfig; +import com.dotcms.inference.model.CallerSafeException; import com.dotcms.ai.client.AIClient; import com.dotcms.ai.client.AIRequest; import com.dotcms.ai.client.JSONObjectAIRequest; @@ -82,7 +83,6 @@ public class LangChain4jAIClient implements AIClient { * Response format asked of an image provider by the {@code /api/inference/v1} family, so the * bytes arrive inline and no separately-addressable artifact is minted upstream. */ - private static final String INLINE_IMAGE_FORMAT = "b64_json"; /** * Cache-key discriminator for models the inference family borrows. Those differ from the ones @@ -214,9 +214,17 @@ public R withEmbeddingModel(final AppConfig appConfig, *

The image counterpart of {@link #withChatModel}, with two differences that are the * caller's request rather than the site's configuration. The requested {@code size} is applied * over the configured one, because a caller who named a size changed both what they receive - * and what the site pays. And {@code responseFormat} is set to {@value #INLINE_IMAGE_FORMAT} - * so a provider that offers the choice never mints a hosted artifact in the first place; - * providers that ignore it still answer, and the caller re-encodes what comes back.

+ * and what the site pays.

+ * + *

The output-format parameter is deliberately not sent. Asking for the + * inline form was the obvious way to stop a provider minting a hosted artifact, and it is what + * this did first — but the current generation of image models rejects the parameter outright + * ("Unknown parameter: 'response_format'"), because they only ever return base64 and there is + * nothing to choose. Sending it broke image generation against every one of them while working + * against the older models this family's tests stubbed. Not sending it is both simpler and + * safer than naming the models that accept it: the set that rejects it grows with every + * release, so any list would rot. Where a legacy model returns a URL instead, the caller still + * receives base64 — the answer is fetched and re-encoded, which this family already does.

* *

Both are folded into the cache key, so a model asked for one size is never handed to a * request that asked for another, and the legacy image endpoint — which wants the provider's @@ -234,8 +242,7 @@ public R withImageModel(final AppConfig appConfig, final ProviderConfig baseConfig = parseSection(appConfig.getProviderConfig(), IMAGE_SECTION); final boolean sized = size != null && !size.isBlank(); final ProviderConfig requestConfig = ImmutableProviderConfig.copyOf(baseConfig) - .withSize(sized ? size : baseConfig.size()) - .withResponseFormat(INLINE_IMAGE_FORMAT); + .withSize(sized ? size : baseConfig.size()); return executeWithFallbackTyped( cacheKeyPrefix(appConfig) + INFERENCE_KEY_SEGMENT @@ -554,7 +561,10 @@ R executeWithFallbackTyped( final BiFunction executor) { final List models = effectiveModels(baseConfig); if (models.isEmpty()) { - throw new IllegalArgumentException( + // CallerSafeException, not a bare IllegalArgumentException: this sentence was written + // here, names the site's own configuration, and is the one thing that tells a caller + // what to fix. The type is what marks it returnable — see CallerSafeException. + throw new CallerSafeException( "No model configured in providerConfig." + section + " — set 'model'"); } // Each failure is logged immediately. The last exception is rethrown only after diff --git a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/ProviderConfig.java b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/ProviderConfig.java index ce389c738af4..ed732cf9ac71 100644 --- a/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/ProviderConfig.java +++ b/dotCMS/src/main/java/com/dotcms/ai/client/langchain4j/ProviderConfig.java @@ -135,7 +135,7 @@ default List allModels() { * made by the caller that builds the model, which is why it is left null by default and the * strategies pass it on only when somebody asked for one. * - *

{@code /api/inference/v1/images/generations} sets {@code b64_json} because FR-012 wants + *

{@code /api/inference/v1/images/generations} sets {@code b64_json} because it returns * no hosted artifact minted upstream at all, not merely none returned to the caller. The * legacy {@code /api/v1/ai/image} endpoint leaves it unset and keeps the provider's default, * because its own response contract is a URL.

diff --git a/dotCMS/src/main/java/com/dotcms/ai/rest/AiHostResolver.java b/dotCMS/src/main/java/com/dotcms/ai/rest/AiHostResolver.java index 0ec0cf64f443..273e4d5318fd 100644 --- a/dotCMS/src/main/java/com/dotcms/ai/rest/AiHostResolver.java +++ b/dotCMS/src/main/java/com/dotcms/ai/rest/AiHostResolver.java @@ -117,11 +117,11 @@ private static Host resolveFromRequest(final HttpServletRequest request) { * *

Deliberately not {@code getCurrentHostNoThrow}, which is the obvious call and the wrong * one: it reads the {@code host_id} and {@code Host} request parameters before it ever looks - * at the server name (`HostWebAPIImpl.getCurrentHostFromRequest`), and FR-025 requires those - * be ignored on this family. They were already ignored whenever the server name matched a - * site, because that path returns before reaching here — so the effect was that a legacy - * override was honoured in precisely the case where it could redirect which site's - * credentials get spent, and ignored everywhere it was harmless.

+ * at the server name (`HostWebAPIImpl.getCurrentHostFromRequest`), and those parameters must + * not decide which site's credentials get spent here. They were already ignored whenever the + * server name matched a site, because that path returns before reaching here — so the effect + * was that a legacy override was honoured in precisely the case where it could redirect + * which site's credentials get spent, and ignored everywhere it was harmless.

* * @return the default site */ diff --git a/dotCMS/src/main/java/com/dotcms/inference/model/CallerSafeException.java b/dotCMS/src/main/java/com/dotcms/inference/model/CallerSafeException.java new file mode 100644 index 000000000000..bfd5c90c5d07 --- /dev/null +++ b/dotCMS/src/main/java/com/dotcms/inference/model/CallerSafeException.java @@ -0,0 +1,31 @@ +package com.dotcms.inference.model; + +/** + * An invalid-request failure whose message dotCMS wrote and is therefore safe to return. + * + *

Exists because "is this text safe to show a caller?" is not a question that can be answered + * by looking at an exception's type. Most of what reaches the error translator is an + * {@link IllegalArgumentException}, and they arrive from two very different places: some carry a + * sentence dotCMS composed about the site's own configuration, which is exactly what a caller + * needs to fix the problem; others carry a provider client's message, which can hold the + * provider's endpoint, its account identifiers, or a fragment of the prompt. Passing the message + * through whenever the type matched meant the second kind reached the caller too.

+ * + *

So the decision is made where the text is written rather than where it is rendered. Throwing + * this type is a statement that the message was authored here and may be returned; everything + * else is answered with a generic sentence and logged in full. Extending + * {@link IllegalArgumentException} keeps existing callers that catch the broader type working + * unchanged, which matters because the throw sites are shared with the older dotAI endpoints, + * whose behaviour must not shift.

+ */ +public class CallerSafeException extends IllegalArgumentException { + + private static final long serialVersionUID = 1L; + + /** + * @param message a message written by dotCMS, safe to return to a caller + */ + public CallerSafeException(final String message) { + super(message); + } +} diff --git a/dotCMS/src/main/java/com/dotcms/inference/model/InferenceError.java b/dotCMS/src/main/java/com/dotcms/inference/model/InferenceError.java index a24e642a6556..e80ace571d53 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/model/InferenceError.java +++ b/dotCMS/src/main/java/com/dotcms/inference/model/InferenceError.java @@ -41,9 +41,17 @@ public static InferenceError invalidRequest(final String message, final String p * @param model the model the caller asked for * @return a 404 error in the shape clients recognise as an unknown model */ - public static InferenceError noSuchModel(final String model) { + public static InferenceError noSuchModel(final String model, final String capability) { + // Naming the capability rather than saying "not configured for this site", which is true + // of the site's chat models only in the sense that matters and false in the sense a caller + // reads. The model listing returns every model the site has configured across all three + // capabilities, because the adopted format has nowhere to record what a model is for — so + // a caller can see an id in the list and then be told it is not configured, with no way to + // resolve the contradiction from the API alone. This refusal is the only place that + // information can live. return new InferenceError("invalid_request_error", - "The model '" + model + "' is not configured for this site", "model", 404); + "The model '" + model + "' is not configured for " + capability + " on this site", + "model", 404); } /** @@ -54,6 +62,94 @@ public static InferenceError upstream(final String message) { return new InferenceError("api_error", message, null, 502); } + /** + * Translates a provider failure into the status a standard client's back-off keys off. + * + *

A provider refusing on rate limit and a provider genuinely broken are different + * events with different right answers, and collapsing both into 502 tells a client to retry a + * throttled request on the wrong schedule while telling it nothing about when to come back. + * Rate limiting is 429; anything else upstream stays 502.

+ * + *

The provider's own message is never carried out of here — it can hold its endpoint, its + * account identifiers, and occasionally a fragment of the prompt. Only the status is taken + * from it, which is why the caller supplies the wording.

+ * + *

Three outcomes, not two, because "the provider said no" splits along a line a client + * acts on. A rate limit is temporary: 429, come back later. A refusal the provider will + * repeat — an exhausted account, a rejected key, a model the provider does not serve — is + * permanent until somebody changes something, and answering 502 there tells a standard + * client's back-off to retry a request that cannot ever succeed. Everything left is a + * genuine upstream fault, which is what 502 is for.

+ * + *

Permanence is read from the provider library's own classification rather than from a + * list of exception types or status codes: it already sorts its failures into retriable and + * non-retriable, and keying off that means a type added in a later release is classified + * correctly without this method being touched.

+ * + *

The cause chain is walked rather than the top exception inspected, because the provider + * client wraps: a rate limit surfaces as its own type or as a generic HTTP failure carrying + * 429, and by the time a retrying fallback chain has rethrown it, either can be two or three + * causes down.

+ * + * @param failure the exception the provider client raised + * @param upstreamMessage the wording to return for a non-rate-limit failure + * @return the error to answer with + */ + public static InferenceError fromProviderFailure(final Throwable failure, + final String upstreamMessage) { + for (Throwable current = failure; current != null; current = current.getCause()) { + if (current instanceof dev.langchain4j.exception.RateLimitException) { + return rateLimited(); + } + if (current instanceof dev.langchain4j.exception.HttpException httpFailure) { + if (httpFailure.statusCode() == 429) { + return rateLimited(); + } + // Any other 4xx is the provider saying the request itself is wrong — a parameter + // it does not accept, a model it does not serve, an account it will not bill. It + // will say the same thing to an identical retry, so it must not leave here with a + // retryable status. This arrives as a bare HttpException rather than one of the + // library's classified types, which is how a permanently malformed request used + // to be reported as a temporary upstream fault. + if (httpFailure.statusCode() >= 400 && httpFailure.statusCode() < 500) { + return providerRefused(); + } + } + if (current instanceof dev.langchain4j.exception.NonRetriableException) { + return providerRefused(); + } + if (current.getCause() == current) { + break; + } + } + return upstream(upstreamMessage); + } + + /** + * @return the standard rate-limit refusal, which a client reads as "retry later" + */ + /** + * @return the refusal for a provider failure that a retry cannot fix + */ + public static InferenceError providerRefused() { + // 400 rather than 502, because the status is the only thing carrying retryability and a + // 5xx would keep a client coming back. It is an imperfect fit — the caller has usually + // done nothing wrong, and the thing to fix is the site's provider account or + // configuration — so the message says where to look rather than implying the request was + // malformed. The provider's own wording, which names the account and sometimes the + // prompt, stays in the log. + return new InferenceError("invalid_request_error", + "The model provider refused this request, and will refuse it again on retry. " + + "This is a provider account or site configuration problem rather than " + + "a problem with the request; the details are in the server log.", + null, 400); + } + + public static InferenceError rateLimited() { + return new InferenceError("rate_limit_error", + "The model provider is rate limiting this site's requests", null, 429); + } + /** @return whether a standard client should retry, as implied by the status */ public boolean isRetryable() { return httpStatus == 429 || httpStatus >= 500; diff --git a/dotCMS/src/main/java/com/dotcms/inference/rest/ChatCompletionsResource.java b/dotCMS/src/main/java/com/dotcms/inference/rest/ChatCompletionsResource.java index cfe426f2ac3c..65617e18c5d3 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/rest/ChatCompletionsResource.java +++ b/dotCMS/src/main/java/com/dotcms/inference/rest/ChatCompletionsResource.java @@ -18,6 +18,7 @@ import com.dotcms.inference.rest.view.InferenceErrorView; import com.dotcms.rest.WebResource; import com.dotcms.rest.annotation.NoCache; +import com.dotcms.rest.annotation.NoCors; import com.dotcms.rest.api.v1.DotObjectMapperProvider; import com.dotmarketing.exception.DotSecurityException; import com.dotmarketing.util.Logger; @@ -91,6 +92,7 @@ */ @Path("/inference/v1/chat") @Tag(name = "AI", description = "AI-powered content generation and analysis endpoints") +@NoCors public class ChatCompletionsResource { /** Media type a streamed completion is served as. */ @@ -109,6 +111,21 @@ public class ChatCompletionsResource { private static final String SITE_HEADER = "X-dotCMS-Site"; /** Prefix the completion id carries, as standard clients expect. */ + /** The header a standard client reads to learn how long to wait before retrying. */ + private static final String RETRY_AFTER_HEADER = "Retry-After"; + + /** + * How long to tell a caller to wait when this node is at its streaming ceiling. + * + *

Five seconds, which is a judgement rather than a measurement: a streamed completion runs + * for seconds to minutes, so no single number is right for every deployment. It is chosen to be + * long enough that a refused client does not immediately return and re-refuse — turning one + * ceiling into a retry storm that keeps the node at capacity — and short enough that capacity + * freed a moment later does not sit idle. It is not a configurable limit because it is not a + * limit: the ceiling itself is configurable, and this only says how long to wait for it.

+ */ + private static final int CAPACITY_RETRY_AFTER_SECONDS = 5; + private static final String COMPLETION_ID_PREFIX = "chatcmpl-"; /** What a caller is told when the provider failed; never the provider's own words. */ @@ -186,6 +203,17 @@ public class ChatCompletionsResource { @JSONP @NoCache @InferenceEndpoint + // One flat price for an operation whose real cost varies by orders of magnitude, and it + // under-counts a stream worst of all: a streamed completion holds a request thread for the + // whole generation rather than for one round trip, and the tokens it spends are unknown until + // it ends. Deliberately not "fixed" here by picking a larger constant — a bigger flat number + // is the same mistake scaled, and it would silently re-price every non-streaming caller too. + // Pricing this honestly means charging on what a request actually consumed, token counts being + // the unit that matches what the provider bills, which is a design this endpoint cannot settle + // on its own: it needs a decision about where usage is metered, what happens when a stream + // fails halfway, and how it reconciles with per-site spend attribution, which is not + // something this endpoint family answers. Until that exists, the flat price stands and is + // known to be wrong for streams. @RequestCost(Price.HTTP_FETCH) @Path("/completions") @Consumes(MediaType.APPLICATION_JSON) @@ -277,7 +305,7 @@ private Response completeWhole(final ResolvedAiContext context, // occasionally a fragment of the prompt, so it is logged and never returned. Logger.error(this, "Chat completion failed for site " + AiHostResolver.sanitize(context.servingSiteId()), e); - return errorResponse(InferenceError.upstream(UPSTREAM_FAILURE_MESSAGE)); + return errorResponse(InferenceError.fromProviderFailure(e, UPSTREAM_FAILURE_MESSAGE)); } } @@ -302,12 +330,17 @@ private Response streamCompletion(final ResolvedAiContext context, ACTIVE_STREAMS.decrementAndGet(); Logger.warn(this, "Refusing a streamed completion: the node is already serving its " + "ceiling of " + maxConcurrentStreams + " concurrent streams"); + // Carries Retry-After because this refusal is dotCMS's own: the wait is until a + // stream on this node finishes, not until some provider decides to let us back in. + // "Retry shortly" in the message is prose a client cannot act on; the header is the + // same instruction in the form every standard client already honours. return errorResponse(new InferenceError( "rate_limit_error", "This node is already serving its ceiling of " + maxConcurrentStreams + " concurrent streamed completions; retry shortly", null, - 429)); + 429), + CAPACITY_RETRY_AFTER_SECONDS); } // Fixed for the whole stream: a client correlates the chunks it stitches together by the @@ -359,7 +392,7 @@ private Response refuseUnconfiguredModel(final ResolvedAiContext context, if (!configuredModels.contains(requestedModel.trim())) { Logger.warn(this, "Site " + AiHostResolver.sanitize(context.servingSiteId()) + " has no chat model matching the requested one"); - return errorResponse(InferenceError.noSuchModel(echoable(requestedModel))); + return errorResponse(InferenceError.noSuchModel(echoable(requestedModel), "chat")); } return null; @@ -440,6 +473,26 @@ private static Response errorResponse(final InferenceError error) { .build(); } + /** + * Renders a refusal that tells the caller when to come back. + * + *

Only for refusals whose duration dotCMS actually knows — its own capacity ceilings. An + * upstream rate limit is not one of those: the provider knows how long it wants to be left + * alone and says so in a header the provider abstraction discards before dotCMS sees it, and + * inventing a number there would be a guess presented to the client as an instruction.

+ * + * @param error the refusal + * @param retryAfterSeconds how long the caller should wait before trying again + * @return the refusal, carrying {@code Retry-After} + */ + private static Response errorResponse(final InferenceError error, final int retryAfterSeconds) { + return Response.status(error.httpStatus()) + .entity(InferenceErrorView.of(error)) + .type(MediaType.APPLICATION_JSON) + .header(RETRY_AFTER_HEADER, retryAfterSeconds) + .build(); + } + /** * Writes stream events out as server-sent event frames, and decides how the stream ends. * diff --git a/dotCMS/src/main/java/com/dotcms/inference/rest/EmbeddingsResource.java b/dotCMS/src/main/java/com/dotcms/inference/rest/EmbeddingsResource.java index d478c1c3c4e8..cc5608f621fd 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/rest/EmbeddingsResource.java +++ b/dotCMS/src/main/java/com/dotcms/inference/rest/EmbeddingsResource.java @@ -13,6 +13,7 @@ import com.dotcms.inference.rest.view.InferenceErrorView; import com.dotcms.rest.WebResource; import com.dotcms.rest.annotation.NoCache; +import com.dotcms.rest.annotation.NoCors; import com.dotmarketing.exception.DotSecurityException; import com.dotmarketing.util.Logger; import com.fasterxml.jackson.databind.JsonNode; @@ -72,6 +73,7 @@ */ @Path("/inference/v1/embeddings") @Tag(name = "AI", description = "AI-powered content generation and analysis endpoints") +@NoCors public class EmbeddingsResource { /** Section of the site's {@code providerConfig} JSON that configures embeddings. */ @@ -232,6 +234,7 @@ private Response embed(final ResolvedAiContext context, Logger.error(this, "The embeddings provider returned " + batch.vectors().size() + " vectors for " + inputs.size() + " inputs on site " + AiHostResolver.sanitize(context.servingSiteId())); + // No exception to inspect here: the provider answered, it just answered short. return errorResponse(InferenceError.upstream(UPSTREAM_FAILURE_MESSAGE)); } @@ -243,7 +246,7 @@ private Response embed(final ResolvedAiContext context, // occasionally a fragment of the input, so it is logged and never returned. Logger.error(this, "Embeddings failed for site " + AiHostResolver.sanitize(context.servingSiteId()), e); - return errorResponse(InferenceError.upstream(UPSTREAM_FAILURE_MESSAGE)); + return errorResponse(InferenceError.fromProviderFailure(e, UPSTREAM_FAILURE_MESSAGE)); } } @@ -300,7 +303,7 @@ private Response refuseUnconfiguredModel(final ResolvedAiContext context, if (!configuredModels.contains(requestedModel.trim())) { Logger.warn(this, "Site " + AiHostResolver.sanitize(context.servingSiteId()) + " has no embeddings model matching the requested one"); - return errorResponse(InferenceError.noSuchModel(echoable(requestedModel))); + return errorResponse(InferenceError.noSuchModel(echoable(requestedModel), "embeddings")); } return null; diff --git a/dotCMS/src/main/java/com/dotcms/inference/rest/ImagesResource.java b/dotCMS/src/main/java/com/dotcms/inference/rest/ImagesResource.java index 21d5ac2ebb7b..aede5ab1f2c7 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/rest/ImagesResource.java +++ b/dotCMS/src/main/java/com/dotcms/inference/rest/ImagesResource.java @@ -14,6 +14,7 @@ import com.dotcms.inference.rest.view.InferenceErrorView; import com.dotcms.rest.WebResource; import com.dotcms.rest.annotation.NoCache; +import com.dotcms.rest.annotation.NoCors; import com.dotmarketing.exception.DotSecurityException; import com.dotmarketing.util.Logger; import com.liferay.portal.model.User; @@ -86,6 +87,7 @@ */ @Path("/inference/v1/images") @Tag(name = "AI", description = "AI-powered content generation and analysis endpoints") +@NoCors public class ImagesResource { /** Section of the site's {@code providerConfig} JSON that configures image generation. */ @@ -298,7 +300,7 @@ private Response generate(final ResolvedAiContext context, // occasionally a fragment of the prompt, so it is logged and never returned. Logger.error(this, "Image generation failed for site " + AiHostResolver.sanitize(context.servingSiteId()), e); - return errorResponse(InferenceError.upstream(UPSTREAM_FAILURE_MESSAGE)); + return errorResponse(InferenceError.fromProviderFailure(e, UPSTREAM_FAILURE_MESSAGE)); } } @@ -328,7 +330,7 @@ private Response refuseUnconfiguredModel(final ResolvedAiContext context, if (!configuredModels.contains(requestedModel.trim())) { Logger.warn(this, "Site " + AiHostResolver.sanitize(context.servingSiteId()) + " has no image model matching the requested one"); - return errorResponse(InferenceError.noSuchModel(echoable(requestedModel))); + return errorResponse(InferenceError.noSuchModel(echoable(requestedModel), "images")); } return null; diff --git a/dotCMS/src/main/java/com/dotcms/inference/rest/InferenceRequestAttributes.java b/dotCMS/src/main/java/com/dotcms/inference/rest/InferenceRequestAttributes.java index 319b5facc87b..3fae3e4a697b 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/rest/InferenceRequestAttributes.java +++ b/dotCMS/src/main/java/com/dotcms/inference/rest/InferenceRequestAttributes.java @@ -5,7 +5,7 @@ * *

The resolved site is published here rather than returned, so the response filter can report * it even on a response the resource never produced — an exception mapped to a 4xx, or a stream - * that failed after it began. FR-020 asks for the serving site on every response, and a + * that failed after it began. The serving site is reported on every response, and a * value only the happy path can set would not deliver that.

*/ public final class InferenceRequestAttributes { diff --git a/dotCMS/src/main/java/com/dotcms/inference/rest/ModelsResource.java b/dotCMS/src/main/java/com/dotcms/inference/rest/ModelsResource.java index e2f2f4a31207..daf87c88ee90 100644 --- a/dotCMS/src/main/java/com/dotcms/inference/rest/ModelsResource.java +++ b/dotCMS/src/main/java/com/dotcms/inference/rest/ModelsResource.java @@ -10,6 +10,7 @@ import com.dotcms.inference.rest.view.ModelListView; import com.dotcms.rest.WebResource; import com.dotcms.rest.annotation.NoCache; +import com.dotcms.rest.annotation.NoCors; import com.dotmarketing.exception.DotSecurityException; import com.dotmarketing.util.Logger; import com.liferay.portal.model.User; @@ -34,6 +35,7 @@ import javax.ws.rs.core.Response; import java.time.Instant; import java.util.ArrayList; +import java.util.LinkedHashSet; import java.util.List; import java.util.Optional; @@ -42,7 +44,7 @@ * *

This endpoint is the discovery mechanism the rest of the family depends on. There is no * implicit default model and no reserved alias, so the names listed here are exactly the names - * {@link ChatCompletionsResource} accepts — a caller who wants "whatever this site runs" reads the + * this family accepts — a caller who wants "whatever this site runs" reads the * list and takes the first entry. Two consequences follow, and both are deliberate.

* *