Add opt-in completion metadata and harden prompt delimiters

README.md

@@ -96,6 +96,56 @@ candidate's custom metadata.
 Use `Success` to distinguish outcomes, not the presence of a value. Rejected data and response text
 may contain sensitive output; do not log or serialize them automatically.
 
+### Optional completion metadata
+
+The existing `ILanguageModel.CompleteAsync` method still returns `Task<string>`.
+Existing model implementations and callers do not need to change. Models can opt into
+[ILanguageModelWithMetadata](./src/typechat/ILanguageModelWithMetadata.cs) to return completion
+text together with [CompletionInfo](./src/typechat/CompletionInfo.cs):
+
+```cs
+ILanguageModel model = new LanguageModel(config);
+string text = await model.CompleteAsync(prompt, settings, cancelToken);
+LanguageModelResponse completion = await model.CompleteWithMetadataAsync(prompt, settings, cancelToken);
+TokenUsage? usage = completion.Info?.Usage;
+```
+
+The extension method uses the optional capability when available. Otherwise it calls the original
+text-only method and returns null metadata. The REST provider supplies normalized metadata and the
+raw API response. The Microsoft.Extensions.AI provider supplies the model, usage, and finish reason
+reported by its client. Semantic Kernel and other text-only implementations use the fallback;
+missing usage is not reported as zero.
+
+For translation results with metadata, use the additive `TranslateToResultAsync` API:
+
+```cs
+Result<CalendarActions> result = await translator.TranslateToResultAsync(requestText);
+if (result.Success)
+{
+    CalendarActions actions = result.Value;
+    TokenUsage? finalCompletionUsage = result.Info?.Usage;
+}
+else
+{
+    string? validationError = result.Message;
+    FailureInfo? rejectedOutput = result.Failure;
+}
+```
+
+`Info` describes the final completion on success or validation failure. `RepairAttempts` counts
+repairs actually performed; token usage is for that final completion, **not** the sum of all
+attempts. Provider metadata is copied before repair counts are attached, and no shared
+"last response" state is used.
+
+Only exhausted validation/repair failures become failed results. Transport errors, cancellation,
+and unexpected exceptions still propagate, with the failure context described above when available.
+The original `TranslateAsync` methods keep their throwing behavior and still call the protected
+`GetResponseAsync` hook returning `Task<string>`. Subclasses customizing the new opt-in translation
+path should override `GetResponseWithMetadataAsync` instead.
+
+User requests and validation errors in repair prompts are encoded as JSON strings. This hardens
+prompt delimiters; it is not a guarantee against prompt injection.
+
 ## Microsoft.TypeChat.Program ##
 TypeChat.Program translates natural language requests into simple programs (***Plans***), represented as JSON. 
 
@@ -265,6 +315,18 @@ config = OpenAIConfig.FromEnvironment();
 var model = new LanguageModel(config);
 ```
 
+### Responses API
+
+`LanguageModel` infers the Responses API from an endpoint whose path ends in `/responses`.
+Set `OpenAIConfig.UseResponsesApi` to explicitly select Responses (`true`) or Chat Completions
+(`false`); leave it null to infer the API from the endpoint.
+
+For Azure, prefer a v1 endpoint such as
+`https://YOUR_RESOURCE_NAME.openai.azure.com/openai/v1/responses`, with `Model` set to the
+deployment name. Switching between the v1 APIs preserves the v1 path. For older resource-scoped
+Azure endpoints, enabling Responses also requires an `ApiVersion` that supports Responses,
+such as `2025-04-01-preview`; the legacy default API version is not suitable.
+
 ## Using Semantic Kernel
 You can also access a LanguageModel using the SemanticKernel Kernel object you created using a KernelBuilder.
 ```cs

src/typechat.meai/ChatLanguageModel.cs

@@ -7,7 +7,7 @@ namespace Microsoft.TypeChat;
 /// <summary>
 /// A LanguageModel that uses IChatClient
 /// </summary>
-public class ChatLanguageModel : ILanguageModel
+public class ChatLanguageModel : ILanguageModelWithMetadata
 {
     private readonly IChatClient _chatClient;
     private readonly ModelInfo _model;
@@ -38,12 +38,60 @@ public ChatLanguageModel(IChatClient chatClient, ModelInfo model)
     /// <param name="cancelToken"></param>
     /// <returns></returns>
     public async Task<string> CompleteAsync(Prompt prompt, TranslationSettings? settings = null, CancellationToken cancelToken = default)
+    {
+        LanguageModelResponse response = await CompleteWithMetadataAsync(prompt, settings, cancelToken).ConfigureAwait(false);
+        return response.Text;
+    }
+
+    /// <summary>
+    /// Return a completion and metadata reported by the chat client.
+    /// </summary>
+    public async Task<LanguageModelResponse> CompleteWithMetadataAsync(Prompt prompt, TranslationSettings? settings = null, CancellationToken cancelToken = default)
     {
         List<ChatMessage> history = ToHistory(prompt);
         ChatOptions? options = ToRequestSettings(settings);
         var response = await _chatClient.GetResponseAsync(history, options: options, cancellationToken: cancelToken).ConfigureAwait(false);
 
-        return response.Text;
+        return new LanguageModelResponse(response.Text, BuildInfo(response));
+    }
+
+    private static CompletionInfo BuildInfo(ChatResponse response)
+    {
+        var info = new CompletionInfo
+        {
+            Model = response.ModelId,
+            FinishReason = MapFinishReason(response.FinishReason)
+        };
+        UsageDetails? usage = response.Usage;
+        if (usage is not null)
+        {
+            info.Usage = new TokenUsage(
+                (int)(usage.InputTokenCount ?? 0),
+                (int)(usage.OutputTokenCount ?? 0),
+                (int)(usage.TotalTokenCount ?? 0));
+        }
+        return info;
+    }
+
+    private static CompletionFinishReason? MapFinishReason(ChatFinishReason? reason)
+    {
+        if (reason is null)
+        {
+            return null;
+        }
+        switch (reason.Value.Value?.ToLowerInvariant())
+        {
+            case "stop":
+                return CompletionFinishReason.Stop;
+            case "length":
+                return CompletionFinishReason.Length;
+            case "content_filter":
+                return CompletionFinishReason.ContentFilter;
+            case "tool_calls":
+                return CompletionFinishReason.ToolCalls;
+            default:
+                return CompletionFinishReason.Other;
+        }
     }
 
     private List<ChatMessage> ToHistory(Prompt prompt)

src/typechat/CompletionFinishReason.cs

@@ -0,0 +1,26 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+namespace Microsoft.TypeChat;
+
+/// <summary>
+/// Normalized reason a completion stopped generating, mapped from the underlying API so callers can
+/// detect conditions such as truncation (<see cref="Length"/>) or filtered output
+/// (<see cref="ContentFilter"/>) without special-casing each API variant.
+/// </summary>
+public enum CompletionFinishReason
+{
+    /// <summary>The model stopped naturally (Chat Completions "stop" / Responses "completed").</summary>
+    Stop,
+
+    /// <summary>Output was truncated by a token limit (Chat "length" / Responses "max_output_tokens").</summary>
+    Length,
+
+    /// <summary>Output was withheld or truncated by a content filter.</summary>
+    ContentFilter,
+
+    /// <summary>The model emitted tool/function calls.</summary>
+    ToolCalls,
+
+    /// <summary>A provider-specific reason that doesn't map to one of the common values.</summary>
+    Other,
+}

src/typechat/CompletionInfo.cs

@@ -0,0 +1,47 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+using System.Text.Json;
+
+namespace Microsoft.TypeChat;
+
+/// <summary>
+/// Metadata about a single completion. Translation results carry the final completion's metadata,
+/// on success or validation failure. Usage does not include earlier repair attempts.
+/// </summary>
+public class CompletionInfo
+{
+    /// <summary>
+    /// The model that actually produced the completion, as reported by the API. This can differ from
+    /// the requested model (for example, when an alias resolves to a dated model version).
+    /// </summary>
+    public string? Model { get; set; }
+
+    /// <summary>
+    /// Normalized token usage for the request, when the API reports it.
+    /// </summary>
+    public TokenUsage? Usage { get; set; }
+
+    /// <summary>
+    /// Normalized reason the completion stopped, when the API reports it.
+    /// </summary>
+    public CompletionFinishReason? FinishReason { get; set; }
+
+    /// <summary>
+    /// Number of repair attempts TypeChat made before producing this result (0 = succeeded on the
+    /// first attempt). Populated by <see cref="JsonTranslator{T}"/>, not by the language model.
+    /// </summary>
+    public int? RepairAttempts { get; set; }
+
+    /// <summary>
+    /// The raw, unmodified response body returned by the API, when available. Use this to access
+    /// provider- or API-specific fields that are not normalized onto the other properties.
+    /// </summary>
+    public JsonElement? Raw { get; set; }
+
+    internal CompletionInfo WithRepairAttempts(int repairAttempts)
+    {
+        var result = (CompletionInfo)MemberwiseClone();
+        result.RepairAttempts = repairAttempts;
+        return result;
+    }
+}

src/typechat/HttpEx.cs

@@ -50,6 +50,51 @@ internal static async Task<Response> GetJsonResponseAsync<Request, Response>(thi
         return default;
     }
 
+    /// <summary>
+    /// Like <see cref="GetJsonResponseAsync"/>, but also returns the raw (unparsed) JSON response body
+    /// so callers can surface it (for example, on <see cref="CompletionInfo.Raw"/>).
+    /// </summary>
+    internal static async Task<(TResponse Response, string Raw)> GetJsonResponseWithRawAsync<TRequest, TResponse>(this HttpClient client, string endpoint, TRequest request, int maxRetries, int retryPauseMs, string? apiToken = null)
+    {
+        int retryCount = 0;
+        while (true)
+        {
+            HttpRequestMessage httpRequest = new HttpRequestMessage(HttpMethod.Post, endpoint)
+            {
+                Content = Json.ToJsonMessage(request)
+            };
+            try
+            {
+                if (!string.IsNullOrEmpty(apiToken))
+                {
+                    httpRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", apiToken);
+                }
+                HttpResponseMessage response = await client.SendAsync(httpRequest).ConfigureAwait(false);
+                if (response.StatusCode == HttpStatusCode.OK)
+                {
+                    string raw = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
+                    return (Json.Parse<TResponse>(raw), raw);
+                }
+                if (!response.StatusCode.IsTransientError() || retryCount >= maxRetries)
+                {
+                    // Let HttpClient throw an exception
+                    response.EnsureSuccessStatusCode();
+                    break;
+                }
+                if (retryPauseMs > 0)
+                {
+                    await Task.Delay(retryPauseMs).ConfigureAwait(false);
+                }
+                retryCount++;
+            }
+            finally
+            {
+                httpRequest.Dispose();
+            }
+        }
+        return default;
+    }
+
     internal static bool IsTransientError(this HttpStatusCode status)
     {
         switch (status)

src/typechat/ILanguageModel.cs

@@ -19,6 +19,6 @@ public interface ILanguageModel
     /// <param name="prompt">prompt</param>
     /// <param name="settings">translation settings such as temperature</param>
     /// <param name="cancelToken">cancellation token</param>
-    /// <returns></returns>
+    /// <returns>The completion text.</returns>
     Task<string> CompleteAsync(Prompt prompt, TranslationSettings? settings, CancellationToken cancelToken);
 }

src/typechat/ILanguageModelWithMetadata.cs

@@ -0,0 +1,19 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+namespace Microsoft.TypeChat;
+
+/// <summary>
+/// Optional capability for models that return completion metadata in addition to text.
+/// Existing models only need to implement <see cref="ILanguageModel"/>.
+/// </summary>
+public interface ILanguageModelWithMetadata : ILanguageModel
+{
+    /// <summary>
+    /// Get completion text and any metadata reported by the model.
+    /// </summary>
+    /// <param name="prompt">prompt</param>
+    /// <param name="settings">translation settings such as temperature</param>
+    /// <param name="cancelToken">cancellation token</param>
+    /// <returns>The completion text and optional metadata.</returns>
+    Task<LanguageModelResponse> CompleteWithMetadataAsync(Prompt prompt, TranslationSettings? settings = null, CancellationToken cancelToken = default);
+}

src/typechat/JsonResponse.cs

@@ -82,10 +82,10 @@ public static JsonResponse Parse(string responseText)
             // Prologue precedes Json
             response.Prologue = responseText.Substring(0, iJsonStartAt);
         }
-        if (iJsonEndAt < 0)
+        if (iJsonEndAt < iJsonStartAt)
         {
             // Incomplete json. Treat remaining text as epilogue
-            response.Json = responseText.Substring(iJsonStartAt, lastIndex - iJsonStartAt);
+            response.Json = responseText.Substring(iJsonStartAt);
             return response;
         }