fix(memory): skip unhealthy query embeddings

docs/architecture/memory-quality-gates-and-observability/metrics.md

@@ -18,6 +18,8 @@ SQL, stacks, or other free text.
 | `agent.retrieval.*.{ftsCandidates,vectorCandidates,selected}` | Count | Authoritative revalidation and fusion completion | Per-Agent counter; same Agent cleanup | Aggregate count |
 | `agent.retrieval.*.outcomeCounts.*` | Count by closed outcome | Operation `finally` | Per-Agent counter; same Agent cleanup | Closed enum |
 | `agent.retrieval.*.degradationCounts.*` | Count by closed cause | Operation `finally`; one operation may record multiple causes | Per-Agent counter; same Agent cleanup | Closed enum |
+| `agent.queryEmbeddingCircuit.state` | Closed state (`closed`, `open`, `halfOpen`) | Query-embedding circuit transition | Current provider/model only; reset by configuration change and Agent cleanup | Closed enum |
+| `agent.queryEmbeddingCircuit.{failures,openCount,skipped}` | Count | Qualifying provider failure, open transition, or skipped query embedding | Per-Agent counter; same configuration and Agent cleanup | Aggregate count |
 | `agent.extraction.{chunksCompleted,chunksCancelled,chunksFailed,llmCalls,casRetries}` | Count | Chunk settlement or actual second CAS apply | Per-Agent counter; same Agent cleanup | Aggregate count |
 | `agent.embedding.batchSize` | Row distribution | Embedding drain batch settlement | 256 samples; same Agent cleanup | Aggregate count |
 | `agent.embedding.drainDurationMs` | Millisecond distribution | Embedding drain batch settlement | 256 samples; same Agent cleanup | Content-free timing |

docs/architecture/memory-quality-gates-and-observability/spec.md

@@ -102,6 +102,8 @@ metric calculation remain isolated from credentials, providers, and external net
 - Recorders accept only numbers, booleans, timestamps, and closed enums.
 - Diagnostics never retain query text, memory content, prompts, vectors, provider responses, API keys, SQL,
   stacks, exception messages, or other free text.
+- Query-embedding circuit diagnostics retain only closed/open/half-open state and aggregate failure, open, and
+  skip counts; provider/model identity remains internal to the circuit owner.
 - Collector failures are swallowed and cannot change business results.
 
 ### AC-6 — Metric Semantics
@@ -117,6 +119,8 @@ metric calculation remain isolated from credentials, providers, and external net
 - Maintenance reports cheap/heavy outcomes, calls, tokens, and every denied budget step.
 - Vector warmup distinguishes succeeded, deferred, and failed outcomes.
 - Provider diagnostics separate admission decisions from deadline, abort, and late-settle race events.
+- Query-embedding circuit skips use a closed retrieval degradation cause; cancellation and local control
+  rejection never increment its provider-health failure count.
 - Process gauges receive absolute values from resource owners and never aggregate retained Agent state.
 
 ### AC-7 — Typed Health Contract and UI

docs/architecture/memory-system.md

@@ -93,6 +93,12 @@ Memory contribution 必须等待到 soft deadline，成功时限制 token/字符
 文本、selection manifest 与成功持久化的 `memory/view_assembled` anchor ID；不能接收或重写 base
 system prompt。
 
+Warm recall 的 query embedding 按 Agent 与当前 provider/model identity 使用进程内有界熔断：短窗口内
+连续 deadline/transport failure 会临时跳过 vector path 并直接使用已生成的 FTS candidates；冷却后只
+允许一个 half-open probe，成功自动恢复。取消和本地 capacity rejection 不计 provider health failure；
+配置切换、Agent cleanup 与 presenter disposal 清除旧状态。熔断不应用于 embedding batch、warmup、
+dimension discovery 或 text generation，也不改变健康路径的 scoring。
+
 FTS 在 SQL `LIMIT` 前应用 Agent 和 scope predicate。Vector store 仍按 Agent namespace 查询，使用
 有上限的 oversampling，并在 ranking 前通过 SQLite authoritative row 重新校验 owner、scope、
 lifecycle、revision 和 embedding identity；不得依赖 vector candidate 本身做授权判断。过滤后不足

docs/issues/memory-query-embedding-circuit-breaker/spec.md

@@ -0,0 +1,107 @@
+# Memory Query Embedding Circuit Breaker
+
+Status: implemented and validated.
+
+GitHub issue: [#2118](https://github.com/ThinkInAIXYZ/deepchat/issues/2118)
+
+## Issue
+
+Warm Memory recall always starts query embedding when the vector store is ready. The provider
+gateway and retrieval service bound that call to 800 ms and correctly degrade to FTS, but neither
+owner retains provider health across turns. A repeatedly slow embedding path therefore adds the
+same pre-stream delay to every Agent turn even though FTS candidates are already available.
+
+## Impact
+
+- Every affected turn pays a predictable first-token delay before the existing FTS fallback.
+- Requests that are already known to be unhealthy add provider load and repeated warnings.
+- Agent and benchmark latency includes avoidable query-embedding deadlines rather than useful work.
+- The successful final answer hides the failure mode from users and makes it difficult to diagnose.
+
+## Root Cause
+
+- `MemoryProviderGateway` owns per-request deadlines, cancellation, admission, and capacity, but it
+  intentionally has no cross-request policy and serves all Memory provider purposes.
+- `RetrievalService` owns query-embedding de-duplication and FTS degradation, but its retained state
+  records only in-flight requests. A settled timeout is forgotten before the next turn.
+- Runtime diagnostics count individual deadlines and retrieval degradations but expose no current
+  query-embedding health state or circuit skips.
+
+## Fix Plan
+
+Keep the circuit in `RetrievalService`, immediately before query embedding starts. This is the
+narrowest owner that can skip only query vectors while preserving the already-computed FTS path.
+Do not put the policy in the shared provider gateway, where it could suppress embedding batches,
+warmup, dimension discovery, extraction, decisions, or maintenance.
+
+For each Agent, retain at most the current effective embedding provider/model state:
+
+- open after two deadline or transport failures in a 30-second window;
+- while open, skip new query embedding and continue with FTS immediately;
+- after a 30-second cooldown, admit one half-open probe and skip concurrent turns;
+- close and reset the failure window after a successful probe;
+- reopen after a failed probe without exponential or unbounded cooldown growth.
+
+Memory cancellation, generic `AbortError`, and local capacity rejection do not count as provider
+health failures. Shared in-flight requests settle circuit health only once. Configuration changes,
+Agent cleanup, and service disposal remove stale circuit and in-flight state.
+
+Expose only the closed/open/half-open state and cumulative failure, open, and skip counts through
+the existing bounded Agent diagnostics. Add a closed retrieval degradation for circuit skips so an
+injection manifest can report that vector recall was intentionally bypassed. Do not retain query
+text, memory content, vectors, provider/model identifiers, responses, errors, or other free text in
+the circuit or diagnostics.
+
+## Compatibility And Non-Goals
+
+- Preserve healthy FTS/vector scoring, candidate limits, adaptive refill, and access accounting.
+- Preserve the 800 ms per-request absolute and soft deadlines.
+- Do not change provider retry or rate-limit policy.
+- Do not persist circuit state or add a database migration.
+- Do not share failures across Agents, provider/model identities, or non-query provider purposes.
+- Do not add an external telemetry or polling path.
+
+## Acceptance Criteria
+
+1. Two qualifying failures open the circuit, and later turns return existing FTS results without
+   starting or waiting for query embedding.
+2. Exactly one probe is admitted after cooldown; success closes the circuit and restores vector
+   recall, while probe failure reopens it.
+3. Agent/provider-model isolation is preserved, and provider changes, Agent cleanup, and disposal
+   invalidate stale state.
+4. Cancellation and local control errors do not increment failure counts or open the circuit.
+5. Health diagnostics expose current closed/open/half-open state and content-free failure, open, and
+   skip counts; skipped retrieval records a closed degradation cause.
+6. Deterministic fake-timer tests cover threshold, cooldown, recovery, isolation, cleanup, and
+   cancellation without changing healthy retrieval results.
+
+## Task Checklist
+
+- [x] Validate the issue against the production recall, provider, diagnostics, and lifecycle paths.
+- [x] Implement the bounded query-embedding circuit and lifecycle invalidation.
+- [x] Extend typed content-free diagnostics and maintained Memory documentation.
+- [x] Add focused fake-clock regression coverage.
+- [x] Run formatting, i18n, lint, type checking, and relevant Memory/renderer tests.
+- [x] Review the staged commit for side effects, compatibility, boundaries, performance, security,
+  naming, coverage, and maintenance cost before committing.
+
+## Validation
+
+Completed on 2026-08-10:
+
+```bash
+pnpm run i18n
+pnpm run lint
+pnpm run typecheck
+pnpm run format:check
+pnpm run test:memory
+pnpm run test:main
+pnpm exec vitest run --config vitest.config.renderer.ts \
+  test/renderer/components/MemoryDiagnosticsPanel.test.ts
+```
+
+- Memory suite: 54 files, 888 tests passed.
+- Main suite: 541 files and 6,622 tests passed; 28 files and 371 tests skipped by the existing
+  suite configuration.
+- Renderer diagnostics panel: 1 file, 22 tests passed.
+- Formatting, i18n validation, lint, and Node/Web type checking passed.

src/main/memory/index.ts

@@ -458,6 +458,7 @@ export class MemoryService implements MemoryRuntimePort {
       agentId,
       resolveMemoryEmbedding(config)
     )
+    if (embeddingIdentityChanged) this.retrieval.onEmbeddingConfigChanged(agentId)
     if (embeddingIdentityChanged && observation !== 'changed') {
       this.runtime.invalidateAgentOperations(agentId)
     }
@@ -742,7 +743,10 @@ export class MemoryService implements MemoryRuntimePort {
   }
 
   getHealth(agentId: string): MemoryHealthDto {
-    return this.management.getHealth(agentId)
+    const health = this.management.getHealth(agentId)
+    health.runtime.agent.queryEmbeddingCircuit.state =
+      this.retrieval.getQueryEmbeddingCircuitState(agentId)
+    return health
   }
 
   async deleteMemory(agentId: string, memoryId: string): Promise<MemoryCommandResult> {

src/main/memory/infra/diagnostics/memoryDiagnosticsCollector.ts

@@ -74,6 +74,12 @@ type RetrievalDiagnosticsState = {
 type AgentDiagnosticsState = {
   lastTouchedAt: number
   retrieval: Record<MemoryRetrievalPurpose, RetrievalDiagnosticsState>
+  queryEmbeddingCircuit: {
+    state: 'closed' | 'open' | 'halfOpen'
+    failures: number
+    openCount: number
+    skipped: number
+  }
   extraction: {
     chunksCompleted: number
     chunksCancelled: number
@@ -173,6 +179,36 @@ export class MemoryDiagnosticsCollector {
     })
   }
 
+  recordQueryEmbeddingCircuitEvent(
+    agentId: string,
+    event: 'failure' | 'opened' | 'halfOpen' | 'closed' | 'probeCancelled' | 'skipped'
+  ): void {
+    this.safely(() => {
+      const circuit = this.agent(agentId).queryEmbeddingCircuit
+      if (event === 'failure') circuit.failures += 1
+      else if (event === 'opened') {
+        circuit.state = 'open'
+        circuit.openCount += 1
+      } else if (event === 'halfOpen') circuit.state = 'halfOpen'
+      else if (event === 'closed') circuit.state = 'closed'
+      else if (event === 'probeCancelled') circuit.state = 'open'
+      else circuit.skipped += 1
+    })
+  }
+
+  resetQueryEmbeddingCircuit(agentId: string): void {
+    this.safely(() => {
+      const state = this.agents.get(agentId)
+      if (!state) return
+      state.queryEmbeddingCircuit = {
+        state: 'closed',
+        failures: 0,
+        openCount: 0,
+        skipped: 0
+      }
+    })
+  }
+
   recordExtraction(agentId: string, sample: MemoryExtractionDiagnosticSample): void {
     this.safely(() => {
       const counters = this.agent(agentId).extraction
@@ -300,6 +336,7 @@ export class MemoryDiagnosticsCollector {
               ]
             })
           ) as MemoryRuntimeDiagnosticsDto['agent']['retrieval'],
+          queryEmbeddingCircuit: { ...state.queryEmbeddingCircuit },
           extraction: { ...state.extraction },
           embedding: {
             batchSize: distribution(state.embedding.batchSize.snapshot()),
@@ -392,6 +429,12 @@ export class MemoryDiagnosticsCollector {
       retrieval: Object.fromEntries(
         MEMORY_RETRIEVAL_PURPOSES.map((purpose) => [purpose, retrievalState()])
       ) as Record<MemoryRetrievalPurpose, RetrievalDiagnosticsState>,
+      queryEmbeddingCircuit: {
+        state: 'closed',
+        failures: 0,
+        openCount: 0,
+        skipped: 0
+      },
       extraction: {
         chunksCompleted: 0,
         chunksCancelled: 0,

src/main/memory/runtimeConstants.ts

@@ -67,6 +67,9 @@ export const MEMORY_HEALTH_RECENT_FAILURES_LIMIT = 5
 export const MEMORY_CREATED_IDS_EVENT_LIMIT = 50
 
 export const RECALL_QUERY_EMBEDDING_TIMEOUT_MS = 800
+export const RECALL_QUERY_EMBEDDING_BREAKER_FAILURE_THRESHOLD = 2
+export const RECALL_QUERY_EMBEDDING_BREAKER_FAILURE_WINDOW_MS = 30 * 1000
+export const RECALL_QUERY_EMBEDDING_BREAKER_COOLDOWN_MS = 30 * 1000
 export const RECALL_QUERY_EMBEDDING_STALE_MS = 30 * 1000
 export const RECALL_QUERY_EMBEDDING_MAX_CONCURRENT = 2
 export const RECALL_VECTOR_QUERY_TIMEOUT_MS = 2 * 1000