fix(errors): stop masking query-level errors as 'database_not_connected' - #448
Merged
Conversation
_classify_error matched the substrings "operationalerror", "role", "does not exist", and a bare "timeout" when routing exceptions to the PostgreSQL setup guide. Those match ordinary query failures, not connection failures: - sqlite3.OperationalError's class name is literally "OperationalError", so type(exc).__name__.lower() == "operationalerror" — EVERY SQLite query error (FTS5 syntax error, "no such table", "database is locked") was classified as database_not_connected and handed a `brew install postgresql@17` guide. On the SQLite backend that guide is doubly wrong, and it buried the real error (a FTS5 syntax error in get_causal_chain surfaced exactly this way during benchmarking). - bare "role" matched "control", "payroll", "role-based"; bare "does not exist" matched `column "x" does not exist` (a query/schema bug); bare "timeout" matched statement/lock timeouts. Tightened the bucket to unambiguous connection/auth phrases only (connection refused, could not connect, could not translate host name, server closed the connection, the database system is starting up, password authentication failed, connection timed out, timeout expired). Genuine create-db/create-role setup conditions remain fully actionable from their own honest `OperationalError: ... FATAL: database "cortex" does not exist` text, which the fall-through returns verbatim. Regression tests pin that FTS5 syntax errors, no-such-table, column-does-not- exist, a "role" substring in unrelated text, and a locked-db error all fall through to their honest '<Type>: <message>', while the seven genuine connection failures still get the setup guide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fnomaXS71hgxMw2zr7HGR
Owner
Author
|
ZETETIC-REVIEW: REQUEST_CHANGES Independent review (fresh-context reviewer, read-only via gh, head 7a0e1db). Sound and kept: the masking diagnosis is correct (SQLite Blocking (one item):
Non-blocking:
|
_classify_error grew to a 58-line AST span (cap: 40 local / 50 hard, High stakes — tool_error_handler.py is imported by 7 production modules registering ~50 tools), flagged blocking in the PR #448 review. The violation was pre-baselined debt the diff-scoped craftsmanship gate could not see growing. Hoist the missing_extension and database_not_connected keyword lists into module-level named constants (_MISSING_EXTENSION_PHRASES, _CONNECTION_FAILURE_PHRASES), moving the connection-failure rationale comment onto its constant. Pure Extract Variable — classification logic and phrase lists are unchanged; span now 21 lines. Prune the now-stale baseline entry for _classify_error (safe_handler's separate, untouched entry stays). Also close the non-blocking truth-table gap: add the three untested database_not_connected phrases (no such host, connection reset, timeout expired) to the genuine-connection-failures test loop. Co-Authored-By: Claude <noreply@anthropic.com>
Owner
Author
|
ZETETIC-REVIEW: APPROVE Delta re-review of commit e74ee3a (same fresh-context reviewer as the prior REQUEST_CHANGES):
Green AND verified — mergeable. |
cdeust
added a commit
that referenced
this pull request
Aug 24, 2026
… drift) origin/main advanced past this branch's fork point (#447 FTS5 fix, #448 error-classification fix) and #448 fixed mcp_server/tool_error_handler.py _classify_error's method-size violation (was in the branch-point baseline; now 21 lines on main, verified by AST), pruning it from main's .craftsmanship-baseline.json. This branch's copy of the file still carried the stale entry, which the gate's ratchet rule flags as "added without a base-ref match" once compared against a freshly fetched origin/main (the ratchet may only shrink within a PR). Not something this PR's own changes caused — reproduced by stashing every change in this branch back to dddc001 and re-running the gate against fresh origin/main; the same failure appears. Synced this branch's baseline to match origin/main's exactly for this one entry (diffed byte-for-byte confirmed no other divergence in either direction) rather than grandfathering it or regenerating blind. Co-Authored-By: Claude <noreply@anthropic.com>
cdeust
added a commit
that referenced
this pull request
Aug 24, 2026
…s instead of hanging (#449) * fix(ap): bound interactive AP read calls so a wedged pipeline degrades The AP MCP client runs with callTimeoutMs=0 — deliberately unbounded, because a fresh index_codebase of a large tree legitimately exceeds any fixed wall clock. The only backstop is the 600s wedge-silence window. That is correct for ingestion but wrong for the interactive read path: search_codebase (behind unified_search) and the get_symbol / get_context / get_impact / get_processes / health_check lookups inherited the same 600s, so an AP that connects but then wedges made unified_search / get_causal_chain hang for up to 10 minutes instead of degrading to Cortex-only results — the "unreachable => status=partial" promise in unified_search's docstring was not actually kept. Fix: APBridge.call gains an optional timeout_s that wraps the client call in asyncio.wait_for and degrades a timeout to None (recorded reason + stderr note), exactly like every other AP failure. The interactive read wrappers pass interactive_call_timeout_s() (30s default, env-overridable via CORTEX_AP_INTERACTIVE_TIMEOUT_S, 20x below the 600s indexing window). The indexing/write wrappers (index_codebase, analyze_codebase, resolve_graph, cluster_graph, detect_changes) and the build-loop query_graph stay unbounded. A None from a timed-out search_codebase flows through as_list(None) -> [] -> unified_search returns status=partial with Cortex-only hits. Tests: a hanging fake client returns None within a 50ms ceiling; the interactive wrapper forwards a positive timeout while index_codebase forwards none; the default ceiling is positive and strictly below the 600s wedge window and honors/validates the env override. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017fnomaXS71hgxMw2zr7HGR * fix(ap): extract APBridge._degrade to clear the 40-line method-size gate APBridge.call spanned 42 lines (repo cap: 40, scripts/craftsmanship_rules.py METHOD_LINE_LIMIT). Both except-branches duplicated the same "record _unavailable_reason + stderr note + return None" degrade logic; pulled into one _degrade(reason, note) helper, called from both branches. Fixes the CI craftsmanship-gate FAILURE on 3debfda (PR #449 review finding 1). Co-Authored-By: Claude <noreply@anthropic.com> * fix(ap): surface AP call failure in unified_search status, not just the config flag unified_search.handler derived status solely from the static is_enabled() config check. An AP that is enabled but times out or errors on the actual search_codebase call returned [] indistinguishably from AP genuinely finding nothing: status="ok", sources=["cortex","ap"], counts.ap=0 either way. This silently hid the exact failure mode #3debfda's timeout fix was meant to expose. WorkflowGraphASTSource gains last_search_degraded_reason, reading the outcome APBridge._degrade already records on its own call path (no widened return type, no second round-trip) — None when the call succeeded or was never attempted (AP disabled / no graph configured), set when it genuinely failed. unified_search now sets status="partial" and a degraded: {source, reason} field whenever AP was attempted but failed, keeping status="ok", degraded=null for a genuine empty AP result. PR #449 review finding 2. Co-Authored-By: Claude <noreply@anthropic.com> * docs(ap): remove false get_causal_chain AP-hang claim from source comments get_causal_chain (mcp_server/handlers/get_causal_chain.py) imports only MemoryStore and core.knowledge_graph — no AP dependency, so it cannot hang on a wedged AP call. mcp_call_timeout.py's interactive-ceiling comments and the interactive-timeout test's module docstring both claimed otherwise; corrected to name unified_search only, with an explicit note that get_causal_chain is unaffected. PR #449 review finding 3 (PR body corrected separately via gh pr edit). Co-Authored-By: Claude <noreply@anthropic.com> * fix(ap): extract search_codebase hit normalization to clear the 300-line/40-line gates The degraded-tracking addition (bd74351) pushed workflow_graph_source_ast.py to 316 lines (cap 300) and search_codebase's body to 45 lines (cap 40) — a NEW craftsmanship violation not in the base-ref baseline, caught by CI on 30b2d20. Local `python3 scripts/check_craftsmanship.py` (no args) at the time compared against a stale local origin/main; re-running after `git fetch origin main` reproduces CI's FAILURE exactly. Root fix: search_codebase's row-normalization loop (id/qualified_name/ file_path/score/snippet construction from the raw AP response) is lifted into workflow_graph_ast_response.normalize_search_hits — the same module that already owns "normalize an AP response shape" for the symbol- and edge-loading concerns (issue #275's split), so this is the same seam, not a new one. search_codebase now just resolves the graph path, calls the bridge, and delegates. Docstrings on search_codebase/last_search_degraded_reason trimmed to essentials. workflow_graph_source_ast.py: 316 -> 288 lines. search_codebase: 45 -> delegates (no longer a > 40-line method). Behavior-preserving; targeted tests (ap_bridge/mcp_client/workflow_graph + the two new degraded-status test files + response_budget_wiring) all still green. Co-Authored-By: Claude <noreply@anthropic.com> * fix(ap): reset APBridge._unavailable_reason at the top of every call() _unavailable_reason was only cleared in connect()'s slow (reconnect) path. connect()'s fast path (client already connected) skipped the clear, and call()'s success branch never reset it either. On a reused instance — the codebase already does this, wiki_verify.py reuses one WorkflowGraphASTSource across its candidate loop — a stale reason from an earlier failed call leaked into a later successful one, so last_search_degraded_reason could report a call as failed when it had actually just succeeded. Contradicted its own docstring ("None if it succeeded"). Fix: call() resets self._unavailable_reason = None right after the tool-allowlist check, before touching connect()/the RPC — each call's own outcome is now authoritative regardless of prior calls on the same bridge instance. call() stays at 39 lines (cap 40). New regression test: two calls on one APBridge instance, first via a hanging client (fails), second via a client that succeeds, asserting unavailable_reason is set after the first and None after the second. Fails on pre-fix code (verified via git stash), passes after. PR #449 review round 3 (Medium finding, house "no deferred coverage on new code" rule). Co-Authored-By: Claude <noreply@anthropic.com> * chore(craftsmanship): sync baseline ratchet to origin/main (unrelated drift) origin/main advanced past this branch's fork point (#447 FTS5 fix, #448 error-classification fix) and #448 fixed mcp_server/tool_error_handler.py _classify_error's method-size violation (was in the branch-point baseline; now 21 lines on main, verified by AST), pruning it from main's .craftsmanship-baseline.json. This branch's copy of the file still carried the stale entry, which the gate's ratchet rule flags as "added without a base-ref match" once compared against a freshly fetched origin/main (the ratchet may only shrink within a PR). Not something this PR's own changes caused — reproduced by stashing every change in this branch back to dddc001 and re-running the gate against fresh origin/main; the same failure appears. Synced this branch's baseline to match origin/main's exactly for this one entry (diffed byte-for-byte confirmed no other divergence in either direction) rather than grandfathering it or regenerating blind. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_classify_errorrouted exceptions to the PostgreSQL setup guide (database_not_connected) when their text matched any of"operationalerror","role","does not exist", or a bare"timeout". Those substrings match ordinary query-level failures, not connection failures.The worst offender:
sqlite3.OperationalError's class name is literallyOperationalError, sotype(exc).__name__.lower() == "operationalerror". Every SQLite query error — a FTS5syntax error,no such table,database is locked— matched and was handed abrew install postgresql@17guide. On the SQLite backend that guide is doubly wrong, and it buried the real error.This is how it was found: while running the harness-comparison benchmark on a real SQLite store, a FTS5 syntax error in
get_causal_chain(fixed in #447) came back to the client as "Cortex could not connect to PostgreSQL" — sending diagnosis in exactly the wrong direction.The other keywords were similarly broad:
"role"matchedcontrol/payroll/role-based;"does not exist"matchedcolumn "x" does not exist(a query/schema bug); bare"timeout"matched statement/lock timeouts.Fix
Tighten the
database_not_connectedbucket to unambiguous connection/auth phrases only:Genuine create-db/create-role first-run conditions stay fully actionable from their own honest
OperationalError: ... FATAL: database "cortex" does not existtext, which the fall-through (type(exc).__name__, str(exc)) returns verbatim — precision over hand-holding, so a real bug is never masked.Tests
Added
TestQueryErrorsAreNotMaskedAsDbNotConnected: FTS5 syntax error,no such table,column "x" does not exist, arolesubstring in unrelated text, and a locked-db error all fall through to<Type>: <message>; a companion test asserts the seven genuine connection failures still get the setup guide. The pre-existingtest_generic_connection_refused_still_classified_as_db_not_connectedcontinues to pass unchanged.Gates:
ruff check✓ ·ruff format --check✓ · craftsmanship gate ✓ ·pytest tests_py/server/test_tool_error_handler_classification.py→ 13 passed.Pairs with #447 (the FTS5 crash this bug was hiding).
🤖 Generated with Claude Code
Generated by Claude Code