Fix uncaught pydantic ValidationError crash on malformed LLM response#255
Fix uncaught pydantic ValidationError crash on malformed LLM response#255Ruanyuxi1337 wants to merge 1 commit into
Conversation
|
Thanks for taking this on. One gap worth covering before merge: this only wraps the async In that sync loop, a malformed structured response from one batch still raises out of So I think the same per-batch isolation should be added to |
keshprad
left a comment
There was a problem hiding this comment.
Requesting changes because the async retry still aborts the scan after two malformed responses and changes the documented handling of ordinary ValueError. I also agree with @koriyoshi2041 that the synchronous run_batches() path used by semantic_security_discovery remains fail-fast and needs equivalent per-batch isolation. Please add regressions for persistent malformed async responses, ordinary ValueError propagation, and a three-batch sync run with a malformed middle response.
| batch.file_label, | ||
| retry_exc, | ||
| ) | ||
| raise retry_exc |
There was a problem hiding this comment.
A persistent malformed response still reproduces #250 here. Pydantic ValidationError subclasses ValueError, so after the retry re-raises it, the aggregation loop matches it as ValueError and aborts the entire scan. Please handle exhausted Pydantic validation failures as an isolated batch failure while preserving propagation for ordinary configuration ValueError, and add a regression where both attempts are malformed.
| except NotImplementedError as exc: | ||
| # Do not retry on configuration / code errors | ||
| raise exc | ||
| except Exception as exc: |
There was a problem hiding this comment.
This broad catch changes the documented ValueError contract. If the first call raises an ordinary configuration ValueError (for example, a missing API key) and the retry succeeds, arun_batches() returns successfully instead of propagating the configuration error. Please catch Pydantic ValidationError explicitly for retry and let non-Pydantic ValueError bypass retry.
Fixes #250. Wraps the per-batch LLM request and parse loop in a try-except guard. If a parsing or network validation failure occurs, it warns, retries exactly once, and falls back gracefully, preventing the entire scan from crashing and losing all report progress.