Repository navigation
feat(detection): yield the event loop, cap per-request scan time, add semantic-only header opt-out - #167
Conversation
…can time, add semantic-only header opt-out Await the shared regex executor in the async engine and yield between pattern slices so one request's scan no longer freezes the event loop (sync mirror keeps blocking). Add detection_max_scan_seconds (default 10.0), a per-request total regex scan budget carried in request-scoped state. When spent, remaining patterns and values are skipped and a weighted pattern_timeout threat (scan_time_budget_exhausted) is recorded like a per-pattern timeout. Add excluded_semantic_detection_headers (and detection_exclusion(semantic_headers=...)): opt-in header names whose values skip the semantic analyzer only; regex scanning is unchanged.
… a request When a per-pattern timeout or the spent scan budget was what pushed a request over the threshold, the trigger named only the first threat, so the block read as an ordinary pattern match. The trigger now appends the first pattern_timeout threat behind a non-timeout first threat. The sync tree is regenerated with scripts/unasync.py.
detection_max_scan_seconds summed the wall time of each pattern, so on a busy or CPU-starved host ordinary requests exceeded the default and failed closed (600 benign query params: 26.7 s wall for 3.7 s of CPU). The budget now accumulates thread CPU time: inline scans measure time.thread_time() deltas, and executor-run scans measure it inside the worker and return it, since contextvars do not cross into the executor. A scan that hits detection_compiler_timeout is charged that timeout. The per-pattern timeout is no longer clamped to the budget left (the units no longer match), and event loop slicing stays wall based. The sync tree is regenerated with scripts/unasync.py.
github-actions
Bot
added
documentation
…tic_detection_headers
This comment has been minimized.
This comment has been minimized.
Detection gate reportAll detection gate shards passed. Shard
|
| Category | Recall | False positives |
|---|---|---|
| cmd_injection | 38/46 (82.6%) | 10/370 (2.7%) |
| cms_probing | 36/36 (100.0%) | 0/370 (0.0%) |
| code_injection | 3/3 (100.0%) | 0/370 (0.0%) |
| deserialization | 18/18 (100.0%) | 0/370 (0.0%) |
| dir_traversal | 10/10 (100.0%) | 0/370 (0.0%) |
| file_inclusion | 16/16 (100.0%) | 3/370 (0.8%) |
| file_upload | 32/32 (100.0%) | 0/370 (0.0%) |
| http_split | 4/4 (100.0%) | 0/370 (0.0%) |
| ldap | 25/25 (100.0%) | 0/370 (0.0%) |
| nosql | 10/10 (100.0%) | 0/370 (0.0%) |
| path_traversal | 13/13 (100.0%) | 0/370 (0.0%) |
| proto_pollution | 8/8 (100.0%) | 0/370 (0.0%) |
| recon | 23/23 (100.0%) | 0/370 (0.0%) |
| sensitive_file | 16/16 (100.0%) | 0/370 (0.0%) |
| sqli | 38/38 (100.0%) | 6/370 (1.6%) |
| ssrf | 32/32 (100.0%) | 4/370 (1.1%) |
| template | 13/13 (100.0%) | 1/370 (0.3%) |
| xml | 4/4 (100.0%) | 0/370 (0.0%) |
| xss | 39/39 (100.0%) | 0/370 (0.0%) |
Full per-case output is in the job log.
Shard builtin-safety
Gate green: no category regressed against its committed baseline and no benign string is newly flagged.
Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.
Full per-case output is in the job log.
Shard cpu-load-builtin-safety
Gate green: no category regressed against its committed baseline and no benign string is newly flagged.
Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.
Full per-case output is in the job log.
Shard hardening-canaries
Gate green: no category regressed against its committed baseline and no benign string is newly flagged.
Total recall: n/a. Total false-positive rate: n/a.
Documented known gaps: 0. Documented known false positives: 0. Both stay in the denominators.
Full per-case output is in the job log.
1 participant
Closes #166
A single request's regex scan could hold the event loop, nothing capped total scan time per request, and a header could not be exempted from semantic scoring alone. Scans now yield the event loop, detection_max_scan_seconds (default 10.0, range 0.1-60.0) caps per-request regex CPU time, and the trigger names the budget when it tips a request. excluded_semantic_detection_headers (opt-in, empty by default, with a route-level counterpart) skips only the semantic analyzer while regex scanning is unchanged. I added a test that the opt-out applies to each Cookie pair.