fix(core): stop rejecting requests based on global wait estimates

[poroh]

Admission control schedules separate per-client queues using round-robin, but the global wait estimator treated all pending requests as one flat FIFO. As a result, non-runnable backlog from one client could cause EstimatedQueueDelay(Global) for an unrelated client whose actual wait did not depend on that backlog.

Predictive rejection now uses only the incoming client’s FIFO queue. Global concurrency and pending limits, the actual pending timeout, and global EWMA-based retry advice remain unchanged. A regression test covers the issue under global saturation.

Related issues

Introduced by #4562
Fixes: #5662

Type of Change

• Add - New feature or capability
• Change - Changes in existing functionality
• Fix - Bug fixes
• Remove - Removed features or deprecated functionality
• Internal - Internal changes (refactoring, tests, docs, etc.)

Breaking Changes

• This PR contains breaking changes

Testing

• Unit tests added/updated
• Integration tests added/updated
• Manual testing performed
• No testing required (docs, internal refactor, etc.)

Additional Notes

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 85882bec-61bb-48b4-a354-d6e68d376e49

📥 Commits

Reviewing files that changed from the base of the PR and between eafb6b9 and 0b60919.

📒 Files selected for processing (2)

• crates/api-core/src/admission/engine.rs
• crates/api-core/src/admission/peak_ewma.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Summary by CodeRabbit

• Bug Fixes

  • Improved admission fairness so a blocked client’s backlog does not prevent other eligible clients from being admitted when capacity becomes available.
  • Refined estimated wait handling to reflect each client’s own queue, avoiding predictive rejections based solely on overall system pressure.
  • Retry guidance now accounts for both client-specific and overall capacity conditions without imposing additional execution limits.

• Documentation

  • Clarified queue-delay estimates, admission decisions, and retry behavior.

Walkthrough

The admission engine no longer rejects requests from global queue-delay estimates. Per-client FIFO estimates still reject requests that exceed the pending timeout. Tests verify admission for an independent client despite unrelated backlog.

Changes

Admission queue-delay evaluation

Layer / File(s)	Summary
Per-client predictive rejection crates/api-core/src/admission/engine.rs, crates/api-core/src/admission/peak_ewma.rs	Predictive queue-delay rejection now uses the requesting client’s FIFO. Global pressure remains available for retry advice and does not impose execution limits.
Independent client admission coverage crates/api-core/src/admission/engine.rs	Tests poll pending futures and verify that an independent client receives capacity when another client has a non-runnable backlog.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 0b609

The change limits predictive queue-delay rejection to the requesting client while preserving global capacity limits, timeouts, and retry behavior. No actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary fix: stopping predictive rejection based on global wait estimates.
Description check	✅ Passed	The description accurately explains the global wait-estimation bug, the client-local predictive rejection fix, preserved limits, and regression testing.
Linked Issues check	✅ Passed	The changes satisfy issue #5662 by removing global predictive queue-delay rejection, preserving per-client rejection and hard limits, retaining pending timeouts and global EWMA retry advice, and addin…
Out of Scope Changes check	✅ Passed	The code, documentation, and tests directly support the linked issue and stated objectives. No unrelated changes are evident.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files.

Full details: Linked Issues check

Explanation

The changes satisfy issue #5662 by removing global predictive queue-delay rejection, preserving per-client rejection and hard limits, retaining pending timeouts and global EWMA retry advice, and adding regression coverage.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}