[release/11.0] Increase spinning/polling aggressiveness in the thread pool in low-saturation scenarios - #132908
Open
github-actions[bot] wants to merge 1 commit into
Open
Conversation
…turation scenarios (#132765) We made some changes in the threadpool to reduce spinning. In particular to reduce fruitless spinning - when a worker thread scanned through the work queue and found no work whatsoever. We would park such thread as a matter of throttling pointless scanning. The change helped in high saturation scenarios as reducing spurious scans reduces waste and lets other threads do useful work. Unfortunately, in some low-saturation scenarios those spurious scans were load bearing. In such scenarios some redundancy in terms of spurious scans must be tolerated to provide good latency. If we park workers too aggressively when we do not have many workers in the first place we will need to rely on waking them up to serve incoming requests. In a bursty case this could be a noticeable regression. In bursty ping-pong kind of scenario, if this happens on both the app and the client ends, the result could be amplified further. Here we are tuning the heuristic that parks threads after spurious scans to be enabled only when we have more than 2/3 of the proc count workers. There could be better ways to make use of this signal selectively and we should explore further. This is a simple enough change that we can do for net11. The change also increases allowed spin time and lowers the max delay between polls to cap the impact on latency from longer spin, if such happens. (it makes sense to have per iteration cap lower than the total, we had it the other way) --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @VSadov |
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #132765 to release/11.0
/cc @VSadov
Customer Impact
Too aggressive parking of threadpool threads in response to spurious wake ups may result in having too few active workers in bursty low saturation scenarios and cause massive regressions.
Some tests like Websockets show nearly 4x less RPS compared to net10.
Regression
Introduced in #128606
The change resultied in improvements in high saturation throughput-sensitive scenarios. It was later discovered that in low staturation latency-sensitive scenarios it resulted in regressions.
Testing
The fix was validated with various benchmarks both high-saturation and low, on both x64 and arm.
Risk
Low.
This is an intentionally small change. The new parking policy will only be enabled when the threadpool has more than 2/3 proc count workers.