Performance: Implement worker auto-scaler

[AJenbo]

This implements worker auto scaling. Rather then simply setting a fixed number of max 8 workers this looks at the available CPU threads to determin the best number of active workers.

This means large systems won't be performance caped by a lower number of workers, containerized systems won't overcommit and waist resources scheduling more proesses then are allowed to run simultaneously.

On my workstation with 32 available threads it results in a 36% performance lift by better utilizing the available threads instead of being caped to 8 cores.

For CI where there can be a discrepancy between nproc reported cores and available cores this implementation takes cgroup and CFS in to account to determin effective limits avoiding the issues that the fixed 8 default was trying to be a happy medium for.

If it's unable to determin the CPU count it fallback to the old path.

[AJenbo]

I have now verified that it also works as expected on macOS and Docker.

[ondrejmirtes]

The 30 % improvement would of course be very welcome, but the more I thought about this, the less sense the infinite scaling makes to me.

Please describe the actual algorithm of how the scaling works. Give some examples of different project sizes / number of CPU cores / available memory and other variables going into the formula and what the result looks like.

I've set the limit to 8 recently because in my measurements, after that the returns were diminishing. It didn't seem worth it to consume twice the CPU for 10 % of speedup for example.

Maybe we can make a chart of how the performance scales? X axis number of processes, Y axis time spent analysing a project.

I also recently took advantage of the improvements in cpu-core-counter, 89f1382, hoping it'd solve a problem for some users where their PC would be unusable during PHPStan run. So I definitely don' want to completely exhaust a machine when running PHPStan.

Are you somehow estimating how much memory would each worker use? That doesn't seem possible to me.

Also - in some CI environments the CPU is shared, so although 32 cores are available, we shouldn't use all of them because the machine should also be available for other processes / adjacent runners.

[ondrejmirtes]

Maybe we can get the 90 % of your improvement just by increasing the default max from 8 to 16? Just an idea...

[AJenbo]

The 30 % improvement would of course be very welcome, but the more I thought about this, the less sense the infinite scaling makes to me.

Right, there of cause is no infinit scale to extract here :) I only have 16 cores and 32 threads to test with, the efficiency of my cores drops once i go past 16, but from my tests there is still a lot of performance to extract by improving the distribution here.

The 30% is specific to my main system, but I did see improviserets on a wide range of system configurations and no slow downs across the 25 real projects i measured.

I can try and see if I can get someone to run a test on a 128 core machine later so we can see if there should be an upper limit to how many cores it tries to scale to.

One thing to note is that I do have two followups planned which aims to improve how well PHPStan scales across multiple cores, which would improve this further. If there was no cost to horizontal scaling and things where perfectly balanced it should have been a roughly 300% improvement so there is some further potential here.

I've set the limit to 8 recently because in my measurements, after that the returns were diminishing. It didn't seem worth it to consume twice the CPU for 10 % of speedup for example.

Core scaling is a diminishing returns game since there are overheads, but without it it's going to be hard to pull 10% out of a hat.

Also - in some CI environments the CPU is shared, so although 32 cores are available, we shouldn't use all of them because the machine should also be available for other processes / adjacent runners.

That's exactly why I implemented reading of cgroup and CFS limits, these systems should actually see an improvement (I did test this) from the change since it now avoids managing 8 processes on systems that only allow for 2-4 concurrent threads per tenant.

Maybe we can get the 90 % of your improvement just by increasing the default max from 8 to 16? Just an idea...

This can hurt performance where there are fewer available threads, in fact on one resource constrained system where I tested this PR it went from 5m20s to 4m40s from the better balance (mainly by not over committing memory).

Bumping it to 16 would cause more low memory systems to start thrashing and even crash, and low core systems would be wasting time managing processes without any performance (at least true with shared systems with the current solution). I think 8 is a good fallback value, but it's not a good one-fits-all and 16 is worse.

I'll get back to you with graphs and the other things you asked for.

[AJenbo]

Oh one good news already with the pcntl_fork enablement you did the thread overhead is already reduced noticeably (13% improvement).

1-4 threads looks to be effectively liner scaling and after that it starts to taper off with a 2.2x overhead at 32 threads (something that can be improved by future work).

[staabm]

That's exactly why I implemented reading of cgroup and CFS limits, these systems should actually see an improvement (I did test this) from the change since it now avoids managing 8 processes on systems that only allow for 2-4 concurrent threads per tenant.

Doesn't fidry/cpu-counter also read these? (At least I expected it)

[AJenbo]

I have scaled back the PR to only implement CPU based scaling, mostly to simplify evaluation. Once merged I'll make a follow up for taking memory constraints in to consideration, that one is mostly about avoiding crashes so it's a different set of valuations then this one which is now mostly about performance.

The CPU algorithm is pretty simple, it's fidry/cpu-core-counter capped by cgroup CPU quotas.

Most of my data is gathered from my workstation (AMD) which is a 16 core CPU with SMT allowing up to 32 threads. It's cores are spread over 2 CCDs which, this is relevant for the performance number I have since there are hardware inefficiencies both when crossing the at 8 and 16 thread count per workload. In effect this means the graph will flatten soner then it would on a faster system.

Workstation

First here are the numbers for running things directly on my desktop with various CPU limits:

threads	fixed 8: wall	auto: wall	Δ wall	auto: step efficiency	fixed 8: CPU-s	auto: CPU-s
1	120.2 s	121.3 s	+0.9 %	--	121	122
2	64.1 s	63.8 s	±0	47%	126	125
4	34.9 s	35.0 s	±0	45%	131	131
8	20.7 s	20.5 s	±0	41%	145	144
16	20.8 s	13.9 s	−33 %	32%	146	174
32	20.4 s	13.1 s	−36 %	6%	144	302

(the numbers here are without turbo enabled, but I saw exactly the same relative scaling with turbo enabled)

The efficiency drop from 41 to 32% and 32% to 6% are largely because of my CPU topology, so I would not recommend setting a cap at 16 or trying to detect SMT cores. When testing on a laptop (Intel) with only a singe CPU die SMT still yielded a 40% performance increase. A system with 16 cores per CCD or a larger number of physical cores would be able to pull a lot more performance at high worker counts, but it is worth nothing that memory consumption will increase so hopefully no one is running a 64 thread system with 16 GB of memory (if they do that would be addressed by my follow up).

On small project worker count continues to be limited by files / 40.

Currently the per worker cost is 2.5% when not affected by the hardware topology, but it grows to 4.8% because of CCD and SMT bottlenecks on my CPU. If we want to set an upper efficiency limit I would suggest extrapolating from the 2.5% though I do hope to reduce this in future PRs so that it scales better with higher core counts.

Besides testing on PHPStan's own source I also ran it on two Laravel application where I got a 14 to 25% performance increase, the overhead here is higher because of some of the plugins which is out of scope here but also something I'm looking at improving.

Containers

Next is what the changes means for containerized setup (which is relativly common on CIs). Since fidry/cpu-core-counter doesn't take limits in to consideration PHPStan currently over commits resulting in 9-23% performance loss purly from thread management overhead, this also leads to additional memory usage which isn't helping either.

cpu	wall	auto: wall	Δ wall	fixed: CPU-s	auto: CPU-s	fixed: peak	auto: peak
1	159.2 s	122.6 s	−23 %	160	123	1.66 GB	0.59 GB
2	76.4 s	63.2 s	−17 %	151	124	1.65 GB	0.79 GB
4	38.5 s	34.9 s	−9 %	146	131	1.66 GB	1.12 GB
8	20.5 s	20.5 s	±0	144	144	1.65 GB	1.65 GB
16	20.4 s	13.7 s	−33 %	143	173	1.66 GB	2.60 GB

On systems like this PHPStan currently only performs properly when all 3 values align on the configured 8 worker limit.

%%{init: {"themeVariables": {"xyChart": {"plotColorPalette": "#eb6834, #2a78d6"}}}}%%
xychart-beta
    title "Cold self-analysis wall time - container CPU limit (orange = fixed 8, blue = auto)"
    x-axis "--cpus" [1, 2, 4, 8, 16]
    y-axis "seconds" 0 --> 165
    line [159.2, 76.4, 38.5, 20.5, 20.4]
    line [122.6, 63.2, 34.9, 20.5, 13.7]

Loading

(orange is current, blue is auto / this PR )

[AJenbo]

That's exactly why I implemented reading of cgroup and CFS limits, these systems should actually see an improvement (I did test this) from the change since it now avoids managing 8 processes on systems that only allow for 2-4 concurrent threads per tenant.

Doesn't fidry/cpu-counter also read these? (At least I expected it)

No, that's largely the problem on the container side of things.