From f17e8b548f75960a4df44edde24a547d80a2b7ed Mon Sep 17 00:00:00 2001 From: NorwinYu Date: Sat, 24 Oct 2026 14:19:12 +0800 Subject: [PATCH] docs(MULTI_GPU): say when a layer split pays on a slow CPU The page says a second card is on par with the faster card alone for decode, and that a third, slower card makes a pair slower. Both were measured on a fast desktop CPU (9950X3D), so a reader with two identical older cards on a slower workstation CPU cannot tell which side of that line they are on, and on that machine the answer is not on par. Adds one bullet that names the condition (the CPU pool is what the window waits on), the measured numbers on two RTX 2080 Ti over a dual-socket Xeon E5-2696 v4 at 180 W and engine 0.1.49, and the mechanism from STRATA_DECODE_TIMING=1: decode 35.2 to 51.0 tok/s, an 11.6K prompt 672 to 908 tok/s, the window 70.06 to 37.96 ms, and the CPU pool 9.52 to 0.79 ms. Docs only; no behavior change and no default touched. --- docs/MULTI_GPU.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/MULTI_GPU.md b/docs/MULTI_GPU.md index 123654286f8..397ea4c9f0f 100644 --- a/docs/MULTI_GPU.md +++ b/docs/MULTI_GPU.md @@ -423,6 +423,14 @@ The Coder on an RTX 5080 + RTX 3090 (Ryzen 9 9950X3D), 32K context; details in - Leave out a much slower card when two already hold the model. An RTX 2080 Ti as a third card made the 5080 + 3090 pair slower (68 / 90 tok/s decode): every extra card costs its own round per window. - More cards pay off when the model's routed experts do not fit the faster ones. +- **The split pays most when the CPU pool is the bottleneck.** On two identical RTX 2080 Ti + 22 GB over a dual-socket Xeon E5-2696 v4 (AVX2 only, 88 logical CPUs), a plain split moved + decode from 35.2 to 51.0 tok/s (+45%) and an 11.6K-token prompt from 672 to 908 tok/s (+35%) + at 180 W, engine 0.1.49, six 300-token coding prompts. `STRATA_DECODE_TIMING=1` shows why: + the window fell from 70.06 to 37.96 ms, almost entirely because the CPU pool fell from 9.52 + to 0.79 ms per window. The "on par" number above is a fast CPU (9950X3D); when the CPU pool + is what the window waits on, moving half the layers to a second identical card removes that + work and pays about 1.45x. ## One conversation with both cards busy (`--pipeline-windows`, default on two or more AMD gfx12 stages)