Every bar on this page is a row in our speed ledger: its id, when it ran, on which cards, against which baseline, and how busy the box was. No number here is rounded up.
One test box: 4x Tesla P100 16 GB, 2x Tesla V100 16 GB, every card on PCIe gen3 x4; dual-socket Xeon E5-2699 v3. Every card sits on a PCIe gen3 x4 link (about 3.2 GB/s), so streaming and card-to-card costs are real here; faster slots lift the multi-card numbers.
The probe
llama-server, greedy, 256 tokens per request, two prompt classes (prose and code), REPS 3 per class per arm; board cells against a competitor run in the same bracket with its own build, file and flags. Arms alternate ABBA in one window, after a warm-up. Each speed probe also hashes a 512-token greedy answer to prove a lever did not change the output.
Busy box, honest label
The box also runs other work. A bracket with box load above a quiet threshold is marked NOT-QUIET in each cell: the deltas inside one bracket stay valid, absolute tok/s are lower bounds. A monotone drift right after a cold load is treated as a warm-up ramp and judged on the close brackets.
Percentages
Percentages are computed from the ledger values and truncated to one decimal (floor), never rounded up. Where an arm has two ledger rows (ABBA pairs) the value is their mean. A lever ships on by default only after determinism (12 identical greedy answers, long and short), a needle-in-a-haystack test and smoke prompts; kernels and quants are bit-identical by default, anything lossy is opt-in.
Against a popular community fork
Head to head on Pascal
a popular community fork of llama.cpp tuned for Pascal (its patched build, its own files, its README flags). We never name it.
Qwen3.8-27B dense, 2x P100
Board cells: same box, same bracket, their best build and flags on their file, ours on ours; percentages truncated to one decimal.
shipping MTP ramp (depth 1 below 8k), same engine prose 59.50 / code 65.45 same bracket as ours (arms F1.1 / F1.4 around F4p5.2)
Box
NOT-QUIET box (load 7-24)
Status
Gated (det 12/12, needle, smoke) as an opt-in preset for contexts up to 8k (msg 20964); the depth cutoff 0.5 preset is gated at 8k (+8.3% prose over the shipping preset); the 32k preset gate is owed.
Messages
#20905, #20964
Class
Ours
Baseline
Change
Rows
prose
62.43
59.50
+4.9%
row 7228, row 7240 | row 7238, row 7244
code
84.38
65.45
+28.9%
row 7229, row 7241 | row 7239, row 7245
V100 27B plain decode
default on
Qwen3.8-27B one token at a time, no drafting, one V100 (llama-server, the shipping SA1 shape).
V100 27B plain decode
prose +2.9% · code +3.2%
1x Tesla V100 16 GB · Qwen3.8-27B PXQN4 · product of two same-bracket steps
After the step
Before (same binary, levers off)
NOT-QUIET box (load 7-15 across both windows)rows 7177, 7178, 7179, 7180, 7268, 7269, 7270, 7271
Step 1
step 1 card 2, 2026-10-14 08:43 EDT, ABBA O N N O, load 12-15, REPS 3
Step 2
step 2 card 2, 2026-10-14 10:56 EDT, ABBA, REPS 3
Box
NOT-QUIET box (load 7-15 across both windows)
Status
Default on, bit-identical (greedy512 sha 9f7c2fec01e4 in every arm), det 12/12 on the shipping arm. Plain V100 decode is GPU-bound (about 84% of HBM bandwidth): this cell is closed; the 27B speed comes from MTP.
Caveat
The total is the product of two same-bracket steps; each step compares the same binary with the levers off and on.
Messages
#20838, #20950
Class
After
Before
Change
Rows
step 1 prose
40.9
40.34
+1.3%
row 7179 / row 7177
step 1 code
40.98
40.32
+1.6%
row 7180 / row 7178
step 2 prose
41.81
41.19
+1.5%
row 7270 / row 7268
step 2 code
41.82
41.17
+1.5%
row 7271 / row 7269
total prose
+2.9%
product of the steps
total code
+3.2%
product of the steps
P100 27B
opt-in
Qwen3.8-27B plain decode on one P100, the stacked sm_60 kernel levers on vs off in one binary.
Bit-identical (det long 12/12 on the measured sha, msg 21389); default in every Flash-Next preset. Tensor split on the V100 pair ties layer under MTP, so the pair stays on layer.
Gated on the tensor arm: det long 12/12, det short 12/12, smoke 5/5, needle 14k and 30k PASS; the registry auto-picks it on a P100 pair (msg 21414). Tensor and layer outputs differ in summation order (near-tie flips), each deterministic.
Messages
#21392, #21414, #21428
Class
Ours
Baseline
Change
Rows
prose
40.94
38.11
+7.4%
row 7466 | row 7468
code
47.29
44.88
+5.3%
row 7467 | row 7469
pp8k
300.0
250.0
+20.0%
row 7474 | row 7477
pp16k
290.0
235.0
+23.4%
row 7475 | row 7478
pp30k
267.0
208.0
+28.3%
row 7476 | row 7479
Flash-Next 28k prefill
default on
Reading a 28,000-token prompt with Flash-Next on one V100 while its experts stream over PCIe x4 (the long-context complaint).
inside a posted QUIET bracket on cores 54-71; box load not recordedrow 7402, row 7401
Ours
opt-in preset: -ub 4096 when experts stream (dev build b2f25f3daa) pp28k 597.3 card 2, 2026-10-15 06:00-06:23 EDT, server Overdrive preset, ABBA 4 requests over 2 boots
Baseline
same server, today's automatic -ub 2048 pp28k 324.3 same bracket
Box
inside a posted QUIET bracket on cores 54-71; box load not recorded
Status
Gated det 12/12 (long + short) and needle 28k PASS; default ON for one card whose experts stream (dev build b0bdef21d6). The gate's needle requests read 332.3 vs 664.1 t/s (one request each, msg 21318).
Messages
#21318
Class
Ours
Baseline
Change
Rows
pp28k
597.3
324.3
+84.1%
row 7402 | row 7401
P100 27B MTP
no number yet
Qwen3.8-27B with MTP on P100s.
No v3.1 comparison yet: the depth sweep is owed. Against the community fork the MTP arm is already measured (see Rivals).
GLM
no number yet
GLM-5.3-Flash, the file we ship as N2.
No number yet: the RAM-heavy cell waits for a free P100 pair.
Appendix
Every cited ledger row
Row ids are the speeds ledger's primary keys; build.py --verify re-reads each one and fails on any mismatch.
