Found 7 posts
Same hardware, three models, completely different performance profiles. GPT-OSS-120B is fastest despite 117B params. Gemma4 has the best TTFT. Nemotron never loses to shuffle. The right model depends on the workload.
The smallest model benefits most from tensor parallelism: +50% at c1. Cluster TP=2 with --no-ray dominates decode through c32. Also: Ray vs PyTorch distributed -- the 2-8% you're leaving on the table.
The model where cluster never loses. Nemotron-3-Super benefits more from TP=2 than any other model tested -- and the SM12.1 CUTLASS patch doubles performance vs FlashInfer.
One formula tells you whether your hardware is working correctly or misconfigured. The community used this to diagnose the 40 t/s problem.
Cluster TP=2 dominates decode up to c32, then simple-shuffle takes over. Least-busy routing collapses under load. Full topology comparison with four configurations.
Neither topology dominates — cluster wins decode at every concurrency level, but 2x Solo wins prefill and TTFT under load. The right choice depends on the workload.
Post 3 hit 4,158 t/s at c64. llama-benchy puts those numbers under the microscope. Same hardware, two tools, 25x difference in TTFT.