Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

[D] - 1M tokens/second serving Qwen 3.5 27B on B200 GPUs, benchmark results and findings

Via r/MachineLearning
Thursday, Mar 26, 2026 · 7:52PM
Summary

Wrote up the process of pushing Qwen 3.5 27B (dense, FP8) to 1.1M total tok/s on 96 B200 GPUs with vLLM v0.18.0. DP=8 nearly 4x'd throughput over TP=8. Model is too small for tensor parallelism to help on B200s. MTP-1 mattered more than anything else (GPU utilization was 0% without it). MTP-5 crashe

Continue reading the full article
Read at r/MachineLearning
www.reddit.com
Back to all stories