Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

Qwen3.8-Flash-Next 177B NVFP4(119GiB): SSD streaming at 9-10 tok/s on one 16 GB RTX 5060 Ti + 32 GB RAM

Via r/LocalLlama
Sunday, Sep 27, 2026 · 10:13PM
Summary

We built an inference engine for MoE models that don't fit in VRAM + RAM. Most of the model stays on the SSD, and experts are read as tokens need them. This started as a proof of concept, and poc worked, we are getting 9-10 tok/s decode on Qwen3.8-Flash-Next NVFP4 (9.06 on the benchmark turn, 10.4 o

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories