Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

300b on 32gb MoE-streaming findings + optimisations

Via r/LocalLlama
Sunday, Aug 9, 2026 · 10:14AM
Summary

The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter more than the kernels; repacking to enable sequential reads rather than random reads works, pipelining

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories