Best AI News โ€” Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

DSv4-0731 on 32gb RAM - 70tok/s prefill, 1.5tok/s decode

Via r/LocalLlama
Sunday, Aug 9, 2026 ยท 9:53AM
Summary

The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter more than the kernels; repacking to enable sequential reads rather than random reads works, pipelining

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories