Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

Running Qwen3.8-27B-Q4 at max context on a 32 GB GPU while avoiding kvcache quantization

Via r/LocalLlama
Friday, Sep 11, 2026 · 7:41PM
Summary

I can't stand kv cache quantization. Even at q8_0, I can feel the difference. But realistically, when running Qwen3.8-27B-UD-Q4_K_XL on my 32 GB GPU, I only have room for ~170k tokens (with mtp and mmproj enabled). It's a lot of context, but Qwen3.8 eats through it on xhigh effort. I've been wanting

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories