Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

To llama.cpp: I love you. I owe my entire local inference experience to you. But can we have quantization recipes that actually mean something?

Via r/LocalLlama
Thursday, Sep 3, 2026 · 2:53AM
Summary

Not even sure if that plea should be addressed at llama.cpp but, look at the Q4KM quants of Qwen3.8-Flash-Next: Quantizer Size Unsloth: q4km is non-existent but the closest is q4kxl 111 GB Lmstudio 119 GB AtomicChat 94.5 GB Bartowski 120 GB AesSedai 135 GB Ggml. The goat itself. Only Q8 but not far

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories