Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

Let’s talk quants of Gemma and Qwen - 16 vs Q8 vs Q4 - any experiences?

Via r/LocalLlama
Wednesday, May 20, 2026 · 1:52AM
Summary

Some people say they’d never go under Q8, and others say they find Q3 acceptable! What’s your take?

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Related in Community
LM Studio finally added support for MTP Speculative Decoding
r/LocalLlama
Decent deal on RTX 3080 20GB on ebay - $30 per GB
r/LocalLlama
Running DeepSeek-V4 locally with 4x legacy RTX 2080 Ti ($2k budget setup). Custom Turing kernels, W8A8 quantization, and 255 prefill tok/s!
r/LocalLlama
HRM Seems To Be Going Off Right Now
r/LocalLlama
48GB VRAM users, what are your daily drivers? Do you wish you had more VRAM? What would you run if you did?
r/LocalLlama
More from Best AI News
Position: Let's Develop Data Probes to Fundamentally Understand How Data Affects LLM Performance
ArXiv cs.AI · Papers
Operationalizing Document AI: A Microservice Architecture for OCR and LLM Pipelines in Production
ArXiv cs.AI · Papers
Evaluating the Utility of Personal Health Records in Personalized Health AI
ArXiv cs.AI · Papers
Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency
ArXiv cs.AI · Papers
Back to all stories