Best AI News โ€” Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

In Q8_0 weight quantization, why can't we just skip blocks of 32 that have very large outliers?

Via r/LocalLlama
Tuesday, Jun 2, 2026 ยท 8:51PM
Summary

Looking for someone with an expert-level understanding. I understand that we can skip layers and sub-layers when doing quantization, but why can't we skip blocks? I am using Q8_0 as it's a simple example. Every block of 32 values has a scale. If we find that at least 1/32 values meets the criteria o

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories