Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

Top-N-Sigma: Remove unconditional softmax+sort by TimNN · Pull Request #22645 · ggml-org/llama.cpp

Via r/LocalLlama
Monday, Jun 22, 2026 · 5:18PM
Summary

Overview Currently, the Top-N-Sigma sampler does an unconditional softmax+sort at the end. In the (common, I believe) case of Top-N-Sigma being followed by Dist, this expensive work is completely wasted. Additional information On my M3 Max MacBook Pro, this PR increases the t/s for google_gemma-4-E4

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories