Best AI News โ€” Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

kernel-anvil: 2x decode speedup on AMD by auto-tuning llama.cpp kernels per model shape

Via r/LocalLlama
Monday, Mar 30, 2026 ยท 7:02AM
Summary

Built a tool that profiles your GGUF model's layer shapes on your AMD GPU and generates optimal kernel configs that llama.cpp loads at runtime. No recompilation needed. The problem: llama.cpp's MMVQ kernels use the same thread/block configuration for every layer regardless of shape. A 1024-row GQA p

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories