Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

Would extremely high decode tok/s even be useful?

Via r/LocalLlama
Thursday, Jul 30, 2026 · 6:51PM
Summary

If you were able to get an inference machine that could do decode at 1k toks/s or even 10k tok/s, would that even be helpful? Would it unlock any new use cases? Let’s assume that this is for actually useful models and fairly large models like Qwen 3.5 397B, GLM-5.2, etc Or at that speed would it jus

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories