Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

Is there a limit on the number of active parameters in an MoE model?

Via r/LocalLlama
Thursday, May 14, 2026 · 5:26PM
Summary

Hi. We recently had MoE models as big as 1T and 1.6T total parameters. My expectation on the proportion between total and active parameters so far was around 10 to 1, that we save on smaller, "actually local" models. However, these new huge models have a much smaller number of active parameters for

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories