I designed a CPU-native LLM architecture that hits 100+ tok/s on a 10B parameter model (the quality is the problem)

I've been working on an architecture designed to work with CPU memory constraints, not against it. The 10B parameters runs at 113-130 tok/s on a Ryzen 5 3600X, no GPU but the weights are garbage. To verify whether the quality would also hold at scale, I wanted to train an LM. While the training was