everyone's talking about the claude code stuff (rightfully so) but this paper came out today, and the claims are pretty wild: 1-bit 8b param model that fits in 1.15 gb of memory ... competitive with llama3 8B and other full-precision 8B models on benchmarks runs at 440 tok/s on a 4090, 136 tok/s on
