Wanting to make sure I’m not missing something here. I see a lot of posts around performance on new hardware and it feels like it’s always on a small context at missing the information around quantization. I’m under the impression that use cases for llms generally require substantially larger contex
