Best AI News โ€” Updated Every 3 Hours
Story Page
← All Stories
Home Community Story
Community

[PAPER] GPQA, MMLU-Pro, and MMMU-Pro were audited for broken questions, and up to 12% of them had to be removed. New drop in clean versions released

Via r/LocalLlama
Tuesday, Jul 28, 2026 ยท 7:58PM
Summary

I was very curious why all the models were topping out on GPQA-Diamond around 92 or 93% (AA) and spent the last few weeks pouring over GPQA (Diamond and Extended), and then expanded to auditing MMLU-Pro and MMMU-Pro. It was quite frankly shocking just how many questions were malformed, had wrong ans

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories