This post is 8,000 words long—it is our new collaborative paper on an emerging type of AI evaluation. The paper is also published in a PDF format here.Summary: AI models have started to saturate most major benchmarks. But does that mean they can build and ship a real product, or conduct a scientific
