Benchmarks
All entries tagged with Benchmarks.
8 min read
AI / LLMs / Benchmarks
AI / LLMs / BenchmarksBenchmarks Are Thermometers, Not Report Cards
LLM benchmarks are useful when you treat them like instruments, not trophies. Here is how to read MMLU, Arena, SWE-bench, HELM, and your own evals without turning the leaderboard into a religion.
5 min readAI / Benchmarks / OpenAI
AI / Benchmarks / OpenAIOpenAI Just Retired Its Own Report Card
OpenAI says SWE-bench Verified — the benchmark every coding model has been bragging about — is no longer measuring frontier capability. Here's what the new scoreboard looks like, and why the old one stopped being honest.