SemiAnalysis launches GDPx, a brutal AI agent benchmark where the top score is 12%

Every new model launch comes with a chart where it wins, so I've learned to get excited about benchmarks that make everyone look bad. SemiAnalysis just released one. It's called GDPx, and on its first leaderboard the best model clears just 12.0 percent.

What GDPx tests

GDPx has 206 tasks across 11 domains, built with 14 specialist data partners: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (basically ML research engineering), hardware design, materials science, alignment, and software engineering. SemiAnalysis deliberately left out sectors like government, real estate, and insurance. Their argument is that early AI revenue will cluster where frontier models pay off most, not spread evenly across the economy.

These aren't quiz questions. The default time limit is eight hours per task, models run in their own native agent tools where they have them, and the headline number is pass@1 across three attempts, with every domain weighted equally. The team also says its quality checks flagged nearly half the tasks in one popular existing benchmark as broken, which is a big part of why they built their own.

The early leaderboard

Claude Opus 5.5 sits on top at 12.0, followed by Gemini 4 Carbon at 10.9 and GPT-6 Astra at 9.6. The site also maps cost per task against score, which honestly might be the more useful chart for anyone paying the bills.

My take

Low scores are the point here. A benchmark that's already saturated can't tell you much, while this one leaves plenty of room to track real progress. SemiAnalysis plans to refresh it every three to six months, so I'll be checking how fast that 12 percent climbs. You can read the SemiAnalysis launch post or browse the GDPx leaderboard.