SemiAnalysis launches GDPx agent benchmark, where the top model scores 12%

Semiconductor and AI research firm SemiAnalysis published GDPx on November 2, a benchmark meant to measure how well AI agents handle long, realistic tasks in the fields where the authors expect AI to create the most economic value. The first results are low across the board: the best-performing model scores 12%.

What the benchmark covers

Version 1.0 contains 206 tasks spread over 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. The tasks were written by 14 specialist data partners, with SemiAnalysis providing editorial support. The authors, Max Kan, Andrew Megalaa and Dylan Patel, say they deliberately left out sectors such as government, real estate and insurance, and present GDPx as a response to OpenAI's GDPval, which they criticize for lacking interactive environments and relying on noisy grading.

Agents get up to eight hours per task by default and run in their makers' own tools where available, such as Claude Code, Codex and Antigravity, with OpenCode used otherwise. The headline score averages single-attempt success over three tries and weights each domain equally. Each domain publishes at least one public task, while most tasks are kept private.

The results

On the public leaderboard, Claude Opus 5.5 ranks first with 12.0%, followed by Gemini 4 Carbon at 10.9%, GPT-6 Astra at 9.6% and Muse Wonder Full at 9.0%. GLM 5.3 closes the 13-model ranking at 2.7%. SemiAnalysis describes Opus 5.5 as the strongest all-round model, leading in quant trading, hardware design and chip design, while GPT-6 Astra leads in recursive self-improvement and materials science and Gemini 4 Carbon in biology.

Some results show how hard the tasks are. No model passed any finance or legal task under strict binary grading, so partial scores are shown instead. In hardware design, none of 12 runs produced a verified full pass, and in quant trading 70 of 111 graded runs did worse than the reference strategy.

Caveats

The authors say safety classifiers artificially lowered the overall scores of Opus 5.5 and GPT-6 Astra, because refusals sometimes triggered in the cyber, biology and alignment domains. Using different harnesses for each model also complicates direct comparison. SemiAnalysis acknowledges that its partner-vetting process can be gamed and that it lacks in-house expertise in every field. The firm calls GDPx a living benchmark and, according to Unite.AI, plans new versions every three to six months.