The Code/X ArchiveView on X
SemiAnalysis

@SemiAnalysis_

Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.

GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
Image from the post
3483083.2K1.3K
SemiAnalysis

@SemiAnalysis_

GDPx covers the following domains. Each was a collaboration with at least one specialized data partner. SemiAnalysis was responsible for final QA and heavily involved with task iteration. (2/7)
Image from the post
52741493
SemiAnalysis

@SemiAnalysis_

We believe the first trillion dollars of AI revenue will be concentrated in sectors that can deploy frontier intelligence at high ROI, not broadly diffused throughout the whole economy. Building a general-purpose benchmark based on share of GDP or the SOC/O*NET occupational taxonomy is therefore a mistake. If your goal is to track the progress of frontier models, you should focus on the domains where they will be most impactful. This is why we include chip design, cyber, and bio over domains like government, real estate, and insurance. (3/7)
31630935
SemiAnalysis

@SemiAnalysis_

It’s worth emphasizing that certain models like Opus 5.5 and GPT-6 Astra have sandbagged overall scores due to their safety classifiers. If we exclude cyber, biology, and alignment—the three domains where safety classifiers sometimes kick in—you get the following results instead.

We were pleasantly surprised by how well this ranking lines up with our subjective impression of the models. (4/7)
Image from the post
162135564
SemiAnalysis

@SemiAnalysis_

We believe it’s underrated how relevant safety classifiers are when picking the best model for certain use cases. For example, Muse Wonder Full is actually the best available cyber model today by a meaningful margin. (5/7)
Image from the post
6829041
SemiAnalysis

@SemiAnalysis_

We would like to give a huge thank you to our data partners for contributing to GDPx

Arcverex: @ARCVEREX
Aria: @ariadata_ @nratliff9
Arithmetic: @arithmeticlabs1 @uri_rolls
Beautiful Machines: @btfl_machines @akarshaurora
Chakra: @chakra_ai @27upon2
Datacurve: @datacurve @serenaa_ge
Emulated: @emulated_ai @_josephwang_
Ergodex: @ergodexlabs
Extrapolate: @qasim31wani
Fleet: @fleet_ai @Yngkvn
Latch: @LatchBio @kenbwork
Living Context: @livingcontextai @_rahulparikh
Third Moment: @thirdmomentres @adamkhakhar
ValenceForge: @valenceforge @bryancsk

As well as @FireworksAI_HQ for providing inference for the open source models and @modal for providing sandbox infra (6/7)
1417746
End of thread