Epoch AI ✓

@EpochAIResearch uncorrected CritPt is pretty much cooked though? https://t.co/VAoTWiHwuT

4.2K views · 74 likes · 1 reposts · 1 replies Open on X →

While we are launching with 15 benchmarks, we plan to continue reviewing benchmarks on an ongoing basis, prioritizing those with the most impact and reach, as well as highlighting benchmarks we think are high quality that might be overlooked. 6.9K views · 67 likes · 1 reposts · 1 replies Open on X →

Each review is of a specific benchmark version, and if a benchmark is updated to fix errors we may review the new version. We share findings with developers and will publish a link to their response, if they’d like. Read our full methodology here: https://t.co/OWQJd3zQco 8.3K views · 67 likes · 2 reposts · 1 replies Open on X →

If we aren’t able to access enough information to review a benchmark, we’ll designate it ‘Not Enough Info.’ We will try to work with the creators of private benchmarks to conduct reviews while keeping the questions/tasks outside of public knowledge. 7.7K views · 64 likes · 1 reposts · 2 replies Open on X →

Flawed benchmarks have one or more substantive flaws we believe users need to be aware of to accurately interpret results, most commonly that >20% of the tasks have accuracy-impacting errors. In this case, we publish a limited writeup of the flaws we found. 8.3K views · 84 likes · 1 reposts · 1 replies Open on X →

Verified benchmarks can broadly be interpreted as described, and any errors that exist do not substantially affect the results. Alongside any verified benchmark we publish a full review and assessment of the benchmark, including any weaknesses and limitations we think it has. 9.1K views · 83 likes · 2 reposts · 1 replies Open on X →

We assign each benchmark a verdict based on our rubric: Flawed, Verified, or Not Enough Info. To avoid conflicts of interest we do not review Epoch-created benchmarks, but welcome external reviews. https://t.co/MqS2w6ZJS7 12.3K views · 109 likes · 4 reposts · 1 replies Open on X →

Benchmarks assess AI capabilities, but the benchmarks themselves vary substantially in quality. We hope to be a consistent source of information on benchmark quality. See our reviews here: https://t.co/p6lXXbWHOC 15.4K views · 152 likes · 5 reposts · 2 replies Open on X →

Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review. https://t.co/xU43tv32gH

609.8K views · 1.9K likes · 187 reposts · 115 replies Open on X →

@EpochAIResearch The ultimate tier is the AI finding the best problems to solve 640 views · 3 likes · 0 reposts · 1 replies Open on X →

Our ongoing math benchmarking efforts draw on unsolved problems from real research mathematics. Check out our website for more! https://t.co/ahukc3C1rO 9.9K views · 113 likes · 2 reposts · 0 replies Open on X →

When we launched Tier 4 on July 11th, 2025, the top score was 5%. Less than 14 months later, the top score is 98% and we consider the benchmark saturated. https://t.co/Ta0Cxx5BBn

52.8K views · 433 likes · 33 reposts · 7 replies Open on X →

Every FrontierMath Tier 4 problem has now been solved by AI, with GPT-6 Astra solving the last problem standing. Mathematicians often commented that AI found unintended shortcuts when solving their Tier 4 problems. Not so for this last one, which was created by Jay Pantone. https

327.7K views · 1.5K likes · 164 reposts · 36 replies Open on X →

Epoch AI ✓@EpochAIResearch8 Sep

OpenAI GPT-5.6 models and Anthropic Claude 5 models have different pricing structures at long context lengths. GPT model costs increase in price past 272k input tokens, while Claude model costs remain fixed. Does this reflect an underlying difference in the architecture of these

20.3K views · 313 likes · 33 reposts · 12 replies Open on X →

Epoch AI ✓@EpochAIResearch4 Sep

AI data centers have been getting larger at an exponential rate. The IT power capacity of the largest data centers has doubled every 10 months since mid-2024. But will that trend continue? https://t.co/hsdphxeNLF

13.9K views · 204 likes · 22 reposts · 18 replies Open on X →

Epoch AI ✓@EpochAIResearch4 Sep

Will Huawei catch up to Nvidia by 2030? Almost certainly not. We analyzed Huawei’s chip roadmap and supply chain, from the Ascend 950 to its 3D-stacking bet and domestic HBM supply. Our estimate: Huawei will produce less than 4% of Nvidia's AI compute in 2026. Relying on https:/

206.6K views · 504 likes · 95 reposts · 31 replies Open on X →

Epoch AI ✓@EpochAIResearch3 Sep

GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our ht

632.8K views · 2.2K likes · 323 reposts · 64 replies Open on X →

Epoch AI ✓@EpochAIResearch26 Aug

@EpochAIResearch @brightertmrrw @DanCareyEcon @pawtrammell @ansonwhho Nice. GDP misses a ton of other measures such as well-being. 263 views · 1 likes · 0 reposts · 0 replies Open on X →

Epoch AI ✓@EpochAIResearch25 Aug

@EpochAIResearch @brightertmrrw @DanCareyEcon @pawtrammell @ansonwhho @threadreaderapp unroll 170 views · 1 likes · 0 reposts · 1 replies Open on X →

@EpochAIResearch Nvidia is so big it's literally breaking US GDP accounting. Designed in America, counted as a foreign import. The AI boom is even larger than the official numbers say. 264 views · 25 likes · 3 reposts · 1 replies Open on X →

Read the full report by @brightertmrrw, @DanCareyEcon, @pawtrammell, and @ansonwhho: https://t.co/Tnb6IJJqHV 22.3K views · 108 likes · 3 reposts · 5 replies Open on X →

A fix exists: 2025 IMF/UN guidelines reclassify factoryless goods producers like Nvidia as domestic manufacturers, so their foreign sales count as exports. But the last such change took the BEA 5 years to implement. Until then, GDP growth estimates will remain biased downward, 11.8K views · 91 likes · 4 reposts · 1 replies Open on X →

To be certain, we also asked the BEA. They confirmed that, when production follows the route we described, the domestic value of the US-based design work does not enter GDP. This is because the BEA lacks data on overseas transactions of US-owned goods that are “transformed” by 6.4K views · 97 likes · 1 reposts · 2 replies Open on X →

Nvidia's value-add isn't recorded anywhere else, either: not in service exports (Nvidia sells goods, not services), nor in "merchanting" (chips are transformed abroad, and that category tracks <$1B/yr). 5.1K views · 75 likes · 2 reposts · 1 replies Open on X →

Goods exports? Also, no. Customs records only record goods that cross the US border, and Nvidia's chips are sold while still overseas. US computer exports have spiked, but that's mostly foreign-made GPUs passing through the US to Mexico for server assembly (e.g. Foxconn, for htt 5.6K views · 73 likes · 1 reposts · 2 replies Open on X →

We were surprised to discover tens of billions missing from US GDP accounting. So we checked every place the export could plausibly hide: IP exports? No. Nvidia generally doesn’t sell or license its IP. While we do see a recent $45B spike in IP exports, it traces mostly to tax h 6.5K views · 95 likes · 4 reposts · 1 replies Open on X →

This accounting blind spot is a critical missing piece of a widely accepted story about AI’s economic impact: that AI investment is enormous, but its effect on GDP growth is modest because much of the spending goes to imported technology goods, which subtract from GDP. What's ht 9.9K views · 128 likes · 4 reposts · 1 replies Open on X →

The result: servers enter the US at a price that includes Nvidia's design markup, which is fully counted as an import. But no export is ever recorded; no physical good leaves the US, and no one explicitly pays Nvidia for its IP. The value added by US design simply vanishes from 11K views · 148 likes · 5 reposts · 2 replies Open on X →

Why? It comes down to the supply chain for Nvidia chips. Nvidia chips are designed in the US, but manufactured, assembled, and sold abroad. The chips never touch US soil until they arrive inside imported servers. https://t.co/aN9YCvYuZN 12.5K views · 127 likes · 3 reposts · 4 replies Open on X →