Erik Wang (@erikyw26) / X

Erik Wang

102 posts

Erik Wang

@erikyw26

AI PhD @stanford research @benchmark prev @uniofoxford @harvard

erikyw.com

Joined September 2022

Pinned
Erik Wang
@erikyw26
Mar 17
Can AI make real mathematical discoveries? If so, how do we measure progress? Recent results on Erdős problems and First Proof are promising, but we still lack a rigorous framework for evaluating research ability in agents. HorizonMath takes a step toward resolving this. A 🧵
12K
Erik Wang
@erikyw26
Jun 16, 2025
Replying to @ben_j_todd and @tobyordoxford
Current models also aren't very capable of backtracking or actively reviewing their code, which is a major difference from humans. This is one reason why multi-agent approaches could be incredibly valuable
4.5K
Erik Wang
@erikyw26
Feb 14, 2025
We’re excited to share that HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematics has been accepted to #ICLR2025! 🧵
9.3K
Erik Wang
@erikyw26
Aug 23, 2025
Replying to @chetanp
Delighted to be here!
995
Erik Wang
@erikyw26
Aug 7, 2025
oh the irony of these plots being on the deception evals slide
2.6K
Erik Wang
@erikyw26
Feb 14, 2025
Replying to @erikyw26
OpenAI has pointed that benchmarks like MATH and GSM8K "are no longer effective at differentiating models.” We are excited to help fill the need for challenging, advanced math benchmarks that can differentiate between frontier LLMs.
1.3K
Erik Wang
@erikyw26
Feb 1, 2025
Replying to @hyhieu226
This seems especially true given o3’s ~0% performance on resolving OpenAI PRs. Intrinsic agentic capability is another facet of overall model performance that currently seems difficult to align with these new “reasoning” models
359
Erik Wang
@erikyw26
Feb 14, 2025
Replying to @erikyw26
HARDMath includes 1426 graduate-level applied math problems across 6 problem types, targeting approximation reasoning & asymptotic methods, and an algorithmic framework that generates questions and solutions with quality guarantees.
190
Erik Wang
@erikyw26
Feb 14, 2025
Replying to @erikyw26
When you think of math, you might picture neat equations with exact analytical solutions. However, scientists and engineers often face complex systems that lack easy, closed-form solutions and use clever mathematical approximations instead. Current LLM math benchmarks
172
Erik Wang
@erikyw26
Feb 14, 2025
Replying to @erikyw26
We find that frontier models across the board (including recent reasoning models) exhibit much worse performance on HARDMath than on existing, widely-used benchmarks like GSM8K and MATH.
115
Erik Wang
@erikyw26
Jul 9, 2023
Replying to @climate
reminds me of the ice cube from Artemis Fowl
33
Erik Wang
@erikyw26
Feb 15, 2025
Replying to @erikyw26
Read more about the project here! arXiv: arxiv.org/abs/2410.09988 Code: github.com/sarahmart/HARD… Many thanks to my awesome collaborators: @fjxdaisy, @BrennerGroup1, Sarah Martinson, and Kaylie Hausknecht.
979
Erik Wang
@erikyw26
Jan 31, 2025
Replying to @AravSrinivas
won't this essentially require solving continual learning? ie, how to stop catastrophic forgetting in an online scenario like this?
145
Erik Wang
@erikyw26
May 21, 2025
Replying to @pfau
this reminds of the s1 paper from @Muennighoff, where they find large performance gains despite using incorrect gemini reasoning streams
476