Quentin Anthony (@QuentinAnthon15) / X

Quentin Anthony

362 posts

Quentin Anthony

@QuentinAnthon15

I make models more efficient. Google Scholar: scholar.google.com/citations?user…

quentin-anthony.github.io

Joined June 2019

Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
I was one of the 16 devs in this study. I wanted to speak on my opinions about the causes and mitigation strategies for dev slowdown. I'll say as a "why listen to you?" hook that I experienced a -38% AI-speedup on my assigned issues. I think transparency helps the community.
METR
@METR_Evals
Jul 10, 2025
We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.
1.8M
Quentin Anthony
@QuentinAnthon15
Jul 10, 2025
The very first task I usually give new pretraining people is to run a tiny transformer, profile it, and understand it deeply. I wrote up a small tutorial covering this exact workflow. I talk about how to measure GPU perf, how to estimate tensor core speedup, etc. Take a look:
183K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
I think cases of LLM-overuse can happen because it's easy to optimize for perceived enjoyment rather than time-to-solution while working. Me pressing tab in cursor for 5 hours instead of debugging for 1:
00:00
159K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
Firstly, I think AI speedup is very weakly correlated to anyone's ability as a dev. All the devs in this study are very good. I think it has more to do with falling into failure modes, both in the LLM's ability and the human's workflow. I work with a ton of amazing pretraining
71K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
Second, LLMs today have super spiky capability distributions. I think this has more to do with: 1) what coding tasks we have lots of clean data for, and 2) what benchmarks/evals LLM labs are using to measure success. As an example, LLMs are all horrible at low-level systems
59K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
Third, it's super easy to get distracted in the downtime while LLMs are generating. The social media attention economy is brutal, and I think people spend 30 mins scrolling while "waiting" for their 30-second generation. All I can say on this one is that we should know our own
45K
Quentin Anthony
@QuentinAnthon15
Apr 16, 2024
Zyphra is pleased to announce Zamba-7B: - 7B Mamba/Attention hybrid - Competitive with Mistral-7B and Gemma-7B on only 1T fully open training tokens - Outperforms Llama-2 7B and OLMo-7B - All checkpoints across training to be released (Apache 2.0) - Achieved by 7 people, on 128
185K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
Some final statements: - METR is a wonderful organization to work with, and they are strong scientists. I've loved both participating in this study and reading their results. - I am not some LLM guru trying to preach. Think of this as me publishing a personal diary entry, and
34K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
Along this point, there's a long tail of issues that cause an LLM to choke: - "Context rot", where models become distracted by long+irrelevant contexts (especially from long conversations). See x.com/simonw/status/…. You need to open a new chat often. This effect is worsened if
Simon Willison
@simonw
Jun 18, 2025
Workaccount2 on Hacker News just coined the term "context rot" to describe the thing where the quality of an LLM conversation drops as the context fills up with accumulated distractions and dead ends news.ycombinator.com/item?id=443087…
75K
Quentin Anthony
@QuentinAnthon15
Feb 3, 2024
State-space models (SSMs) like Mamba and mixture-of-experts (MoE) models like Mixtral both seek to reduce the computational cost to train/infer compared to transformers, while maintaining generation quality. Learn more in our paper: zyphra.com/blackmamba
53K
Quentin Anthony
@QuentinAnthon15
Jan 30, 2024
Getting the most out of your hardware when training transformers requires thinking about your model as a sequence of GPU kernel calls. This mindset, common in HPC, is rare in ML and leads to inefficiencies in LLM training. Learn more in our paper arxiv.org/abs/2401.14489
54K
Quentin Anthony
@QuentinAnthon15
Jul 10, 2025
Replying to @QuentinAnthon15
GitHub - Quentin-Anthony/torch-profiling-tutorial
From github.com
17K
Quentin Anthony
@QuentinAnthon15
Jun 4, 2025
Inspired by “minimal implementation“ projects in AI such as @karpathy’s nanoGPT, I worked to bring this concept to the HPC world! I’ve built a minimal implementation of an MPI library called nanoMPI, which focuses on clarity, simplicity, and easy installation.
38K
Quentin Anthony
@QuentinAnthon15
Jul 12, 2025
Replying to @QuentinAnthon15
LLMs are a tool, and we need to start learning its pitfalls and have some self-awareness. A big reason people enjoy @karpathy's talks is because he's a highly introspective LLM user, which he arrived at a bit early due to his involvement in pretraining some of them. If we
37K