Lewis Tunstall (@_lewtun) / X

Lewis Tunstall

5,682 posts

Lewis Tunstall

@_lewtun

🤠 post-training @huggingface

Berne, Switzerland

Joined August 2018

Pinned
Lewis Tunstall
@_lewtun
Oct 30, 2025
We've just published the Smol Training Playbook: a distillation of hard earned knowledge to share exactly what it takes to train SOTA LLMs ⚡️ Featuring our protagonist SmolLM3, we cover: 🧭 Strategy on whether to train your own LLM and burn all your VC money 🪨 Pretraining,
143K
Lewis Tunstall
@_lewtun
Jan 25, 2025
We are reproducing the full DeepSeek R1 data and training pipeline so everybody can use their recipe. Instead of doing it in secret we can do it together in the open! 🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1. 🧠
GitHub - huggingface/open-r1: Fully open reproduction of DeepSeek-R1
From github.com
278K
Lewis Tunstall
@_lewtun
Feb 22, 2023
One exciting feature of our partnership with AWS is that we now have 1000+ GPUs to start training really large models at @huggingface 🔥🔥🔥! We’ll be working hard to make closed models open, starting with LLMs and friends 🤓 Which closed models would you like to be open?
00:00
208K
Lewis Tunstall
@_lewtun
Dec 16, 2024
We outperform Llama 70B with Llama 3B on hard math by scaling test-time compute 🔥 How? By combining step-wise reward models with tree search algorithms :) We show that smol models can match or exceed the performance of their much larger siblings when given enough "time to
192K
Lewis Tunstall
@_lewtun
Jan 4, 2024
I no longer fear regular expressions 😂
Greg Brockman
@gdb
Jan 3, 2024
How has ChatGPT changed your life?
83K
Lewis Tunstall
@_lewtun
Oct 10, 2023
Here's a simple recipe to train a 7B model that outperforms Llama2 70B on MT Bench 🥇 1. SFT Mistral 7B on the UltraChat dataset 2. Align the SFT model to the UltraFeedback dataset with "direct preference optimisation" (DPO) Demo: huggingfaceh4-zephyr-chat.hf.space More details in the 🧵
463K
Lewis Tunstall
@_lewtun
Jul 21, 2024
We have just released the ✨NuminaMath datasets: the largest collection of ~1M math competition problem-solution pairs, ranging in difficulty from junior challenge to Math Olympiad preselection. These datasets were used to win the 1st Progress Prize of the AI Math Olympiad and
145K
Lewis Tunstall
@_lewtun
Jul 4, 2024
After 3 months of hard work, I'm heaps excited to share that our team won the first progress prize of the AI Math Olympiad 🥇! kaggle.com/competitions/a… This challenge involved fine-tuning open LLMs to solve 50 difficult math problems spanning geometry to number theory 🤓 Our
76K
Lewis Tunstall
@_lewtun
Aug 5, 2025
One line of code is all it takes to fine-tune the gpt-oss models from @OpenAI 🔥 > Support to target the MoE expert layers with PEFT > Kernels for FlashAttention3 & MegaBlocks > Fast inference with MXFP4 quantization format In our testing, these models are extremely efficient
77K
Lewis Tunstall
@_lewtun
Nov 10, 2023
🪁 Today we're releasing the code to train your very own Zephyr models! We've worked hard to make this as accessible as possible, so you can run: 🏋️‍♂️ Full fine-tuning with @MSFTDeepSpeed ZeRO-3 on A100s 🐭 LoRA or QLoRA fine-tuning on consumer GPUs Code: github.com/huggingface/al…
135K
Lewis Tunstall
@_lewtun
Mar 8, 2023
For everyone building ChatGPT at home, there's now a very cool dataset on the Hub that allows you to train instruction models at comparable quality to OpenAI's InstructGPT 🤯 How long before someone trains a certain 🌸 or 🦙 on it? Download it here 👉: huggingface.co/datasets/yizho…
99K
Lewis Tunstall
@_lewtun
Mar 16, 2023
A certain LLaMa has just landed on the main branch of 🤗 Transformers! If you have access to the weights, you can finetune it on Stanford's Alpaca dataset to create models of similar quality to GPT-3.5 🤯 Dataset: huggingface.co/datasets/tatsu… Training code: github.com/tatsu-lab/stan…
115K
Lewis Tunstall
@_lewtun
Mar 17, 2025
It's pretty outrageous that a 250M parameter model can correctly convert screenshots of quantum field theory equations to LaTeX 🤯 Wish I had this when I was a student!
38K
Lewis Tunstall
@_lewtun
Oct 27, 2023
Excited to release Zephyr-7b-beta 🪁 ! It pushes our recipe to new heights & tops 10x larger models 💪 📝 Technical report: huggingface.co/papers/2310.16… 🤗Model: huggingface.co/HuggingFaceH4/… ⚔️Evaluate it against 10+ LLMs in the @lmsysorg arena: arena.lmsys.org Details in the 🧵
184K