Stuart Sul (@stuart

Stuart Sul

94 posts

Stuart Sul

@stuart_sul

ml research @cursor_ai, cs phd @Stanford, @HazyResearch

Joined July 2024

Pinned
Stuart Sul
@stuart_sul
Feb 20
(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS! Alongside this release, we’re equally excited to share some insights we learned while squeezing every
62K
Stuart Sul
@stuart_sul
Aug 19, 2025
MoE layers can be really slow. When training our coding models @cursor_ai, they ate up 27–53% of training time. So we completely rebuilt it at the kernel level and transitioned to MXFP8. The result: 3.5x faster MoE layer and 1.5x end-to-end training speedup. We believe our
163K
Stuart Sul
@stuart_sul
Sep 22, 2025
(1/6) We’re happy to share that ThunderKittens now supports writing multi-GPU kernels, with the same programming model and full compatibility with PyTorch + torchrun. We’re also releasing collective ops and fused multi-GPU GEMM kernels, up to 2.6x faster than PyTorch + NCCL.
32K
Stuart Sul
@stuart_sul
Sep 28, 2025
We released a TP-8 Llama-70B Megakernel! A single kernel runs the entire Llama 70B, with fine-grained overlap of compute, memory, and communication (without external dependencies like NCCL or NVSHMEM). Check out the blog post and the code from @HazyResearch:
Benjamin F Spector
@bfspector
Sep 28, 2025
(1/8) We’re releasing an 8-GPU Llama-70B inference engine megakernel! Our megakernel supports arbitrary batch sizes, mixed prefill+decode, a paged KV cache, instruction pipelining, dynamic scheduling, interleaved communication, and more! On ShareGPT it’s 22% faster than SGLang.
17K
Stuart Sul
@stuart_sul
Nov 17, 2025
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new
156K
Stuart Sul
@stuart_sul
May 27, 2025
GPU kernel launches are expensive--so we fused the entire Llama-1B into a single kernel. Very excited to kick off our megakernel framework series with Thunderkittens @HazyResearch. More coming soon!
Benjamin F Spector
@bfspector
May 27, 2025
(1/5) We’ve never enjoyed watching people chop Llamas into tiny pieces. So, we’re excited to be releasing our Low-Latency-Llama Megakernel! We run the whole forward pass in single kernel. Megakernels are faster & more humane. Here’s how to treat your Llamas ethically: (Joint
4.6K
Stuart Sul
@stuart_sul
Oct 29, 2025
Composer is our first model built in-house at Cursor. It’s designed specifically for real-world coding and is blazingly fast. Very excited to see what people think of it!
Cursor
@cursor_ai
Oct 29, 2025
Introducing Cursor 2.0. Our first coding model and the best way to code with agents.
00:00
2.6K
Stuart Sul
@stuart_sul
Sep 22, 2025
Replying to @stuart_sul
(6/6) Check out our kernels, the new ThunderKittens update, and a full demystification of NVLink/NVSwitch communication in our blog:
hazyresearch.stanford.edu
One Kernel for All Your GPUs
4.1K
Stuart Sul
@stuart_sul
Sep 22, 2025
Replying to @stuart_sul
(2/6) You don’t actually need NCCL for multi-GPU workloads. With custom kernels you can often beat its performance, and writing one is much simpler than it appears! All it takes is some virtual memory setup and NVSwitch acceleration for in-fabric reductions + multicast.
1.4K
Stuart Sul
@stuart_sul
Sep 22, 2025
Replying to @stuart_sul
(5/6) Another reason TK runs faster is reduced overhead. TK can operate directly on peer tensors instead of going through pre-allocated buffers like NCCL. TK can also directly perform fine-grained collectives (e.g., on the tensor dimension) without costly reshaping.
1.8K
Stuart Sul
@stuart_sul
Aug 7, 2025
We worked closely with the OpenAI team to make sure GPT-5 is the best coding agent ever on Cursor. For me, it’s the first AI model that actually provides meaningful help with GPU kernels (especially on finding race conditions). Everyone should give it a try.
Cursor
@cursor_ai
Aug 7, 2025
GPT-5 is now available in Cursor. It’s the most intelligent coding model our team has tested. We're launching it for free for the time being. Enjoy!
3K
Stuart Sul
@stuart_sul
Nov 17, 2025
Replying to @stuart_sul
(6/6) Read more on our blog and paper: Intro: tinyurl.com/3y4d9jst Part 1: tinyurl.com/23afs6kj Part 2: tinyurl.com/bddyad6p Paper: tinyurl.com/45532xxz
hazyresearch.stanford.edu
ParallelKittens: Simple and Fast Multi-GPU AI Kernels
7.9K
Stuart Sul
@stuart_sul
Aug 20, 2025
Replying to @tejasybhakta and @cursor_ai
So in microscaled matmuls, you need to wait for the block matmul to finish before dequantizing. The slowdown comes from two factors: moving data out of TMEM stalls tcgen05.mma (the next matmul can’t start until it clears), and CUDA cores are now too slow compared to
2.7K
Stuart Sul
@stuart_sul
Sep 22, 2025
Replying to @stuart_sul
(3/6) ThunderKittens (TK) now handles all of this setup for you. You can write multi-GPU kernels as if they were single-GPU. All TK ops now fully work on peer memory, automatically triggering NVLink transfers. We’ve also added new ops specifically for NVSwitch acceleration.
1.3K