Omead Pooladzandi (@HessianFree) / X

Omead Pooladzandi

10.4K posts

Omead Pooladzandi

@HessianFree

Co-Founder @PrismML PSGD

Newport Beach, CA

Joined February 2010

Pinned
Omead Pooladzandi
@HessianFree
Mar 31
your spotify cache is bigger than our largest AI model. Bonsai: 1-bit weights. 1.7B to 8B params. 14x compression vs bf16. 8x faster on edge. 256 MB to 1.2GB. Based on Qwen 3. we just came out of stealth. intelligence belongs at the edge and we're going to put it there.
PrismML
@PrismML
Mar 31
Today, we are emerging from stealth and launching PrismML, an AI lab with Caltech origins that is centered on building the most concentrated form of intelligence. At PrismML, we believe that the next major leaps in AI will be driven by order-of-magnitude improvements in
207K
Omead Pooladzandi
@HessianFree
Jan 29, 2025
We've been cooking. No this is not PSGD. Yes, I have trained a 3b parameter model with it.
163K
Omead Pooladzandi
@HessianFree
Jul 4, 2025
This will probs be one of the most influential papers of the year
136K
Omead Pooladzandi
@HessianFree
Dec 29, 2024
Momentum in optimization isn't always your best friend. In dynamic systems, even tiny momentum can be catastrophically destructive. 🐾 Here we're using PSGD's Dense Newton method as the base optimizer—an 'old school' method that's reliable, like a wise old dog. 🐕‍🦺 For static
20K
Omead Pooladzandi
@HessianFree
Aug 13, 2025
Wait what how did I miss this one @aaron_defazio at it again
11K
Omead Pooladzandi
@HessianFree
May 10, 2025
Replying to @janonacct
Unionically this would probably save them a lot of money for people's spam messages to gpt's
12K
Omead Pooladzandi
@HessianFree
Dec 30, 2024
Alright let's put Muon and SOAP head-to-head with PSGD to solve a simple loss = (1 - (x^T * x))^2, such that x is initialized as a 2x2 random matrix. At high beta both all optimizers fail but, otherwise PSGD is able to reduce the loss down exactly to zero whereas Muon can
62K
Omead Pooladzandi
@HessianFree
Feb 14, 2025
I'm reviewing for icml right now. Muon clones everywhere. It's actually so sad to see. This is a bad look for the community. If ur doing a Muon colne and u don't use modded nanoGPT. It's literally just an auto reject from me. Sheesh. @kellerjordan0
13K
Omead Pooladzandi
@HessianFree
Feb 24, 2025
A fun plot today from a convo with @evaninwords @torchcompiled @leloykun and @rami_mmo. It shows Muon's Newton-Schulz iterations flips the sign of the gradient update ~25% of the time. The cosine similarity is conserved only ~63%. While update directions being flipped are not of
17K
Omead Pooladzandi
@HessianFree
May 2, 2024
Since we were rejected from ICML here are some results we are proud of. We find that the Shampoo and CASPR preconditioners leave a lot of performance on the table when estimating the Hessian. We propose a much cheaper Hessian estimate. No expensive matrix inverse is required!
30K
Omead Pooladzandi
@HessianFree
Jun 5, 2025
@aaron_defazio sensational
2.9K
Omead Pooladzandi
@HessianFree
Dec 29, 2024
Good list of 2nd order optimizers
GitHub - riverstone496/awesome-second-order-optimization
From github.com
3.8K
Omead Pooladzandi
@HessianFree
Jul 4, 2025
Replying to @HessianFree
arxiv.org/abs/2507.02754 @Happylemon56775 @_arohan_ @dvsaisurya and team did a great job with this one
arxiv.org
Fast and Simplex: 2-Simplicial Attention in Triton
Recent work has shown that training loss scales as a power law with both model size and the number of tokens, and that achieving compute-optimal models requires scaling model size and token count...
5.4K
Omead Pooladzandi
@HessianFree
Jan 29, 2025
when ur scientific plot is so good it becomes memeable
gerred
@sloppenheimer
Jan 29, 2025
when everyone and their mom became an expert in distillation by 9am
5.9K