AI moves so fast now that by the time we understand something it is no longer being used
Xavier (Xavi) Amatriain
1,261 posts
- My Transformers Catalog has become one of my most popular posts ever. Some of you told me that you turned into a pdf for easier reading. I thought I should make it into an arXiv preprint. Here you go: 60 Transformers in 36 pages 🤖 🎉 arxiv.org/abs/2302.07730
- Pretty big update to my Transformer Catalog. I added ChatGPT, Sparrow, and Stable Diffusion among others. I also included a section about RLHF and Diffusion models and a new timeline view. Enjoy! amatriain.net/blog/transform…
- Today I had coffee with an MIT PhD who, in an effort to build AGI and mathematically prove free will, is coming up with an alternative to back propagation. How was your Monday?
- A lot has been speculated about TikTok's recommendations. This is the first paper I've read by the team, and it has many interesting details: expirable embeddings, parameter server, online training... Good #recsys stuff
- As many of you know, over the past few months I have been sharing Prompt Engineering resources in different forms. I have now compiled them all into a cohesive publication and uploaded to arxiv: arxiv.org/abs/2401.14423
- Excited to share I've joined Google as VP of Product for Core ML/AI! Dream job at the perfect time, blending cutting-edge AI with direct user impact across Google's product portfolio.
- AI is going to kill search as we know it. What's going to be fascinating is watching Google miss that boat.
- You have probably heard recently that Direct Preference Optimization (DPO) is taking over RLHF as the preferred method to align LLMs to human preferences (arxiv.org/abs/2305.18290). Well, that is "old news" now. The newest/coolest thing now is Kahneman Tversky optimization (KTO)
- Our new survey on LLMs is now available in arXiV. Great team work with awesome collaborators. Our goal is to give a comprehensive overview of LLMs (including forward looking work like post-attention, SLMs and agents) while keeping it very readable. arxiv.org/abs/2402.06196
- Google assistant, a product developed by a company with thousands of AI researchers and engineers, cannot auto detect language or describe an image you get on text. That's how hard deploying AI in product is.
- Ten years ago @JustinBasilico & me published a blog post describing an architectural blueprint for Recommender Systems. I'm now revisiting it by including several alternatives published since, and a new one that in some ways includes all the previous ones: amatriain.net/blog/RecsysArc…
- Thompson Sampling has been one of my favorite algorithms due to its efficiency and simplicity. It turns out that it also works for LLM alignment! Great paper by Deepmind on an extension to DPO
