Durk Kingma (@dpkingma) / X

Durk Kingma

706 posts

Durk Kingma

@dpkingma

@AnthropicAI. Prev. @Google Brain/DeepMind, founding team @OpenAI. Computer scientist; inventor of the VAE, Adam optimizer, and other methods. ML PhD.

Joined March 2009

Pinned
Durk Kingma
@dpkingma
Oct 1, 2024
Personal news: I'm joining @AnthropicAI! 😄 Anthropic's approach to AI development resonates significantly with my own beliefs; looking forward to contributing to Anthropic's mission of developing powerful AI systems responsibly. Can't wait to work with their talented team,
350K
Durk Kingma
@dpkingma
Apr 9, 2022
Generative models (such as Dall-E 2 and PaLM) are becoming just such an insanely powerful, almost magic-like technology, it's completely NUTS. And it seems like most (non-ML) people still don't fully grasp the implications. This technology will thoroughly transform society.
Durk Kingma
@dpkingma
May 29, 2022
It was 16 years ago, in 2006, that @geoffreyhinton et al released their demo of deep belief nets. Undergrad me was highly impressed, and helped convince me that deep learning was the way to go. I refreshed Geoff's website almost every day checking for new papers... (1/n)
00:00
Durk Kingma
@dpkingma
Sep 30, 2017
"Variational Inference and Deep Learning: A New Synthesis", written by yours truly, is now available for D/L here: goo.gl/6aGYZ1.
Durk Kingma
@dpkingma
Dec 10, 2023
I hope this document ends up in LLM training data 😂
114K
Durk Kingma
@dpkingma
Jul 9, 2018
Check out blog.openai.com/glow/, my work with @prafdhar on improving flow-based generative models with invertible 1x1 convolutions. youtu.be/exJZOC3ZceA
00:00
Durk Kingma
@dpkingma
Mar 3, 2023
New theoretical work on diffusion objectives: arxiv.org/abs/2303.00848 We e.g. show that under a simple condition (monotonic weighting, satisfied by e.g. the v-prediction loss), diffusion objectives equal the ELBO with data augmentation, namely additive noise. 1/2
147K
Durk Kingma
@dpkingma
Jul 6, 2021
New paper: Variational Diffusion Models (VDMs)! arxiv.org/abs/2107.00630 ✅ New general insights into diffusion models ✅ Simple objective ✅ Fast optimization & anytime synthesis ✅ SotA likelihoods & lossless compression Work with @TimSalimans @poolio @hojonathanho (1/n)
Durk Kingma
@dpkingma
May 7, 2024
Thanks to the ICLR Award Committee! And thank you for the kind words, Max! You were the perfect Ph.D. advisor and collaborator, kind and inspiring. I really couldn't have wished for better.
Max Welling
@wellingmax
May 7, 2024
Thank you Yisong and the Award Committee for choosing the VAE for the Test of Time award. I like to congratulate Durk who was my first (brilliant) student when moving back to the Netherlands and who is the main architect of the VAE. It was absolutely fantastic working with him.
67K
Durk Kingma
@dpkingma
Aug 31, 2019
Someone is obviously really close to solving AGI: adamoptimizer.com
Durk Kingma
@dpkingma
Jul 29, 2017
A figure I made for explaining variational autoencoders (VAEs) as part of a larger work-in-progress.
Durk Kingma
@dpkingma
Jul 6, 2020
Are nonlinear features learned by deep discriminative, contrastive, autoregressive etc. models arbitrary? No! We show (theoretically and empirically) that, under mild conditions, you will learn the same features every time you train, up to only a linear transformation.
Durk Kingma
@dpkingma
Jan 27, 2020
Another great result demonstrating that VAEs (deep learning + amortized variational inference) make a lot of sense for data compression. Its loss function directly maximizes compressibility, and the resulting codec is fully parallelizable.
Taco Cohen
@TacoCohen
Jan 27, 2020
Short but sweet paper on recurrent autoencoder architectures for speech compression. We systematically explore the space of RNN-AEs and show that the best method, dubbed FRAE, outperforms classical codecs by a large margin. Check it out!
Durk Kingma
@dpkingma
Jan 8, 2020
Our paper "Variational Autoencoders and Nonlinear ICA: A Unifying Framework" has been accepted to AISTATS'20. With @ilkhem, Ricardo Pio Monti and Aapo Hyvarinen (UCL). Surprisingly strong and general identifiability results, with rigorous proofs! arxiv.org/abs/1907.04809