I’m excited to join @DeepMind as a research scientist next week :)
Conor Durkan
148 posts
Conor Durkan
@conormdurkan
Research @GoogleDeepMind
- I like the Bayesian framing of reward-based post-training (i.e. reward-maximization with a KL penalty). Up to an additive constant, reward functions are log-likelihoods, and the pre-trained model is a prior. Then the posterior target is the product of the likelihoods and prior
- "On Maximum Likelihood Training of Score-Based Generative Models" -- a new technical note with @YSongStanford showing how score-based generative models can be fit using maximum likelihood. 1/7
- In motivating their 'grad' API, the JAX docs point to the prologue of 'Functional Differential Geometry' (mitpress.mit.edu/books/function…), which shows how the Euler-Lagrange equations can be written in a purely functional form, and is worth a read.
- "On Contrastive Learning for Likelihood-free Inference" - new work with @driainmurray and @gpapamak. arxiv.org/abs/2002.03712
- I'm delighted to launch Udio this week with fantastic co-founders @DavidDingAI @charlietcnash @yaroslav_ganin @avincentsanchez I've wanted to work on music models for a long time, and it's pretty surreal to this in the wild. It's also been great to work on something that's justIntroducing Udio, an app for music creation and sharing that allows you to generate amazing music in your favorite styles with intuitive and powerful text-prompting. 1/11
00:00 - This paper addresses a major drawback of likelihood-based generative models, which traditionally assume a distribution with full support in the ambient data space, whereas we believe this is likely not the case for highly-structured data like e.g. images. Very exciting work.Johann & I released v2 of our paper "Flows for simultaneous manifold learning and density estimation" with more experiments. We dubbed the class of model ℳ-Flows Here you see the flow learning the 2d manifold and the density for the Lorenz attractor! 1/n arxiv.org/abs/2003.13913
GIF - This model has a magical quality that’s hard to convey without trying it yourself. It’s musical and idiomatic in ways that I thought were years away. It was an extraordinary thing to work on.Today with @YouTube, we’re announcing Lyria: our most advanced music generation model to date. 🎶 We’re also releasing 2️⃣ AI experiments in close collaboration with participating artists and creators to bring their ideas to life responsibly. → deepmind.google/discover/blog/…
GIF - Chatting with Flamingo about images is definitely the most organic experience I’ve had with an ML model. The ability to readily describe output from e.g. DALL-E 2 might be the closest we’ve come to two independently-trained large-scale models having a conversation 👀Introducing Flamingo 🦩: a generalist visual language model that can rapidly adapt its behaviour given just a handful of examples. Out of the box, it's also capable of rich visual dialog. Read more: dpmd.ai/dm-flamingo 1/
- "On Contrastive Learning for Likelihood-free Inference" will appear at ICML 2020. #icml2020"On Contrastive Learning for Likelihood-free Inference" - new work with @driainmurray and @gpapamak. arxiv.org/abs/2002.03712
- Neural spline flows replace affine transformations in coupling and autoregressive layers with flexible monotonic rational-quadratic splines, significantly boosting density estimation performance without sacrificing analytic invertibility.
