Log inSign up
user avatar
Dan Roberts
@danintheory
Scientist @OpenAI. Prev. co-founder @diffeo, acquired by @salesforce // co-authored The Principles of Deep Learning Theory // studied gravity.
San Francisco, CA
danintheory.com
Joined June 2009
749
Following
6,787
Followers
  • Pinned
    user avatar
    Dan Roberts
    @danintheory
    Jun 4
    TL; DW I say things (on reinforcement learning, the process of scientific discovery, and how physicists approach work on AI).
    user avatar
    Matt Turck
    @mattturck
    Jun 4
    Why AI Can Now Make Discoveries - my conversation with @danintheory, Lead of the Foundations of Reinforcement Learning team at @OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum
    00:00
    8.3K
  • user avatar
    Dan Roberts
    @danintheory
    Jun 18, 2021
    I'm delighted to share publicly "The Principles of Deep Learning Theory," co-written with @ShoYaida, and based on work also with @BorisHanin. It will appear on the @arxiv on Sunday and will be published by @CambridgeUP early next year: deeplearningtheory.com 1/
  • user avatar
    Dan Roberts
    @danintheory
    May 26, 2022
    The Principles of Deep Learning Theory, by @ShoYaida, @BorisHanin, and myself, is available in print today from @CambridgeUP! You can find links to purchase (as well as a link to a free arxiv draft) here: deeplearningtheory.com 1/4
  • user avatar
    Dan Roberts
    @danintheory
    May 16, 2024
    Update on this: I joined @OpenAI.
    user avatar
    Dan Roberts
    @danintheory
    Apr 27, 2024
    Finally tried the new AI assistant: seems like it has a long way to go before it matches even the most basic capabilities of the old AI assistant…
    228K
  • user avatar
    Dan Roberts
    @danintheory
    May 9, 2025
    My short talk at @sequoia AI Ascent on how we at @OpenAI are attempting to flip the old "cherry-on-top" meme about "Scaling RL"
    00:00
    138K
  • user avatar
    Dan Roberts
    @danintheory
    Mar 27, 2024
    Do LLMs really need to be so L? That's a rejected title for a new paper w/ @Andr3yGR, @kushal_tirumala, @Hasan_Shap, @PaoloGlorioso1 on pruning open-weight LLMs: we can remove up to *half* the layers of Llama-2 70B w/ essentially no impact on performance on QA benchmarks. 1/
    93K
  • user avatar
    Dan Roberts
    @danintheory
    Jun 18, 2021
    Replying to @danintheory
    This formula accurately gives the output of a deep MLP after training. 11/
  • user avatar
    Dan Roberts
    @danintheory
    Feb 20, 2022
    I respectfully disagree w/ this and @ylecun's perspectives on causality: (a) causality is an essential part of microscopic physics and (b) even at the effective level of Newtonian physics, F=ma is *not* symmetric since it's a differential equation, not an algebraic equality. 1/
    user avatar
    Judea Pearl
    @yudapearl
    Feb 19, 2022
    You don't need to go to CPT to argue that "deep down" there is no such thing as causality. Newtonian mechanics is also governed by algebraic equalities, so it is symmetric, telling us that f=ma is as valid as a=f/m. Where causality comes in is when we venture to model the 1/2
  • user avatar
    Dan Roberts
    @danintheory
    May 8, 2025
    I gave a talk where the only real thing to say is that we are inverting the meme
    user avatar
    Sonya Huang 🐥
    @sonyatweetybird
    May 7, 2025
    Replying to @sonyatweetybird @sequoia and 7 others
    @danintheory of @OpenAI Strawberry 🍓: * Reinforcement learning is shifting from a “cherry on top” of the pre-training cake, to the main ingredient driving breakthroughs going forward * Imagine einstein-v1907-super-high had access to unlimited test-time compute. What new science
    30K
  • user avatar
    Dan Roberts
    @danintheory
    Jan 11, 2023
    If you like both physics and AI -- and in particular if you are interested in using physics tools to study AI -- please apply to the "Theoretical Physics for Deep Learning" workshop at the Aspen Center for Physics this summer (May 28 to June 18): aspenphys.org/physicists/sum…
    21K
  • user avatar
    Dan Roberts
    @danintheory
    Nov 1, 2022
    New work on the origin of @OpenAI's neural scaling laws w/ Alex Maloney and @jamiesully2: we solve a simplified model of scaling laws to gain insight into how scaling behavior arises and to probe its behavior in regimes where scaling laws break down. arxiv.org/abs/2210.16859 1/
  • user avatar
    Dan Roberts
    @danintheory
    Aug 6, 2021
    This week @ShoYaida & I gave 5 lectures based on our book at the Princeton Deep Learning Theory (PDLT) Summer School (…-learning-summer-school.princeton.edu) masterfully organized by @BorisHanin. For those interested, videos are online (linked below), & next week I'll summarize our talks here.
  • user avatar
    Dan Roberts
    @danintheory
    Dec 22, 2022
    Replying to @djstrouse
    I thought when you catch an AI in a logical paradox it’s supposed to self destruct?
    12K
  • user avatar
    Dan Roberts
    @danintheory
    Jan 24, 2025
    I’d be very glad to vouch for @shaunmmaguire ‘s work with DARPA and in physics: I affirm he did a tour with DARPA in Afghanistan, his startup Qadium worked with DARPA as part of the Memex program along with my startup @diffeo (cf. forbes.com/sites/thomasbr…), and he did his PhD in
    user avatar
    Steven Balik
    @laurenbalik
    Jan 23, 2025
    The beautiful thing about Shaun Maguire is nobody can confirm date/time specifics about his DARPA tour in Afghanistan. He was about to flunk out of every school he attended before his PhD from Caltech. Nobody from his background can confirm anything. I've done the work.
    41K