user avatar
Sanjeev Arora
@prfsanjeevarora
Director, @PrincetonPLI and Professor @PrincetonCS. Seeks math/conceptual understanding of deep learning and large AI models. Also on the "other" social network
New Jersey, USA
Joined July 2017
Posts
  • Pinned
    user avatar
    Really excited about the launch of this research initiative. Hiring Research Scientists now. Research Software Engineers and postdocs over next few months. 300 H100 GPUs. Multidisciplinary teams. Princeton helps keep AI expertise in the open sphere. More: pli.princeton.edu
    “The dramatic rise of AI capabilities…is a watershed event for humanity…It is also sure to transform research and teaching in every academic discipline.” – @prfsanjeevarora, director of the new @Princeton Language and Intelligence initiative. For more: pli.princeton.edu
    Sanjeev Arora, director of Princeton Language and Intelligence (PLI)
  • user avatar
    An old friend working at a big-3 frontier AI lab asked me recently about their agenda to create research agents that could do research as good as (or better than) grads or faculty. My reply was essentially similar to @karpathy 's : please work on an agent that can boost my
    My pleasure to come on Dwarkesh last week, I thought the questions and conversation were really good. I re-watched the pod just now too. First of all, yes I know, and I'm sorry that I speak so fast :). It's to my detriment because sometimes my speaking thread out-executes my
  • user avatar
    Conventional wisdom: "Not enough data? Use classic learners (Random Forests, RBF SVM, ..), not deep nets." New paper: infinitely wide nets beat these and also beat finite nets. Infinite nets train faster than finite nets here (hint: Neural Tangent Kernel)! arxiv.org/abs/1910.01663
  • user avatar
    "Is optimization the right language to understand the brain?" is a famous controversy in neuroscience. My new blog post asks if optimization is the right language even to understand deep learning? (TL;DR: let's think: trajectories!)
  • user avatar
    Princeton has a new Center for Language and Intelligence, researching LLMs + large AI models, as well as their interdisciplinary applications. Looking for postdocs/research scientists/engineers; attractive conditions. nlp.cs.princeton.edu/center-languag…
  • user avatar
    Completely misses the point. Nobody is suggesting that solving IMO problems is useful for math research. The point is that AI has become really good at complex reasoning, and is not just memorizing its training data. It can handle completely new IMO questions designed by a
    Quote of the day: I certainly don't agree that machines which can solve IMO problems will be useful for mathematicians doing research, in the same way that when I arrived in Cambridge UK as an undergraduate clutching my IMO gold medal I was in no position to help any of the
  • user avatar
    Conventional wisdom: slowly decay learning rate (lr) when training deep nets. Empirically, some exotic lr schedules also work, eg cosine. New work with Zhiyuan Li: exponentially increasing lr works too! Experiments + surprising math explanation. See tinyurl.com/y3s62jbw
  • user avatar
    Blogpost on our new theory for word2vec-like representation learning methods for images, text, etc. Explains why representation do well on previously unseen classification tasks offconvex.org/2019/03/19/CUR… Relevant to meta learning, transfer learning? Paper arxiv.org/abs/1902.09229
  • user avatar
    Workshop: "Theory of Deep Learning: Where Next?" at the Institute for Advanced Study, Tuesday--Friday this week. Amazing schedule of talks! math.ias.edu/wtdl Registration is closed (sorry), but follow livestream here ias.edu/livestream
  • user avatar
    Off to ICML'18 to present a tutorial on "Toward Theoretical Understanding of Deep Learning" Tuesday 1pm. Lecture slides and bibliography here.unsupervised.cs.princeton.edu/deeplearningtu…
  • user avatar
    Big congratulations to Avi Wigderson of IAS Princeton for winning the Turing Award in CS. Truly an all-time great in theoretical computer science and discrete math. Also one of the nicest human beings I know --friend and mentor to so many (including me) tinyurl.com/fz5vxxaf
  • user avatar
    Our long-delayed blogpost on ICLR20 paper that shows current deep nets can be trained with learning rate that is exponentially increasing. Not just experiments but also a mathematical proof that this is at least as powerful as usual LR tuning.
  • user avatar
    Feels like a passing of the torch between fields. When I was a teenager in the 1980s, after a half-century of monumental progress powered by theoretical physics, most smart high schoolers wanted to do physics. Upon arriving as an undergrad at MIT in 1988, it quickly became clear
    BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”
  • user avatar
    Major news in AI today. Hinton is the father of modern deep learning and AI. Lecun and Bengio were his postdocs. @ilyasut of OpenAI was his student.