user avatar
Christopher Manning
@chrmanning
Founder @stanfordnlp & cs224n—Senior Fellow @StanfordHAI—Prof. CS & Linguistics @Stanford—GP @aixventureshq—MTS @moonlake—Australian🇦🇺—Do #NLProc & #AI 👋
Palo Alto
Joined September 2014
Posts
  • user avatar
    Re: “Every major breakthrough in AI has been American”: America does itself no favors when it overestimates its specialness. Yes, the center of the AI industry is the US (California!), but many of the breakthroughs of (neural, gradient-based) AI happened elsewhere: • LSTMs,
    DeepSeek is a wake up call for America, but it doesn’t change the strategy: - USA must out-innovate &race faster, as we have done in the entire history of AI - Tighten export controls on chips so that we can maintain future leads Every major breakthrough in AI has been American
  • user avatar
    I do not believe human-level AI (artificial superintelligence, or the commonest sense of #AGI) is close at hand. AI has made breakthroughs, but the claim of AGI by 2030 is as laughable as claims of AGI by 1980 are in retrospect. Look how similar the rhetoric was in @LIFE in 1970!
  • user avatar
    Machine Learning just ate Algorithms in one large bite, thx to @tim_kraska, @alexbeutel, @edchi, @JeffDean & Polyzotis at @Google—faster, smaller trees, hashes, bloom filters arxiv-vanity.com/papers/1712.01…
  • user avatar
    I’ve kept quiet on the @OpenAI fiasco, since I also don’t know what’s going on, 🤷 but I can’t possibly support today’s interim CEO—the below in a thread on “50/50 everyone gets paperclipped & dies”—or a residue board that believes in these EA-infused fantasy lands. HT @vkhosla.
    Replying to @BarbettiJames @ApriiSR and @BellaRudd1
    The Nazis were very evil, but I'd rather the actual literal Nazis take over the world forever than flip a coin on the end of all value.
  • user avatar
    “The fact that [transformer neural nets] model language is probably one of the biggest discoveries in history. That you can learn language by just predicting the next word with a Markov chain—that’s just shocking to me,” Mikhail Belkin says. By @strwbilly.
  • user avatar
    This paper gives some really nice insights and mathematical depth to what had previously (for us) been “the mystery of squared distance” in revealing the representation of parse trees in deep contextual representations (BERT, ELMo, etc.). Great to read!
    How does a neural net represent language? See the visualizations and geometry in this PAIR team paper arxiv.org/abs/1906.02715 and blog post pair-code.github.io/interpretabili…
  • user avatar
    This is truly an opinion piece. Not even a cursory attempt is made to check easily refutable claims (“they may well predict, incorrectly”). Melodramatic claims of inadequacy are made not of specific current models but any possible machine learning approach
  • user avatar
    Artificial Intelligence Definitions: This (northern) summer, I spent more time than I’d like to admit coming up with a handout defining key terms in AI in 1 page, trying to be informative and suitable for non-specialists – let me know if you like them! hai.stanford.edu/sites/default/…
  • user avatar
    Meanwhile at @Stanford, we just encourage all students to take as many CS courses as they would like …
  • user avatar
    There are 2 mistakes you can make about LLMs: ① Thinking everything LLMs say is correct, they can reason, and with a bit more scale they’ll get us to superintelligence ② Thinking LLMs are good for almost nothing—they are FAR better at all #NLProc tasks than previous methods
  • user avatar
    Replying to @chrmanning
    But most AI people work in the quiet middle: We see huge benefits from people using AI in healthcare, education, …, and we see serious AI risks & harms but believe we can minimize them with careful engineering & regulation, just as happened with electricity, cars, planes, ….
  • user avatar
    Dear @emilymbender—and @Abebab—you need to keep “reminding” people of your viewpoint because it is not an argument that is convincing to all or a self-evident truth. It is a particular academic position, which lots of people support but a good number of others disagree with. 1/8
    Yes, exactly this. I wish we didn't need to keep reminding people, and @Abebab is commendable for being gentle about it! For the long form of this argument, see Bender & @alkoller 2020: aclanthology.org/2020.acl-main.…
  • user avatar
    Reflecting again on how knowing all the architecture & equations of the Transformer model is really of no use at all in convincingly explaining to someone how an LLM like ChatGPT can write paragraphs of lucid text in response to a prompt. I guess I’m saying “Beware reductionism”.
  • user avatar
    Replying to @chrmanning
    LLMs and other generative AI are enormously powerful, because they soak up, abstract, and can mashup the work of millions of humans. But they are only a bit more intelligent than an encyclopedia. Central to intelligence is the ability to learn, adapt, and act in novel situations.