John Hewitt (@johnhewtt) / X

John Hewitt

244 posts

John Hewitt

@johnhewtt

Assistant Prof @columbia CS. Visiting Researcher @ Google DeepMind. PhD from @stanfordnlp. Language x Neural Nets.

New York, NY

cs.columbia.edu/~johnhew

Joined February 2015

Pinned
John Hewitt
@johnhewtt
Apr 29
New paper! Subliminal learning—transferring hidden signals between language models—is more powerful than we thought. By biasing the teacher with a steering vector instead of a prompt, we achieve strong, consistent transfer, which we use to study its mechanisms. w/@GeorgeMorgulis
20K
John Hewitt
@johnhewtt
Sep 4, 2025
My first NLP lectures at Columbia are in the books! In our first two lectures, we went over (1) learning from text with a simple word vector language model, and (2) tokenization of text. Lecture notes are brand new and freely available on my website (links in thread.)
73K
John Hewitt
@johnhewtt
Jun 12, 2024
I’m joining the Columbia Computer Science faculty as an assistant professor in fall 2025, and hiring my first students this upcoming cycle!! There’s so much to understand and improve in neural systems that learn from language — come tackle this with me!
100K
John Hewitt
@johnhewtt
Nov 25, 2024
I’m hiring PhD students in computer science at Columbia! Our lab will tackle core challenges in understanding and controlling neural models that interact with language. for example, - methods for LLM control - discoveries of LLM properties - pretraining for understanding
107K
John Hewitt
@johnhewtt
Apr 5, 2019
Does my unsupervised neural network learn syntax? In new #NAACL2019 paper with @chrmanning, our "structural probe" can show that your word representations embed entire parse trees. paper: nlp.stanford.edu/pubs/hewitt201… blog: nlp.stanford.edu/~johnhew/struc… code: github.com/john-hewitt/st… 1/4
John Hewitt
@johnhewtt
Feb 3, 2023
For this year's CS 224n: Natural Language Processing with Deep Learning, I've written notes on our Self-Attention and Transformers lecture. web.stanford.edu/class/cs224n/r… Topics: Problems with RNNs, then self-attention, then a 'minimal' self-attention architecture, then Transformers.
87K
John Hewitt
@johnhewtt
Jun 24, 2025
I’m beginning to share notes from my upcoming fall 2025 NLP class, Columbia COMS 4705. First up, some notes to help students brush up on math. Vectors, matrices, eigenstuff, probability distributions, entropy, divergences, matrix calculus cs.columbia.edu/~johnhew/coms4…
32K
John Hewitt
@johnhewtt
May 29, 2023
#acl2023! To understand language models, we must know how activation interventions affect predictions for any prefix. Hard for Transformers. Enter: the Backpack. Predictions are a weighted sum of non-contextual word vectors. -> predictable interventions! backpackmodels.science
GIF
107K
John Hewitt
@johnhewtt
Nov 15, 2023
I'm on the faculty market! My goal is to build language systems that we understand deeply through discovery and by design, so we can precisely control them and treat their failures. Let's tackle this grand challenge of science and engineering together. nlp.stanford.edu/~johnhew/
97K
John Hewitt
@johnhewtt
Oct 19, 2020
#emnlp2020 paper: we give some theoretical insight into the syntactic success of RNN LMs: we prove they can implement bounded-size stacks in their states to generate some bounded hierarchical langs with optimal memory! paper arxiv.org/pdf/2010.07515… blog nlp.stanford.edu/~johnhew/rnns-…
John Hewitt
@johnhewtt
Sep 24, 2024
If I finetune my LM just on responses, without conditioning on instructions, what happens when I test it with an instruction? Or if I finetune my LM just to generate poems from poem titles? Either way, the LM will roughly follow new instructions! Paper: arxiv.org/pdf/2409.14254
45K
John Hewitt
@johnhewtt
Jul 10, 2023
Our paper on Backpacks has won an Outstanding Paper Award at ACL 2023! If you're excited about both fascinating learned structure in language models, and designing architectures to enable interpretability while maintaining expressivity, take a read! backpackmodels.science
Stanford NLP Group
@stanfordnlp
Jul 9, 2023
Our papers of #ACL2023NLP: Backpack Language Models @johnhewtt, @jwthickstun, @chrmanning, @percyliang backpackmodels.science Mon July 10, poster 14:00-15:30, Frontenac Ballroom and Queen’s Quay
48K
John Hewitt
@johnhewtt
Dec 4, 2023
It’s conference time! Come say hello at EMNLP to hear my hot takes on understanding LMs Is your CS department hiring? Hey nice come talk to me! Do you know few people at EMNLP? Not for long; come talk to me! Here’s what I look like at a poster session when the lights go out
55K
John Hewitt
@johnhewtt
Jun 8, 2025
I wrote a note on linear transformations and symbols that traces a common conversation/interview I've had with students. Outer products, matrix rank, eigenvectors, linear RNNs -- the topics are really neat, and lead to great discussions of intuitions. cs.columbia.edu/~johnhew//fun-…
22K