user avatar
Jacob Andreas
@jacobandreas
Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL / @NLP_MIT (he/him). lingo.csail.mit.edu web.mit.edu/jda/www
Cambridge, MA
Joined March 2007
947
Following
23.6K
Followers
Posts
  • Pinned
    user avatar
    Jacob Andreas
    @jacobandreas
    Mar 5
    👉 New preprint: how do we make LMs more reliable once there's no more training data? Enforcing *consistency* of LM predictions across inputs lets us unsupervisedly optimize for factual accuracy & faithful explanation (& get a unifying view on many existing post-training algs)
    user avatar
    Itamar Pres
    @PresItamar
    Mar 5
    New paper: It's time to optimize for 🔁self-consistency 🔁 We’ve pushed LLMs to the limits of available data, yet failures like sycophancy and factual inconsistency persist. We argue these stem from the same assumption: that behavior can be specified one I/O pair at a time. 🧵
    14K
  • user avatar
    Jacob Andreas
    @jacobandreas
    Dec 6, 2022
    Speculative (!!!) paper arguing that big LMs can model agency & communicative intent: arxiv.org/abs/2212.01681 (somehow in EMNLP findings). Briefly: 1. LMs do not in general have beliefs or goals. An LM trained on the Internet models a distribution over next tokens *marginalized*
    First page of the paper "Language Models as Agent Models" (available at link in tweet)
  • user avatar
    Jacob Andreas
    @jacobandreas
    Oct 28, 2017
    Deep RL is popular because it's the only area in ML where it's socially acceptable to train on the test set.
  • user avatar
    Jacob Andreas
    @jacobandreas
    Jun 14, 2018
    Excited to announce that I'll be joining the @MIT_CSAIL faculty in fall 2019!
  • user avatar
    Jacob Andreas
    @jacobandreas
    Nov 3, 2018
    ATTENTION TWITTER I AM LOOKING FOR STUDENTS: Are you interested in doing things with language & machine learning? Doing things to machine learning using language? Doing language learning to machine things? Apply to MIT! gradapply.mit.edu/eecs/apply/log…
  • user avatar
    Jacob Andreas
    @jacobandreas
    Jan 5, 2021
    very relieved, after all these years of working on blocks world, that "a daikon in a tutu walking a dog" is easier to model than "a large red block sitting below a small blue block"
  • user avatar
    Jacob Andreas
    @jacobandreas
    Oct 4, 2023
    Excited to see that it’s that time of the year when we reinvent probing again
    user avatar
    Dan Hendrycks
    @hendrycks
    Oct 3, 2023
    AI models are not just black boxes or giant inscrutable matrices. We discover they have interpretable internal representations, and we control these to influence hallucinations, bias, harmfulness, and whether a LLM lies. 🌐: ai-transparency.org 📄: arxiv.org/abs/2310.01405
    83K
  • user avatar
    Jacob Andreas
    @jacobandreas
    Dec 25, 2023
    When Pratyusha showed me the results from her summer internship my head 🤯. Incredibly surprising results about knowledge representation, robustness, and rank in neural sequence models.
    user avatar
    Pratyusha Sharma
    @pratyusha_PS
    Dec 24, 2023
    What if I told you that you can simultaneously enhance an LLM's task performance and reduce its size with no additional training? We find selective low-rank reduction of matrices in a transformer can improve its performance on language understanding tasks, at times by 30% pts!🧵
    59K
  • user avatar
    Jacob Andreas
    @jacobandreas
    Jul 26, 2024
    Some thoughts on how to think about "world models" in language models and beyond: lingo.csail.mit.edu/blog/world_mod…
    35K
  • user avatar
    Jacob Andreas
    @jacobandreas
    Jul 28, 2018
    Some thoughts on meaning representations in models without logical forms: blog.jacobandreas.net/meaning-belief…
  • user avatar
    Jacob Andreas
    @jacobandreas
    Dec 16, 2024
    Is your CS dept worried about what academic research should be in the age of LLMs? Hire one of my lab members! Leshem Choshen (@LChoshen), Pratyusha Sharma (@pratyusha_PS) and Ekin Akyürek (@akyurekekin) are all on the job market with unique perspectives on the future of NLP: 🧵
    35K
  • user avatar
    Jacob Andreas
    @jacobandreas
    Sep 21, 2020
    New TACL paper from Semantic Machines! A first look at the approach we've been developing for modeling complex, task-oriented conversations using dataflow graphs. aka.ms/AA9oxf3 I am extremely excited about this work for several reasons:
    Creating a more natural conversational AI with dataflow graphs
    From microsoft.com
  • user avatar
    Jacob Andreas
    @jacobandreas
    Dec 8, 2018
    hottest trend at #NeurIPS2018 is calling every learning problem a meta-learning problem
  • user avatar
    Jacob Andreas
    @jacobandreas
    May 6, 2022
    New NAACL paper by @belindazli! We know lots about LM generalization at the example level, but comparatively little about LM "adaptability" at the task level. This paper uses a huge set of *randomly generated* tasks to explore the limits of LM adaptability. 🧵 w/ key findings:
    Hundreds of randomly generated tasks. Example: `map(seq, \x. occupation_of(father_of(x)))`.