user avatar
Michael C. Frank
@mcxfrank
Cognitive scientist at Stanford. Open science advocate. @stanfordsymsys director. Bluegrass picker, slow runner, dad.
Palo Alto, CA
stanford.edu/~mcfrank/
Joined June 2012
1,447
Following
14.3K
Followers
Posts
  • Pinned
    user avatar
    Michael C. Frank
    @mcxfrank
    Jan 20, 2023
    Do you want to do a psychology experiment while following best practices in open science? My collaborators and I have created Experimentology, a new open web textbook (to be published by MIT Press but free online forever). experimentology.io Some highlights! 🧵
    239K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 4, 2023
    People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵
    549K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Mar 27, 2023
    How do we compare the scale of language learning input for large language models vs. humans? I've been trying to come to grips with recent progress in AI. Let me explain these two illustrations I made to help. 🧵
    2.6M
  • user avatar
    Michael C. Frank
    @mcxfrank
    Aug 17, 2022
    Wordbank is an open database of children's vocabulary development across languages, archiving parent reports of what their children say. Today we are updating Wordbank, adding 10k children, 9 new languages, and data on bilinguals. wordbank.stanford.edu
    Wordbank homepage
  • user avatar
    Michael C. Frank
    @mcxfrank
    Jun 22, 2018
    Everyone makes mistakes during data analysis. Literally everyone. The question is not what errors you make, it's what systems you put into place to prevent them from happening. Here are mine. [a thread because I'm sad to miss #SIPS2018]
    user avatar
    simine vazire
    @siminevazire
    Jun 21, 2018
    Some thoughts as I get ready for #SIPS2018 : Some people say the credibility revolution has made researchers afraid of making mistakes. 1/4
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 10, 2023
    What does it mean for a large language model (LLM) to "have" a particular ability? Developmental psychologists argue about these questions all the time and have for decades. There are some ground rules. 🧵
    diagram of developmental change referenced later in the post
    diagram of abstractions linking to observations referenced later in the post
    188K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Aug 6, 2018
    A thought on grad advising. When I was a second year, an announcement went out to our dept. with the abstract for a talk I was giving in the area talk series. A senior faculty member wrote back with a scathing critique (cc'd to my advisor, @LanguageMIT). /1
  • user avatar
    Michael C. Frank
    @mcxfrank
    Jun 27, 2023
    How do we use methods from developmental psychology to assess AI models? My comment, "'Baby steps' in evaluating the capacities of large language models" is now out in Nature Reviews Psychology: nature.com/articles/s4415…
    75K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Dec 19, 2018
    Errors in data analysis code are a major problem for scientists and for science. Two practices that can help you avoid them (or track them down more easily) : 1. never copy and paste code, and 2. never write a code block longer than your screen. [thread]
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 19, 2023
    Can a large language model be used as a "cognitive model" - meaning, a scientific artifact that helps us reason about the emergence of complex behavior and abstract representations in the human mind? My answer is YES. Why and under what conditions? 🧵
    Figure showing a computational artifact being used as a model of a human system, with potential interventions of interest listed
    106K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Sep 24, 2018
    What is "the open science movement"? It's a set of beliefs, research practices, results, and policies that are organized around the central roles of transparency and verifiability in scientific practice. An introductory thread. /1
  • user avatar
    Michael C. Frank
    @mcxfrank
    Feb 27, 2018
    Mixed effects models: is it time to go Bayesian by default? New blogpost musing about what our default statistical choices should be: babieslearninglanguage.blogspot.com/2018/02/mixed-…
  • user avatar
    Michael C. Frank
    @mcxfrank
    Sep 19, 2023
    Coming soon: the Open Encyclopedia of Cognitive Science, an open access reference work devoted to the study of the mind. @asifa_majid and I are co-editors in chief – articles will be posted starting in 2024. oecs.mit.edu
    Open Encyclopedia of Cognitive Science logo
    32K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Sep 23, 2019
    The first ManyBabies project, "Quantifying sources of variability in infancy research using the infant-directed speech preference," has been accepted at AMPPS! psyarxiv.com/s98ab This 67 lab (!) paper shows the power of collaboration to advance methods and theory!