user avatar
Joshua Batson
@thebasepoint
trying to understand evolved systems (🖥 and 🧬) interpretability research @anthropicai formerly @czbiohub, @mit math
Oakland, CA
Joined February 2012
679
Following
6,207
Followers
Posts
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 13, 2020
    Alexis, I just spoke with a few of my colleagues who were on the panel, who said "these inaccurate notes were not reviewed, endorsed" and are missing significant context. In particular, the large % infected is not a forgone conclusion; coordinated action now can help.
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 13, 2020
    Replying to @thebasepoint and @AlexisMadrigal
    In China, Korea, and Singapore, we have seen that 1) this virus is containable, at least in the short term, and 2) that 1) is true in diverse population structures, political systems, using different primary response methods. Good take on response:
    washingtonpost.com
    Opinion | Quarantining cities isn’t needed. But a fast, coordinated response to covid-19 is...
    There is still time to implement tough mitigation measures that would reduce the scope of a nationwide epidemic. But that opportunity narrows with each passing day.
  • user avatar
    Joshua Batson
    @thebasepoint
    Apr 29, 2025
    Great post "So you want to work in mechanistic interpretability" about skills to develop and resources to use, whether you're coming more from research or engineering. (link in thread)
    37K
  • user avatar
    Joshua Batson
    @thebasepoint
    Oct 1, 2019
    Single-Cell RNA-seq data is notoriously noisy. There are dozens of methods for cleaning it, but how do you know when they’re working right? How clean is too clean? Presenting...Molecular Cross-Validation biorxiv.org/content/10.110…
    GIF
  • user avatar
    Joshua Batson
    @thebasepoint
    Oct 5, 2023
    In writing this paper, there were countless features we thought might be bugs. After careful inspection, ~all of them revealed surprising and subtle model properties. To me this capacity for surprise is the true test of a new technique. This thread is about my favorite finding.
    user avatar
    Anthropic
    @AnthropicAI
    Oct 5, 2023
    The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.
    105K
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 13, 2020
    Replying to @thebasepoint and @alexismadrigal
    Hospitals are sensibly preparing for incoming cases by clearing floors, postponing procedures, setting up intake stations to evaluate people w/ symptoms (the vast majority of which still don't have COVID19).
  • user avatar
    Joshua Batson
    @thebasepoint
    Feb 1, 2019
    I'm happy to share Noise2Self, a framework for blind-denoising high dimensional measurements. We calibrate classical image denoisers and train deep neural nets; the same idea works on matrices of single-cell gene expression. Find out how: arxiv.org/abs/1901.11365 w/ @loicaroyer
    GIF
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 13, 2020
    Replying to @thebasepoint and @alexismadrigal
    What's missing is additional logistical resources for those institutions (who can requisition a gym?) and support for its staff (school's canceled, doctors have kids), together with financial support for the many whose livelihoods are going to be severely disrupted.
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 13, 2020
    Replying to @thebasepoint and @alexismadrigal
    Leadership to coordinate btw hospitals, public health depts, testing labs, cities, and schools is sorely needed, and to set clearer guidelines so everyone isn't trying to figure this out for themselves. A few governors are stepping up (@GovInslee especially). Hoping for more.
  • user avatar
    Joshua Batson
    @thebasepoint
    Jun 30, 2020
    I want to talk about a toy model for reasoning about what viral genomics can and cannot tell us about #SARSCoV2 transmission. Suppose viral isolates from two people have *identical* genotypes. How many transmission events separate them? 1/13
  • user avatar
    Joshua Batson
    @thebasepoint
    Jul 31, 2025
    One of our major blindspots in the last circuits paper was attention -- we could see what information attention moved, but not *why* it moved it. Today, the team is sharing an update interpreting attention patterns using features.
    7.5K
  • user avatar
    Joshua Batson
    @thebasepoint
    May 29, 2025
    Our recent circuit work has two steps: 1. Sparsely approximate the transformer model 2. Generate attribution graphs for prompts using those approximations We're releasing code for step 2, and using building on open-weights versions of step 1. But soon....
    user avatar
    Anthropic
    @AnthropicAI
    May 29, 2025
    Our interpretability team recently released research that traced the thoughts of a large language model. Now we’re open-sourcing the method. Researchers can generate “attribution graphs” like those in our study, and explore them interactively.
    6.7K
  • user avatar
    Joshua Batson
    @thebasepoint
    Mar 15, 2020
    Hi genomics/covid twitter! We are gearing up to do a lot of COVID19 mNGS of nasal + oral swabs, with goals of getting high-quality whole genomes and detecting secondary pathogens. Looking for advice on best practices for assembly and qc of genomes. Can you share your workflows?
  • user avatar
    Joshua Batson
    @thebasepoint
    Oct 5, 2023
    Replying to @thebasepoint
    It turns out the letters vs digits features were driven by *properties of the tokenizer*. Even on random alphanumeric strings, the tokenizer leaks information about future tokens! If you see a "7" the next token can't be an "8" because it would have been tokenized as "78".
    10K