Lisa Dunlap (@lisabdunlap) / X

Lisa Dunlap

587 posts

Lisa Dunlap

@lisabdunlap

messing around with model auditing | PhD @berkeley_ai and @BerkeleySky | prev @lmarena_ai

Joined October 2021

Pinned
Lisa Dunlap
@lisabdunlap
Dec 15, 2025
🧵Tired of scrolling through your horribly long model traces in VSCode to figure out why your model failed? We made StringSight to fix this: an automated pipeline for analyzing your model outputs at scale. ➡️Demo: stringsight.com ➡️Blog: blog.stringsight.com
00:00
28K
Lisa Dunlap
@lisabdunlap
Nov 2, 2025
So is the formula to just name the most famous institutions and call it an X paper? Neither the first or last author are from Anthropic or Stanford. I get that reputation matters for publicity but it does seem a little disrespectful
Rohan Paul
@rohanpaul_ai
Nov 2, 2025
New Stanford+Anthropic paper shows long step-by-step prompts can break model safety and trigger harmful answers. 😟 Long reasoning can quietly neutralize safety checks that people assume are working. The trick adds a benign puzzle and long reasoning before the harmful ask, plus
84K
Lisa Dunlap
@lisabdunlap
Aug 22, 2025
Don't know what to get your advisor for their birthday? Give them the best gift of all: their time back. Happy (late) birthday @profjoeyg :)
12K
Lisa Dunlap
@lisabdunlap
Dec 15, 2023
Highly recommend you ask people to sign your poster tube at conferences. I tried it for the first time this NeurIPS and it feels like I’m that cool kid who had a cast signed by the whole class :D
28K
Lisa Dunlap
@lisabdunlap
Oct 28, 2024
🧵We love measuring accuracy, but what about the vibes? Intro VibeCheck—a system that discovers and measures qualitative differences in LLMs. VibeCheck shows Llama is friendlier while GPT-4 focuses on ethics; these vibes can even predict model identity and user preference.
29K
Lisa Dunlap
@lisabdunlap
Dec 6, 2023
I finally feel like an ML researcher.
AK
@_akhaliq
Dec 6, 2023
Describing Differences in Image Sets with Natural Language paper page: huggingface.co/papers/2312.02… How do two sets of images differ? Discerning set-level differences is crucial for understanding model behaviors and analyzing datasets, yet manually sifting through thousands of
23K
Lisa Dunlap
@lisabdunlap
Dec 6, 2023
[1/5] Introducing VisDiff - an #AI tool that describes differences in image sets with natural language. VisDiff can summarize model failures, compare models, find nuanced dataset differences, discover what makes an image memorable, and so much more! …derstanding-visual-datasets.github.io/VisDiff-websit…
58K
Lisa Dunlap
@lisabdunlap
Dec 14, 2024
Replying to @sunjiao123sun_ and @NeurIPSConf
im confused, was there a reason she even mentioned race at all in this slide? What was this talk about?
37K
Lisa Dunlap
@lisabdunlap
Sep 7, 2024
Turns out you can just book a meeting room and announce an "invited talk" about whatever you want. Here is my talk and taste test of all the goldfish cracker flavors, with a goldfish arena so we could determine the best fish. God I love the PhD.
18K
Lisa Dunlap
@lisabdunlap
Dec 8, 2024
This week in the @berkeley_ai CompVision meeting we did 5 min presentations: short powerpoint presentations about anything you want, as long as it isn't related to work. Here is a sampling of what transpired:
15K
Lisa Dunlap
@lisabdunlap
Mar 10, 2025
New lab theme just dropped: Alice in wonderland
13K
Lisa Dunlap
@lisabdunlap
Jan 22, 2025
In honor of VibeCheck™️ getting accepted to ICLR, I have reduced my horrifically messy repo to like 3 files, added a demo notebook, improved the visualizations, and even added a nice gradio option to look through model vibes:
github.com
GitHub - lisadunlap/VibeCheck: Automated Qualitative Analysis of LLMs (ICLR 2025)
Automated Qualitative Analysis of LLMs (ICLR 2025) - lisadunlap/VibeCheck
4.2K
Lisa Dunlap
@lisabdunlap
Oct 19, 2025
I am at ICCV this week! Hit me up if you want to chat about research (or if you would like to give me a job) :D
11K
Lisa Dunlap
@lisabdunlap
Jul 24, 2024
Are Vision and Language Models Colorblind? Here at Chatbot Arena we do high-impact research, but sometimes you gotta have fun, ya know? Sometimes you have to waste a few hours of your life testing VQA models for color blindness. Enjoy. TL;DR plot of model responses per image
10K