Stephen Bates (@stats

Stephen Bates

279 posts

Stephen Bates

@stats_stephen

Assistant Professor, MIT EECS. Rigorous stats & ML methods for data-driven science and reliable AI systems. My research group is hiring postdocs & PhDs!

stephenbates19.github.io

Joined April 2018

Pinned
Stephen Bates
@stats_stephen
Oct 15, 2021
📰 Excited to share our new work on risk control in prediction! Multiple testing leads to practical calibration algorithms with PAC guarantees for any statistical error rate. Works with any model + data distribution! arxiv.org/abs/2110.01052 #Statistics #MachineLearning
Anastasios Nikolas Angelopoulos
@ml_angelopoulos
Oct 15, 2021
Thrilled to share Learn then Test, a tool to calibrate any model to control risk (eg. IOU, recall in object detection). No assns on model/data. See arXiv arxiv.org/abs/2110.01052 + Colab colab.research.google.com/github/aangelo… ✍️w/@stats_stephen, E.J. Candes, M.I. Jordan, @lihua_lei_stat! 🧵1/n
00:00
arxiv.org
Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees. Our calibration algorithms work with any...
Stephen Bates
@stats_stephen
Sep 13, 2023
Excited to share that I've joined MIT as an assistant professor in EECS! I'm thrilled to join many thoughtful, inspiring colleagues. Looking ahead, I'm working to develop statistical principles for AI models so that we can use them for science and reliable automated systems.
89K
Stephen Bates
@stats_stephen
Dec 13, 2022
Want to learn about concentration inequalities and high-dimensional statistics? Roman Vershynin just released 41 lecture videos! They go along with his beautiful book. This is an amazing new resource! math.uci.edu/~rvershyn/teac…
Stephen Bates
@stats_stephen
Nov 19, 2024
Our new textbook Theoretical Foundations of Conformal Prediction is out! Conformal prediction is a a statistical technique that augments ML systems with uncertainty information for safe deployment. This book lays out the core theory.
arxiv.org
Theoretical Foundations of Conformal Prediction
This book is about conformal prediction and related inferential techniques that build on permutation tests and exchangeability. These techniques are useful in a diverse array of tasks, including...
41K
Stephen Bates
@stats_stephen
Feb 10, 2022
Excited to share a new, simple regression adjustment to get causal estimates with longitudinal data! arxiv.org/abs/2201.13451 Causal inference with longitudinal data is hard!! Why? A 🧵👇 w/ @edwardhkennedy, @robtibshirani, V Ventura, and L Wasserman 1/n
Stephen Bates
@stats_stephen
Dec 5, 2023
If you're a prospective PhD student interested in statistics & uncertainty for ML 🤖⁉️, AI for science 🧪, or data with strategic behavior ♟️, I have openings in my group! Consider applying to the MIT EECS PhD program and mention me in your application. gradapply.mit.edu/eecs/apply/log…
46K
Stephen Bates
@stats_stephen
Jul 2, 2021
Need to give your ML model 🤖 reliable uncertainty quantification? Check out our new Gentle Intro to Conformal Prediction tutorial + video. You get valid confidence sets with any model for any (unknown) distribution, no retraining. sites.google.com/berkeley.edu/d… with @ml_angelopoulos
Stephen Bates
@stats_stephen
Sep 16, 2024
🚨Postdoc positions🚨 Excited to announce two postdoc openings in my research group at @MITEECS & @MITLIDS! We're working on statistical uncertainty quantification with AI models, AI for science, and statistics with strategic agents. Join us😀 stephenbates19.github.io/postdoc_call.h…
18K
Stephen Bates
@stats_stephen
Nov 17, 2023
Excited to share that our work on prediction-powered inference just came out in Science!
Science Magazine
@ScienceMagazine
Nov 16, 2023
A new statistical framework dubbed “prediction-powered inference” could enable researchers to draw valid and more data-efficient scientific conclusions using datasets enriched with machine-learning predictions. Learn more in Science: scim.ag/4QJ
23K
Stephen Bates
@stats_stephen
Sep 7, 2022
Deep learning predictions should *always* come with error bars. Conformal prediction is a practical, easy-to-use statistical technique for this. Check out our tutorial 👇 for a simple introduction. Lots of real examples with Jupyter notebooks!! 🦾 arxiv.org/abs/2107.07511
Anastasios Nikolas Angelopoulos
@ml_angelopoulos
Sep 7, 2022
📢Huge update to Gentle Introduction to Conformal Prediction📢 arxiv.org/abs/2107.07511 Notebooks for EVERY example, easy-2-run WITHOUT model/data download. Open+run in Colab!✅ New repo here: github.com/aangelopoulos/… New sections on time-series and risk control!✅ More in 🧵
Stephen Bates
@stats_stephen
Apr 11, 2021
What is cross-validation estimating? It turns out the answer is *not* "the accuracy of the model from my data" but is instead "the average accuracy over many unseen training sets" (at least for regression). New work with Trevor Hastie and Rob Tibshirani.
rob tibshirani
@robtibshirani
Apr 1, 2021
With postdoc Stephen Bates and Trevor Hastie, I have just completed a new paper "Cross-validation: what does it estimate and how well does it do it?" statweb.stanford.edu/~tibs/ftp/NCV.…
Stephen Bates
@stats_stephen
Jul 23, 2021
🚨The Distribution-free Uncertainty Quantification ICML workshop kicks off tomorrow!🚨 Leading off the morning session will be Rina Barber, Michael Jordan, Vladimir Vovk, Larry Wasserman, and Leying Guan. sites.google.com/berkeley.edu/d…
Stephen Bates
@stats_stephen
Jan 16, 2025
Announcing ICLR '25 workshop: Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI How can we trust large language models (LLMs) when they generate text with confidence, but sometimes hallucinate or fail to recognize their own
5.3K
Stephen Bates
@stats_stephen
Jan 25, 2023
If you want to use ML outputs in your regression model but need valid confidence intervals, we have a solution for you. Check out Prediction-Powered Inference! New work now online: arxiv.org/abs/2301.09633 Valid with any ML model and any data set -- no assumptions. 🤖🙌
Anastasios Nikolas Angelopoulos
@ml_angelopoulos
Jan 25, 2023
📯Prediction-Powered Inference📯 arxiv.org/abs/2301.09633 With the rise of AlphaFold etc., people are using ML predictions to replace costly experimental data. But predictions aren't perfect; can we still use them for rigorous downstream inferences? The answer: yes. A 🧵
arxiv.org
Prediction-Powered Inference
Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework...
11K