Julia Kempe (@KempeLab) / X

Julia Kempe

231 posts

Julia Kempe

@KempeLab

Silver Professor at NYU Courant & CDS Director & Research Scientist at MetaFAIR Research in Machine Learning, past in Quantum Computing & Finance Posts my own.

Joined April 2024

Julia Kempe
@KempeLab
Oct 22, 2024
Looking for a PhD intern in my team at @AIatMeta in Paris starting Spring (12-24 weeks, 24 better). With Yann Ollivier, we tackle LLM reasoning/planning via RL training (System 1) with test-time optimization (System 2). Motivated students, please apply: metacareers.com/jobs/173828249…
68K
Julia Kempe
@KempeLab
Nov 6, 2024
My team @AIatMeta Paris reviews PhD interns now! Core Learning: synthetic data, generalization, feedback, generative models, inference: metacareers.com/jobs/522530270… RL reasoning: metacareers.com/jobs/173828249… Good fit, w strong publication record? Send short email (only if you haven't yet)
38K
Julia Kempe
@KempeLab
Oct 24, 2024
Recruiting another PhD intern in my team at @AIatMeta in *Paris* in Core Learning, starting Spring (12-24 weeks, 24 better). Topics: synthetic data, generalization, feedback, generative models, inference...Expertise in Diffusion Models a plus. Email&apply: metacareers.com/jobs/522530270…
36K
Julia Kempe
@KempeLab
Jun 12, 2024
How to leverage AI-synthesized data without catastrophic degradation? Rank-and-prune feedback, from humans or even weaker models, provably restores and even surpasses original performance! See arxiv.org/abs/2406.07515 @AIatMeta @feeelix_feng @dohmatobelvis @f_charton @yangpuPKU
72K
Julia Kempe
@KempeLab
Jul 27, 2024
1/ Model Collapse: Lots of great papers on model collapse & effects of mixing original & synthetic data (Nature by @iliaishacked et al out!). When Gerstgrasser et al first sent us their v1, few of them were discussed, eg: arxiv.org/pdf/2402.04376 arxiv.org/abs/2310.00429… ...
Rylan Schaeffer
@RylanSchaeffer
Jul 25, 2024
For anyone interested in model collapse, I strongly urge people to look at our COLM 2024 paper arxiv.org/abs/2404.01413 Model collapse appears when researchers intentionally induce it in ways that simply don't match what is actually done practice @alexandr_wang is wrong
83K
Julia Kempe
@KempeLab
Apr 12, 2025
Our ICLR25 papers: 🎉ICLR Spotlight: Strong Model Collapse arxiv.org/abs/2410.04840 🎉ICLR Spotlight: DRoP: Distributionally Robust Data Pruning arxiv.org/abs/2404.05579 Beyond Model Collapse arxiv.org/abs/2406.07515 Flavors of Margin arxiv.org/abs/2410.22069 More details here soon!
arxiv.org
Strong Model Collapse
Within the scaling laws paradigm, which underpins the training of large neural networks like ChatGPT and Llama, we consider a supervised regression setting and establish the existance of a strong...
20K
Julia Kempe
@KempeLab
Jul 26, 2024
#icml24 How to leverage AI-synthesized data without model collapse? Rank-and-prune feedback, from humans or even weaker models, provably restores and even surpasses original performance! See our work at TF2M tomorrow! @AIatMeta @feeelix_feng @dohmatobelvis @f_charton @yangpuPKU
42K
Julia Kempe
@KempeLab
Dec 15, 2024
If you - like many - believe that using more (good) data is better than repeating on less - guess again! And come to our poster "Emergent Properties..." at the SciForDL @scifordl workshop West Mtg Room 205-207 this afternoon (after 4pm)! With @f_charton @AIatMeta @NYUDataScience
11K
Julia Kempe
@KempeLab
Jul 24, 2024
#ICML24 Training on AI-generated data destroys scaling laws; mixing of real & AI-data leads to transient training plateaus! Interested? Come to our poster Thu 1:30pm "Tale of Tails: Model Collapse as a Change of Scaling Laws" w @dohmatobelvis @feeelix_feng @yangpuPKU @f_charton
35K
Julia Kempe
@KempeLab
Oct 23, 2024
Dropping data intelligently is the way forward for data-efficient foundation models. We show that current data pruning methods exacerbate class bias and introduce DRoP: Distributionally Robust Pruning arxiv.org/abs/2404.05579 @arvysogorets @KartikAhuja7 @AIatMeta @NYUDataScience
8.5K
Julia Kempe
@KempeLab
Feb 8, 2025
PILAF (Policy-Interpolated Learning for Aligned Feedback): our response sampling scheme that provably aligns LLM preference learning w maximizing the underlying oracle reward! arxiv.org/abs/2502.04270 @feeelix_feng @ArielKwiatkowsk @KunhaoZ @YaqiDuanPKU @AIatMeta @NYUDataScience
6.1K
Julia Kempe
@KempeLab
Nov 2, 2024
Optimization induces implicit bias. We study general steepest descent in homogeneous nets & show (generalized) convergence to a (generalized) KKT pt. Adam presents a curious case between l2 & l1: arxiv.org/abs/2410.22069 With Nikos Tsilivis & @galvardi @NYUDataScience @AIatMeta
5.8K
Julia Kempe
@KempeLab
Jul 26, 2024
#ICML2024 Mechanisitc study on how iterative tasks are learned with a transformer, leading to what we call "iteration head"! To be presented at the Mechanistic Interpretability workshop Sat: @CabannesVivien @CharlesArnal @f_charton @_Vassim @alicey_ang from @AIatMeta
7.3K
Julia Kempe
@KempeLab
Jan 15, 2025
Submit to our New Frontiers in Associative Memories workshop @iclr_conf. New architectures & algorithms, memory-augmented LLMs, energy-based models, Hopfield networks, assoc. memory & diffusion.. nfam.vizhub.ai openreview.net/group?id=ICLR.… Organizing with @DimaKrotov et al
5.4K