Joel Jang (@jang

Joel Jang

460 posts

Joel Jang

@jang_yoel

prev: Research Scientist @nvidiaai GEAR Lab, world modeling lead. LAPA, DreamGen, DreamDojo, and DreamZero. PhD dropout @uwcse

San Francisco, CA

Joined March 2021

Pinned
Joel Jang
@jang_yoel
Feb 6
Robotics right now feels like peak entropy. Everyone has a different bet on what will work, and they're all confident, which is why doing robotics research right now is so fun. I wrote an essay on the question that's been driving our work from DreamGen → DreamZero → what’s
World Models and the Data Problem in Robotics
From joeljang.github.io
57K
Joel Jang
@jang_yoel
May 20, 2025
Introducing 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧! We got humanoid robots to perform totally new 𝑣𝑒𝑟𝑏𝑠 in new environments through video world models. We believe video world models will solve the data problem in robotics. Bringing the paradigm of scaling human hours to GPU hours. Quick 🧵
00:00
117K
Joel Jang
@jang_yoel
Mar 17, 2025
Some personal life update: I have joined @NVIDIAAI GEAR lab as a full-time Research Scientist last month (after one year as a research intern)! I’ll continue to be working on developing general-purpose robot foundation models. Stay tuned for some exciting updates!
25K
Joel Jang
@jang_yoel
Feb 8, 2023
Scaling 📈 the total # of tasks during instruction tuning has been known to unlock new abilities in LMs. However, we find that an LM trained on a single task outperforms an LM trained on 300+ tasks on unseen tasks 🤯 📝:arxiv.org/abs/2302.03202 1/8
75K
Joel Jang
@jang_yoel
Oct 19, 2023
🎯 Tired of one-size-fits-all AI chatter? ChatGPT tends to generate verbose & overly informative responses. This is because the current RLHF pipeline only allows aligning LLMs to the general preferences of the population. However, in the real world, people may have multiple,
76K
Joel Jang
@jang_yoel
Oct 16, 2024
Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: arxiv.org/abs/2410.11758 🧵 1/9
00:00
46K
Joel Jang
@jang_yoel
Oct 5, 2022
(1/9) Pretrained LMs memorize a vast amount of world knowledge, including private & sensitive information that we might want to erase post hoc. We propose 🌟knowledge unlearning🌟 as the solution. 📝 : arxiv.org/abs/2210.01504
Joel Jang
@jang_yoel
Mar 17, 2023
✨Life Update✨ I am joining @uwcse @uwnlp as a PhD student working with @LukeZettlemoyer and @HannaHajishirzi this fall!! 🥳 I will mostly be working on making the current LLMs much more modular and dynamic. Super excited for the next steps to come!
19K
Joel Jang
@jang_yoel
May 3, 2022
How can we update LMs *whenever* we want on up-to-date knowledge with *minimal* cost? Furthermore, how can we evaluate if each update was successful or not? We introduce ✨ TemporalWiki ✨, a lifelong benchmark providing the solution. 📝 arxiv.org/abs/2204.14211 🧵 1/8
Joel Jang
@jang_yoel
Jun 20, 2025
🚀 GR00T Dreams code is live! NVIDIA GEAR Lab's open-source solution for robotics data via video world models. Fine-tune on any robot, generate 'dreams', extract actions with IDM, and train visuomotor policies with LeRobot datasets (GR00T N1.5, SmolVLA).
Joel Jang
@jang_yoel
May 20, 2025
Introducing 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧! We got humanoid robots to perform totally new 𝑣𝑒𝑟𝑏𝑠 in new environments through video world models. We believe video world models will solve the data problem in robotics. Bringing the paradigm of scaling human hours to GPU hours. Quick 🧵
00:00
GitHub - NVIDIA/GR00T-Dreams: DreamGen: Nvidia GEAR Lab's initiative to solve the robotics data...
From github.com
29K
Joel Jang
@jang_yoel
Jan 21, 2023
Excited to share that I'll be joining @ai2_mosaic at @allen_ai this summer as a research intern working with @rajammanabrolu and @xiangrenNLP! Can't wait to explore Seattle and work on super interesting research with amazing people!! :)
7.8K
Joel Jang
@jang_yoel
Oct 27, 2021
Pretrained language models encode world knowledge 🌐 📚 in their parameters. Then, can we train ever-changing LMs 🤖 that update their implicit knowledge as the world changes? Joint work w/ @SeonghyeonYe @soheeyang_ LG AI Research @seo_minjoon 📰 arxiv.org/abs/2110.03215 1/N
Joel Jang
@jang_yoel
Nov 12, 2024
Excited to share that 𝐋𝐀𝐏𝐀 has won the Best Paper Award at the CoRL 2024 Language and Robot Learning workshop, selected among 75 accepted papers! Both @SeonghyeonYe and I come from NLP backgrounds, where everything is built around tokenization. Drawing inspiration from
00:58
Joel Jang
@jang_yoel
Oct 16, 2024
Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: arxiv.org/abs/2410.11758 🧵 1/9
14K
Joel Jang
@jang_yoel
Apr 26, 2023
Happy to share that our ELM project has been accepted to @icmlconf 🎉!! See you in Honolulu!
Joel Jang
@jang_yoel
Feb 8, 2023
Scaling 📈 the total # of tasks during instruction tuning has been known to unlock new abilities in LMs. However, we find that an LM trained on a single task outperforms an LM trained on 300+ tasks on unseen tasks 🤯 📝:arxiv.org/abs/2302.03202 1/8
9.7K