Jason Liu (@JasonJZLiu) / X

Jason Liu

313 posts

Jason Liu

@JasonJZLiu

PhD student @CMU_Robotics | Prev @NvidiaAI @UofT

jasonjzliu.com

Joined September 2014

Pinned
Jason Liu
@JasonJZLiu
Jun 11
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors. We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies. FACTR 2 consists of: 1. Neural External Torque (NEXT):
00:00
97K
Jason Liu
@JasonJZLiu
Mar 31, 2025
FACTR is now open source! You can now build your own low-cost force-feedback teleop system. Hardware: github.com/JasonJZLiu/FAC… Teleop Code: github.com/RaindragonD/fa… Training Code: github.com/RaindragonD/fa…
Jason Liu
@JasonJZLiu
Feb 25, 2025
Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A
00:00
GitHub - JasonJZLiu/FACTR_Hardware: FACTR Hardware
From github.com
27K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Low-cost teleop systems have democratized robot data collection, but they lack any force feedback, making it challenging to teleoperate contact-rich tasks. Many robot arms provide force information — a critical yet underutilized modality in robot learning. We introduce: 1. 🦾A
00:00
152K
Jason Liu
@JasonJZLiu
May 13, 2025
Robot data is expensive and hard to scale But what if we could collect rich, diverse demos—with just our hands? 🙌 Our latest work, DexWild, shows how large-scale human data 💪 + robot data 🦾 co-training enables strong generalization across tasks, scenes, and embodiments
Tony Tao
@_tonytao_
May 13, 2025
Training robots for the open world needs diverse data But collecting robot demos in the wild is hard! Presenting DexWild 🙌🏕️ Human data collection system that works in diverse environments, without robots 💪🦾 Human + Robot Cotraining pipeline that unlocks generalization 🧵👇
00:00
5.7K
Jason Liu
@JasonJZLiu
Apr 28, 2025
So cool to see end-to-end dexterous grasping being applied to humanoids
Ritvik Singh
@ritvik_singh9
Apr 28, 2025
Over the past few months we've been working with Boston Dynamics on end-to-end dexterous manipulation for the EAtlas:
00:00
2.9K
Jason Liu
@JasonJZLiu
Oct 27, 2022
Excited to share what we have been working on over the past months! We demonstrate agile in-hand manipulation achieved via GPU accelerated sim-to-real transfer. Both the RL policy and the pose estimation are entirely trained in simulation using @NVIDIA Isaac Sim and Gym!
Ankur Handa
@ankurhandos
Oct 26, 2022
DeXtreme is our new work on scaling sim-to-real for contact-rich manipulation with a vision-based state estimation on a robot hand with the infrastructure we have been developing with Isaac Gym over the past one year. arxiv.org/abs/2210.13702 dextreme.org
00:00
Jason Liu
@JasonJZLiu
May 7, 2025
Super impressive pipeline!
Arthur Allshire
@arthurallshire
May 7, 2025
our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy (w/ @redstone_hong @junyi42 @davidrmcall)
00:00
1.7K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🥊 Naively adding force to policy learning does not guarantee performance improvement. BC policies often overfit to vision input and ignore force when added naively, limiting performance in contact-rich tasks. (3/N)
00:00
2.2K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🦾Our teleop system (~$1K per leader arm) lets users intuitively feel forces experienced by robots, without additional sensor hardware. This can be done for many arms. We also integrate gravity comp, redundancy resolution, etc. in the teleop leader arm, augmenting the teleop
00:00
3.8K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🥊 We propose FACTR, a curriculum that corrupts vision with decreasing intensity during training. This prevents overfitting to vision and guides the policy to properly attend to force. (4/N)
1.7K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🥊 By attending to force, FACTR policies also exhibit emergent recovery behavior (not in training data), even for unseen objects. We do not observe this in baseline policies. (7/N)
00:00
1.6K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
Both our teleop system and autonomous policies significantly outperform baselines. FACTR is easy to set up on existing systems. All components will be open-sourced soon! w/ @yulongli42 @kenny__shaw @_tonytao_ @rsalakhu @pathak2206 (8/N)
1.4K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🥊 We visualize cross attention of action tokens to force (blue) and vision (orange) tokens. [Left] Without FACTR, policy does not pay much attention to force. [Right] FACTR policies properly attend to force, even learning mode-switches, indicated by force attention outweighing
00:00
1.4K
Jason Liu
@JasonJZLiu
Feb 25, 2025
Replying to @JasonJZLiu
🥊 FACTR policies generalize to unseen objects significantly better than baseline policies (vision+force without FACTR). (5/N)
00:00
1.5K