Chen Wang (@chenwang

Chen Wang

321 posts

Chen Wang

@chenwang_j

Final-year CS PhD @Stanford. Prev @GoogleDeepMind @NVIDIA @MIT_CSAIL. Robotics/Manipulation

Stanford, CA

Joined January 2021

Pinned
Chen Wang
@chenwang_j
Mar 14, 2024
Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced
00:00
235K
Chen Wang
@chenwang_j
Feb 23, 2023
How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇
00:00
289K
Chen Wang
@chenwang_j
Sep 6, 2023
How to chain multiple dexterous skills to tackle complex long-horizon manipulation tasks? Imagine retrieving a LEGO block from a pile, rotating it in-hand, and inserting it at the desired location to build a structure. Introducing our new work - Sequential Dexterity 🧵👇
00:00
160K
Chen Wang
@chenwang_j
Sep 28, 2021
Can robots learn hand-eye coordination simply from teleoperated human demonstrations? Our new #IROS2021 paper presents a novel action space to enable this! Website: sites.google.com/stanford.edu/h… 1/9
00:00
Chen Wang
@chenwang_j
Jul 12, 2022
1/ Can we improve the generalization capability of a vision-based task planner with representation pretraining? Check out our RAL paper on learning to plan with pre-trained object-level representation. Website: sites.google.com/view/gentp
GIF
Chen Wang
@chenwang_j
Jul 7, 2023
The combination of LLM and VLM shows great potentials in grounding “Where” and “How” to act in 3D observation space. Such capability allows the robot to perform visuomotor manipulation in a zero-shot fashion! Checkout VoxPoser, amazing work led by @wenlong_huang at @StanfordSVL
Wenlong Huang
@wenlong_huang
Jul 7, 2023
How to harness foundation models for *generalization in the wild* in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 voxposer.github.io 🧵👇
00:00
24K
Chen Wang
@chenwang_j
Jan 3, 2024
Everything is soooo fast. Are these all happen within 3 months???
Zipeng Fu
@zipengfu
Jan 3, 2024
Introduce 𝐌𝐨𝐛𝐢𝐥𝐞 𝐀𝐋𝐎𝐇𝐀🏄 -- Learning! With 50 demos, our robot can autonomously complete complex mobile manipulation tasks: - cook and serve shrimp🦐 - call and take elevator🛗 - store a 3Ibs pot to a two-door cabinet Open-sourced! Co-led @tonyzzhao, @chelseabfinn
00:00
10K
Chen Wang
@chenwang_j
Mar 13, 2024
wow 200hz low-level control frequency! Thanks for sharing details, learned a lot!
Corey Lynch
@coreylynch
Mar 13, 2024
Replying to @coreylynch
Finally, let's talk about the learned low-level bimanual manipulation. All behaviors are driven by neural network visuomotor transformer policies, mapping pixels directly to actions. These networks take in onboard images at 10hz, and generate 24-DOF actions (wrist poses and
00:00
8.4K
Chen Wang
@chenwang_j
May 17, 2024
sim2real is still tough, but at least now we know how to make it work with minimal human help (< 1 minute teleoperated interventions) — which feels way better than back-and-forth tuning of envs, PD, rewards... all that stuff seldomly mentioned in papers..
Yunfan Jiang
@YunfanJiang
May 17, 2024
Does your sim2real robot falter at critical moments 🤯? Want to help but unsure how, all you can do is reward tuning in sim 😮‍💨? Introduce 𝐓𝐑𝐀𝐍𝐒𝐈𝐂 for manipulation sim2real. Robots learned in sim can accomplish complex tasks in real, such as furniture assembly. 🤿🧵
00:00
4.8K
Chen Wang
@chenwang_j
Mar 14, 2024
Replying to @chenwang_j
Motion capture gloves, unlike vision-based tracking, are not affected by occlusions during hand-object interactions, perfect for mocap in daily activities. With an RGB-D camera, DexCap reconstructs 3D scenes and aligns motion data, all powered by a mini-PC in the backpack. 2/
00:00
4.9K
Chen Wang
@chenwang_j
Mar 14, 2024
Replying to @chenwang_j
We then retarget the mocap data to the robot embodiment. This includes (1) Observation retargeting by switching the camera system from human to robot. (2) Action retargeting by matching fingertip positions with IK. (3) Bridging the visual gap by including robot point clouds. 3/
00:00
12K
Chen Wang
@chenwang_j
Mar 14, 2024
Exactly! Checking out dex-cap.github.io for all the hardware/code/data details!
Jim Fan
@DrJimFan
Mar 14, 2024
DexCap: a $3,600 open-source hardware stack that records human finger motions to train dexterous robot manipulation. It's like a very "lo-fi" version of Optimus, but affordable to academic researchers. This isn't teleoperation: data collection is decoupled from the robot
00:00
4.3K
Chen Wang
@chenwang_j
Sep 6, 2023
Replying to @chenwang_j
We hope Sequential Dexterity paves the path for future research on long-horizon dexterous manipulation. Feel free to check out our code! Website & Paper: sequential-dexterity.github.io Code: github.com/sequential-dex… Work done w/ Yuanpei Chen, @drfeifei, and Karen Liu at @StanfordAILab.
00:00
2.1K
Chen Wang
@chenwang_j
Sep 28, 2023
Designing such a wearable and portable exoskeleton system is extremely hard. And its low-cost ($300 per arm)! Another great work from @haoshu_fang!
Hao-Shu Fang
@haoshu_fang
Sep 28, 2023
🤖Joint-level control + portability = robot data in the wild! We present AirExo, a low-cost hardware, and showcase how in-the-wild data enhances robot learning, even in contact-rich tasks. A promising tool for large-scale robot learning & TeleOP, now at airexo.github.io!
00:00
3K