Matthieu LC (@matthieulc) / X

Matthieu LC

158 posts

Matthieu LC

@matthieulc

robotics @gen_intuition / built typeless, acq by @Doctolib

Joined January 2013

Pinned
Matthieu LC
@matthieulc
Jun 9, 2025
worked on having gpt4o drive open-loop motor primitives like waving or grabbing from stereo vision. special tokens like <distance: 10cm> get streamed straight into the transcript stream, which gpt4o reacts too. next up is integrating the closed-loop hand tracking. main challenge
00:00
2.5M
Matthieu LC
@matthieulc
Jun 2, 2025
sim2real achieved! mini shoggoth can now (slowly) track hands. to what purpose i do not yet know stack is PPO + mujoco + dynamics randomization + yolo11n
00:00
30K
Matthieu LC
@matthieulc
Apr 29, 2025
hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i
00:00
00:00
46K
Matthieu LC
@matthieulc
Jul 15, 2025
Releasing Shoggoth Mini!🐙 Soft tentacle robot meets GPT-4o & RL. I built it to explore the boundaries of weird: expressiveness, aliveness, and AI embodiment. Blogpost: matthieulc.com/posts/shoggoth…
00:00
43K
Matthieu LC
@matthieulc
Jun 23, 2025
approaching the end of v1 here :) worked on some new primitives, and integrating them all together. making the gpt4o realtime api consistent in its use of the primitives was harder than i thought. i will now start writing the blogpost and cleaning the vibecode a bit for
00:00
9.3K
Matthieu LC
@matthieulc
Jul 11, 2023
Got stable diffusion to dynamically produce a virtual desktop interface controlled via mouse inputs! No frontend code, no frontend rendering engine, pure image generation. Blogpost: mlecauchois.github.io/posts/deep-gui/ #stablediffusion
GIF
62K
Matthieu LC
@matthieulc
Jul 3, 2025
in dexterity tests, shoggoth mini reveals strong passive grasping generalization
00:00
11K
Matthieu LC
@matthieulc
Jun 9, 2025
Replying to @MattVMacfarlane
Love to hear it, will try to do that before end of June!
2.8K
Matthieu LC
@matthieulc
May 5, 2025
got a solid initial tracking policy ready for real-world testing! spent a lot of time refining the mujoco simulation to better reflect reality, particularly around tendon, joints and actuator dynamics. also added simple dynamics randomization to improve sim2real transfer.
00:00
7.9K
Matthieu LC
@matthieulc
May 26, 2025
redesigned the mini shoggoth body to fit the stereo camera inside. also added a little spike at the tip to make it feel more alien. my roommate also made me notice that it looks a lot like a little green men head with a weird giger-like mandible when held upside down,
00:00
00:00
00:12
00:34
00:16
Matthieu LC
@matthieulc
May 19, 2025
finally got the rl tracking policy to work on the real robot this week! the tentacle loosely follows hands based on pose estimation of both hands and the tentacle tip. to do that i had to enhance/simplify the tip pose estimation + improve the hardware by covering the spool to
2.5K
Matthieu LC
@matthieulc
Jun 10, 2025
Replying to @fluiddesign201
the goal is to have audio input and proper conversations, which i'll show next week :)
463
Matthieu LC
@matthieulc
Jun 10, 2025
Replying to @devinaconley
from my understanding there is still only audio (which i use but not shown here) and transcript input (which i use for visual) that are available in the realtime api, for output there is audio (which i deactivate for now) and tool use (which i use for the control of motor
590
Matthieu LC
@matthieulc
Aug 28, 2025
Please give this new model a try, it is so so good!
Peter Bakkum
@pbbakkum
Aug 28, 2025
My favorite demo of the new gpt-realtime model from @matthieulc -- Shoggoth Mini using Realtime API with image input
00:00
5.2K
Matthieu LC
@matthieulc
Jun 9, 2025
Replying to @MattVMacfarlane
definitely! i was thinking of making it "walk" too. would need to do some cable management and optimize the insides to fit 3*3 tendons
2K