Justin Alvey (@justLV) / X

Justin Alvey

276 posts

Justin Alvey

@justLV

• head down building @sesame • prev @CovariantAI & hardware founder • techno-optimist

San Francisco, CA

Joined March 2009

Justin Alvey
@justLV
Jul 18, 2023
I “jailbroke” a Google Nest Mini so that you can run your own LLM’s, agents and voice models. Here’s a demo using it to manage all my messages (with help from @onbeeper) 🔊 on, and wait for surprise guest! I thought hard about how to best tackle this and why, see 🧵
00:00
1.7M
Justin Alvey
@justLV
Feb 16, 2023
I wanted to imagine how we’d better use #stablediffusion for video content / AR. A major obstacle, why most videos are so flickery, is lack of temporal & viewing angle consistency, so I experimented with an approach to fix this See 🧵 for process & examples
00:00
525K
Justin Alvey
@justLV
Mar 20, 2023
We are getting closer to “Her” (part 2!) Conversationally do anything with emails, using LLM chaining & few-shot prompting for tool use (@langchain inspired) This is now realtime (ish), thanks to #OpenAI gpt-3.5-turbo 🔈 on for voice realism! 🧵
00:00
481K
Justin Alvey
@justLV
Feb 27, 2025
Excited to share a peek of what I’ve been working on We @sesame believe voice is key to unlocking a future where computers are lifelike Here’s an early preview you can try! 👇 We’ll be open sourcing a model, and yes… we’re building hardware! 🧵
00:00
452K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
I’m working on open sourcing the PCB design, build instructions, firmware, bot & server code - expect something in the next week or so. If you don't want to source Nest Mini's (or shells from AliExpress) it's still a great dev platform for developing an assistant! Stay tuned!
68K
Justin Alvey
@justLV
Aug 9, 2023
Here's one of my modded Google Nest Mini's using @OpenAI function calling to take notes & control 💡 I’m releasing all code & docs to get this exact demo running yourself, including: 💬 Messages 🏡 Home Assistant integration 📝 Note-taking github.com/justLV/onju-vo…
00:00
131K
Justin Alvey
@justLV
Feb 2, 2023
We are getting closer to “Her” where conversation is the new interface. Siri couldn’t do it, so I built an e-mail summarizing feature using #GPT3 and life-like #AI generated voice on iOS. (🔈Audio on to be 🤯with voice realism!) How did I do this? 👇
00:00
204K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
I've been experimenting with multiple of these, announcing important messages as they come in, morning briefings, noting down ideas and memos, and browsing agents. I couldn’t resist - here's a playful (unscripted!) video of two talking to each other prompted to be AI’s from "Her
00:00
88K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
After looking into jailbreaking options, I opted to completely replace the PCB. This let’s you use a cheap ($2) but powerful & developer friendly WiFi chip with a highly capable audio framework. This allows a paradigm of multiple cheap edge devices for audio & voice detection…
170K
Justin Alvey
@justLV
Nov 1, 2022
1/ I created this with Stable Diffusion using image inpainting and “walking through the latent space” Without using tweening, every frame is generated by an interpolated embedding and variable denoising strength, so keeping continuity was tricky See 🧵for process
Karen X. Cheng
@karenxcheng
Nov 1, 2022
Testing out Stable Diffusion inpainting on a video - by @justLV See below for his process #inpainting #stablediffusion #artificialintelligence
00:00
Justin Alvey
@justLV
Dec 20, 2022
I used the #StableDiffusion 2 Depth Guided model to create architecture photos from dollhouse furniture. By using a depth-map you can create images with incredible spatial consistency without using any of the original RGB image. See 🧵
Karen X. Cheng
@karenxcheng
Dec 20, 2022
Using AI for design inspiration We used Stable Diffusion Depth to Image to get the consistency - collab with @justLV See below for our process #ArtificialIntelligence #stablediffusion #interiordesign
00:00
173K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
The custom PCB uses @EspressifSystem's ESP32-S3 I went through 2 revisions from a module to a SoC package with extra flash, simplifying to single-sided SMT (< $10 BOM) All features such as LED’s, capacitive touch, mute switch are working, & even programmable from Arduino (/IDF)
74K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
& offloading large models to a more powerful local device (whether your M2 Mac, PC server w/ GPU or even "tinybox"!) In most cases this device is already trusted with your credentials and data so you don’t have to hand these off to some cloud & data need never leave your home
71K
Justin Alvey
@justLV
Jul 18, 2023
Replying to @justLV
For this demo I used a custom “Maubot” with my @onbeeper credentials (a messaging app which securely bridges your messaging clients using the Matrix protocol & e2e encryption) which runs locally serving an API I’m then using GPT3.5 (for speed) with function calling to query this
58K