Sieve

We just hosted an amazing multimodal AI lunch at Y Combinator HQ, bringing together tons of multimodal AI researchers across many different frontier AI labs. We talked about everything from benchmarking creativity to robot policies to computer use agents. Thanks to Diana Hu, Audrey Pe, and Arthita Ghosh for making it a successful event. If you're a researcher or builder working in the multimodal space, drop us a DM or send us message here: https://sieve.ai/contact. We'd love to chat and invite you to the next one!

Sieve hosted a multimodal brunch at Copra SF last Sunday bringing together a small group of researchers, engineers, and builders all deep in the multimodal space. Great people, great food, and some of the best back-and-forth we've had! Thanks Arthita Ghosh, Auriel W., Ishan Dhawan for organizing this, and thanks to everyone who joined us and helped make it such a huge success. If you're a researcher or builder working in the multimodal space, drop us a DM or send a message here: https://sieve.ai/contact. We'd love to chat and hope to see you at the next one!

This Wednesday, we’re hosting our first Multimodal Dinner at Berkeley with Launchpad (UC Berkeley). If you’re building multimodal systems (research or production) or thinking deeply about multimodal data, come meet others working at the frontier and swap notes on what’s working, what’s broken, and what’s next. If you’re in the Bay Area and want to join, comment or DM for details/RSVP.

Sieve is headed to NeurIPS next week! If you're around, I'd love to meet and chat about video models and where they're headed in the next year. Our team spends all of our time thinking about 1) how to create, aggregate, or acquire as much useful raw video as possible and 2) how to build the best possible search and understanding systems on top of all that raw content to deliver the highest-quality video datasets in the world at petabyte-scale (we've processed hundreds of PBs in the last few months!). Shoot me a message or hit us up at neurips@sievedata.com if you want to meet! Ai Ishiguro Gaurang Bharti

Simulation and human videos are an exciting data well for robot world models due to the particular difficulty of collecting real-world data, but Sergey Levine (UC Berkeley professor + co-founder of Physical Intelligence) just published an incredibly sobering take on the danger of venturing too deeply in this direction. Some researchers have gotten so excited in other data sources to the extent that they're being treated as a replacement to the real thing. But just like how LLMs use lots of text data and VLMs use text-image pairs, VLAs (vision-language-action) models in robotics need a lot of data of robots performing real-world tasks. Instead of treating simulation or human video (i.e. FPV videos posted online) as a complete replacement, we should treat it the same way we treat internet data in LLM and VLM pre-training - something less relevant to the ultimate goals of the model but still relevant enough to provide useful world knowledge. At Sieve, we're excited to be contributing to this problem area through our early work with robotics labs making use human videos for VLA pre-training. If you're interested in learning more about Sergey's take or our human video offering, check out the links in comments.

Stoked to welcome Ahi to the team at Sieve. Ahi's background is unique - from building the world's smallest batteries to training high quality TTS and diffusion models in the Computational Image Group at Rice. If you're interested in joining a fast-growing team working on internet-scale computer vision problems, DM me :)

Transparency in how AI capabilities work and the way in which they're evaluated help developers build trust in using them. At Sieve, we're constantly developing internal evaluation systems that enable to us to ship the highest quality AI video capabilities in the world. Tomorrow, we'll be sharing a behind-the-scenes look at how we evaluate one of our most nuanced solutions and the specific optimizations we made to score highly.