Volant

• 

  Setup page

• 

  Screen capture and VTuber posing

• 

  Resulting livestream on YouTube

Inspiration

VTubers have been progressively increasing in popularity since their introduction in the mid-2010s.

What it does

Volant converts a webcam feed to a VTuber model, merges it with a recording of the user's screen, then streams it to a website such as Twitch or YouTube, achieving what would normally require dedicated software and technical knowledge in a website.

How we built it

We used React and TypeScript throughout the project, Next.js and Tailwind CSS for the front end, MediaPipe Holistic for body recognition, and Socket.IO and WebRTC for communication.

Challenges we ran into

The slow performance of the project, combined with a lack of documentation for some interfaces, such as Holistic, slowed down development. Additionally, most of our team was unfamiliar with the project's toolchain and had to learn as they went.

Accomplishments that we're proud of

We managed to successfully put many tools together, and work in parallel to create a viable result in time. All team members were able to contribute tangibly to the project.

What we learned

We learned about the VRM file format, communication via Sockets.IO and WebRTC, and optimizing TypeScript code. Additionally, most of the team learned basic React skills.

What's next for Volant

Stability, performance, customizability, and possibly removing the backend?

Built With

• mediapipe
• next.js
• node.js
• react
• socket.io
• tailwind
• three.js
• typescript
• webrtc