Inspiration
We wanted to explore how artificial intelligence could enhance voice synthesis, unlocking new opportunities for accessibility and media production. This project emerged from our shared interest in combining technology with creativity in innovative and practical ways. We aimed to create a tool that could manipulate and generate voices, ultimately making voice-related applications more versatile and impactful, even though we were new to this specific field.
What it does
Upload any audio sample or YouTube link, type your text, and our AI analyzes the reference voice's unique characteristics—pitch, speaking rate, and prosody patterns. Within seconds, it generates new audio matching those characteristics while saying your custom text. Useful for content creators, language learners, accessibility tools, and anyone who needs custom voice generation without expensive studio time.
How we built it
We used Python for the backend, integrating machine learning models such as YourTTS for voice cloning. For real-time audio processing, we used CUDA for GPU acceleration, Librosa for audio manipulation. The Streamlit framework was employed for building an interactive interface that allows users to input their voice and control various settings for voice cloning.
Challenges we ran into
Achieving high-quality voice cloning proved to be a challenge, as the output quality was not initially as high as expected. We faced difficulties fine-tuning the models (only using samples) to get more accurate and natural-sounding results. Additionally, the process of integrating the different technologies and ensuring smooth functionality took considerable time and effort.
Accomplishments that we're proud of
We successfully built a voice cloning system that can generate customized speech synthesis
What we learned
We learned a lot about the complexities of speech synthesis and real-time AI model deployment. Combining machine learning with real-time applications challenged our skills in optimization, latency management, and user experience design. We also learned how to better handle and process audio data to achieve natural-sounding speech clones that remain adaptable to various input sources.
What's next for Voice Cloner
We plan to further enhance the quality and customizability of the cloned voices, by adding options like voice training, allowing users to fine-tune the voice cloning system to their unique speech patterns. We also want to explore integration with platforms such as virtual assistants and gaming for more practical applications.
Log in or sign up for Devpost to join the conversation.