Inspiration
We wanted to create an app where a person can sing, and no matter how amazing or how poorly they sound, the app can generate a background track to make them sound better.
What it does
A user records themselves singing. After the user has stopped recording, the program processes the audio by extracting the tempo (how fast) of the singing as well as the key of the singing. Then, using these parameters extracted through audio processing, the program will generate a background track that matches the tempo and key of the singing. The app plays this audio back to the user, with the user's original track playing on top of the background music. Voila! Orchestrate has created background music to a user's singing.
How we built it
Java - we used Java to record the audio from the user Python - we used python to process the audio. We used a beat API which determines the beat frequency of an audio file. In addition, we implemented a real-time tuner that runs in parallel during the audio recording. This real-time tuner logs each note that is recognized. Then, we process these notes through an algorithm that sorts out the notes that appear more frequently, and the notes that occur most often are used to determine the key. We also used python to create the final background audio with another API that generates notes. We combined multiple notes based on the key and then add this new audio to the user's singing.
Challenges we ran into
Processing and working with audio is a very difficult process. We had issues with processing audio and extracting useful information as well as using python libraries to effectively generate background tracks. We originally wanted to break down the recorded audio to determine individual frequencies of pitches in order to determine the pitch. Due to the physics of audio, this proved to be a very difficult task. We worked around this by implementing the real-time tuner and algorithm to determine the key.
Accomplishments that we're proud of
We're proud of creating a useful app that effectively creates background audio to a user's singing. We were able to overcome our difficulties in processing audio and learned a lot about working with audio files. The various APIs we utilized proved successful in achieving our end goals.
What we learned
We learned a lot about processing and working with audio. Not only did we learn various techniques for processing audio and generating audio, we also learned a lot about flexible programming. We had to change our methodology for achieving various tasks in order to overcome some of the difficulties involved in working with audio. We learned that there are multiple solutions to a task, and sometimes the more creative ideas prove to be very effective.
What's next for Orchestrate
We're planning to improve some of our audio generation in order to offer better background tracks and user customization. There are many opportunities in terms of audio, so we're looking to give the user lots of options and customizations for their background track.
Log in or sign up for Devpost to join the conversation.