deepmusic

Rich music created by deep learning.

Comment

Inspiration

We were inspired by the art generator deepart.io that generates new art based off two input images allowing for unique art in a style you want. Our focus on music came from the fact that we enjoy music (one of us goes onto spotify almost every 2 minutes). We though if this worked on art this can be applied to music and this was the fruits of our labor.

What it does

Uses machine learning to synthesize audio remixes of a song in the style of another. The algorithm simply takes in two audio files and generates an output audio file. The output is the first audio file in the musical stylings of the second. It can easily generate seamless remixes for users to enjoy and music artists to thrive.

How We built it

The structure of the machine learning was developed using a 19 layer deep convolutional neural network as a visual graphic group. The algorithm is worked the same way that deepart.io works based off their scientific paper. Deep convolutional neural networks work fantastically for images and we therefore converted audio to spectrograms due to much evidence in the machine learning community that it is a very effective way to train audio machine learning. The algorithm converts the two audio inputs to spectrogram images, feeds it through the convolutional neural network, and takes the output file (which has been trained to be output as a spectrogram) is converted back to audio.

Challenges we ran into

  1. Training multiple neural networks in such a short span of time.
  2. Converting a spectrogram image back into an audio file in .wav format.
  3. Regulating the volume of the result due to unknown reasons.
  4. Properly formatting the website in an aesthetic manner.

Accomplishments that We are proud of

  1. Creating a functional machine learning algorithm in 24 hours.
  2. Managing to modify a pre-trained neural network.
  3. Training it correctly to 7.5% error within a 24 hour period.
  4. Creating a nice looking website displaying our data.

What we learned

  1. Audio machine learning requires a lot of learning and hard work.
  2. Never directly listen to a first time converted spectrogram to audio.
  3. Learned to create a visually appealing website.
  4. Learned to understand audio machine learning using spectrograms.

What is next for deepmusic

  1. Apply the algorithm to our static website so that it is possible to feed forward with the trained network.
  2. Change from a static website to a dynamic website on a proper server for users to use.
  3. Code the algorithm in lower level for higher optimization and faster response time to user requests.
  4. Continuing to train the algorithm to reduce the error because the error percentage should get better.
  5. Apply our application to markets such a charging for API calls, have a monthly subscription for aspiring music artists to create useful remixes, and have it open to investment due to the uniqueness of the implementation to be adapted to other problems.

Built With

Share this project:

Updates