Kids Jam

An interactive site where kids can discover and create their own beats!

Comment

Inspiration

Music defines who you are. From your favorite streaming app, artist, genre, instrument, etc. Everyone has their own taste in music, even children. As kids, we were all interested in making music by playing an instrument, humming along to a song, or tapping our pencils to the rhythm. But, combining everything to create a whole musical piece was impossible. Applications like Garage Band are complex, non intuitive, and designed for adults. Buying physical instruments can be expensive. That is why we created Kids Jam. Children can log on and have many musical experiences and bring their mental compositions to life.

What it does

Kids Jam has two main features. A simplified version of Garage Band and an interactive drum set. The simplified version of Garage Band allows users to create musical beats. Different sounds are represented as images in order to make the app more intuitive. This removes the language barrier allowing younger children to be able to use the app. The other aspect of the app, the interactive drumset, allows for users to play the drums without a physical drum set which can be expensive. Users can place an open hand in a circle for it to play the drum sounds. Both hands can be used to play the drums to create a realistic feel to it. Through these immersive experiences, anyone can experience the joy of creating music.

How we built it

We built this application using Expo Go because it allows us to deploy for iOS, Android, and web. We built the Studio Jam feature using a basic React Native grid and implemented the Play feature using simple JavaScript and AsyncStorage. For the drums, we used mediapipe’s hand detection algorithm. This gave us the key points necessary for finding the position of the hands in relation to the drums, represented by the circles. The process as a whole involved searching for hands, drawing the landmarks, detecting if the hand was open or closed and playing sounds.

Challenges we ran into

Throughout our project, we encountered several challenges:

  • Updating XP Points: We struggled with properly getting the XP points of the player to update so that their level updates as well. For a while, the user would be able to click the “Claim XP” button successfully, however the points wouldn’t update anywhere. We fixed this by setGame with the new XP and claimed status to force state synchronization.
  • Seamlessly integrate the audio to the drum set: The audio for the drum set was challenging to implement. The drums would sometimes randomly go off or not turn on at all. One big issue was determining how to allow users to move their hands without setting off the audio. We decided if the hand is closed, the audio should not play. The challenging aspect was determining which hand landmarks would most accurately give information on the hands state. We concluded the distance between the time of the middle finger and bottom of the palm could give insight on if the hand was in a balled up position. Then, we compared the distance to a set number of pixels and if the distance was smaller than the number, the audio would not play.
  • Scaling texts and drums: Scaling the text and drum shapes on the drum set required many steps and calculations to be taken along the way. Because of this, it was a tedious task. Ensuring that the scaling algorithm worked also required us to use different cameras from each of our computers which led to slower testing. By working diligently together and setting times to test the code on everyone's computers, we were able to properly scale it.

Accomplishments that we're proud of

  • Creating and Generating Cool Beats: One of our biggest worries was that we would not be able to make quality music. So, we are very proud of the programming it took in order to make the beats sound pretty.
  • Drum Set has little to no lag: While creating the drum sets the audio would start late or just not start at all. But, we managed to get it to work and it is very consistent with the hand motions.
  • Clean, Simple Interface: The user interface is very minimalist and clean making it easier for younger children to navigate. The use of images for the music composer instead of text allows those who cannot read to be able to use our website too.

What we learned

Through the hackathon we learned about using mediapipe in javascript and using Expo Go to create websites. We also improved our skills in communication and problem solving through group work.

What's next for Kids Jam

  • Adding audio recorder: Adding a recorder would allow users to seamlessly input their own recordings. This could be a song they sang, beatboxing, other audios they like, etc. This would allow for more creativity and would give the opportunity for users to create songs.
  • Adding more audios: The music editor currently has a limited number of audio selections. The goal is to add more types of sounds and allow users to be able to select audio through a dropdown menu.
  • Adding a streak: Adding a streak element so users practice with music consistently. Repetition is important for becoming a good composer, so with the streak, users should be motivated to practice daily.
  • More Interactive Instruments: Due to time limitations we were only able to make a virtual drumset. The same is possible for xylophone, piano, etc..
  • Allow for iOS and Android Deployment: Due to the time constraint we were not able to properly fix the app and have it run on iOS and Android so we hope to do that soon.

Built With

Share this project:

Updates