BeatDocs

Have you ever wanted a step sequencer built into google docs? We sure didn't but now we have it! BeatDocs allows you to create beats directly within google docs including extensive effects and samples

Comment

Inspiration

We were trying to come up with fun project ideas that really stretched how something could be used and after a lot of ideas Adam came up with the basic idea of a music production tool in google docs. At first the idea started as a super simple step sequencer that could play a couple notes from a synth but over the course of the hackathon we just kept adding support for more features!

What it does

BeatDocs parses google docs into MIDI like objects and then plays them using advanced web audio features. BeatDocs supports 5 built in synths and 7 built in effects but is extremely extensible and new instruments can be implemented by adding a new case to the respective instrument/effect factory and by updating the parser to properly parse the required attributes from the google doc. We also support external samples so that users can download and record their own clips to make their beats extra special.

How we built it

We built a google chrome extension using typescript which talks to the google docs API to first sign a user in with OAuth and then read the “sheet music” from the page. Once we parse the “sheet music” we turn it into our own custom MIDI like objects that are then processed by our player subsystem. The player subsystem then creates the appropriate Tone.js instruments and effects and then uses those instruments to create sequences using the provided notes. We then use Tone.js’ Web Audio integrations to play that audio live for a user!

Challenges we ran into

While working on BeatDocs we ran into A LOT of problems. First of all, neither of us know anything about music tech so we had to learn everything on the fly as we were implementing it. This turned out to be quite the challenge when there’s 5 different oscillators, 4 different reverbs, and 3 different delays. While trying to define the syntax for creating music in google docs we ran into a number of issues such as determining the best way to store metadata within a single character. Google docs offers several options such as italics, underline, bold, text color, superscript, subscript, or even comments. In the end we chose italics, underline, and text color as our primary metadata but throughout the entire hackathon we kept discussing because it feels somewhat slow to type when creating music. We also ended up with an extra google docs tab for extra metadata because we couldn’t fit all of the possible attributes for each instrument and all of the possible effects that can be put on each instrument all on one page. Another major challenge we had while making this was creating an efficient parser for google docs. You would think that you could just scrape it using something like selenium but google docs uses a pre-compiled page so the actual text is not always accessible from within the html. As an alternative we ended up using the google docs API but we found it unfriendly and very tedious to use. Many times text would be right next to each other on the doc but they were nowhere near each other when working with the API. Our biggest challenge of the hackathon was trying to figure out how to deal with running Tone.js from a browser extension. We discovered that Audio Runners cannot run on a Web Runner which meant that from the extension we could not directly play any audio. We ended up spending nearly 6 hours trying to figure out a solution for this and in the end we found a way to inject the script to play the audio into the document and we use the chrome browser message broker to send play and stop commands to the injected code.

Accomplishments that we're proud of

We are extremely proud of how this project turned out. We are especially proud of how extensible and modifiable this project is. With the built in instruments and effects and all of their settings there are already thousands of different sounds that we can make and it has been a lot of fun experimenting with everything. Combined with our sampling system we have been able to make some really cool beats and we are excited about how well the project came together. We are also proud of ourselves for programming for nearly 22 hours non-stop and having fun the whole time!

What we learned

We both learned a lot about music theory and the science behind how music is made. Learning about different types of synths and oscillators was not something we expected to do but was very interesting. Neither of us had created a browser extension before so it was cool to see just how easy it was to get started. We both learned a ton about both Tone.js and the google docs API and Max had never worked with any of the google APIs before so it was a great learning opportunity.

What's next for BeatDocs

Up next for BeatDocs would be to put this on the chrome web store for anyone to download and play with. We already integrated with google’s OAuth so it shouldn’t be too difficult to get the last steps up and working. We would also love to continue implementing instruments and effects to improve our existing library. We also created an open source sample library for people to use and we would love to keep working on it and increase our storage capacity. We also want to try to integrate our sample library directly into BeatDocs so that users can browse and choose the right sample for them directly in their own documents.

Built With

Share this project:

Updates