Keyvolution

sample misstype correction
The word technological being matched

Inspiration

We thought keyboards hadn't had significant improvements in the past couple of years and we seek to change that. We've designed a keyboard that can leverage machine learning to adapt and learn how you type.

What it does

We know that a lot of people like the qwerty layout and wouldn't want to change, so whats the point of our keyboard then? Our keyboard doesn't have a home row. You can type anywhere on the keyboard, and it will be able to recognize the words you type. Instead of any one key being associated with a single letter, any key could be every letter. Our model is even able to correct basic miss-types. Our keyboard could even be used to type multiple languages or layouts on the same keyboard without any need to change anything! Imagine the impact this could have on international business!

How we built it

We decided to leverage machine learning and apply it to a keyboard. Our current application is written in javascript for our demo to best showcase how our project works but is something that could be easily be ported over to work entirely on the keyboard. In our current design, our keyboards input coordinates instead of characters which are then processed to record the keys typed in relation to one another. We take keystrokes, convert them into distance and angle vectors, and use that data to train a machine learning model. This kind of learning is similar to standard image recognition, except it uses the path that a particular typed word would create instead of pixel data. After our model is trained, it will then be able to predict words typed using training data to pick a word that best matches the path you input. Because our system looks at paths on a word by word basis, you could even type anywhere on the keyboard and still get the correct result.

Challenges we ran into

The first main challenge that we ran into was how we should save the keyboard input in a way that can be used well for machine learning. We felt that by using angles and vectors, we can make the most flexible system that can leverage our model.

A physical limitation of our project was the processing power of the microcontrollers built into the keyboards. The keyboards run off of ATMEGAU34 chips with limited processing power, so we weren't able to run our machine learning components of the project on it. This forced us to develop software that would run on a computer and handle the custom input that our keyboards produce. In the future, we could get more processing power built into the keyboard which would allow us to run the entire system on the keyboard itself.

The largest challenge that we faced was how to tell the difference between different words if they would produce the same path when typed. Our current solution is to select the word that is most commonly used, but we have also thought of adding a cycle functionality to cycle through the possible matched words as a user is typing. While this is not a feature that is currently implemented, it is something that can easily be added down the line.

Accomplishments that we're proud of

Our keyboard is exceptionally accurate at what it does and works better than we thought it ever could.

We built our keyboards during the hackathon as well.

The only library our project uses is Node.js. Everything else, including the machine learning model, was developed by us.

What we learned

While most of us had never done any machine learning before, we were still able to not only write our model but make something that was extremely accurate.

What's next for Keyvolution

The first thing that we could add to our project would be the support for punctuation and numbers which would allow us to completely replace the need for any other keyboard.

We could then also design a keyboard with more processing power which would allow us to run our entire project inside the keyboard that could then be plugged into any computer without additional software.

Try it out

Because our keyboards use a system that doesn't have a layout, we can't share a demo version online for you to try. Come by our booth at the hackathon to try typing on our keyboards and see the magic for yourself.