Beat the Python

Inspiration

We had been wanting to get our feet wet with Machine Learning in games for a while and this seemed like a good opportunity. We had a couple more of ideas in mind before starting the hackathon but the organizer team saying we should learn and not be afraid of failing while doing it was what made us end up going for this.

What it does

It's an AI that takes part in a multiplayer Snake game, against you. The cool thing is the AI learnt how to play by itself, using Q-learning. By setting the possible follow-up states and rewards for each situation the machine found itself in, it quickly improved and raised to be better than most players who tried it during the hackathon. It also has an online global ranking, so you can test yourself against anyone else in the world!

How we built it

We used the PyGame library for the media aspect of the game, which'd be the graphics part and the cool 8-bit soundtrack it rocks, as well as adding controls for the player. Once we could interact with it, we proceeded to make the snake learn using Q-learning. Since the implementation of the algorithm was "simple" we did not use any libraries like TensorFlow. Finally we felt like we wanted a bit more competition involved in the game, so we added a ranking where you can log your name and score and check how good you and your friends are!

Challenges we ran into

Implementing the Q-learning algorithm by hand (using numpy) was good.Nonetheless, since it was our first time with this type of algorithms, we needed to spend a good amount of time to learn about it. Another problem we faced during the development of the game was that we first coded it without thinking properly how to manage the state of the game, so in order to train the agent we needed to later do some modifications to the code, leading us to spending some extra time.

Accomplishments that we are proud of

We are happy to have a finished and working project. We wanted to learn about this family of algorithms and being able to play with our own trained bot is so satisfying :D

What we learned

We learned that PyGame is very powerful, intuitive and for projects like this it is very versatile. Also we learned about Q-learning and Reinforcement Learning which was one of our main goals.

What's next for Beat the Python

We want to do some updates like changing how we represent the state of the snake during the Q-learning step, which would lead us to being able to use bigger maps because of the improved memory efficiency.