A Multi-Agent's Paradise

Been training most their lives, simulating in a multi-agent's paradise.

40x60 Map of the Game with 2 players
GIF
GIF representation of shrinking map

Inspiration

Using browser games as our inspiration for the style, we wanted to create a toy environment for the development of agents that can learn to cooperate and communicate.
Research works such as Emergent Tool Use from Multi-Agent Interaction by OpenAI (Hide and Seek video) and RL networks such as DRQN (https://arxiv.org/pdf/1507.06527.pdf) were key in leading the project direction.

What it does

Simulates most their lives of livin' in a multi-agent's paradise

Game server that can take connections from both real players and AI agents. The intention is to use the game server in multi-agent training, with the plan to teach the agents the ability to communicate, hopefully generating their own language.
In the theme of this year we wanted to build everything from the ground up as much as possible and even "attempt" to follow proper software development protocols. (Setup travis, unittests, git branches)

The Game

Teams of equal sizes are placed in a randomly generated world. They need to stay on the path and avoiding falling into the void or being trapped by the world map getting smaller. They can push other agents and "reinforce" their team mates by standing behind them and stopping them being pushed back, or help attack an enermy player at the same time.
The last team alive wins the game!
Agents are rewarded for their team performance, and intermidate rewards can be added in such as failing to respond to message of nearby friendlies.

How I built it

Python server backend, websockets for communicating between server backend and javascript frontent, tensorflow backend, also websockets between agents and server

Challenges I ran into

Making the python asyncio library do what you want, making a consistent environment that works for both ai and humans.
Designing the network that reads and generates text within the time constraint.

Accomplishments that I'm proud of

Walking through the valley of the shadow of death

The game engine managed to get working with web UI and multiple clients

What I learned

I take a look at my life and realize there's not much left

Working with multi client systems in an asynchronous manner with two very different client types

What's next for A Multi-Agent's Paradise

Gonna spend the rest our lives livin' in a multi-agent's paradise

Implement standard baselines such as PPO, SAC, A3C
Implement example network for reading and generation of messages
Add functionality for mobile
Add functionality for agents to communicate and to visualise the messages

Built With

Submitted to

Hack Cambridge 101
- Winner Hack Cambridge People's Choice

Created by

Mainly, I worked on the frontend. Developed connection, movement, graphics etc.

sirvan3tr Almasi
Developed backend server in python. This takes the inputs from the users and runs the game simulation to resolve movements, collisions, deaths, and winning conditions. The backend also used websockets to communicate state to all clients in real-time. In future this code should be redesigned to give a more consistent structure and sane set of data types.

Edward Stables
Worked on Multi-Agent RL environment and DRQN.

Next steps:
- Implement recurrence to read messages and generate messages to send to near by agents.

Joe Arrowsmith
Worked on initialising the game environment map in the back end as well supporting on the development of the Multi-Agent RL environment.

Tudor Trita

Updates

Edward Stables started this project — Jan 19, 2020 05:12 AM EST

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.