Inspiration
We implemented a content moderation bot for Discord. It can be deployed on any server to flag hate speech, misinformation, and harmful content. Our bot tracks the number of times a user posted hate speech on a specific server. If a user violates the hate speech policy multiple times the bot will time out the user ( after 5 offenses) and can remove the user (after 10 offenses).
What it does
Fairness - Our bot accounts for contextual biases, as well as multiple languages ( such as French, Spanish, etc.) Resourcefulness - We leveraged open source APIs such as GPT Moderation Model, and GPT 3.5 for content moderation. We used an open-source template as the foundation for the discord bot. Accessibility - Our bot is easy to use on multiple discord servers and it uses discord frontend so its users are familiar with it. It is an easy replacement for human admin. Social impact - Our discord bot accounts for toxicity in discord communities making this popular platform more welcoming. Its moderation capabilities do a great job of identifying hate speech and misinformation even in complicated contexts. Our bot tracks the number of times a user’s message is flagged in a given server and takes disciplinary actions, hence keeping the users accountable at all times.
How we built it
We based our discord bot on the python framework and used GPT APIs to moderate the content for harmful content as well as misinformation. Our discord bot takes a two-layer approach to content filtering. We used the GPT Moderation Model API to detect hate speech, harmful content, harassment, self-harm, sexual content and violence. We then implemented misinformation detection capability by using GPT 3.5. Upon detecting misinformation or hate speech, our bot could time out or remove users based on the number of offenses.
Challenges we ran into
Implementing the misinformation detection was a challenge since we first had to determine what is considered misinformation.
Accomplishments that we're proud of
We are proud being able to use more than one suitable API to implement a broad range of content moderation. We are also glad to make discord a safer and better place for all people
What we learned
We learned how to work with GPT APIs and deploy a discord bot using Python. We learned how to work with MongoDB to store user information.
What's next for ForkYouDiscordBot
We will work towards implementing models which fine tuned to allow to more specific content moderation. It would be ideal to allow users to choose specific tags they want content moderation for.
Log in or sign up for Devpost to join the conversation.