We will be undergoing planned maintenance on January 16th, 2026 at 1:00pm UTC. Please make sure to save your work.

CommenTidy

Regulate the youtube comments!

Comment

Inspiration

In today’s increasingly digital and socially connected world, cyberbullying, harassment, and trolling have become pressing challenges. YouTube, as one of the largest content platforms, is particularly vulnerable to such issues in its comment sections. Currently, YouTubers have limited control — they can disable comments entirely, maintain a list of banned words, or manually moderate comments, which becomes impractical at scale. Recognizing this gap, I set out to build a tool that could assist creators in automating and enhancing comment moderation. The goal is to eventually integrate this solution into the YouTube Studio ecosystem, enabling intelligent, AI-driven moderation to ensure healthier interactions in comment sections.

What it does

CommenTidy is a web application that analyzes YouTube video comments using AI, enables dynamic rule creation for moderation, and offers insightful visualizations to help creators manage community engagement effectively.

Fetch comments

  1. The user inputs a YouTube video URL along with the number of comments to fetch (up to 250).
  2. The application fetches top-level comments using the YouTube API.
  3. Video metadata (video ID, channel ID, next page token, comment count, timestamp) is stored in MongoDB.
  4. Each comment is analyzed using Gemini AI to classify it into one of the categories: Positive, Negative, Spam, Question, or Feedback, and is also assigned toxicity and sentiment scores.
  5. Embeddings for each comment's text are generated using Jina AI.
  6. All relevant comment information is stored in MongoDB, including author details, timestamps, like counts, reply counts, categories, and vector embeddings.

Subcategorize comments

  1. When a video is selected, all its associated comments are retrieved from MongoDB.
  2. Using Gemini AI, each category is further broken down into subcategories based on the content and tone of the comments.
  3. Each subcategory’s average embedding is calculated and stored.
  4. If a comment matches a subcategory (cosine similarity ≥ 0.7), it is grouped under that subcategory. If no match is found, Gemini AI generates a new subcategory, and its average embedding is updated dynamically.

Visualize the data

  1. MongoDB aggregation pipelines are used to process and organize the data efficiently.
  2. Average sentiment and toxicity scores are shown as range sliders.
  3. A pie chart displays the distribution of main categories.
  4. A dynamic tree map visualizes subcategories within each category using Plotly.
  5. A bar chart highlights the most common keywords from the comments.

Creating Comment Moderation Rules

  1. Users can review all comments associated with a video.
  2. Comments can be filtered by category or subcategory, with AI-generated summaries provided for each.
  3. Users can define rules by:
    1. Selecting categories or subcategories to block
    2. Specifying banned keywords
    3. Writing custom rules that comments must satisfy; Gemini AI determines if a comment adheres to these rules.
  4. Upon rule submission, each comment is processed and evaluated.
  5. The interface displays which comments are permitted or blocked, along with the reasons of blocking the comment.

How I built it

  • Backend Framework: Developed a lightweight backend using Flask, handling routes for comment processing, embedding generation, and rule enforcement.
  • Frontend Visualization: Built with HTML, CSS, and vanilla JavaScript. Used Plotly.js for dynamic visualizations such as treemaps, bar charts, and pie charts.
  • YouTube Data: Used the YouTube Data API v3 to fetch top-level comments for a given video. Limited to 250 comments per request for quick analysis.
  • Comment Categorization: Leveraged Gemini Pro (Google’s LLM) to classify each comment into categories: Positive, Negative, Question, Spam, or Feedback.
  • Text Embedding: Used Jina AI Embeddings to generate vector representations of comment texts for similarity-based subcategorization.
  • Subcategorization Logic:
    • Grouped comments under finer subcategories using cosine similarity thresholds.
    • If no match found, Gemini created a new subcategory and updated average embeddings.
    • All category and subcategory data were saved in the corresponding video document.
  • MongoDB & Vector Search:
    • Used MongoDB Atlas to store structured video, comment, and embedding data.
    • Performed aggregation pipelines to generate grouped statistics for visualizations.
    • Stored average embeddings for subcategories and enabled efficient filtering.
  • Rule Engine:
    • Developed a dynamic rule system allowing users to block comments based on category, subcategory, keywords, and LLM-verified conditions.
    • Gemini was used to validate if a comment follows a specific textual rule.
  • Comment Moderation View:
    • Displayed comments with styling indicating whether they would be shown or blocked.
    • Provided visual reasons for blocking (e.g., rule violations, keyword matches).
  • Deployment:
    • Deployed on Google Cloud Run.
    • Faced and overcame challenges with handling environment variables and deployment configs.

Challenges I ran into

  • Managing Environment Variables on Google Cloud Run: Initially faced difficulty loading the .env file properly in the Cloud Run environment. After some exploration, discovered a more effective way using Cloud Run’s built-in secrets and environment variable configuration. This was a valuable learning experience in cloud deployment practices.
  • Frontend Integration for Similar Comment Search: Although a vector-based similar comment search was successfully implemented on the backend using MongoDB and embeddings, I was unable to complete the frontend integration within the given timeframe.

Accomplishments that I am proud of and what I learned

  • First-Time Use of Gemini Agent: Successfully integrated Gemini Pro with structured outputs to perform nuanced classification, subcategorization, and rule validation tasks.
  • Flask Development Experience: Developed a full-fledged Flask application from scratch for the first time. The lightweight nature of Flask allowed for quick prototyping, rapid iteration, and clear separation of concerns.
  • Cloud Deployment Proficiency: Gained hands-on experience deploying to Google Cloud Run, understanding containerization, environment configuration, and debugging deployment issues. This significantly improved my confidence with serverless infrastructure for small to mid-scale projects.

What's next for CommenTidy

  • Sentiment Analysis: Currently the sentiment analysis is done by Gemini flash. However, Google offers models for sentiment analysis in particular. So, next step will be to implement Google Sentiment analysis for finer understanding of comment sentiments.
  • Enhancing LLM Consistency: Improve the reliability and determinism of Gemini-based categorizations and rule enforcement by refining prompt structures and incorporating feedback loops.
  • Dynamic and Custom Rule System: Expand the rule engine to support more complex, multi-layered logic—enabling content creators to design personalized moderation workflows.
  • Frontend Integration of Similarity Search: Enable users to search for comments similar to any selected comment or phrase directly from the frontend, powered by vector similarity.
  • YouTube Studio Integration: Explore pathways to integrate the tool into the YouTube Studio ecosystem via API hooks or extensions, turning CommenTidy into a seamless content moderation assistant.

Built With

Share this project:

Updates