FirstShot

Inspiration

We came up with FirstShot to enhance precision and efficiency in first-person shooters, specifically Counter-Strike 2. The ability to make that first shot often determines who wins the exchange in competitive gameplay, especially where split-second decisions and precise accuracy are critical. But most gamers cannot point out why they continuously miss those critical shots.

What it does

FirstShot is a tool created for the players of Counter-Strike 2, focusing on improving their aim and performance. It seamlessly integrates with your actual gameplay, recording every shot you fire to provide feedback and insight.

How we built it

We created FirstShot by training an AI to recognize the heads of enemies and the player's crosshair and calculate the distance between them to create a heat map of how far the crosshair is from the enemy's head. We trained Detectron2 to detect the heads of enemy players with high accuracy in a game environment. For this purpose, a good dataset of in-game screenshots with different scenarios—different maps, lighting conditions, stances of enemies, and their positions—was needed to train Detectron2.

Challenges we ran into

During the development of our FPS game analysis system, we encountered several significant technical challenges across our technology stack. Integrating Detectron2 required careful optimization to balance processing speed with accurate player detection across diverse game environments and character models. Our Flask server implementation faced complexity in managing asynchronous image processing while maintaining stable connections, particularly when handling concurrent requests under load. The integration of ngrok introduced additional challenges in maintaining persistent connections and handling session timeouts, requiring robust reconnection logic to ensure reliable tunneling. On the database side, implementing PostgreSQL demanded thoughtful schema design to efficiently store and retrieve both image processing results and performance metrics while maintaining query performance as the dataset grew. These challenges pushed us to implement comprehensive error handling, optimize our data pipeline, and create a more resilient system architecture that could handle real-time game analysis effectively.

Accomplishments that we're proud of

As a team, we’re incredibly proud of what we accomplished with FirstShot. Together, we successfully trained a Detectron2 model to recognize enemy heads and crosshairs in Counter-Strike 2, tackling the challenges of creating and labeling a diverse dataset. We integrated advanced machine learning with real-time gameplay analysis, enabling precise distance calculations and intuitive heatmap visualizations to provide players with actionable feedback.

What we learned

Through developing FirstShot, we learned a great deal about training machine learning models like Detectron2 and tailoring them for real-world applications. We gained experience in creating and managing diverse datasets, optimizing models for real-time performance, and integrating AI into gaming environments.

What's next for a good project

We plan to expand what games this program applies to. For example, we want to train it for other shooter games like Valorant or Apex Legends so that we can help all different types of gamers.

Built With

• detectron2
• flask
• json
• postgresql
• pyside6
• python
• sqlite