Trash Cam

Inspiration

According to GlobalNews Canada, a staggering one-third of recyclable materials are mistakenly sent to landfills each year. This issue has been compounded by the COVID-19 pandemic, leading to a notable increase in litter on Toronto's streets, as highlighted in a report by the Toronto Star. It's evident that Toronto is facing a significant waste disposal problem, posing environmental concerns and carrying substantial financial implications. The annual cost of citywide clean-up efforts amounts to millions of dollars.

This website application follows the trend of explosive growth of IoT by utilizing the webcam to capture images for the purpose of connecting data with other systems. It recognizes the corresponding item using Artificial Intelligence Model. Furthermore, the user can file a report about street littering and access the real-time map about the location of reported street littering cases. Consequently, the city council can analyze the data and increase productivity in monitoring and assist in environmental protection.

What it does

This website application employs webcam technology to capture images, which are then processed by the GPT-4-Vision-Preview model for recognition of corresponding items. Additionally, the model is provided with a predetermined list of classifications in a specific format, ensuring deterministic responses. This streamlined approach enhances accuracy and efficiency in item identification and classification.

Moreover, users have the option to report instances of street littering and access a real-time map displaying the locations of reported cases. This data empowers the city council to analyze trends and enhance monitoring efforts, thereby boosting productivity and supporting environmental protection initiatives.

How we built it

Front End: We initiated the development process by crafting an intuitive UI using Figma, a design tool that helped us conceptualize and refine the user interface. With our design in place, we translated it into code using React, a powerful framework known for its ability to create dynamic and interactive web applications. React enabled us to build a user-friendly interface that responds quickly to user interactions, enhancing the overall user experience.

Back End: In the backend infrastructure, we leveraged a combination of technologies to ensure robust functionality. MongoDB served as our primary database solution, offering scalability and flexibility for storing various types of data. We harnessed its document-oriented architecture to efficiently manage data related to user reports and disposal information.

Additionally, we utilized Flask, a lightweight web framework in Python, for API routing and handling HTTP requests. Flask provided a streamlined approach to developing RESTful APIs, allowing us to define endpoints and manage communication between the frontend and backend components effectively.

Furthermore, we integrated pre-trained models from OpenAI for image processing and classification, enhancing our system's ability to recognize and categorize items from user-uploaded images. This integration enabled us to provide users with accurate disposal information based on real-time analysis of uploaded images.

In the event of a user filing a report through the web application, pertinent details such as location data and waste type were collected and securely stored in the MongoDB Database. We then leveraged the Google Maps API to visualize reported cases in real-time, providing users with a comprehensive overview of waste management activities in their area directly from the frontend interface.

Challenges we ran into

Non-deterministic Responses from ChatGPT Model: The replies from the ChatGPT image recognition model were inconsistent and sometimes deviated from the specified format. We had to implement prompt engineering techniques to stabilize the responses.
1. Conversion Issues from Figma to HTML/CSS: Converting designs from Figma to HTML/CSS proved problematic, as it didn't translate seamlessly. Consequently, we had to extensively revise and essentially redesign the UI from scratch.
2. Limitations of ChatGPT Model Token: The ChatGPT model had a restricted token limit, which was insufficient for our needs. We couldn't provide it with the full list of classifications due to the token cap, hindering our ability to leverage the model to its fullest potential.
3. Mismanagement of API Keys: Managing API keys posed a challenge, as we inadvertently exposed secret keys on GitHub during our initial commits. As a result, our first ChatGPT API keys were revoked, necessitating additional security measures and caution moving forward. Consequently, we made the decision to utilize ChatGPT instead, leveraging its cloud-based processing capabilities. This shift enabled us to overcome processing limitations while still achieving our project goals efficiently.
4. Initial Plan with Zero-Shot Image Classification: Initially, we intended to employ a zero-shot image classification model from Hugging Face. However, due to constraints in processing power, classifying each image locally wasn't feasible.

Accomplishments that we're proud of

Successful Implementation of Webcam Image Recognition: We take pride in successfully implementing a robust image recognition system utilizing a webcam, providing users with seamless and efficient functionality.
Accurate Map Plotting from User Data: Our achievement in developing an active and precise map plotting feature, which seamlessly integrates user-generated data, underscores our commitment to providing valuable and relevant information in real-time.
Utilization of Image Recognition Model for Garbage Prediction: We're proud of successfully integrating an image recognition model into our system, enabling accurate prediction of the type of garbage from user-uploaded images. This innovative approach enhances the efficiency and effectiveness of waste management processes, contributing to a cleaner and more sustainable environment.
Intuitive and User-Friendly Design: We're proud to have created an intuitive and user-friendly design and UI, catering to users of all experience levels and ensuring a positive and engaging user experience.

What's next for Trash Cam

Moving forward, our team is eager to expand our project into mobile application development, offering users even more accessible solutions to address waste management challenges. We are dedicated to enhancing the accuracy of our image recognition model, particularly in navigating "gray area" cases like determining the proper disposal method for contaminated pizza boxes. Additionally, we plan to integrate advanced machine learning technologies, such as the K-Nearest Neighbors (KNN) algorithm, to predict the type or location of potential littering cases. By incorporating these innovations, we aim to propel the urban technological revolution forward and accelerate the development of sustainable cities