App Home Screen
App Upload Screen
App Record Screen
App Camera Screen
Website Home Page
Website DBSCAN

Fresh

By Autin Mitra, Andrew Lam, Andrew Xie, and David Liu

What is Fresh?

Fresh is a tool that is used to autonomously record and display locations of trash found outside, providing powerful and valuable analysis tools for researchers, volunteers, and environmentalists alike.

Why?

Scientists approximate we have 5-6 years left to save the environment. In order to help combat this issue, a tool is needed to speed up data collection and analysis - making volunteering and researching much easier.

The problem with conventional volunteer work and environmental analysis is the method of data it is recorded in: often paper. This format limits the types of data usable for analysis, is subject to human error (tallying), is time consuming to process, and is subject to wear/degradation. Our objective was to create a more autonomous platform for trash data collection, and providing the necessary visual analysis tools for both environmentalists and researchers. The provided visualization tools provide valuable input towards small-scale and large-scale trash distribution, allowing researchers to identify trash hot-spots with density-clustering algorithms such as K-means, DBSCAN, and OPTICS.

With fresh, we believe that we can help mitigate the process of climate change and accelerate research.

What technologies does it use?

We used Flutter for the mobile app and used Google's MobileNet in combination with TFLite to detect trash in an image.
For the backend, we utilized ExpressJS and MongoDB to host our website & API.
For the front-end visuals, we used GatsbyJS with theme-UI. google-map-react was used to display the map and framer-motion was used for fancy-pants animations.

How does it work?

The fresh mobile app can be used to scan the environment for trash. Upon finding a piece of trash, the app will identify the type of trash (plastic bag, bottles, etc.) and allows the user to upload an image of the trash, the GPS coordinates, and the type of trash to a server.

The backend receives the information and stores the GPS coordinates, image, and trash type in a database. The front-end retrieves this data, providing scientists with a multitude of analysis tools.

In the front-end, researchers/scientists/analysts/environmentalists can use various tools at their disposal to further analyze patterns in trash distribution. We provide density-clustering/clustering algorithms, including DBSCAN, OPTICS, and kMeans. Each algorithm can be customized with arguments, such as the # neighbors, neighbor radius, and the number of clusters. The data points are plotted in a Google Maps components, with different colors representing the labels and clusters. Furthermore, users can filter out different types of trash for a more precise analysis of distribution patterns.

What improvements could we make to this?

Add more analysis tools
"Spaces" concept (multiple maps per organization)

What challenges did we have?

The project was more complex than the usual projects we've accomplished, mainly due to the plethora of codebases we had to write. Usually, we would focus on a single platform (web, backend, app, etc.), but this time around, we wrote ALL 3. So in 24 hrs (alongside some sleep), we somehow managed to make a coherently functional integration of all three platforms.

Also, it's our first time integrating TFlite into a Flutter app, and due to weird dependency issues with TFLite and our models, we spent quite a bit of time debugging that.

TLDR: Lot's of code, so little time, but we did it 😳

Bro, why didn't you use the state-of-the-art google-nvidia-stanford-carnegiemellon-georgiainsituteoftech-harvard-uva-nova-model for trash recognition, bro?!?!?!?!? 👁️👄👁️

This is a hackathon; we got ~24 hours (some of which I'd like to use for sleep), and gathering training data, waiting for it to train, and then building the necessary infrastructure to make it run is a huge PITA. Also, TFlite for Flutter is notoriously wacky sometimes, may as well run with something works (mobilenet).

sheeeesh UI looking clean 😳😳😳

thanks we made it 😎

Figma Link

Screenshots

App Home Screen App Upload Screen App Record Screen App Camera Screen Website Home Page Website DBSCAN

We hope you enjoyed our project! 😳

Built With

Submitted to

HackTJ 8.0
- Winner Best Mobile App

Created by

I wrote the app-components in Flutter, using TFLite to integrate the MobileNet ML model. Also wrote the backend components, built on MongoDB, NodeJS, and Experss. Designed and wrote the front-end UI in Figma and GatsbyJS, using Google Maps for the visualization.

Autin Mitra
Mostly helped with the frontend using GatsbyJS. Also helped with presentation.

Andrew Lam
I helped work on the backend and helped integrating MobileNet model. Worked on map layout and clustering for trash hotspots. I assisted with the frontend and made contributions to the presentation and project files. I helped debug the files.

David Liu
I contributed to writing the front-end and researched the machine learning model we used. I also assisted in creating the presentation.

Andrew X