Cleanr

What Inspired Us

The main goal of this project was to learn more about machine learning and its practical applications. Having little to no experience on machine learning between team members, we decided to go simple and build a classification model based on existing neural nets. After settling on machine learning, we were inspired to tackle classification problems that were simple but relevant. One of the many shared nitpicks between team members was the sheer amount of clutter in terms of the camera roll on a person's phone. Countless scanned homeworks, memes, and screenshots blotted out more personal photos of family and friends. We wanted to make something that would quickly be able to filter out and delete irrelevant information

What it does

The app scans through a users camera roll and sends it to a server that categorizes and then determines if it should be kept. The user can see a summary of their photos and filter out unwanted photos based on categories.

How we built it

The app was built using Swift. The classification model is a transfer learning model built using Tensorflow. It relies on the pre-trained MobileNetV2 with the top layer removed. The top layer is then trained using a new dataset with more specific categories. The app will upload the images to the flask server, which will use the classification model to predict the category of the photo. This information is sent to the app, which shows the user which categories

Challenges we ran into

We wanted to work with React Native, but we were unable to integrate it with the camera roll. Additionally, we wanted the app to be run locally, but had difficulties using tensorflow lite and porting the learning model to mobile. We chose to go with a flask server as an alternative. Another challenge was the time pressure of training the neural net. We ended up training the model on approximately ~100,000 images, which was reduced from our dataset of about ~300,000 images. Additionally, we reduced the number of epochs to reduce time spent training, and did not fine tune our model to adjust the initial weights of the MobileNetV2 neural net.

Accomplishments that we're proud of

The neural network was surprisingly accurate, ~97% accuracy, though we trained it using a limited number of categories.

What we learned

We learned the basics of neural networks and how to integrate neural networks with mobile apps. Additionally, we (Akshit) learned about developing in Swift and Flask.