Supplyit!

Supply Management at the Press of a Button

Comment

Backend for Supplyit! (Your AI-Powered Supply Cabinet for School Supply Management!)

Bigggggg idea ☁️

Hey... YOU there. Have you ever wandering into the forbidden cabinet of yours and saw the MOUNTAIN HEAP of RANDOMLY PLACED school supplies???? (ik some people will answer no). I have. This is why I wanted something to HELP ME sort + keep track of stuff with the TAP of a button... quite literarly.

⭐Basically: This app allows people to index their supplies in the real word with the tap of a button! (instead of counting manually...)⭐

What I wanted to achieve or was prototyping was a system where many different clients would connect to one server (like in a school setting. Think high school).

These clients would respectively contain different supplies and have their own hardware + software (like a barcode scanner + touch screen + raspberry pi) and manage their own borrows and returns.

The admin side would basically have the ability to monitor all the logs, supplies, and have the ability to montior the client's location (which is why in the app there is a gps feature)

Features

Entire app

  • Firebase integration! ✅
  • FCM Messaging (Basically meaning push notifications work)✅
  • Local notifications✅
  • Camera + Barcode + Summarizing (yes, I use AI to categorize items) = AI integration ✅
  • IT HAS DARKKK MODEEEE✅
  • don't forget light mode too✅
  • it also has a sick logo (trust)✅
  • Portrait AND landscape mode for the app (except for camera part 🥲)✅

Overview

Admin side Client Side
Monitor GPS location + be able to get overview in gps mode (so you won't loose any of your supply clients) List of supplies in the client
Overview of Storage Clients Ability to add supplies to the client through camera or barcode on just a device!
Color indicators Filtering ability (invovles AI... (mystically)woooooo)
Search for supplies based on categories Shopping list generation + copy to clip board
Search results autodirect to map of gps locations Borrowing and returning items (emphasis on the UI ⭐)
Explore to find out more! Notifications delivered when supplies are running low
Cute icons that list the supplies based on category
Explore to find out more!

Small side note: many of the implemented scripts are not hard coded so to boast a little more it can support as many supplies and supply clients as firebase can hold 😉

Backend

  • One yolov8 model finetuned with custom dataset images (dataset has ~2k images)
  • One chatgpt 4.0 model to do image inferencing WITH the finetuned model (its actually slightly better lol but much slower than the yolo)
  • One chatgpt 3.5 model to do category inferencing based on product names (customize your own categories)
  • Flask server (highly customizable!)

Future features

  • Integrate hardware. This may include theft detection (basically pass a current through the supply client's cabinet's locks and see when it disconnects) + card id systems or facial recognition systems. This also includes a barcode scanner and raspberry pi (im broke ☹️)
  • More sophisticated Borrow/Return system which includes the end user
  • Actually buy a good barcode-to-item api (because the api im using has a kinda-bad-+-lacking database)

Backend Setup

All setup is done with the main directory (directory which contains main.ipynb) as root.

Initial

git clone https://github.com/hackathonprojects4eded/reccesshacks4-backend.git
cd reccesshacks4-backend

Python Enviornment

python -m venv .venv
# WINDOWS: .venv\scripts\activate 
# MAC: . .venv/bin/activate ( not verified)
.venv\scripts\activate
pip install -r requirements.txt

Server Enviornment

  1. Set up a firebase project
  2. Go to settings and download a firebase sdk json file
  3. Move the firebase sdk file to the main directory, rename it to firebasesdk.json
  4. Open the folder server
  5. Create a file .env
  6. In .env put OPENAI_API_KEY=COPYPASTEYOUROPENAIKEYHERE
  7. Create a folder named images
  8. Exit the folder server, back to main directory.

Running

cd server
python app.py

Customization

Note: you can change the port exposed - in this case I'm running on my local IP/network so my phone can communicate w/ the windows pc

If you do, remember to change the url in constants.dart in the frontend`

Supplyit! Frontend

For the project overview + features please find our backend folder for reference

This tutorial is for ios only and commands are for unix system (I'm writing this from a macbook pro 🤣)

Setup

Folder

git clone https://github.com/hackathonprojects4eded/reccesshacks4-frontend.git
cd reccesshacks4-frontend

Coding Enviornment

  • Preferrably you should have vscode flutter extension downloaded and you should be sort of familiar with the flutter development process
  • Once you open the folder, flutter should automatically detect your packages and run flutter pub get

Set up Firebase

  • Add Realtime database to your firebase project in your console.
  • Also add Authentication - Email&password to your firebase project via the console.
  • After you have your firebase project (check backend instructions for more details), please follow this tutorial to learn how to configure your flutter project with firebase

Note: You should see a file named firebase_options.dart after you ran fireflutter configure (you might also see a firebase.json I forgot what step that appeared)

Set up FCM (Push notifications)

This step is a bit tricky since you actually need to have an apple developer account:

You would need to follow these instructions to upload a .p12 file/certificate in the settings-cloud messaging tab:

Upload your APNs authentication key

Before you use FCM, upload your APNs certificate to Firebase. If you don't already have an APNs certificate, create one in the Apple Developer Member Center.

Inside your project in the Firebase console, select the gear icon, select Project Settings, and then select the Cloud Messaging tab. Select the Upload Certificate button for your development certificate, your production certificate, or both. At least one is required. For each certificate, select the .p12 file, and provide the password, if any. Make sure the bundle ID for this certificate matches the bundle ID of your app. Select Save.

Other:

None! (Hopefully... please contact me if there are other instructions I missed)

Running

You can run this on your own iphone by running flutter run --profile. This will make the app presistent. To debug, run flutter run in the terminal.

Customization:

So I actually added a noise factor to mapscreen.dart when displaying all the locations because its a hackathon project and I didn't want to dox myself (If I did that would have been bad 🙁). To remove this, simply do this:

//before (lines 21, 22 of screens/mapscreen.dart)
final randomLatDis = Random().nextDouble() / 100.0;
final randomLongDis = Random().nextDouble() / 100.0;
//after 
final randomLatDis = 0;
final randomLongDis = 0;

You can also customize the app name/constants and where you are connecting to the api by modifying contents in constants.dart.

Devpost Reflections

Challenges we ran into

  • I had to reset my entire firebase (meaning re-configuring and basically redownloading all setup files) because I accidently deleted the entire google cloud project because I was messing around with google map's api 😞
  • Null operator issues (dart people will understand)
  • YolovX models not compatible with the library I am using from ultralytics.... so I had to retrain my models three SEPERATE times. The one in the github repository is the latest one!
  • FCM for some reason extremely slow + inconsistent???? (I tested this and originally it works but for some reason its not now...)
  • Integrating maps + map markers into the app (surprisingly difficult but sort of worked!)

Accomplishments

I'm afraid that my project does not have a lot to offer since I'm not really rushing when I'm writing this (in all my previous hackathons I had like 30 minutes to write and record something for the submission 💀). Basically, I'm surprsied that I managed to 'finish' my project goal with 6 hours to spare.

Built With

Share this project:

Updates