Inspiration
Ever realized how we sometimes lose track of our spendings? We do have a budget, but there is no way we can limit our expenses based off of that? Instead of the old-school ways of writing down all the expenses of a week on paper and categorizing them, we can now use FinanceOps.ai by only taking a picture of any receipt!!! (made in less than a day :) )
What it does
FinanceOps.ai is a mobile friendly web app built to categorize and keep a database of your daily receipts using computer vision and Natural language processing allowing you to track and visualize each category of your spendings on a monthly basis and compare it to a personalized budget. The user sets a budget for their expenses, and will be able to scan a receipt. Using computer vision and natural language processing, the receipt will be saved in the database and the AI generated tags for each product will allow the user to search for their history of spendings on each category. In addition to the comparison between the budget and spendings, FinanceOps provides an interactive visualization of the history of user's spendings on each category. Our app, takes care of you if you forget your budget! FinanceOps sends you periodic SMS with the status of your spendings to keep you in (financial) shape!
The implementation is adapted to present a novel usage of Co:here API as well as the best usage of Twilio and has a super friendly user interface to qualify for the best design challenge.
How we built it
OCR (optical character recognition) is to convert an image data to machine-encoded text. We use VeryFi API to do an OCR on every receipt image the user uploads to analyze. The text will be generated and the backend saves the information of a receipt and every item on it to the database. This information includes a general tag associated to each item. The tag is generated using Co:here API's generate functionality. After spending some time doing prompt engineering, we ended up getting the most optimal prompt to associate tags to items of a receipt.
In order to display the update, the backend built by node/express in Typescript uses sqlite3 to store the database and provide the user with a visualization of the status on the front end which is React.
Twilio's API will be used later to notify the user once about their spending status in a while when they leave the website. This is done by sending an API call whenever the client leaves the website.
The messages will be generated by Co:here in a funny manner!
The front end is built in React and is adapted from the very specific prototyping we did on Figma.
Stack: Typescript, ExpressJS, React Libraries of Interest, frameworks, and tools: Veryfi, Co:here, Twilio, MUI for React, Figma
Challenges we ran into
A major challenge of this project was creating the tags of each item in a receipt. Coming up with an engineered prompt for the generate function of Co:here API required us to learn more about how generative ML models work.
Accomplishments that we're proud of
Working with a wide range of APIs and creating our own backend based off of those. Prototyping in Figma before implementing the frontend in addition to a dark mode.
What we learned
We learned to work in an agile system even though we didn't have much time to add all the properties we wanted to our website. Working with new libraries and APIs in typescript and unifying them to react components.
What's next for FinanceOps.ai
Expanding the ideas of finance handling and financial personalization to broader extent by learning from user's history and suggest them recommendations. Creating the web app as a mobile app.
Built With
- co:here
- figma
- mui
- node.js
- react
- twilio
- typescript
- veryfi
Log in or sign up for Devpost to join the conversation.