Aware

In-depth analysis UI
Scam awareness page
Learn page

Inspiration

This project was inspired by the increasing number of scams targeting seniors. The goal was to create a simple, easy-to-use tool that could help older adults quickly identify potential online threats, like phishing emails and fake links, without feeling overwhelmed. The idea was to provide peace of mind and make the internet a safer place for them.

What it does

The web app analyses texts, emails and whatnots using AI to prevent potential scam opportunities.

How I built it

The project was built using the Next.js app router, utilising Gemini AI to analyse the text via Vercel AI SDK, Radix UI for the base UI components with Tailwind CSS for styling. The AI analyzes emails or messages to identify potential scams, by checking for phishing patterns, dangerous links, and suspicious requests for personal information.

AI system instructions

To orient/modify the AI model’s behaviour to be more suitable to the app's theme, system instructions alongside the intended data structure output had to be provided

Image & text analysis can be done through two interfaces — quick & in-depth analysis featuring different function calls, different system instructions.

In-depth analysis

Chat-based so no structured response, continuous interaction

Quick analysis

Structured AI output, one-time analysis — single session, required additional prompt to the in-depth analysis as it requires structured data from the AI to the UI.

Structured AI output

The Vercel AI SDK did most of the heavy lifting as regards structuring the AI’s response but there was an issue, when a user adds an image for quick analysis there was an invalid JSON error from the server, skimming through the error log, I realised the error occurred from the SDK attempting to parse the model’s response; since the error only happened when I used an image as the input, I modified the system instruction for the quick analysis interface to return the AI response as the intended data structure provided to it beforehand even when it receives image as an input.

Note: this was only happening because the AI response is required to be structured, this didn't happen for the in-depth analysis interface.

so ultimately, In addition to the provided zod schema, I reinforced the AI’s structured output capability with updated system instructions.

In relation to the above, there was a slight issue, it was an error from Gemini itself when it receives only an image as input, there was an error, the error log was definitely descriptive; apparently it required a text parameter. So the simple & obvious fix was to add a default text when a user uploads only an image & no accompanying text input.

There were sporadic/random errors like incomplete AI responses, at first I thought there was a CSS text-clipping issue; the issue was due to incompatibility from the AI model itself which were bypassed by setting a MaxToken value, playing around with the SDK’s config but it wasn't stable. To fix this I had to check from the Gemini website & used the updated AI model which fixed the random errors.

As with all errors & issues encountered, this fixes didn't come particularly easy.

Error handling

Potential errors are handled gracefully, the UI responds with appropriate text to the user.

User experience

To eliminate/reduce friction in the app’s user experience, I utilised the Clipboard API to enable copy&paste media functionality on the web, in addition to the existing drag-and-drop, click-to-upload options.

Features

Input handling for text and images with drag-and-drop, click-to-upload, copy-and-paste images via the Clipboard API to eliminate friction in the user experience.
Text & image analysis capabilities for both quick and in-depth analysis interfaces using Gemini AI via the Vercel AI SDK.
Speech-to-text in the indepth-analysis chat UI to improve UX & accessibility.
Text-to-speech to read analysis results.
Print analysis result to create awareness for other users.
Scam awareness exercise to improve scam detection & recognition skills

Challenges I ran into

One of the biggest challenges was getting structured data from the AI that could easily be displayed on the frontend. Making sure the information was understandable and actionable for users without overcomplicating things took some extra effort. Another issue was using the Web Speech Synthesis API to read the scam analysis result, on Chrome, it time outs after like 15s of activaton; so getting it to work was an hassle really. I thought of splitting the text into chunks to avoid the timeout but ultimately I settled with pausing the speech after 14s and then resuming concurrently

Accomplishments that I'm proud of

I'm grateful for being able to bring the app to fruition, generally speaking; from ideation to building & deploying, anything could go(have gone) wrong.

Particularly, the whole app UI looks good

What I learned

One of the main lessons was understanding how to build something accessible to a non-technical audience. It became clear that making technology both powerful and simple requires extra thought in terms of design and user experience. There was a lot learned about structuring data for better integration with the frontend and ensuring the results were presented in a clear, understandable way.

What's next for Aware

A platform to report confirmed scams
Enabling Generative UI in the AI chat interface for a more interactive experience
Validating any suspicious URL extracted in a suspicious text against an external service