FraudFence

QR Code to Access our WebApp

Inspiration

For college students, food delivery apps are some of the frequently used services on their phones. When they are combined with generative tools like ChatGPT being at everyone’s fingertips, it doesn’t take long for someone to prompt an LLM to alter an image of perfectly good food to now appear moldy or spoiled. This got us wondering how often this happens, and the results were shocking. The food delivery industry loses an estimated $25 billion a year to refund fraud, and AI image generation has made the barrier basically zero. Before the rise in popularity of AI image generation, creating a convincing fake photo took photoshop skills and effort but now anyone could do it with a single prompt.

What inspired us wasn't just the scale of the problem. It was a counterintuitive insight that we can take the exact workflow a scammer uses to fake a photo to make them easier to catch. This is because every step in that process leaves behind detectable signals. For example, downloading an AI-generated image strips its metadata and the model leaves frequency-domain fingerprints. Then, when is reuploaded, double compression artifacts are introduced and reusing an image can also be caught by the method of perceptual hashing. Finally, user data like account age, claim frequency, and device fingerprint are also major indicators of potential fraud. We built our project to exploit every one of those breadcrumbs simultaneously and fight back against scammers.

What it does

FraudFence analyzes uploaded images for signs of fraud, manipulation, synthetic generation, or suspicious reuse. It runs multiple checks across image metadata, compression patterns, perceptual hashing, frequency and noise artifacts, provenance signals, and optional behavioral context like account history or device data, then combines those signals into an overall risk score and confidence level.

The platform gives users a clear plain-language result first, showing whether an image appears low risk, moderately concerning, or highly suspicious, along with the main reasons behind that assessment. For deeper review, it also provides technical evidence such as per-layer scores, weighted contributions, and visual artifacts, making it useful both as a quick fraud-triage tool and as a more explainable investigation aid.

How we built it

We built FraudFence as a full-stack web app with a Next.js frontend and a FastAPI backend. The frontend handles image upload, guided results, and technical review views, while the backend runs the multi-layer fraud-detection pipeline, combines detector outputs into a weighted risk score, and stores supporting data with SQLite.

For the detection system, we used Python-based image and signal-processing tools for metadata, compression, hashing, and frequency analysis, and integrated Gemini for AI-assisted visual reasoning on suspicious images. We also deployed and ran the system on Vultr, which gave us a practical cloud environment for hosting, testing, and making the app accessible as a working product rather than just a local prototype.

Challenges we ran into

One of the biggest challenges was balancing speed, accuracy, and explainability. Different detectors produce very different kinds of signals, and some are much noisier or slower than others, so combining them into one final score without making the system feel inconsistent was difficult. We also had to design the product so it could give a simple result to non-technical users while still exposing enough technical evidence to make the output trustworthy.

We also ran into practical engineering challenges around real-world image handling, deployment, and integration. Supporting different file formats, dealing with missing or unreliable metadata, integrating Gemini into the analysis flow, and getting everything running smoothly in a deployed Vultr environment all took iteration. Making the full pipeline work reliably end to end under hackathon time constraints was a challenge on its own.

Accomplishments that we're proud of

We’re proud that we built a working end-to-end fraud-screening system that can take an uploaded image, run it through multiple forensic checks, and return a usable risk assessment with both confidence scoring and supporting evidence. Instead of relying on one model, we combined metadata analysis, error level analysis, perceptual hashing, frequency-domain checks, provenance-style checks, behavioral context, and Gemini-powered visual reasoning into a single pipeline that produces a more explainable result.

We’re also proud that we made the product practical and review-friendly. The frontend gives users a clean summary first, then lets them drill into technical details like per-layer results and evidence, which makes the tool useful for both quick triage and deeper investigation. On top of that, we got the system deployed on Vultr, which means FraudFence was not just a local prototype but a real working application.

What we learned

We learned that fraud detection works best when multiple signals are combined rather than relying on a single detector or model. Different techniques catch different kinds of problems, so bringing together metadata checks, forensic image analysis, behavioral context, and Gemini-based reasoning produced a much more useful and explainable system than any one method alone.

Another thing we learned how challenging it is to build a system that is both technically strong and easy to use. It forced us to think carefully about calibration, explainability, deployment, and user experience at the same time, and it showed us how important it is to design for real-world issues like inconsistent file formats, noisy evidence, and deployment constraints in environments like Vultr.

What's next for FraudFence

Next, we want to turn FraudFence into an API-first product so companies can integrate fraud screening directly into their existing systems instead of using it only as a standalone app. That means a business could send images, account context, device information, and claim details straight into FraudFence from their own workflow and receive back a structured fraud-risk score, confidence level, and supporting evidence in real time. Making FraudFence available as an API would let insurance companies, retailers, marketplaces, and support teams plug fraud analysis directly into the tools they already use.

We also want to build stronger network intelligence, where fraud patterns detected on one platform can help inform detection across others. From there, we see FraudFence expanding into more domains like insurance, retail returns, warranty programs, and gig economy disputes, while continuing to strengthen the business case that even a single prevented fraudulent claim can pay for hundreds of fraud analyses.