-
-
Skimmer installed on the terminal machine.
-
No skimmer installed. Safe system.
-
Safe status on the iOS app.
-
Danger status. A skimmer was installed.
-
Map of nearby police departments.
-
Suspect's image captured from the webcam during the incident. Along with a physical description using Gemini's API.
-
Skimmer overlay (Failed 3d printed model taped with a notebook's cover)
Inspiration
We've all seen those news stories about credit card skimmers found at gas stations or grocery stores, and the worst part is that by the time anyone notices, the damage is done and the person responsible is long gone. I asked myself, why does a customer have to get their card stolen before anyone finds out? I wanted to solve that entirely. As someone that is very interested in Cybersecurity and Finance, solving this problem was very interesting to me and I wanted to make sure that technology is used as the first and last line of defense in our daily lives.
What it does
BlockOut monitors enterprise payment terminals for any tampering attempts. The moment a skimmer is detected to be physically attached, the hardware immediately turns into a "locked" mode, and shows "Breach detected" on the small screen attached to the terminal. After this point, the emergency phone number that is set from the BlockOut iOS app is already getting called by an AI describing the situation and the suspect. It also simultaneously snaps a picture of whoever is standing there and sends it to Gemini's API for analysis and description. The user would already have selected their preferred police station so that they're immediately contacted by our AI during the incident. The whole chain happens in seconds without anyone having to lift a finger.
How we built it
The skimmer detection in the hardware side works by using an ESP32 microcontroller with an IR sensor that senses proximity using the reflected light that it detects. In a production environment, it would be ideal to have the sensor embedded inside the terminal with the firmware. But for my case, I only used one sensor that is attached externally as payment terminals have a locked firmware by default, meaning I can not modify it.
Upon detection, it changes the screen data to "Breach detected" and it will "lock" the payment terminal. The ESP32 then sends through bluetooth a code message to our iOS app to indicate that a skimming has took place. The iOS app responds by giving us the alert and immediately sending a POST request to our local web app in order to take a picture of the suspect as the incident is still happening (In this case, the laptop's webcam is acting as a security camera, which would be configured the same way).
When successful, we will be able to view the suspect's image from our mobile app (as it retrieves it with a simple GET request from our web app). It also uses Gemini's API to give an accurate description of whoever is on the image.
Simultaneously, if you've linked a phone number through the mobile app, you will get a call from our AI using Vapi AI's API. The AI is instructed to give description of the suspect and report the incident directly to you, and it will prompt you if you want to forward the evidence to a specific police station that you've selected from the app before.
Challenges we ran into
I have never 3d printed before as I don't have a big hardware background, so 3d printing was definitely a new experience for me which failed. I didn't get the dimensions I expected even though I was certain that it looked accurate in Fusion. The model turns out to be much smaller. However, I found that my notebook's cover had a steady material, which I taped with the broken plastic parts of my 3d printed model to reconstruct a dimensionally correct one. It worked just fine!
I've also tried soldering for the first time with a cheap soldering kit, and it was not a successful attempt. I instead used 2 breadboards to get enough room for my VCC and GND and the other GPIO pins.
Best Use of AI (powered by Reach Capital)
AI plays a big part in combatting this type of financial crime. Humans can never have the same reaction time to report an incident, nor the accuracy in description and execution. An image of the suspect is first analyzed by sending it to Gemini's API with a prompt to generate a very clear description of the individual. What would take humans hours: retrieving the footage and reviewing it frame by frame, AI does it instantaneously and plays a big role in our safety.
Using Vapi AI's API, it calls the linked phone number to notify the store manager about the incident. The AI is fully interactive and is capable to have very realistic conversations with the user. It would also simultaneously contact the police station that you've chosen from the iOS app.
Log in or sign up for Devpost to join the conversation.