LeGalería

Inspiration

We were inspired by a simple but alarming observation: every photo we share potentially leaks sensitive information. A colleague accidentally posted a photo with their credit card visible on a desk. A friend's Instagram story showed their license plate clearly. These everyday privacy breaches happen because manually checking every photo is tedious. We realized AI could solve this - but existing solutions either require cloud uploads (defeating the privacy purpose) or expensive manual editing. We wanted to build something that protects privacy without compromising it.

What it does

LeGalería is a smart photo gallery that automatically detects and blurs sensitive content in your photos using on-device AI. When you import photos, our custom-trained YOLOv11 model scans for five categories of sensitive information: faces, documents, license plates, QR codes, and screens. Detected sensitive regions are automatically blurred, and you can unlock them with biometric authentication when needed. Everything happens locally on your device - no cloud uploads, no data transmission, just pure on-device protection.

How we built it

Dataset Creation: Merged 2000+ images from Roboflow covering five target classes Model Training: Fine-tuned YOLOv11-nano on Google Colab's T4 GPU (60 minutes) Model Optimization: Exported to ONNX format (10MB) optimized for mobile inference Mobile Development: Built with Expo/React Native after discovering Lynx lacked image processing capabilities Dual Detection Pipeline: Implemented YOLO for visual detection + OCR fallback for text-based sensitive content Privacy Features: Two-pass detection system - YOLO first, OCR if needed for maximum safety

Challenges we ran into

Framework Pivot: Started with Lynx for the UI challenge but discovered it couldn't handle image tensor conversion needed for AI inference. Pivoted to Expo mid-hackathon Two-Stage Pipeline: Implementing efficient YOLO + OCR detection while keeping processing under 1 second per image (average 0.8s) Tensor Transformation: Converting React Native images to ONNX-compatible tensors required custom implementation OCR Integration: Adding text detection for license plates and documents that YOLO might miss *The AI model have issue of loading on iOS device when testing

Accomplishments that we're proud of

Successfully trained a custom YOLOv11 model achieving 93% mAP Implemented dual detection (YOLO + OCR) with 0.8s average processing time Created a two-pass safety system: visual detection first, text detection fallback Achieved true on-device processing with zero cloud dependencies Successfully pivoted frameworks mid-hackathon without losing momentum

What we learned

Two-stage detection (YOLO + OCR) provides better coverage than either alone On-device AI can handle complex pipelines - 0.8s for dual detection is acceptable UX OCR catches license plates and document text that pure visual detection might miss Framework limitations can make or break an AI project - image processing is crucial Privacy-preserving AI benefits from redundant detection methods

What's next for LeGalería

End-to-End Encryption: Implement AES-256 encryption for sensitive photos with key derivation from biometric data Performance Optimization: Reduce processing from 0.8s to under 0.5s through model quantization Video Support: Extend protection to video files with efficient frame sampling Selective Protection: Allow users to choose which categories to auto-protect

Built With

• expo.io
• javascript
• ocr
• on-device
• onnx
• opencv
• python
• react-native-text-detector
• ts
• ultralytics
• yolo