Skip to content

alekhyap004/NVIDIAxASUS-Hackathon-

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

ShieldView πŸ›‘οΈ

Real-time privacy protection for screen sharing
Powered by NVIDIA Nemotron 3 Nano Omni


What it does

ShieldView runs silently in the background while you screen share. It captures your screen, finds sensitive information, blurs it, and feeds the blurred version to Zoom/Meet/Teams through a virtual camera β€” before the other person ever sees it.

  • API keys & credentials β†’ blurred instantly
  • SSNs, patient data, account numbers β†’ blurred based on your industry
  • Passwords in terminals β†’ blurred
  • Works with: Zoom, Google Meet, Microsoft Teams, Webex, Discord, Slack β€” any app

Hybrid Mac + Brev (optional)

Offload Nemotron to Brev; keep screen capture and virtual camera on your Mac.

# One command on Mac (starts Brev remote + tunnels + pipeline):
./scripts/shieldview up

./scripts/shieldview down    # stop
./scripts/shieldview status  # health

See docs/HYBRID.md for the Mac + Brev hybrid setup.


Test Brev + ShieldView Integration

Use this checklist before a demo or before pushing changes. The Brev environment name used by this project is:

federal-olive-warbler

1. Mac Terminal β€” upload latest code to Brev

Run this from your Mac:

cd /Users/evangelenestanley/shieldview/ShieldView
./scripts/shieldview upload

This copies source files to Brev at ~/ShieldView-NVIDIA without uploading .venv, .deps, .git, local env files, reports, or runtime data.

2. Brev Terminal β€” make sure remote Nemotron is running

Open a Brev shell:

brev shell federal-olive-warbler

Then run this inside Brev:

cd ~/ShieldView-NVIDIA
source .venv/bin/activate
./scripts/shieldview brev
curl http://127.0.0.1:8788/health

Expected health response:

{"ok":true}

Leave this running if it stays in the foreground. If it says the remote service is already running, that is fine.

3. Mac Terminal β€” start the hybrid pipeline

Open a separate Mac terminal:

cd /Users/evangelenestanley/shieldview/ShieldView
./scripts/shieldview up --detach
./scripts/shieldview status

Expected status:

Mac bridge :8787 β€” OK
Remote Nemotron :8788 β€” OK
SSH forward tunnel β€” up
SSH reverse tunnel β€” up
Brev β†’ Mac bridge (reverse) β€” OK

4. Brev Terminal β€” verify Brev can control the Mac bridge

Inside Brev:

curl http://127.0.0.1:8787/health
curl http://127.0.0.1:8787/redaction/status
curl -X POST http://127.0.0.1:8787/briefing

Expected: /health returns "ok": true, /redaction/status shows the active session, and /briefing returns a pre-call risk summary.

5. NemoClaw integration point

Point NemoClaw at the Mac bridge from inside Brev:

http://127.0.0.1:8787

Useful endpoints for the agent:

GET  /health
GET  /redaction/status
POST /redaction/start
POST /redaction/stop
POST /briefing
POST /simulate/unknown_join
GET  /report/{session_id}

6. Stop the demo

On your Mac:

cd /Users/evangelenestanley/shieldview/ShieldView
./scripts/shieldview down

Quick Start

Step 1 β€” Install system dependencies

# macOS
brew install tesseract

# Install OBS (for the virtual camera driver)
# Download from: https://obsproject.com
# Open OBS β†’ Tools β†’ Virtual Camera β†’ Start β†’ close OBS

Step 2 β€” Install Python dependencies

pip install -r requirements.txt

Step 3 β€” Add your NVIDIA API key

Copy .env.example to .env and replace the placeholder:

cp .env.example .env
# edit .env:
NVIDIA_API_KEY=nvapi-YOUR-KEY-HERE

Get your key from: build.nvidia.com β†’ sign in β†’ Get API Key

No credits yet? Use OpenRouter (free) instead:

USE_OPENROUTER     = True
OPENROUTER_API_KEY = "sk-or-YOUR-KEY-HERE"  # openrouter.ai β€” free tier

Step 4 β€” Run one-time setup

python setup.py

This asks about your industry, apps, and sensitive data types. Takes 2 minutes. Only needs to be done once.

Step 5 β€” Run ShieldView

python main.py

Step 6 β€” Select the virtual camera in Zoom

Zoom β†’ Settings β†’ Video β†’ Camera β†’ OBS Virtual Camera

That's it. ShieldView is now protecting your screen share.


Commands (type in terminal while running)

Key Action
b Pre-call briefing β€” scan open windows for risks
r Generate incident report for current session
f Submit feedback to improve detection
s Show session stats
u Simulate unknown attendee joining
q Stop ShieldView

Options

python main.py --debug      # Show bounding boxes instead of blur (testing)
python main.py --briefing   # Run pre-call briefing before starting
python main.py --setup      # Re-run setup wizard
python main.py --fps 5      # Override FPS (lower = less CPU)

How it works

Your screen
    ↓
mss captures frames at 10fps
    ↓
Tesseract OCR reads text + regex finds patterns (every frame, instant)
    ↓
Nemotron 3 Nano Omni deep scan (every 4 seconds, catches what regex misses)
    ↓
OpenCV blurs detected regions
    ↓
pyvirtualcam feeds blurred frames to Zoom as a fake webcam
    ↓
Other person sees your screen with sensitive parts blurred

Privacy guarantee: No raw screen frames are ever stored or sent to the cloud. Nemotron only receives compressed JPEG snapshots for analysis. The actual sensitive values are never logged β€” only metadata (type, location, timestamp).


Troubleshooting

Virtual camera not showing in Zoom:

  1. Make sure OBS is installed
  2. Open OBS β†’ Tools β†’ Virtual Camera β†’ Start
  3. Quit and reopen Zoom
  4. Check Settings β†’ Video β†’ Camera dropdown

pyvirtualcam error on start:

  • The virtual camera must be running before Zoom opens
  • Always start python main.py before joining a Zoom call

Tesseract not found:

brew install tesseract

API key error:

  • Check .env β€” key should start with nvapi-
  • Alternative: set USE_OPENROUTER=true and add an OpenRouter key

Latency / high CPU:

  • Lower FPS: python main.py --fps 5
  • Or edit PIPELINE_FPS = 5 in config.py

Run smoke test

python scripts/smoke_test.py

What is real vs simulated

Feature Status
Screen capture βœ… Real (mss)
OCR detection βœ… Real (Tesseract)
Regex patterns βœ… Real (30+ patterns)
Blur rendering βœ… Real (OpenCV)
Virtual camera βœ… Real (pyvirtualcam + OBS)
Nemotron vision scan βœ… Real (NVIDIA API)
Pre-call briefing βœ… Real (Nemotron)
Post-call report βœ… Real (Nemotron + SQLite)
Unknown attendee detection ⚠️ Simulated (manual trigger with u)
Credential auto-rotation ❌ Not implemented (recommendation only)

Project structure

ShieldView/
β”œβ”€β”€ main.py           ← Run this
β”œβ”€β”€ setup.py          ← One-time setup wizard
β”œβ”€β”€ config.py         ← Settings; secrets come from .env
β”œβ”€β”€ requirements.txt
β”œβ”€β”€ shared/
β”‚   └── state.py      ← Global shared state
β”œβ”€β”€ pipeline/
β”‚   β”œβ”€β”€ capture.py    ← Screen grab (mss)
β”‚   β”œβ”€β”€ detect.py     ← OCR + regex detection
β”‚   β”œβ”€β”€ render.py     ← Blur + overlay
β”‚   └── main.py       ← Main pipeline loop
β”œβ”€β”€ agent/
β”‚   └── nemotron.py   ← Nemotron API client
β”œβ”€β”€ bridge/
β”‚   └── bridge.py     ← FastAPI HTTP bridge
β”œβ”€β”€ memory/
β”‚   β”œβ”€β”€ db.py         ← SQLite helpers
β”‚   └── schema.sql    ← Database schema
└── scripts/
    └── smoke_test.py ← Installation verification

Built for the NVIDIA Hackathon | ShieldView Team

About

UCSC Hackathon

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors