AIID (Team Undefined)

Inspiration

AI-related incidents are increasing rapidly, yet tracking them remains difficult. The AI Incident Database (AIID) has documented over 1,370 real-world AI harms — including autonomous vehicle crashes, deepfake fraud, biased hiring systems, and more. However, this rich dataset is not easily accessible to policymakers, journalists, or researchers who may not have technical skills to query raw data. Our guiding question was simple: What if anyone could explore the full history of AI harm?

What it does

We created the AIID Research Notebook — a fully reproducible Google Colab notebook that transforms the raw AIID dataset (provided in the hackathon) into an interactive research platform. It requires no setup and runs end-to-end in a single session.

Key Features

Automatically fetches the latest AIID snapshot on every run
Cleans and structures 1,370+ incidents and 6,700+ media reports
Integrates four taxonomies (MIT, GMF, CSET) into analysis-ready DataFrames
12 interactive visualizations covering:
- Incident trends over time
- Harm categories
- Geographic distribution
- Organization rankings
- Sector breakdowns
- Media coverage lag
- Emerging risks
Interactive filter panel (year, risk domain, deployer, AI goal, harmed party)
A 6-tab Gradio web interface for non-technical users
An AI Assistant powered by Retrieval-Augmented Generation (RAG)
- Ask questions in plain English
- Answers are grounded in real incidents
- Each response cites specific incident IDs

How we built it

Data Pipeline

Used requests and BeautifulSoup to download the latest dataset snapshot
Extracted archives with tarfile
Loaded data using CSV-first logic with BSON fallback

Preprocessing

Normalized entity names (e.g., “facebook” → “meta”)
Parsed multi-value taxonomy fields
Standardized datetime formats (UTC-aware)
Merged media statistics per incident
Created derived features:
- Era classification
- LLM-era flag
- Physical harm indicator

Visualizations

Plotly for interactive charts
Matplotlib for word clouds

Web Interface

Built with Gradio Blocks, including 6 tabs:

Incident Explorer
Trend Analysis
Entity Deep Dive
Harm Landscape
High-Impact Incidents
AI Assistant

RAG System

Used all-MiniLM-L6-v2 for embeddings
Stored vectors in a FAISS similarity index
Retrieved top-15 relevant incidents per query
Used Qwen3-0.6B for grounded answer generation
Disabled extended reasoning for faster responses

Challenges we ran into

Pandas version changes affecting timezone handling and column naming
Entity fragmentation (same organization appearing under multiple names)
Taxonomy sparsity — not all incidents are labeled across all taxonomies
Context limits in early models — switching to Qwen3 with a 32K window solved multi-incident reasoning limitations

Accomplishments that we're proud of

A fully reproducible notebook that runs top-to-bottom with live data
A grounded AI Assistant that answers nuanced questions like: > “Which deployers caused physical harm before 2020?”
Making 40+ years of documented AI harm accessible to non-technical users
Revealing trends hidden in raw data — especially the sharp rise in LLM-era harms after 2022

What we learned

Smaller retrieval models with large context windows outperform large models with small context in domain-specific RAG
Data cleaning and normalization take more effort than analysis itself
Reproducibility must be designed from the beginning
Fast, grounded responses matter more than long reasoning chains in interactive tools