Global Disaster Monitor

Visualize disaster events around the world with semantic AI, Google Maps, and real-time data — powered by MongoDB.

Comment

🌍 Global Disaster Monitor

✨ Inspiration

The frequency and impact of natural disasters are increasing globally, yet timely insights and actionable intelligence are often missing. I was inspired to create the Global Disaster Monitor to provide a centralized dashboard for tracking disaster events, visualizing affected regions, and extracting patterns — all in real-time. The goal was to help researchers, policymakers, and even the general public stay informed with structured, visual, and data-driven summaries of global disasters.

🧠 What I Learned

While working on this project, I learned to:

  • Integrate MongoDB Atlas with geospatial and vector data indexing.
  • Perform vector-based search using MongoDB’s Atlas Search capabilities.
  • Cluster structured data using K-Means to identify patterns across locations and disaster categories.
  • Deploy a scalable, containerized app on Google Cloud Run using Docker.
  • Use Google Maps API effectively to map disaster locations interactively.
  • Build a real-time data pipeline using Python and Pandas with a clean Streamlit frontend.

🛠️ How I Built It

  • Frontend: Built with Streamlit, allowing interactive filters and dashboard views.

  • Backend:

    • Used pandas for data processing.
    • Connected to MongoDB Atlas for storing disaster records (country, coordinates, date, type, fatalities, etc.).
    • Implemented vector search on structured fields like location, category embeddings, and temporal features using MongoDB's \$vectorSearch.
    • Performed K-Means Clustering to categorize and group disaster-prone zones.

  • Geospatial: Google Maps API integrated to visualize clusters and live event points.

  • Deployment: Dockerized and deployed using Google Cloud Run for scalability and serverless operation.

🚧 Challenges I ran into

As a solo developer, balancing the scope of the project with time and infrastructure constraints was a major challenge. While the GDELT dataset is rich and structured, it posed some key limitations:

  • Frequent Updates: GDELT updates every 15 minutes, and downloading each CSV manually or in batches proved unsustainable without a scalable ingestion pipeline.
  • Storage and Scalability: Archiving and processing a large number of event files became challenging due to limited local resources and the lack of a robust cloud data pipeline setup.
  • Time Constraints: Building both the backend and frontend while managing data engineering alone meant I had to prioritize critical features over some planned enhancements.
  • Deployment Debugging: Ensuring the container responded on the right port (8080) and passed Cloud Run's health checks required several trial-and-error cycles.

Sure! Here's the Accomplishments section tailored to your solo project:

🏆 Accomplishments that I'm proud of

  • Successfully built and deployed a full-stack disaster monitoring system as a solo developer.
  • Integrated the GDELT global event database with MongoDB for real-time structured data tracking.
  • Implemented clustering techniques to group related events and highlight potential disaster zones.
  • Visualized events using interactive maps and time-based filters, enhancing data interpretability.
  • Deployed the app using Docker and Google Cloud Run, handling environment variables and deployment configuration independently.
  • Ensured secure integration of external APIs like MongoDB Atlas and Google Maps.

🔮 What's next for Global Disaster Monitor

  • Build a scalable ingestion pipeline using Cloud Functions or Pub/Sub to automatically download and process GDELT updates.
  • Shift to a cloud-based storage system like BigQuery or Cloud Storage buckets for handling large-scale CSVs efficiently.
  • Improve the clustering model with geo-aware embeddings and better anomaly detection.
  • Introduce user-level customization for region-specific disaster monitoring and notifications.
  • Possibly add historical data archiving and visualization to analyze disaster patterns over decades.

Built With

Share this project:

Updates