ReActure

REACT time screen
Friends Page
Leaderboard
Home Page
Player View

What It Does

ReActure is a gamified 3D disaster simulator that converts rescue missions into AI training data. 🎮 Players act as rescue robots, navigating collapsing cities and blocked paths while making quick, high-stakes decisions.

How We Built It

We built ReActure using a Three.js-powered 3D simulation running entirely in the browser. Players explore dynamic disaster environments where every action — movement, rotation, and decision — is recorded in real time.

A Node.js + Express.js backend processes these interactions, converting them into structured JSONL and NumPy datasets that can be used to train AI models. The game communicates seamlessly with a MongoDB Atlas database through Mongoose, ensuring each player’s decisions and trajectories are securely stored.

We used the Pointer Lock API for immersive first-person controls, Canvas API for frame capture, and Web Crypto API to anonymize and secure gameplay data. For rapid testing and visualization, a lightweight Python HTTP Server supports local deployment, while GitHub handles version control and collaboration.

Built With

Frontend: Three.js — real-time 3D simulation and WebGL rendering Pointer Lock API — immersive first-person navigation Canvas API — visual frame capture for AI dataset generation LocalStorage — player data persistence Web Crypto API — secure hashing and anonymization

Backend: Node.js + Express.js — RESTful API and data streaming MongoDB Atlas + Mongoose — cloud data storage and schema validation CORS + dotenv — configuration and cross-origin management Python HTTP Server — quick local testing and visualization

Data & Infrastructure: JSONL + NumPy — structured AI-ready datasets AWS (Amazon Web Services) — optional scalable storage GitHub — version control and collaboration

Challenges We Ran Into

Capturing human decision data without sensors
Balancing fun gameplay with real-world data fidelity
Keeping 3D environments smooth in-browser

Accomplishments We’re Proud Of

Built a browser-based 3D disaster simulator Collected multimodal human decision data Converted player strategies into labeled AI datasets Recentered AI on human intuition as intelligence

What We Learned

How to turn gameplay into structured AI data
The value of multimodal signals — movement, timing, and voice
That games can act as data engines for robotics research

Full Tech Stack

Frontend

Three.js — 3D rendering and WebGL graphics
JavaScript (ES6+) — core game logic with no framework overhead
Pointer Lock API — immersive first-person controls
Web Crypto API — secure password hashing (SHA-256)
Canvas API — visual frame capture for ML datasets
LocalStorage — lightweight user data persistence

Backend

Node.js — server runtime for real-time logic
Express.js — RESTful API framework
MongoDB Atlas — cloud NoSQL database (M0 free tier)
Mongoose — schema validation and modeling
CORS — secure cross-origin data handling
.env — environment variable management

Data & ML

NumPy — structured array storage (.npy files)
JSONL — time-series streaming format for decision traces

Infrastructure

MongoDB Atlas — scalable cloud database (free M0, 512MB)
GitHub — version control and deployment
Python HTTP Server — lightweight local development

What’s Next

Train robot navigation models using ReActure data Expand to more environments: floods, wildfires, and space missions Add emotion-aware, multilingual AI narration Release the ReActure Dataset publicly Evolve into a full AI–human collaboration platform where play teaches real-world intelligence 🚀