Shatter

🎨 Shatter: Animated Mosaic Rendering with Native macOS Windows

Shatter is a unique visual art project that deconstructs images, videos, and real-time streams into an animated mosaic of native macOS windows. Inspired by creative media like Animator vs Animation on YouTube, we conceived an unconventional rendering software that uses individual, overlapping windows on a user's desktop to form an image.

The result is a dynamic, "shattered" effect created by coloured rectangular blocks, each rendered as a separate, movable macOS window.

💡 The Journey and Technical Breakthroughs

The project began by experimenting with Python's Tkinter on Windows to open windows of different sizes, but the initial performance was extremely poor. First, we used a quadtree algorithm to recursively break down an image until the colour variance within a box was below a threshold. This produced too many windows, leading to severe slowdowns on Windows OS, capping performance at an unusable 1 frame per second (FPS).

Therefore, we shifted development to macOS as it was ironically better than Windows and experimented with Objective-C++ for window handling, despite the language being novel to the entire team. This move immediately produced a significant performance increase, achieving 10-15 FPS with the original quadtree code.

The final performance breakthrough came from implementing a new, optimised approach:

Probabilistic Sampler: Instead of searching the entire image on every frame, the algorithm was modified to focus on regions of high deviation (where the image changed most).
Frame Differencing Optimisation: By iterating upon previous frames and only updating the blocks that had changed, the system achieved an amazing performance boost and seamless transitions between frames.

This architecture requires running the core processing logic in Python (using NumPy), which outputs a JSON file containing the positions and colors of the blocks. This JSON is then efficiently read and rendered by the compiled Objective-C++/Cocoa code.

The overall hackathon experience highlighted the importance of delegation and recognising that changing development environments can be the key to solving a fundamental performance problem.

🌟 Key Features

Native Window Rendering: Utilises a compiled macOS binary (WindowCreator based on the Cocoa framework) to render blocks as actual, movable windows, providing the project's signature effect.
Multiple Input Sources: Supports static images, videos, real-time screen captures, and live webcam feeds.
Optimised Decomposition: Features a frame differencing algorithm combined with a probabilistic sampler that dynamically focuses on regions of high deviation for efficient image deconstruction, resulting in an amazing performance boost and seamless transitions.
Performance Optimisation: Employs Frame Differencing to only update regions that change between frames, dramatically improving video and real-time stream performance.
Web-Based Control Panel: An intuitive HTML/Flask frontend for managing input sources, configuring settings, and starting/stopping the display.

🛠️ How It Works: System Architecture

Shatter operates using a three-part architecture:

Frontend Interface (static/): A user interacts with the Flask web UI at http://localhost:5000 to select media or a live feed.
Flask Backend (server.py):
- Handles HTTP requests and manages file storage.
- Extracts video frames (using OpenCV) or initiates real-time frame capture (using mss for screen or cv2 for webcam).
- Orchestrates the entire process.
Core Processing & Rendering:
- The compiled WindowCreator (Objective-C/C++) application is executed by the server.
- WindowCreator calls the Python processing scripts (get_block2.py / faster_simon.py). These scripts perform the frame differencing and the probabilistic, variance-based decomposition to generate the JSON block data.
- WindowCreator reads the JSON and creates, moves, or updates the native macOS windows on the desktop, realising the "shattered" animation.