Dream Flow

Dream Flow Immersive Experience
project built using visual programming tool TouchDesigner

Inspiration

The idea behind creating this project is to explore the intersection of technology and human experience, specifically the dream state. Dreams are a unique and subjective experience, and this project seeks to use AI and visual art to represent and interpret them in a new way. By using EEG/EOG signals to predict sleep stages and generate the particle system, we aim to create a visual representation of the dream state that is both grounded in science and open to artistic interpretation. The project also seeks to explore the idea of the subconscious mind and the ways in which our thoughts and emotions can manifest in our dreams. Ultimately, the project aims to create a mesmerizing and immersive experience that allows users to explore the depths of their own minds in a new and exciting way.

Our project, Dream Flow , was inspired by Refik Anadol's "Melting Memories" installation. We were fascinated by the idea of using brain signals to create dynamic and immersive art, and we wanted to explore this concept further using AI and 3D particles.

What it does

Dream Flow uses EEG/EOG data to predict sleep stages using AI models. Based on classification results, we segment and extract the part of EEG/EOG signals which corresponds to REM stage along with attention scores from the AI model. Extracted signals are used to control the flow of particles in 3D space, adjusting factors like surface attraction, size of the surface area, drag, and position of particles.

How we built it

We preprocessed the EEG/EOG signals and used a pretrained Transformer model to predict the sleep stage based on the two signals. During inference time, we segmented and extracted the parts of the signals that corresponded to the REM classification output. The extracted signals and the attention scores from the Transformer model were then used to control the visuals.

For the visual representation, we used TouchDesigner, which is a node-based visual programming language. We used its capabilities to create a particle system that could be controlled by the EEG/EOG signals and attention scores. The system was designed to adjust factors like surface attraction, size of the surface area, drag, and position of particles, creating a dynamic and immersive representation of the dream state.

Challenges we ran into

Our main challenge was building a particle system that accurately represented the flow and beauty of dream states. We had to ensure that the system was responsive to the EEG/EOG signals and attention scores while also being visually appealing. Additionally, we struggled to find the best model for sleep stage prediction, which was crucial for segmenting the EEG/EOG signals and generating the 3D particles.

Rendering the scene was also challenging, as we had to adjust the lighting conditions and camera position to achieve the desired effect. Due to the limited resources on our free plan, we were not able to scale our project to higher resolutions, which also posed a challenge.

Accomplishments that we're proud of

We're proud of the progress we've made so far, especially in adopting a state-of-the-art AI model for sleep stage prediction and implementing a functional particle system in limited setting. We're excited to continue refining our project and exploring new possibilities for AI-generated art.

What we learned

Throughout this project, we've learned a great deal about the intersection of AI, neuroscience, and art. We've gained valuable experience in using Pytorch and Transformer for predictive modeling, as well as in working with EEG/EOG data and 3D particle systems.

What's next for Dream Flow

In the future, we plan to continue refining our project and exploring new ways to push the boundaries of AI art. We're excited to experiment with new techniques for controlling particle flow and creating even more dynamic and immersive visualizations. Ultimately, we hope to inspire new forms