Project Page

Authors: Shen Chen¹, Ruiyu Zhao², Zongkai Wu³, Jenq-Neng Hwang⁴, Serge Belongie⁵, Lei Li^4,5*

¹Zhejiang University  ²East China University of Science and Technology  ³skai worldwide   ⁴University of Washington   ⁵University of Copenhagen

^*Corresponding author.

This project introduces CausalStruct, a novel framework that integrates causal reasoning into 3D scene generation to enhance logical coherence, physical plausibility, and adaptability.

Existing 3D scene generation methods often struggle to model the complex logical dependencies and physical constraints between objects, limiting their ability to adapt to dynamic and realistic environments. We propose CausalStruct, a novel framework that embeds causal reasoning into 3D scene generation. Utilizing large language models (LLMs), we construct causal graphs where nodes represent objects and attributes, while edges encode causal dependencies and physical constraints. CausalStruct iteratively refines the scene layout by enforcing causal order to determine the placement order of objects and applies causal intervention to adjust the spatial configuration according to physics-driven constraints, ensuring consistency with textual descriptions and real-world dynamics. The refined scene causal graph informs subsequent optimization steps, employing a Proportional-Integral-Derivative (PID) controller to iteratively tune object scales and positions. Our method uses text or images to guide object placement and layout in 3D scenes, with 3D Gaussian Splatting and Score Distillation Sampling improving shape accuracy and rendering stability. Extensive experiments show that CausalStruct generates 3D scenes with enhanced logical coherence, realistic spatial interactions, and robust adaptability.

.
├── README.md          # Project description file
├── code/              # Code files
│   ├── CLIP_evaluate.py    # Evaluation script
│   ├── MLLM_evaluate.py    # Evaluation script

Our method demonstrates superior spatial consistency and causal alignment compared to existing methods. Below are qualitative results:

Our method supports flexible and controllable scene editing via text descriptions. Objects can be added, removed, or repositioned based on causal relationships, ensuring physical plausibility and semantic coherence.

We conduct ablation studies to evaluate the impact of causal reasoning. The results show that causal reasoning enhances scene coherence.

Thank you for your interest in this project! I hope my research provides valuable insights and inspiration. For any inquiries, please feel free to contact me.