M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation

Yiheng Zhang^1*, Zhuojiang Cai^2*, Mingdao Wang^1*, Meitong Guo¹, Tianxiao Li¹, Li Lin³, Yuwang Wang^1†

¹ Tsinghua University   |   ² Beihang University   |   ³ Migu Beijing Research Institute

* Equal contribution   |   † Corresponding author

We have released our Dataset, Inference, Object Retrieval and Rendering code, come and try it!!!

Installation

conda create -n m3dlayout python=3.10 -y
conda activate m3dlayout
conda install cuda -c nvidia/label/cuda-12.1.0 -y
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt --use-pep517 --no-build-isolation

Download Dataset

Please download our processed dataset from Baidu Netdisk (Google Drive edition will soon be released). Please reserve at least 230 GB of storage space to download these datasets. The following items are included:

Dataset	Rendering Images	Additional Data
Infinigen	• Floor masks • Oblique-view scene renderings • Top-down scene renderings • Multi-view renderings of main objects	• Text descriptions • Detailed per-scene JSON
3D-FRONT	• Floor masks • Top-down scene renderings	—
Matterport3D	• Floor masks for each region • Top-down layout renderings for each region	• Detailed per-scene JSON

Each dataset additionally provides three JSON files with organized and detailed scene info and text descriptions for model training or layout visualization:
<data_source>_train.json, <data_source>_test.json, and <data_source>_val.json.

You can use visualization_mlayout.py in Object-Retrieval-Layout2Scene to visualize above json files to see the layout.

Download Weights

Please download the model weights from Google Drive or Baidu Netdisk, and place them in the ./weights directory.

Inference

To run the Gradio demo for 3d layout generation from arbitrary text:

python gradio_demo.py

TODO

• Release Object Retrieval code of M3DLayout
• Release rendering code of layouts and scenes
• Release inference code of M3DLayout
• Release M3DLayout dataset
• Provide training instruction for M3DLayout

Citation

If you find our work helpful, please consider citing:

@article{zhang2025m3dlayout,
      title={M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation}, 
      author={Yiheng, Zhang and Zhuojiang, Cai and Mingdao, Wang and Meitong, Guo and Tianxiao, Li and Li, Lin and Yuwang, Wang},
      journal={arXiv preprint arXiv:2509.23728},
      year={2025},
      url={https://arxiv.org/abs/2509.23728}, 
}

Acknowledgements

Our code borrows from ATISS and DiffuScene. We thank them for their excellent work. Please follow their licenses when using this part of the code.