Awesome-3D-AIGC

Awesome 3D AIGC Resources

A curated list of papers and open-source resources focused on 3D AIGC, intended to keep pace with the anticipated surge of research in the coming months. If you have any additions or suggestions, feel free to contribute. Additional resources like blog posts, videos, etc. are also welcome.

Table of contents

• Survey
• Text to 3D Generation
• Image to 3D Generation
• Audio to 3D Generation
• 3D Editing
• Human Avatar Generation
• City/Autonomous Driving
• SLAM
• BioMedical
• 4D AIGC
• Misc
• Open Source Implementations
  • Reference
  • Unofficial Implementations
  • Datasets
  • Other
• Practical Tools
• Blog Posts
• Tutorial Videos
• Credits

Mar. 4, 2024: Update several CVPR 2024 papers.

Jan 23, 2024: Update several ICLR 2024 papers.

Jan 19, 2024: Update several ICLR 2024 papers.

Jan 11, 2024: Add AGG and recent papers.

Jan 10, 2024: Add DreamGaussian (3D version) and several avatar papers.

Jan 6, 2024: Add recent papers.

Jan 2, 2024: Add papers to image to 3d generation.

Dec 29, 2023: Contribute to the section on text-to-3d by adding new papers with their publication years.

Dec 27, 2023: Initial list with first 15 papers.

Survey:

1. Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era [arxiv 2023.05]

Authors: Chenghao Li, Chaoning Zhang, Atish Waghwase, Lik-Hang Lee, Francois Rameau, Yang Yang, Sung-Ho Bae, Choong Seon Hong

📄 Paper

2. Deep Generative Models on 3D Representations: A Survey [arxiv 2023.10]

Authors: Zifan Shi, Sida Peng, Yinghao Xu, Andreas Geiger, Yiyi Liao, Yujun Shen

📄 Paper | 🌐 Project Page

3. A survey of deep learning-based 3D shape generation [Computational Visual Media 2023.05]

Authors: Qun-Ce Xu, Tai-Jiang Mu, Yong-Liang Yang

📄 Paper

4. Learning Generative Models of 3D Structures [Computer Graphics Forum 2020.05]

Authors: Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu, Hao Zhang

📄 Paper

5. A Survey on 3D Gaussian Splatting [arxiv 2024.01]

Authors: Guikun Chen, Wenguan Wang

6. A Survey on Deep Generative 3D-aware Image Synthesis [ACM Computing Surveys 2023.11]

Authors: WEIHAO XIA JING-HAO XUE

📄 Paper | 🌐 Project Page

7. Advances in 3D Generation: A Survey [arxiv 2024.01]

Authors: Xiaoyu Li, Qi Zhang, Di Kang, Weihao Cheng, Yiming Gao, Jingbo Zhang, Zhihao Liang, Jing Liao, Yan-Pei Cao, Ying Shan

📄 Paper

8. Advances in Neural 3D Mesh Texturing: A Survey [arxiv 2026.06]

Authors: Sai Raj Kishore Perla, Hao (Richard) Zhang, Ali Mahdavi-Amiri

📄 Paper | 🌐 Project Page

Text to 3D Generation:

1. DreamFusion: Text-to-3D using 2D Diffusion [ICLR 2023]

Authors: Ben Poole, Ajay Jain, Jonathan T. Barron, Ben Mildenhall

📄 Paper | 🌐 Project Page | 💻 Code

2. Shapeglot: Learning Language for Shape Differentiation [CVPR 2019]

Authors: Panos Achlioptas* Judy Fan Robert Hawkins Noah Goodman Leonidas Guibas

📄 Paper | 💻 Code

3. Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings [ACCV 2018]

Authors: Kevin Chen, Christopher B. Choy, Manolis Savva, Angel X. Chang, Thomas Funkhouser, Silvio Savarese

📄 Paper | 🌐 Project Page | 💻 Code

4. ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model [NeurIPS 2022]

Authors: Rao Fu, Xiao Zhan, Yiwen Chen, Daniel Ritchie, Srinath Sridhar

📄 Paper | 🌐 Project Page

5. Magic3D: High-Resolution Text-to-3D Content Creation [CVPR 2023]

Authors: Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, Tsung-Yi Lin

📄 Paper | 🌐 Project Page | [💻 Code][Coming soon.]

6. Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models [CVPR 2023]

Authors: Jiale Xu, Xintao Wang, Weihao Cheng, Yan-Pei Cao, Ying Shan, Xiaohu Qie, Shenghua Gao

📄 Paper | 🌐 Project Page | [💻 Code][Coming soon.]

7. CLIP-Mesh: Generating textured meshes from text using pretrained image-text models [SIGGRAPH ASIA 2022]

Authors: Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, Tiberiu Popa

📄 Paper | 🌐 Project Page | 💻 Code

8. Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation [CVPR 2023]

Authors: Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, Greg Shakhnarovich

📄 Paper | 🌐 Project Page | 💻 Code

9. Dream Fields: Zero-Shot Text-Guided Object Generation with Dream Fields [CVPR 2022 and AI4CC 2022 (Best Poster)]

Authors: Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, Ben Poole

📄 Paper | 🌐 Project Page | 💻 Code

10. RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D [arxiv 2023.11]

Authors: Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mutian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, Xiaoguang Han

📄 Paper | 🌐 Project Page | 💻 Code

11. SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity [arxiv 2024.01]

Authors: Peihao Wang, Zhiwen Fan, Dejia Xu, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, Vikas Chandra

📄 Paper | 🌐 Project Page | 💻 Code

12. Taming Mode Collapse in Score Distillation for Text-to-3D Generation [arxiv 2024.01]

Authors: Peihao Wang, Dejia Xu, Zhiwen Fan, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, Vikas Chandra

📄 Paper | 🌐 Project Page | 💻 Code

13. DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation [arxiv 2023.09]

Authors: Jiaxiang Tang1, Jiawei Ren2, Hang Zhou3, Ziwei Liu2, Gang Zeng1

📄 Paper | 🌐 Project Page | 💻 Code

Note: DreamGaussian also supports image-to-3d and text-to-image-to-3d.

14. HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance　[arxiv 2023.05]

Authors: Junzhe Zhu and Peiye Zhuang

📄 Paper | 🌐 Project Page | 💻 Code

15. Triplane Meets Gaussian Splatting [arxiv 2023.12]

Authors: Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, Song-Hai Zhang

📄 Paper | 🌐 Project Page

16. DMV3D:Denoising Multi-View Diffusion using 3D Large Reconstruction Model [ICLR 2024]

Authors: Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, Kai Zhang

📄 Paper | 🌐 Project Page

Note: DMV3D also supports image-to-3d and text-to-image-to-3d.

17. Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model [arxiv 2023.11]

Authors: Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, Sai Bi

📄 Paper | 🌐 Project Page

18. Instant3D: Instant Text-to-3D Generation [arxiv 2023.11]

Authors: Ming Li, Pan Zhou, Jia-Wei Liu, Jussi Keppo, Min Lin, Shuicheng Yan, Xiangyu Xu

📄 Paper | 🌐 Project Page | 💻 Code

Note: two Instant3D papers :)

19. Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts [ICLR 2024]

Authors: Xinhua Cheng, Tianyu Yang, Jianan Wang, Yu Li, Lei Zhang, Jian Zhang, Li Yuan

📄 Paper | 🌐 Project Page | 💻 Code

20. DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation [ICLR 2024]

Authors: Yukun Huang, Jianan Wang, Yukai Shi, Boshi Tang, Xianbiao Qi, Lei Zhang

📄 Paper

21. TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy TextFields [ICLR 2024]

Authors: Tianyu Huang, Yihan Zeng, Bowen Dong, Hang Xu, Songcen Xu, Rynson W.H. Lau, Wangmeng Zuo

📄 Paper | 🌐 Project Page

22. Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation [arxiv 2024.01]

Authors: Minglin Chen, Longguang Wang, Weihao Yuan, Yukun Wang, Zhe Sheng, Yisheng He, Zilong Dong, Liefeng Bo, Yulan Guo

📄 Paper

23. Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior [arxiv 2024.01]

Authors: Zike Wu, Pan Zhou, Xuanyu Yi, Xiaoding Yuan, Hanwang Zhang

📄 Paper

24. AToM: Amortized Text-to-Mesh using 2D Diffusion [arxiv 2024.02]

Authors: Guocheng Qian, Junli Cao, Aliaksandr Siarohin, Yash Kant, Chaoyang Wang, Michael Vasilkovsky, Hsin-Ying Lee, Yuwei Fang, Ivan Skorokhodov, Peiye Zhuang, Igor Gilitschenski, Jian Ren, Bernard Ghanem, Kfir Aberman, Sergey Tulyakov

📄 Paper | 🌐 Project Page | 💻 Code

25. EscherNet: A Generative Model for Scalable View Synthesis [arxiv 2024.02]

Authors: Xin Kong, Shikun Liu, Xiaoyang Lyu, Marwan Taher, Xiaojuan Qi, Andrew J. Davison

📄 Paper | 🌐 Project Page | 💻 Code

26. TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts [arxiv 2024.01]

Authors: Jingyu Zhuang, Di Kang, Yan-Pei Cao, Guanbin Li, Liang Lin, Ying Shan

📄 Paper

Note: TIP-Editor supports both text-to-3d and image-to-3d

27. BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion [arxiv 2024.01]

Authors: Yonghao Yu, Shunan Zhu, Huai Qin, Haorui Li

📄 Paper

28. AToM: Amortized Text-to-Mesh using 2D Diffusion [arxiv 2024.02]

Authors: Guocheng Qian, Junli Cao, A