PartCrafter

[NeurIPS 2025] PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers

Yuchen Lin*, Chenguo Lin*, Panwang Pan†, Honglei Yan, Yiqiang Feng, Yadong Mu, Katerina Fragkiadaki

This repository contains the official implementation of the paper: PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers. PartCrafter is a structured 3D generative model that jointly generates multiple parts and objects from a single RGB image in one shot. Here is our Project Page.

Feel free to contact me ([email protected]) or open an issue if you have any questions or suggestions.

📢 News

• 2025-09-18: PartCrafter is accepted to NeurIPS 2025.
• 2025-08-15: PartCrafter HuggingFace🤗 demo is available here. Thanks to alexnasa.
• 2025-07-23: The 3D scene version of PartCrafter is released, which is trained on 3D-Front.
• 2025-07-20: A guide for installing PartCrafter on Windows is available in this fork. Thanks to JackDainzh!
• 2025-07-13: PartCrafter is fully open-sourced 🚀.
• 2025-06-09: PartCrafter is on arXiv.

📋 TODO

• Release inference scripts.
• Release training code and data preprocessing scripts.
• Release pretrained checkpoints on both object and scene level.
• Provide a HuggingFace🤗 demo.
• Release preprocessed dataset.

🔧 Installation

We use torch-2.5.1+cu124 and python-3.11. But it should also work with other versions. Create a conda environment with the following command (optional):

conda create -n partcrafter python=3.11.13
conda activate partcrafter
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

Then, install other dependencies with the following command:

git clone https://github.com/wgsxm/PartCrafter.git
cd PartCrafter
bash settings/setup.sh

If you do not have root access and use conda environment, you can install required graphics libraries with the following command:

conda install -c conda-forge libegl libglu pyopengl

We test the above installation on Debian 12 with NVIDIA H20 GPUs. For Windows users, you can try to set up the environment according to this pull request and this fork. We sincerely thank JackDainzh for contributing to the Windows support!

💡 Quick Start

3D Part-Level Object Generation

Generate a 3D part-level object from an image:

python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --num_parts 3 --tag robot --render

The required model weights will be automatically downloaded:

• PartCrafter model from wgsxm/PartCrafter → pretrained_weights/PartCrafter
• RMBG model from briaai/RMBG-1.4 → pretrained_weights/RMBG-1.4

The generated results will be saved to ./results/robot. We provide several example images from Objaverse and ABO in ./assets/images. Their filenames start with recommended number of parts, e.g., np3 which means 3 parts. You can also try other part count for the same input images.

Specify --rmbg if you use custom images. This will remove the background of the input image and resize it appropriately.

VLM-Based Part Suggestion

Instead of manually specifying --num_parts, you can use a VLM to automatically suggest the number of parts:

GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --part_suggest --tag robot --rmbg --render

This sends the image to a VLM (default: gemini-3-flash-preview) which analyzes the object and suggests an appropriate part count. You can override the provider or model:

--part_provider gemini --part_model gemini-3-flash-preview

Style Transfer for Real-World Images

PartCrafter was trained on rendered images from Objaverse. When using real-world photos, you can apply style transfer to bridge the domain gap:

GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --num_parts 4 --style_transfer --rmbg --render

This converts the input photo to an Objaverse-style 3D rendering (default model: gemini-3.1-flash-image-preview) before feeding it to the pipeline. The stylized image is saved as styled_input.png in the output directory. You can override the provider or model:

--style_provider gemini --style_model gemini-3.1-flash-image-preview

Both features can be combined:

GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --part_suggest --style_transfer --rmbg --render

The provider architecture is extensible -- adding a new provider (e.g., OpenAI) requires only a new file in src/utils/providers/ implementing suggest_num_parts() and/or stylize_for_objaverse().

3D Scene Generation

Generate a 3D scene from an image:

python scripts/inference_partcrafter_scene.py \
  --image_path assets/images_scene/np6_0192a842-531c-419a-923e-28db4add8656_DiningRoom-31158.png \
  --num_parts 6 --tag dining_room --render

The required model weights will be automatically downloaded:

• PartCrafter-Scene model from wgsxm/PartCrafter-Scene → pretrained_weights/PartCrafter-Scene

The generated results will be saved to ./results/dining_room. We provide several example images from 3D-Front in ./assets/images_scene. Their filenames start with recommended number of parts, e.g., np3 which means 3 parts. You can also try other part count for the same input images.

The --part_suggest and --style_transfer flags are also available for scene-level generation.

💻 System Requirements

A CUDA-enabled GPU with at least 8GB VRAM. You can reduce number of parts or number of tokens to save GPU memory. We set the number of tokens per part to 1024 on object level and 2048 on scene level by default for better quality.

📊 Dataset

Please refer to Dataset README to download and preprocess the dataset. To generate a minimal dataset, you can run:

python datasets/preprocess/preprocess.py --input assets/objects --output preprocessed_data

This script preprocesses GLB files in ./assets/objects and saves the preprocessed data to ./preprocessed_data. We provide a pseudo data configuration here, which makes use of the minimal preprocessed data and is compatible with the training settings.

🦾 Training

To train PartCrafter from scratch, you first need to download TripoSG from VAST-AI/TripoSG and store the weights in ./pretrained_models/TripoSG.

huggingface-cli download VAST-AI/TripoSG --local-dir pretrained_weights/TripoSG

Our training scripts are suitable for training with 8 H20 GPUs (96G VRAM each). Currently, we only finetune the DiT of TripoSG and keep the VAE fixed. But you can also finetune the VAE of TripoSG, which should improve the quality of the generated 3D parts. PartCrafter is compatible with all 3D object generative models based on vector sets such as Hunyuan3D-2.1. We warmly welcome pull requests from the community.

We provide several training configurations here. You should modify the path of dataset configs in the training config files, which is currently set to ./datasets/object_part_configs.json.

If you use wandb, you should also modify the WANDB_API_KEY in the training script. If you have trouble connecting to wandb, try export WANDB_BASE_URL=https://api.bandw.top.

Train PartCrafter from TripoSG:

bash scripts/train_partcrafter.sh --config configs/mp8_nt512.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --tag scaleup_mp8_nt512

Finetune PartCrafter with larger number of parts:

bash scripts/train_partcrafter.sh --config configs/mp16_nt512.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model scaleup_mp8_nt512 \
  --load_pretrained_model_ckpt 10 \
  --tag scaleup_mp16_nt512

Finetune PartCrafter with more tokens:

bash scripts/train_partcrafter.sh --config configs/mp16_nt1024.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model scaleup_mp16_nt512 \
  --load_pretrained_model_ckpt 10 \
  --tag scaleup_mp16_nt1024

😊 Acknowledgement

We would like to thank the authors of DiffSplat, TripoSG, HoloPart, and MIDI-3D for their great work and generously providing source codes, which inspired our work and helped us a lot in the implementation.

📚 Citation

If you find our work helpful, please consider citing:

@misc{lin2025partcrafter,
  title={PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers}, 
  author={Yuchen Lin and Chenguo Lin and Panwang Pan and Honglei Yan and Yiqiang Feng and Yadong Mu and Katerina Fragkiadaki},
  year={2025},
  eprint={2506.05573},
  url={https://arxiv.org/abs/2506.05573}
}

🌟 Star History