SAX-NeRF

 

 

   

 

Introduction

This repo is a comprehensive toolbox and library for X-ray 3D reconstruction including two tasks, novel view synthesis (NVS) and computed tomography (CT) reconstruction. This repo supports 12 state-of-the-art methods including six NeRF-based methods, three 3DGS-based methods, two optimization-based methods, and one analytical method. We also provide code for fancy visualization such as turntable video and data generation to help your research. If you find this repo useful, please give it a star ⭐ and consider citing our paper. Thank you.

News

• 2026.11.07 : We have released our datasets and models on HuggingFace 🤗
• 2026.06.18 : Our new work Perturbed-Gaussian-Ensemble for active learning 3DGS-based CT reconstruction has been accepted by ECCV 2026. Congrats to Yulun. Code and models will be released at this repo. 🚀
• 2025.09.18 : Our new work CARE for diffusion based Anatomy-aware enhancement of sparse-view CT reconstruction has been accepted by NeurIPS 2025. Congrats to Tianyu. Code and models have been released at this repo. 🍒
• 2025.06.25 : Our new work X2-Gaussian for dynamic human chest breathing CT reconstruction has been accepted by ICCV 2025. Congrats to Weihao. Code and models will be released at this repo. 🚀
• 2024.09.25 : Our new work R2-Gaussian has been accepted by NeurIPS 2024. Congrats to Ruyi. Code and model will be released at this repo. 💫
• 2024.09.01 : Code of our ECCV 2024 work X-Gaussian has been released. Welcome to have a try! 🚀
• 2024.07.09 : Our SAX-NeRF has been added to the Awesome-Transformer-Attention collection 💫
• 2024.06.16 : I will present this work in person. Our poster session is from 10:30 am to 00:30 pm, Jun 20 at Arch 4A-E Poster #147. Welcome to chat with me in Seattle Convention Center. :satisfied:
• 2024.06.16 : More raw data and generation samples are provided. Feel free to use them.
• 2024.06.03 : Code for traditional methods has been released. 🚀
• 2024.06.03 : Code for fancy visualization and data generation has been released. 🚀
• 2024.06.02 : Data, code, models, and training logs have been released. Feel free to use them :)
• 2024.03.07 : Our new work X-Gaussian, the first 3DGS-based method for X-ray imaging, is now on arxiv now. Code, models, and training logs will be released at this repo. Stay tuned. 💫
• 2024.02.26 : Our paper has been accepted by CVPR 2024. Code and pre-trained models will be released to the public before the start date of CVPR 2024 (2024.06.19). Stay tuned! :tada: :confetti_ball:
• 2023.11.21 : The benchmark of X3D at the paper-with-code website has been set up. You are welcome to make a comparison. 🚀
• 2023.11.21 : Our paper is on arxiv now. We will develop this repo into a baseline for X-ray novel view synthesis and CT reconstruction. All code, models, data, and training logs will be released. 💫

Performance

• SAX-NeRF (CVPR 2024)
• X-Gaussian (ECCV 2024)
• R2-Gaussian (NeurIPS 2024)
• PGE (ECCV 2026)
• X2-Gaussian (ICCV 2025)
• CARE (NeurIPS 2025)
• TensoRF (ECCV 2022)
• NAF (MICCAI 2022)
• NeAT (ACM TOG 2022)
• NeRF (ECCV 2020)
• InTomo (ICCV 2021)
• SART (Ultrasonic imaging 1984)
• ASD-POCS (Physics in Medicine & Biology 2008)
• FDK (Josa a 1984)

Coordinate System

The coordinate system in circular cone-beam X-ray scanning follows the OpenCV standards. The transformation between the camera, world, and image coordinate systems is shown below.

 

1. Create Environment:

We recommend using Conda to set up an environment.

# Create environment
conda create -n sax_nerf python=3.9
conda activate sax_nerf

# for users that don't have CUDA-11.3 on their server and
# don't have sudo permission to install, please install CUDA-11.3 in conda environment
# otherwise you can skip these steps
conda install -c nvidia/label/cuda-11.3.1 cuda cudnn cuda-nvcc	# install CUDA-11.3 (and all needed pkg) for conda environment
ln -s $CONDA_PREFIX/lib $CONDA_PREFIX/lib64	# softlink to solve a TIGRE version/path check issue
conda install -c conda-forge gcc=10 gxx=10	# only support gcc <= 10

# Install pytorch (hash encoder requires CUDA v11.3)
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113

# Install other packages
pip install -r requirements.txt

We suggest you install TIGRE toolbox (2.3 version) for executing traditional CT reconstruction methods and synthesize your own CT data if you plan to do so. Please note that TIGRE v2.5 might stuck when CT is large.

# Download TIGRE
wget https://github.com/CERN/TIGRE/archive/refs/tags/v2.3.zip
unzip v2.3.zip
rm v2.3.zip

# Install TIGRE
pip install cython==0.29.25
pip install numpy==1.21.6
cd TIGRE-2.3/Python/
python setup.py develop

 

2. Prepare Dataset:

Download our processed datasets from Google drive or Baidu disk. Put the downloaded datasets into data/.

Alternatively, download the datasets from Hugging Face using Git LFS. With Git LFS installed, run these commands from the project root:

git lfs install
git clone https://huggingface.co/datasets/CaiYuanhao/SAX-NeRF data

This places the processed .pickle files directly in data/, with the following layout:

  |--data
      |--chest_50.pickle
      |--abdomen_50.pickle
      |--aneurism_50.pickle
      |--backpack_50.pickle
      |--bonsai_50.pickle
      |--box_50.pickle
      |--carp_50.pickle
      |--engine_50.pickle
      |--foot_50.pickle
      |--head_50.pickle
      |--leg_50.pickle
      |--pancreas_50.pickle
      |--pelvis_50.pickle
      |--teapot_50.pickle
      |--jaw_50.pickle

 

3. Testing:

You can directly download our pre-trained models from Google drive or Baidu disk. Put the downloaded models into pretrained/.

Alternatively, download the pretrained models from Hugging Face using Git LFS. Run these commands from the project root:

git lfs install
git clone https://huggingface.co/CaiYuanhao/SAX-NeRF pretrained

This places the scene checkpoints directly in pretrained/, including pretrained/chest.tar. With the dataset downloaded above, run:

# SAX-NeRF
python test.py --method Lineformer --category chest --config config/Lineformer/chest_50.yaml --weights pretrained/chest.tar --output_path output 

# FDK
python3 eval_traditional.py --algorithm fdk --category chest --config config/FDK/chest_50.yaml

# SART
python3 eval_traditional.py --algorithm sart --category chest --config config/SART/chest_50.yaml

# ASD_POCS
python3 eval_traditional.py --algorithm asd_pocs --category chest --config config/ASD_POCS/chest_50.yaml

For your convenience, we provide the reconstructed results in Google drive and Baidu disk.

 

4. Training:

We provide the training logs on all scenes for your convenience to debug. Please download the training logs from Google dive or Baidu disk.

# SAX-NeRF
python train_mlg.py --config config/Lineformer/chest_50.yaml

# NeRF
python train.py --config config/nerf/chest_50.yaml

# Intratomo
python train.py --config config/intratomo/chest_50.yaml

# NAF
python train.py --config config/naf/chest_50.yaml

# TensoRF
python train.py --config config/tensorf/chest_50.yaml

You can use this repo to run NeAT. Remember to reprocess the data first.

 

5. Visualization

To render a cool demo, we provide visualization code in the folder 3D_vis

cd 3D_vis
python 3D_vis_backpack.py
python 3D_vis_backpack_gif.py

 

6. Generate Your Own Data

We also provide code for data generation in the folder dataGenerator. To give you a quick start, we provide the raw data for your debugging. Please download the raw data from Google dive or Baidu disk and then put them into the folder dataGenerator/raw_data. Run

cd dataGenerator
python data_vis_backpack.py
cd ..
python dataGenerator/generateData_backpack.py

 

7. Citation

If this repo helps you, please consider citing our works:

# SAX-NeRF
@inproceedings{sax_nerf,
  title={Structure-Aware Sparse-View X-ray 3D Reconstruction},
  author={Yuanhao Cai and Jiahao Wang and Alan Yuille and Zongwei Zhou and Angtian Wang},
  booktitle={CVPR},
  year={2024}
}

# X-Gaussian
@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Angtian Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}

# R2-Gaussian
@inproceedings{r2_gaussian,
  title={R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction},
  author={Ruyi Zha and Tao Jun Lin and Yuanhao Cai and Jiwen Cao and Yanhao Zhang and Hongdong Li},
  booktitle={NeurIPS},
  year={2024}
}

# Perturbed-Gaussian-Ensemble
@inproceedings{pge,
  title={Active View Selection with Perturbed Gaussian Ensemble for Tomographic Reconstruction},
  author={Yulun Wu and Ruyi Zha and Wei Cao and Yingying Li and Yuanhao Cai and Yaoyao Liu},
  booktitle={ECCV},
  year={2026}
}

# X2-Gaussian
@inproceedings{x2_gaussian,
  title={X2-Gaussian: 4D Radiative Gaussian Splatting for Continuous-time Tomographic Reconstruction},
  author={Yu, Weihao and Cai, Yuanhao and Zha, Ruyi and Fan, Zhiwen and Li, Chenxin and Yuan, Yixuan},
  booktitle={ICCV},
  year={2025}
}

# CARE
@inproceedings{lin2025pixel,
  title={Are Pixel-Wise Metrics Reliable for Sparse-View Computed Tomography Reconstruction?},
  author={Lin, Tianyu and Li, Xinran and Zhuang, Chuntung and Chen, Qi and Cai, Yuanhao and Ding, Kai and Yuille, Alan L and Zhou, Zongwei},
  booktitle={NeurIPS},
  year={2025}
}