HDR-GS

 

Scene Bathroom	Scene Chair	Scene Dog

Scene Bear	Scene Desk	Scene Sponza

 

Introduction

This is the official implementation of our NeurIPS 2024 paper "HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting". We have run the SfM algorithm to recalibrate the data. If you find this repo useful, please give it a star ⭐ and consider citing our paper. Thank you.

News

• 2026.11.08 : We have released our dataset and models on HuggingFace 🤗
• 2024.12.01 : We provide code for direct loading model to test and render spiral demo video. Welcome to have a try! 🤗
• 2024.11.30 : We set up a leaderboard on the paper-with-code website! Welcome to submit your entry! 🏆
• 2024.11.26 : Code, recalibrated data following the opencv standard, and training logs have been released. Feel free to check and have a try! 🤗
• 2024.07.01 : Our HDR-GS has been accepted by NeurIPS 2024! Code will be released before the start date of the conference (2024.12.10). Stay tuned. 🚀
• 2024.05.24 : Our paper is on arxiv now. Code, data, and training logs will be released. Stay tuned. 💫

Performance

 

Interactive Results

Scene Bathroom	Scene Chair	Scene Diningroom

Scene Dog	Scene Sofa	Scene Sponza

 

1. Create Environment

We recommend using Conda to set up the environment.

# cloning our repo
git clone https://github.com/caiyuanhao1998/HDR-GS --recursive


SET DISTUTILS_USE_SDK=1 # Windows only

# install the official environment of 3DGS
cd HDR-GS
conda env create --file environment.yml
conda activate hdr_gs

 

2. Prepare Dataset

Download our recalibrated and reorganized datasets from Google Drive or Baidu Disk (code: cyh2). Put the downloaded datasets into data_hdr/.

Alternatively, download the data from Hugging Face using Git LFS. With Git LFS installed, run these commands from the project root:

git lfs install
git clone https://huggingface.co/datasets/CaiYuanhao/HDR-GS data_hdr

This downloads synthetic/ and real/ directly into data_hdr/, with the following layout:

  |--data_hdr
    |--synthetic
      |--bathroom
        |--exr
          |--0.exr
          |--1.exr
          ...
        |--images
          |--0_0.png
          |--0_1.png
          ...
        |--sparse
          |--0
            |--cameras.bin
            |--images.bin
            |--points3D.bin
            |--points3D.ply  
            |--project.ini
      |--bear
      ...
    |--real
      |--flower
        |--input_images
          |--000_0.jpg
          |--000_1.jpg
          ...
        |--poses_bounds_exps.npy
        |--sparse
          |--0
            |--cameras.bin
            |--images.bin
            |--points3D.bin
            |--points3D.ply  
            |--project.ini
      |--computer
      ...

Note: The original datasets are collected by HDR-NeRF. But the camera poses follow the normalized device coordinates, which are not suitable for 3DGS. Besides, HDR-NeRF does not provide the initial point clouds. So we use the Structure-from-Motion algorithm to recalibrate the camera poses and generate the initial point clouds. We also organize the datasets according to the description of HDR-NeRF, which is different from its official repo.

 

3. Testing

We write the code for directly loading the model to test and render spiral video. Please download our pre-trained weights bathroom from Google Drive or Baidu Disk (code: cyh2) and then put it into the folder pretrained_weights.

Alternatively, download the pretrained models from Hugging Face using Git LFS. Run these commands from the project root:

git clone https://huggingface.co/CaiYuanhao/HDR-GS pretrained_weights

This downloads both bathroom/ and sofa_2/ into pretrained_weights/. The bathroom model files are pretrained_weights/bathroom/point_cloud.ply and pretrained_weights/bathroom/tone_mapper.pth.

With the bathroom dataset downloaded above, test and render the model using:

# For synthetic scenes
python3 train_synthetic.py --config config/bathroom.yaml --eval --gpu_id 0 --syn --load_path pretrained_weights/bathroom  --test_only

Besides, if you train a model with config: bathroom.yaml, you will get a profile as:

  |--output
    |--mlp
      |--bathroom
        |--exp-time
          |--point_cloud
            |interation_x
              |--point_cloud.ply
              |--tone_mapper.pth
            ...
          |--test_set_vis
          |--videos
          |--cameras.json
          |--cfg_args
          |--input.ply
          |--log.txt

Then the --load_path should be "output/mlp/bathroom/exp-time/point_cloud/interation_x"

4. Training

We provide training logs for your convenience to debug. Please download them from the Google Drive or Baidu Disk (code: cyh2).

You can run the .sh file by

# For synthetic scenes
bash train_synthetic.sh

# for real scenes
bash train_real.sh

Or you can directly train on specific scenes as

# For synthetic scenes
python3 train_synthetic.py --config config/sponza.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/sofa.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/bear.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/chair.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/desk.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/diningroom.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/dog.yaml --eval --gpu_id 0 --syn

python3 train_synthetic.py --config config/bathroom.yaml --eval --gpu_id 0 --syn

# for real scenes
python3 train_real.py --config config/flower.yaml --eval --gpu_id 0

python3 train_real.py --config config/computer.yaml --eval --gpu_id 0

python3 train_real.py --config config/box.yaml --eval --gpu_id 0

python3 train_real.py --config config/luckycat.yaml --eval --gpu_id 0

 

4. Citation

@inproceedings{hdr_gs,
  title={HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting},
  author={Yuanhao Cai and Zihao Xiao and Yixun Liang and Minghan Qin and Yulun Zhang and Xiaokang Yang and Yaoyao Liu and Alan Yuille},
  booktitle={NeurIPS},
  year={2024}
}