必须用 --recursive 克隆,否则子模块缺失;训练前需自行下载 NeRF Synthetic、MipNeRF360 或 ScanNet++ 等数据集;Playground 需额外安装 OpenGL 头文件(如 libgl1-mesa-dev);ppisp 后处理需单独安装 ppisp 包。

3dgrut
nv-tlabsApache-2.0活跃维护用光线追踪和混合光栅化渲染高斯粒子的 3D 重建工具,能提升高斯泼溅场景的渲染质量与真实感。
Ray tracing and hybrid rasterization of Gaussian particles
git clone --recursive https://github.com/nv-tlabs/3dgrut.git
cd 3dgrut
./install.sh3DGRT 需专用光线追踪硬件,推荐带 RT 核心的 NVIDIA GPU;README 未给出具体显存要求。支持 Linux 与 Windows。
NVIDIA 官方 3DGRT/3DGUT 实现,用光线追踪与混合光栅化渲染高斯粒子,支持畸变相机与二次光线
用光线追踪和混合光栅化渲染高斯粒子的 3D 重建工具,能提升高斯泼溅场景的渲染质量与真实感。
核心能力
- 用光线追踪渲染体积高斯粒子,支持反射、折射与阴影等二次光线
- 3DGUT 在光栅化框架内支持滚动快门等畸变相机与时变效应
- 3DGRUT 混合渲染:主光线走光栅化、二次光线走光追
- 支持 MCMC 稠密化策略与 Selective Adam 优化器
- 支持导出 USD、PLY、NuRec 格式并带交互式 Playground GUI
容易踩的坑
- v1.1.0v1.1.0 - Stable Code Release (June 10, 2026)最新正式版2026-06-10 · 2026-06-10 23:49 UTC
This version introduces a major new export stack, expands dataset support, improves rendering capabilities, and adds broad compatibility fixes for new hardware and installation workflows.
Notable features:
- Added USD, USDZ, PLY, NuRec, and lightfield export infrastructure with mesh injection and roundtrip tests
- Added NCore-v4 dataset support with accelerated simplejpeg decoding and GPU-cached camera rays
- Added playground path tracing, PBR material meshes, Viser UI, and headless mode
- Improved training with MCMC densification/pruning, Sparse Adam, and multi-camera support
- Expanded platform compatibility for Blackwell, CUDA 12/13, DGX Spark/ARM64, Windows, and uv-based installs
- Added code formatting (Black, isort, clang-format) and expanded CI workflows
What's Changed
- Update README.md by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/49
- revise JIT setup, update for new torch & cuda12 by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/46
- Compatibility pytorch 2.7.0 and blackwell by @johnnynunez in https://github.com/nv-tlabs/3dgrut/pull/36
- fix alpha & depth map, enable hit count by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/50
- Decouple polyscope raygen+cameras & move functionality to kaolin by @orperel in https://github.com/nv-tlabs/3dgrut/pull/43
- Headless Mode by @orperel in https://github.com/nv-tlabs/3dgrut/pull/44
- Switch to loading images on the fly by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/52
- Update README.md by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/53
- MCMC densificastion and prunning strategy by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/55
- Add support for image masks by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/56
- Add support for Sparse Adam optimizer. by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/58
- Bug fix for 3DGRT visibility mask. Add & refine benchmark scripts. by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/60
- #45 : Clarify rotation matrix from quaternion by @moennen in https://github.com/nv-tlabs/3dgrut/pull/67
- #45 : Fix rotation matrix namings by @moennen in https://github.com/nv-tlabs/3dgrut/pull/69
- Enable training with all images by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/65
- Fix for colmap dataset.test_split_interval=0 by @orperel in https://github.com/nv-tlabs/3dgrut/pull/75
- fix typo on README.md. mcmck->mcmc by @kikirizki in https://github.com/nv-tlabs/3dgrut/pull/78
- Playground 2.0 - Updated Path Tracer with Cook Torrance, PBR materials, Env Maps by @orperel in https://github.com/nv-tlabs/3dgrut/pull/89
- QOL operations to MixtureOfGaussians - index buffers, clone, join by @orperel in https://github.com/nv-tlabs/3dgrut/pull/90
- Fix memory leak for 3DGS strat and can handle multiple image sizes by @half-potato in https://github.com/nv-tlabs/3dgrut/pull/93
- Add USDZ export by @nvibd in https://github.com/nv-tlabs/3dgrut/pull/97
- Viser gui for playground (community PR) by @orperel in https://github.com/nv-tlabs/3dgrut/pull/98
- [BUG FIX] Multi-threading support for torch dataset loaders during training + Adding installation script [WINDOWS] by @Arcanous98 in https://github.com/nv-tlabs/3dgrut/pull/110
- Multi-Cam Support (Cleaned Up) by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/112
- Move to LFS for Large Files, Add a Rebase Helper Script by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/114
- Remember to build bvh in case model is resumed from a checkpoint file by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/115
- fixing macros issue on Windows by @Arcanous98 in https://github.com/nv-tlabs/3dgrut/pull/118
- Fix for colored background by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/122
- Fix Camera ID by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/123
- Update build scripts and playground build instructions by @wilsonCernWq in https://github
- v1.0.0Stable Code Release2025-04-03 · 2025-04-03 11:52 UTC
查看更新说明
v1.0.0 - Stable Code Release (April 3, 2025)
This version contains the initial code release on github + 2 weeks of followup fixes and additions from the community
Notable features:
- Initial code release of 3DGRT and 3DGUT
- Slang implementation for differentiable tracer
- Interactive playground app
- Installation scripts, docker and basic CI workflow
- Quality fixes
What's Changed
- update instructions for gcc11 by @nmwsharp in https://github.com/nv-tlabs/3dgrut/pull/1
- use conda env vars instead of activate.d for gcc11 by @nmwsharp in https://github.com/nv-tlabs/3dgrut/pull/3
- Update README.md by @CharlesCNorton in https://github.com/nv-tlabs/3dgrut/pull/10
- Dockerfile for 3dgrut release by @orperel in https://github.com/nv-tlabs/3dgrut/pull/4
- Fix for camera up direction affecting playground render trajectory by @orperel in https://github.com/nv-tlabs/3dgrut/pull/11
- Add 3dgut backward pass early return when all thread in warp are done by @moennen in https://github.com/nv-tlabs/3dgrut/pull/12
- Update animation with mesh assets by @orperel in https://github.com/nv-tlabs/3dgrut/pull/14
- fix: incorrect function call for PLY files in playground.py by @ZERO-black in https://github.com/nv-tlabs/3dgrut/pull/19
- Clarify installation instructions with WITH_GCC11 by @jatentaki in https://github.com/nv-tlabs/3dgrut/pull/20
- Revert projection backward optimization by @moennen in https://github.com/nv-tlabs/3dgrut/pull/23
- Dynamic Materials + fix for UVs tensor by @orperel in https://github.com/nv-tlabs/3dgrut/pull/21
- Download mesh script + handle non-triangular meshes by @orperel in https://github.com/nv-tlabs/3dgrut/pull/15
- pin slang version to 1.3.4 by @zgojcic in https://github.com/nv-tlabs/3dgrut/pull/22
- Alias Slang structs to avoid member names changes across versions by @moennen in https://github.com/nv-tlabs/3dgrut/pull/27
- Fix projection sph backward (missing initialization of the active sph degree) by @moennen in https://github.com/nv-tlabs/3dgrut/pull/29
- optimize trainer + colmap initialization fix by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/31
- 3DGRT : Get particles AABBs from BVH by @moennen in https://github.com/nv-tlabs/3dgrut/pull/32
- Add benchmark numbers, fix typo by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/34
- Fix fisheye distortion calculation by @wilsonCernWq in https://github.com/nv-tlabs/3dgrut/pull/39
- Polyscope cam controls by @orperel in https://github.com/nv-tlabs/3dgrut/pull/16
- Add CI pipeline by @m-naumann in https://github.com/nv-tlabs/3dgrut/pull/9
New Contributors
- @nmwsharp made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/1
- @CharlesCNorton made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/10
- @orperel made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/4
- @moennen made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/12
- @ZERO-black made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/19
- @jatentaki made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/20
- @zgojcic made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/22
- @wilsonCernWq made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/31
- @m-naumann made their first contribution in https://github.com/nv-tlabs/3dgrut/pull/9
Full Changelog: https://github.com/nv-tlabs/3dgrut/commits/v1.0.0
用户点评
登录用户的真实使用记录,可按版本和硬件筛选还没有人点评。第一个说出真实体验的人最有价值。
This repository provides the official implementations of 3D Gaussian Ray Tracing (3DGRT) and 3D Gaussian Unscented Transform (3DGUT). Unlike traditional methods that rely on splatting, 3DGRT performs ray tracing of volumetric Gaussian particles instead. This enables support for distorted cameras with complex, time-dependent effects such as rolling shutters, while also efficiently simulating secondary rays required for rendering phenomena like reflection, refraction, and shadows. However, 3DGRT requires dedicated ray-tracing hardware and remains slower than 3DGS.
To mitigate this limitation, we also propose 3DGUT, which enables support for distorted cameras with complex, time-dependent effects within a rasterization framework, maintaining the efficiency of rasterization methods. By aligning the rendering formulations of 3DGRT and 3DGUT, we introduce a hybrid approach called 3DGRUT. This technique allows for rendering primary rays via rasterization and secondary rays via ray tracing, combining the strengths of both methods for improved performance and flexibility.
For projects that require a fast, modular, and production-ready Gaussian Splatting framework, we recommend using gsplat, which also provides support for 3DGUT.
3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes Nicolas Moenne-Loccoz*, Ashkan Mirzaei*, Or Perel, Riccardo De Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp^, Zan Gojcic^ (*,^ indicates equal contribution) SIGGRAPH Asia 2024 (Journal Track) Project page / Paper / Video / BibTeX
3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting Qi Wu*, Janick Martinez Esturo*, Ashkan Mirzaei, Nicolas Moenne-Loccoz, Zan Gojcic (* indicates equal contribution) CVPR 2025 (Oral) Project page / Paper / Video / BibTeX
Neural Harmonic Textures for High-Quality Primitive Based Neural Reconstruction Jorge Condor, Nicolas Moenne-Loccoz, Merlin Nimier-David, Piotr Didyk, Zan Gojcic, Qi Wu arXiv 2026 Project page / Paper / Video / BibTeX
🔥 News
- ✅[2026/06] 3DGRUT v2.0.0: Neural Harmonic Textures support.
- ✅[2026/03] NCore v4: Support for training from NCore v4 datasets (NCore, commands).
- ✅[2026/01] Physically-Plausible ISP support.
- ✅[2025/08] Support for the 3DGRT and 3DGS/3DGRT pipelines is now available with the Vulkan API as part of the Vulkan Gaussian Splatting Project. 3DGUT will also be available soon.
- ✅[2025/07] Support for datasets with multiple sensors (only for COLMAP-style datasets).
- ✅[2025/07] Support for Windows has been added.
- ✅[2025/06] Playground supports PBR meshes and environment maps.
- ✅[2025/04] Support for image masks.
- ✅[2025/04] SparseAdam support.
- ✅[2025/04] MCMC densification strategy support.
- ✅[2025/04] Stable release v1.0.0 tagged.
- ✅[2025/03] Initial code release!
- ✅[2025/02] 3DGUT was accepted to CVPR 2025!
- ✅[2024/08] 3DGRT was accepted to SIGGRAPH Asia 2024!
Contents
- 🔥 News
- Contents
- 🔧 1 Dependencies and Installation
- 💻 2. Train 3DGRT or 3DGUT scenes
- 🎥 3. Rendering from Checkpoints
- 📋 4. Evaluations
- 🛝 5. Interactive Playground GUI
- 📦 6. Asset Preparation and Re-export
- 📄 7. Contributing
- 🎓 8. Citations
- 🙏 9. Acknowledgements
🔧 1 Dependencies and Installation
- Supported CUDA versions: 11.8, 12.4, 12.6, 12.8 (default), 13.0 (experimental)
- For good performance with 3DGRT, we recommend using an NVIDIA GPU with Ray Tracing (RT) cores.
- Both Linux and Windows are supported via UV install scripts.
Using UV
(Kindly contributed by @MasahiroOgawa)
UV provides faster installation and better dependency resolution.
git clone --recursive https://github.com/nv-tlabs/3dgrut.git
cd 3dgrut
The install scripts automatically find or install a GCC version compatible with your chosen CUDA toolkit.
Prerequisites:
- uv installed:
curl -LsSf https://astral.sh/uv/install.sh | sh - A CUDA toolkit — choose one of the sub-options below.
- OpenGL headers for playground:
sudo apt-get install libgl1-mesa-dev
Sub-option A1 — System CUDA (use an existing nvcc in PATH or CUDA_HOME):
./install_env_uv.sh # venv name defaults to "3dgrut"
source .venv/bin/activate
Sub-option A2 — conda-managed CUDA (let conda install the CUDA toolkit):
# Step 1: create a conda environment with the CUDA toolkit
CUDA_VERSION=12 ./scripts/create_conda.sh 3dgrut
conda activate 3dgrut
# Step 2: install Python dependencies
./install_env_uv.sh # or conda run -n 3dgrut ./install_env_uv.sh
Sub-option A3 — Local venv CUDA (download CUDA into .venv/, no system-wide install needed):
# Supported CUDA_VERSION values: 11.8 (or 11), 12.4, 12.6, 12.8 (or 12), 13.0 (or 13)
FORCE_LOCAL_CUDA=1 CUDA_VERSION=12 ./scripts/create_venv_cuda.sh # ~4 GB download on first run
source .venv/bin/activate
./install_env_uv.sh
[!NOTE] Requires wget:
sudo apt-get install wgetThe CUDA toolkit runfile (~4GB) is cached at/tmp/cuda_<version>_linux.runand reused on subsequent runs. The downloaded CUDA toolkit is installed locally to.venv/cuda-{version}/. You can force a local install even with system CUDA available by settingFORCE_LOCAL_CUDA=1in the environment variables.
Prerequisites:
- uv installed:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" - CUDA Toolkit installed from NVIDIA CUDA Downloads
- Visual Studio Build Tools (2019 or later) with the Desktop development with C++ workload.
The script auto-detects
cl.exe,cmake, andninjafrom the VS installation. If both a CUDA-compatible VS (2019–2022) and a newer one are installed, the script prefers the compatible version. For VS 2025+ (not yet officially supported by CUDA), the script automatically adds--allow-unsupported-compilerto nvcc.
From a PowerShell terminal in the project root:
.\install_env_uv.ps1 # auto-detects CUDA, venv name defaults to "3dgrut"
To override CUDA_HOME:
$env:CUDA_HOME = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8"
.\install_env_uv.ps1
After installation, activate the virtual environment (required for every new terminal session):
.venv\Scripts\Activate.ps1
This also sets the build environment variables (TORCH_CUDA_ARCH_LIST, CUDA_HOME, VS Build Tools paths, etc.) that were persisted during installation.
Blackwell / RTX 50 series Support
We support CUDA 12.8 (Blackwell / RTX 50 series) — kindly contributed by @johnnynunez:
Using the legacy script:
CUDA_VERSION=12.8.1 ./install_env.sh 3dgrut_cuda12 WITH_GCC11
Or using the UV script:
FORCE_LOCAL_CUDA=1 CUDA_VERSION=12 ./scripts/create_venv_cuda.sh 3dgrut_cuda12
source .venv/bin/activate
./install_env_uv.sh
Building and Running with Docker
Build the Docker image:
docker build --build-arg CUDA_VERSION=12.8.1 -t 3dgrut:cuda12 .
docker build --build-arg CUDA_VERSION=11.8.0 --build-arg UBUNTU_VERSION=22.04 -t 3dgrut:cuda11 .
docker buildx build --platform linux/amd64,linux/arm64 --build-arg CUDA_VERSION=13.0.2 -t 3dgrut:cuda13 .
Run it:
xhost +local:root
docker run -v --rm -it --gpus=all --net=host --ipc=host -v $PWD:/workspace --runtime=nvidia -e DISPLAY 3dgrut
[!NOTE] Remember to set the DISPLAY environment variable if you are running on a remote server from the command line.
💻 2. Train 3DGRT or 3DGUT scenes
We provide different configurations for training using 3DGRT and 3DGUT models on common benchmark datasets. For example, you can download the NeRF Synthetic dataset, the MipNeRF360 dataset, or ScanNet++, and then run one of the following commands:
# Train Lego with 3DGRT & 3DGUT
python train.py --config-name apps/nerf_synthetic_3dgrt.yaml path=data/nerf_synthetic/lego out_dir=runs experiment_name=lego_3dgrt
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego out_dir=runs experiment_name=lego_3dgut
# Train Bonsai
python train.py --config-name apps/colmap_3dgrt.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgrt dataset.downsample_factor=2
python train.py --config-name apps/colmap_3dgut.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgut dataset.downsample_factor=2
# Train Scannet++
python train.py --config-name apps/scannetpp_3dgrt.yaml path=data/scannetpp/0a5c013435/dslr out_dir=runs experiment_name=0a5c013435_3dgrt
python train.py --config-name apps/scannetpp_3dgut.yaml path=data/scannetpp/0a5c013435/dslr out_dir=runs experiment_name=0a5c013435_3dgut
Training on NCore v4 datasets
Set path to your NCore v4 sequence JSON. Data layout and tooling are described in the open-source NCore repository. Training defaults are in configs/dataset/ncore.yaml.
python train.py --config-name apps/ncore_3dgut.yaml path=<path>/<sequence-meta>.json out_dir=runs experiment_name=ncore_3dgut
python train.py --config-name apps/ncore_3dgut_mcmc.yaml path=<path>/<sequence-meta>.json out_dir=runs experiment_name=ncore_3dgut_mcmc
python train.py --config-name apps/ncore_3dgrt.yaml path=<path>/<sequence-meta>.json out_dir=runs experiment_name=ncore_3dgrt
python train.py --config-name apps/ncore_3dgrt_mcmc.yaml path=<path>/<sequence-meta>.json out_dir=runs experiment_name=ncore_3dgrt_mcmc
# Example overrides: dataset.downsample=0.5 num_workers=8
We also support the MCMC densification strategy and the selective Adam optimizer for 3DGRT and 3DGUT.
To enable MCMC, use:
python train.py --config-name apps/colmap_3dgrt_mcmc.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgrt dataset.downsample_factor=2
python train.py --config-name apps/colmap_3dgut_mcmc.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgut dataset.downsample_factor=2
Training with Neural Harmonic Textures (NHT)
To enable Neural Harmonic Textures (NHT), use:
python train.py --config-name apps/colmap_3dgrt_mcmc_nht.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgrt_nht dataset.downsample_factor=2
python train.py --config-name apps/colmap_3dgut_mcmc_nht.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgut_nht dataset.downsample_factor=2
To enable selective Adam, use:
python train.py --config-name apps/colmap_3dgrt.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgrt dataset.downsample_factor=2 optimizer.type=selective_adam
python train.py --config-name apps/colmap_3dgut.yaml path=data/mipnerf360/bonsai out_dir=runs experiment_name=bonsai_3dgut dataset.downsample_factor=2 optimizer.type=selective_adam
Post-processing (linear-to-sRGB and PPISP)
Hydra key: post_processing.method. Values:
- null (default): no change to rendered RGB before the loss.
- linear-to-srgb: IEC 61966-2-1 piecewise linear-to-sRGB encoding on
pred_rgb. - ppisp: physically plausible image signal processing; requires the
ppisppackage. PPISP learns exposure, color, vignetting, and camera response corrections. When configured with a controller, it can also predict exposure and color latents from each rendered view.
If you use MCMC and Selective Adam in your research, please cite 3dgs-mcmc, taming-3dgs, and the gSplat library from which the code was adopted (links to the code are provided in the source files).
[!Note] For ScanNet++, we expect the dataset to be preprocessed using the method described in FisheyeGS.
[!Note] If you're running from the PyCharm IDE, enable the rich console as follows: Run Configuration > Modify Options > Emulate terminal in output console*
Using image masks
In order to use image masks, you need to provide a mask for each image in the dataset. The mask is a grayscale image (0s and 1s) that masks out the parts of the image that should not be used during training, i.e. all the pixels with value 0 will be ignored in the loss computation.
The provided masks should have the same resolution as their corresponding images and be stored in the same folder with the same name but with _mask.png extension. For example, to mask out the parts of the image path-to-image/image.jpeg, the mask should be stored at path-to-image/image_mask.png.
NOTE: The masks are only used for loss computation and not for computing the metrics.
Exporting trained scenes (USD, PLY, NuRec)
Trained scenes can be exported to USD (ParticleField), NuRec USDZ for Omniverse,
or PLY, and transcoded between these formats. The simplest path is to enable export at the end of
training:
python train.py --config-name apps/colmap_3dgut.yaml path=data/mipnerf360/garden/ out_dir=runs experiment_name=garden_3dgut dataset.downsample_factor=2 export_usd.enabled=true
[!NOTE] While Isaac Sim 6.0 supports both the
ParticleField(standard USD) schema and the custom NuRec USDZ output, custom NuRec USDZ is going to be deprecated and replaced byParticleField. PreferParticleFieldfor new assets.
For the full set of export workflows — standalone USD export, PLY ⇄ USD ⇄ NuRec transcoding,
PLY→USDZ conversion, adding meshes to USDZ, and PPISP post-processing export — see the export
documentation: threedgrut/export/README.md.
🎥 3. Rendering from Checkpoints
Evaluate a checkpoint with splatting, the OptiX tracer, or PyTorch:
python render.py --checkpoint runs/lego/ckpt_last.pt --out-dir outputs/eval
To visualize training progress interactively
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego with_gui=True
[!NOTE] Remember to set the DISPLAY environment variable if you are running on a remote server from the command line.
Alternatively, use the viser GUI contributed by the community (@tangkangqi):
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego with_viser_gui=True
[!NOTE] Remember to install viser first via
pip install viserand forward the port 8080 to your local machine if you are running on a remote server.
To visualize a pre-trained checkpoint
python train.py --config-name apps/nerf_synthetic_3dgut.yaml path=data/nerf_synthetic/lego with_gui=True test_last=False export_ingp.enabled=False resume=runs/lego/ckpt_last.pt
On startup, you might see a black screen, but you can use the GUI to navigate to the correct camera views:
Similarly, you can use the viser GUI by setting with_viser_gui=True instead of with_gui=True.
📋 4. Evaluations
We provide scripts to reproduce results reported in publications.
# Training
bash ./benchmark/mipnerf360_3dgut.sh <config-yaml>
# Rendering
bash ./benchmark/mipnerf360_3dgut_render.sh <results-folder>
NeRF Synthetic Dataset
bash ./benchmark/nerf_synthetic.sh apps/nerf_synthetic_3dgrt.yaml
bash ./benchmark/nerf_synthetic_render.sh results/nerf_synthetic
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Chair | 35.85 | 0.988 | 556.4 | 299 |
| Drums | 25.87 | 0.953 | 462.8 | 389 |
| Ficus | 36.57 | 0.989 | 331.0 | 465 |
| Hotdog | 37.88 | 0.986 | 597.0 | 270 |
| Lego | 36.70 | 0.985 | 469.8 | 360 |
| Materials | 30.42 | 0.962 | 463.3 | 347 |
| Mic | 35.90 | 0.992 | 443.4 | 291 |
| Ship | 31.73 | 0.909 | 510.7 | 360 |
| Average | 33.87 | 0.971 | 479.3 | 347 |
MipNeRF360 Dataset
bash ./benchmark/mipnerf360.sh apps/colmap_3dgrt.yaml
bash ./benchmark/mipnerf360_render.sh results/mipnerf360
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 24.85 | 0.748 | 2335 | 66 |
| Bonsai | 31.95 | 0.942 | 3383 | 72 |
| Counter | 28.47 | 0.905 | 3247 | 62 |
| Flowers | 21.42 | 0.615 | 2090 | 86 |
| Garden | 26.97 | 0.852 | 2253 | 70 |
| Kitchen | 30.13 | 0.921 | 4837 | 39 |
| Room | 30.35 | 0.911 | 2734 | 73 |
| Stump | 26.37 | 0.770 | 1995 | 73 |
| Treehill | 22.08 | 0.622 | 2413 | 68 |
| Average | 27.22 | 0.817 | 2869 | 68 |
NeRF Synthetic Dataset
bash ./benchmark/nerf_synthetic.sh paper/3dgut/unsorted_nerf_synthetic.yaml
bash ./benchmark/nerf_synthetic_render.sh results/nerf_synthetic
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Chair | 35.61 | 0.988 | 265.6 | 599 |
| Drums | 25.99 | 0.953 | 254.1 | 694 |
| Ficus | 36.43 | 0.988 | 183.5 | 1053 |
| Hotdog | 38.11 | 0.986 | 184.8 | 952 |
| Lego | 36.47 | 0.984 | 221.7 | 826 |
| Materials | 30.39 | 0.960 | 194.3 | 1000 |
| Mic | 36.32 | 0.992 | 204.7 | 775 |
| Ship | 31.72 | 0.908 | 208.5 | 870 |
| Average | 33.88 | 0.970 | 214.6 | 846 |
MipNeRF360 Dataset
GS Strategy, Unsorted
bash ./benchmark/mipnerf360.sh paper/3dgut/unsorted_colmap.yaml
bash ./benchmark/mipnerf360_render.sh results/mipnerf360
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 25.01 | 0.759 | 949.8 | 275 |
| Bonsai | 32.46 | 0.945 | 485.3 | 362 |
| Counter | 29.14 | 0.911 | 484.5 | 380 |
| Flowers | 21.45 | 0.612 | 782.0 | 253 |
| Garden | 27.18 | 0.856 | 810.2 | 316 |
| Kitchen | 31.16 | 0.928 | 664.8 | 275 |
| Room | 31.63 | 0.920 | 448.8 | 370 |
| Stump | 26.50 | 0.773 | 742.6 | 319 |
| Treehill | 22.35 | 0.627 | 809.6 | 299 |
| Average | 27.43 | 0.815 | 686.4 | 317 |
MCMC Strategy, Unsorted
bash ./benchmark/mipnerf360.sh paper/3dgut/unsorted_colmap_mcmc.yaml
bash ./benchmark/mipnerf360_render.sh results/mipnerf360
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 25.31 | 0.765 | 502.3 | 361 |
| Bonsai | 32.51 | 0.947 | 670.6 | 274 |
| Counter | 29.40 | 0.916 | 752.7 | 254 |
| Flowers | 21.86 | 0.616 | 553.3 | 298 |
| Garden | 27.06 | 0.852 | 512.7 | 360 |
| Kitchen | 31.71 | 0.930 | 739.6 | 258 |
| Room | 32.04 | 0.928 | 643.7 | 313 |
| Stump | 27.06 | 0.795 | 487.0 | 339 |
| Treehill | 23.11 | 0.650 | 508.6 | 365 |
| Average | 27.78 | 0.822 | 596.7 | 308 |
GS Strategy, Unsorted, Sparse Adam
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 25.04 | 0.759 | 835.2 | - |
| Bonsai | 32.63 | 0.945 | 457.1 | - |
| Counter | 29.12 | 0.911 | 468.8 | - |
| Flowers | 21.55 | 0.614 | 741.7 | - |
| Garden | 27.12 | 0.855 | 757.4 | - |
| Kitchen | 31.37 | 0.929 | 639.3 | - |
| Room | 31.72 | 0.921 | 415.2 | - |
| Stump | 26.58 | 0.774 | 695.7 | - |
| Treehill | 22.30 | 0.625 | 749.8 | - |
| Average | 27.49 | 0.815 | 640.0 | - |
Scannet++ Dataset
bash ./benchmark/scannetpp.sh paper/3dgut/unsorted_scannetpp.yaml
bash ./benchmark/scannetpp_render.sh results/scannetpp
[!Note] We followed FisheyeGS's convention to prepare the dataset for fair comparisons.
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| 0a5c013435 | 29.67 | 0.930 | 292.3 | 389 |
| 8d563fc2cc | 26.88 | 0.912 | 286.1 | 439 |
| bb87c292ad | 31.58 | 0.941 | 316.9 | 448 |
| d415cc449b | 28.12 | 0.871 | 394.6 | 483 |
| e8ea9b4da8 | 33.47 | 0.954 | 280.8 | 394 |
| fe1733741f | 25.60 | 0.858 | 355.8 | 450 |
| Average | 29.22 | 0.911 | 321.1 | 434 |
MipNeRF360 Dataset
RESULT_DIR=results/mipnerf360_3dgut_nht \
bash ./benchmark/mipnerf360.sh apps/colmap_3dgut_mcmc_nht.yaml
bash ./benchmark/mipnerf360_render.sh results/mipnerf360_3dgut_nht
Current validation uses 1M primitives, 30k iterations, MCMC, and 48 NHT features.
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 25.32 | 0.767 | 3177.5 | 230.9 |
| Bonsai | 33.65 | 0.951 | 5583.5 | 141.4 |
| Counter | 29.99 | 0.919 | 7229.4 | 120.5 |
| Flowers | 21.52 | 0.606 | 3916.8 | 179.3 |
| Garden | 27.52 | 0.859 | 3054.4 | 311.2 |
| Kitchen | 32.44 | 0.934 | 5852.5 | 162.3 |
| Room | 32.61 | 0.931 | 5827.3 | 139.6 |
| Stump | 26.68 | 0.779 | 3155.7 | 213.6 |
| Treehill | 23.02 | 0.653 | 3225.3 | 219.2 |
| Average | 28.08 | 0.822 | 4558.0 | 190.9 |
MipNeRF360 Dataset
RESULT_DIR=results/mipnerf360_3dgrt_nht \
bash ./benchmark/mipnerf360.sh apps/colmap_3dgrt_mcmc_nht.yaml
bash ./benchmark/mipnerf360_render.sh results/mipnerf360_3dgrt_nht
Current validation uses 1M primitives, 30k iterations, MCMC, and 48 NHT features.
| PSNR | SSIM | Train (s) | FPS | |
|---|---|---|---|---|
| Bicycle | 25.21 | 0.762 | 3187.1 | 58.1 |
| Bonsai | 33.11 | 0.949 | 6470.9 | 27.4 |
| Counter | 29.41 | 0.915 | 9095.5 | 18.6 |
| Flowers | 21.42 | 0.604 | 3676.9 | 50.9 |
| Garden | 27.12 | 0.850 | 2864.2 | 67.4 |
| Kitchen | 31.47 | 0.930 | 6025.1 | 26.6 |
| Room | 31.61 | 0.929 | 7548.5 | 30.5 |
| Stump | 25.61 | 0.744 | 2751.5 | 49.1 |
| Treehill | 23.05 | 0.653 | 3405.5 | 55.9 |
| Average | 27.56 | 0.815 | 5002.8 | 42.7 |
🛝 5. Interactive Playground GUI
The playground allows interactive exploration of pretrained scenes, with ray-tracing effects such as inserted objects, reflections, refractions, depth of field, and more.
Run the playground UI to visualize a pretrained scene with:
python playground.py --gs_object <ckpt_path>
See Playground README for details.
Update (2025/04): The playground engine is now exposed, and remote rendering is supported; see README for details.
📦 6. Asset Preparation and Re-export
This repository includes tools for converting, combining, partitioning, and re-exporting Gaussian assets between PLY, ParticleField USD, and NuRec. See the asset transcoding and re-export documentation.
📄 7. Contributing
Contributions are welcome! Please feel free to submit a pull request.
Formatting uses black and isort. Please run
black . --target-version=py311 --line-length=120 --exclude=thirdparty/tiny-cuda-nn and
isort . --skip=thirdparty/tiny-cuda-nn --profile=black before submitting a pull request.
🎓 8. Citations
@article{loccoz20243dgrt,
author = {Nicolas Moenne-Loccoz and Ashkan Mirzaei and Or Perel and Riccardo de Lutio and Janick Martinez Esturo and Gavriel State and Sanja Fidler and Nicholas Sharp and Zan Gojcic},
title = {3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes},
journal = {ACM Transactions on Graphics and SIGGRAPH Asia},
year = {2024},
}
@article{wu20253dgut,
title={3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting},
author={Wu, Qi and Martinez Esturo, Janick and Mirzaei, Ashkan and Moenne-Loccoz, Nicolas and Gojcic, Zan},
journal = {Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2025}
}
@article{condor2026nht,
title={Neural Harmonic Textures for High-Quality Primitive Based Neural Reconstruction},
author={Condor, Jorge and Moenne-Loccoz, Nicolas and Nimier-David, Merlin and Didyk, Piotr and Gojcic, Zan and Wu, Qi},
journal = {arXiv preprint arXiv:2604.01204},
year={2026}
}
🙏 9. Acknowledgements
We sincerely thank our colleagues for their valuable contributions to this project.
Hassan Abu Alhaija, Ronnie Sharif, Beau Perschall and Lars Fabiunke for assistance with assets. Greg Muthler, Magnus Andersson, Maksim Eisenstein, Tanki Zhang, Nathan Morrical, Dietger van Antwerpen and John Burgess for performance feedback. Thomas Müller, Merlin Nimier-David, and Carsten Kolve for inspiration and pointers. Ziyu Chen, Clement Fuji-Tsang, Masha Shugrina, and George Kopanas for technical & experiment assistance, and to Ramana Kiran and Shailesh Mishra for typo fixes.