gtsfm

Platform	Build Status
Ubuntu 20.04.3

Georgia Tech Structure-from-Motion (GTSfM) is an end-to-end SfM pipeline based on GTSAM. GTSfM was designed from the ground-up to natively support parallel computation using Dask.

For more details, please refer to our arXiv preprint.

License

The majority of our code is governed by an MIT license and is suitable for commercial use. However, certain implementations featured in our repo (e.g., SuperPoint, SuperGlue) are governed by a non-commercial license and may not be used commercially.

Installation

GTSfM requires no compilation, as Python wheels are provided for GTSAM.

Install and open GTSFM Studio

GTSFM now installs an application-style command. From a recursive checkout (or a published wheel), one install is enough to open the browser workspace:

uv tool install .
gtsfm run

You can use pip install . instead of uv tool install . inside a Python 3.12 environment. The browser opens automatically and provides dataset, model, splat implementation, hardware, and advanced configuration controls. Input can be supplied by dragging in your own folder or selecting a curated GTSFM GitHub sample; sample scenes automatically apply the matching loader and recommended settings and are cached after their first download. The workspace also manages runs, shows logs, visualizes iterative Gaussian optimization, and opens final reconstructions in the built-in 3D viewer. Completed splats can be downloaded as .ply files from either Activity or Results.

The equivalent terminal workflow remains available:

gtsfm execute -- --dataset_dir /path/to/data --config_name vggt --loader olsson

Useful application commands:

gtsfm hardware          # list detected CPU/GPU devices
gtsfm viz               # open existing results
gtsfm run --no-browser  # start the workspace without opening a browser

Results open at the first reconstructed camera, with its photo selected in the collapsible Photos sidebar. Click a thumbnail (or use the camera arrows) to move to that camera, and use the enlarge button to inspect a larger photo. Overview frames the reconstruction and its textured camera frustums. The ground plane starts hidden and can be enabled with Show plane. The viewer treats reconstruction Z as up; the ground plane lies in the source XY plane, and Plane Z adjusts its height. Camera views retain their recorded roll so that the reconstructed points stay aligned with each photo.

Photos are resolved from the nearest saved config.yaml loader paths, so the source dataset must remain available. Older CLI runs with relative dataset paths should be opened from their original working directory. Portable COLMAP exports can instead keep an images folder alongside the model or in its parent folder. Missing photos leave the reconstruction and camera navigation usable.

The photo strip renders only visible rows and their immediate neighbors. The viewer uses 256-pixel thumbnails for frustum textures, loads at most four textures at once, displays at most 32 visible photo planes, and caches at most 48 textures. All wireframes share one draw call; full-size originals are never sent to the viewer. Enlarged previews are limited to 1600 pixels, and the server caches up to 32 MiB of resized JPEGs. Textures are released when changing scenes.

VGGT is the browser's default reconstruction model. Splat generation is an independent choice:

• Optimized Gaussian splats uses the iterative gsplat implementation and publishes live training previews.
• AnySplat uses the feed-forward AnySplat pipeline.
• No splats runs reconstruction only.

Additional Gaussian optimizer presets placed in gtsfm/configs/gaussian_splatting/ automatically appear in the UI. Iterative gsplat and AnySplat currently require Linux with an NVIDIA CUDA GPU; the UI detects this capability and prevents unsupported runs.

The default VGGT checkpoint is downloaded from Hugging Face on first use when it is not already present. An explicitly configured checkpoint path is never replaced automatically.

To submit work to a remote GPU machine, start its workspace with an API key:

GTSFM_API_KEY=your-secret gtsfm run --host 0.0.0.0 --no-browser

Choose Remote VM in the workspace, then select an API or SSH connection. Modal is the first hosted API option: enter its token ID and token secret separately, or paste the full modal token set --token-id … --token-secret … command into either field and the workspace will split it automatically.2`

GTSFM sample datasets are downloaded directly by the remote workspace. User-selected folders are archived and streamed to the protected workspace before the job starts. Live preview splats and the final .ply result are mirrored back to the local viewer. Modal account credentials and the derived workspace API key are kept in memory and are not included in public job state. Lambda Cloud, RunPod, Vast.ai, and AWS EC2 are listed as coming soon.

Use HTTPS or a trusted private network when exposing a remote workspace; the built-in server is intended for local or controlled-network use.

GTSFM Studio is a strict TypeScript React application built with Vite and lightweight Radix UI primitives. Its FastAPI/Uvicorn backend publishes typed OpenAPI documentation at /docs and streams job, log, and live-splat state over WebSockets instead of polling the server. Release wheels contain the compiled frontend and local 3D viewer vendor assets, so Node.js and external CDNs are not required to run the installed application. To work on the interface itself, start the Python API and Vite development server in separate terminals:

gtsfm run --no-browser

cd visualization/frontend
pnpm install
pnpm dev       # opens the development UI on http://127.0.0.1:5174
pnpm check     # strict TypeScript check
pnpm build     # refresh visualization/static/studio.{js,css}

Initialize Git submodules

This repository includes external repositories as Git submodules, so, unless you cloned with git clone --recursive you need to initialize:

git submodule update --init --recursive

GTSfM supports two installation methods. Choose the one that best fits your workflow:

Option 1: Conda Setup (Recommended for most users)

For detailed Conda installation instructions, see conda-setup.md

Option 2: UV Setup (Fast alternative package manager)

For detailed UV installation instructions, see uv-setup.md

Both methods will allow you to run GTSfM successfully.

Try It on Google Colab

For a quick hands-on example, check out this Colab notebook

Usage Guide (Running 3D Reconstruction)

Before running reconstruction, if you intend to use modules with pre-trained weights (e.g., SuperPoint, SuperGlue, or PatchmatchNet), first download the model weights by running:

bash scripts/download_model_weights.sh

Running SfM

GTSfM provides a unified runner that supports all dataset types through Hydra configuration.

To process a dataset containing only an image directory and EXIF metadata, ensure your dataset follows this structure:

└── {DATASET_NAME}
       ├── images
               ├── image1.jpg
               ├── image2.jpg
               ├── image3.jpg

Then, run the following command:

./run --config_name {CONFIG_NAME} --loader olsson --dataset_dir {DATASET_DIR} --num_workers {NUM_WORKERS}

Loader Options

The runner exposes five portable CLI arguments for dataset selection and universal loader configuration:

• --loader — which loader to use (e.g., olsson, colmap)
• --dataset_dir — path to the dataset root
• --images_dir — optional path to the image directory (defaults depend on loader)
• --max_resolution — maximum length of the image’s short side (overrides config)
• --input_worker — optional Dask worker address to pin image I/O (advanced; runner sets this post‑instantiation)

All other loader‑specific settings (anything beyond the five above) must be specified using Hydra overrides on the nested config node loader.*. This is standard Hydra behavior: use dot‑notation keys with = assignments.

To discover all available overrides for a given loader, open its YAML in gtsfm/configs/loader/

Required Image Metadata

Currently, we require EXIF data embedded into your images. Alternatively, you can provide:

• Ground truth intrinsics in the expected format for an Olsson dataset
• COLMAP-exported text data

Additional CLI Arguments

• --run_mvs — enables dense Multi-View Stereo (MVS) reconstruction after the sparse SfM pipeline.
• --run_gs — enables Gaussian Splatting for dense scene representation.

Many other dask-related arguments are available. Run

./run --help

for more information.

Examples

Example (deep front-end on Olsson, single worker):

./run --dataset_dir tests/data/set1_lund_door \
      --config_name deep_front_end.yaml \
      --loader olsson \
      --num_workers 1 \
      loader.max_resolution=1200

For a dataset with metadata formatted in the COLMAP style:

./run --dataset_dir datasets/gerrard-hall \
      --config_name deep_front_end.yaml \
      --loader colmap \
      --num_workers 5 \
      loader.use_gt_intrinsics=true \
      loader.use_gt_extrinsics=true

You can monitor the distributed computation using the Dask dashboard.
Note: The dashboard will only display activity while tasks are actively running, but comprehensive performance reports can be found in the dask_reports folder.

Comparing GTSFM Output with COLMAP Output

To compare GTSFM output with COLMAP, use the following command:

./run --config_name {CONFIG_NAME} --loader colmap --dataset_dir {DATASET_DIR} --num_workers {NUM_WORKERS} --max_frame_lookahead {MAX_FRAME_LOOKAHEAD}

Visualizing Results with Open3D

To visualize the reconstructed scene using Open3D, run:

python gtsfm/visualization/view_scene.py

Speeding Up Front-End Processing

For users who work with the same dataset repeatedly, GTSFM allows caching front-end results for faster inference.
Refer to the detailed guide:
📄 GTSFM Front-End Cacher README

Running GTSFM on a Multi-Machine Cluster

For users who want to run GTSFM on a cluster of multiple machines, follow the setup instructions here:
📄 CLUSTER.md

Where Are the Results Stored?

• The output will be saved in --output_root, which defaults to the results folder in the repo root.
• Poses and 3D tracks are stored in COLMAP format inside the ba_output subdirectory of --output_root.
• You can visualize these using the COLMAP GUI.

Nerfstudio

We provide a preprocessing script to convert the camera poses estimated by GTSfM to nerfstudio format:

python scripts/prepare_nerfstudio.py --results_path {RESULTS_DIR} --images_dir {IMAGES_DIR}

The results are stored in the nerfstudio_input subdirectory inside {RESULTS_DIR}, which can be used directly with nerfstudio if installed:

ns-train nerfacto --data {RESULTS_DIR}/nerfstudio_input

More Loader Details

The runner supports all loaders through --loader, --dataset_dir, and --images_dir. Any additional, loader‑specific settings are passed as Hydra overrides on the nested node loader.* (this is standard Hydra usage).

General pattern

./run \
  --config_name <config_file> \
  --loader <loader_type> \
  --dataset_dir <path> \
  [--images_dir <path>] \
  [--max_resolution <int>] \
  [--input_worker <address>] \
  loader.<param>=<value> \
  [loader.<param2>=<value2> ...]

Available Loaders

The following loader types are supported:

• colmap - COLMAP format datasets
• hilti - Hilti SLAM challenge datasets
• astrovision - AstroVision space datasets
• olsson - Olsson format datasets
• argoverse - Argoverse autonomous driving datasets
• mobilebrick - MobileBrick datasets
• one_d_sfm - 1DSFM format datasets
• tanks_and_temples - Tanks and Temples benchmark datasets
• yfcc_imb - YFCC Image Matching Benchmark datasets

For the complete list of available arguments for each loader, run:

./run --help

Example: Olsson Loader (images + EXIF)

./run \
  --config_name sift_front_end.yaml \
  --loader olsson \
  --dataset_dir /path/to/olsson_dataset \
  loader.max_resolution=1200

Example: Colmap Loader (COLMAP text export)

./run \
  --config_name sift_front_end.yaml \
  --loader colmap \
  --dataset_dir /path/to/colmap_dataset \
  loader.use_gt_intrinsics=true \
  loader.use_gt_extrinsics=true

Tip: consult gtsfm/configs/loader/<loader_name>.yaml for the full set of fields supported by each loader.

Repository Structure

GTSfM is designed in a modular way. Each module can be swapped out with a new one, as long as it implements the API of the module's abstract base class. The code is organized as follows:

• gtsfm: source code, organized as:
  • averaging
    • rotation: rotation averaging implementations (Shonan, Chordal, etc)
    • translation: translation averaging implementations (1d-SFM, etc)
  • bundle: bundle adjustment implementations
  • common: basic classes used through GTSFM, such as Keypoints, Image, SfmTrack2d, etc
  • data_association: 3d point triangulation (DLT) w/ or w/o RANSAC, from 2d point-tracks
  • densify
  • frontend: SfM front-end code, including:
    • detector: keypoint detector implementations (DoG, etc)
    • descriptor: feature descriptor implementations (SIFT, SuperPoint etc)
    • matcher: descriptor matching implementations (Superglue, etc)
    • verifier: 2d-correspondence verifier implementations (Degensac, OA-Net, etc)
    • cacher: Cache implementations for different stages of the front-end.
  • loader: image data loaders
  • utils: utility functions such as serialization routines and pose comparisons, etc
• tests: unit tests on every function and module

Contributing

Contributions are always welcome! Please be aware of our contribution guidelines for this project.

Citing this work

If you use GTSfM, please cite our paper:

@misc{Baid23_distributedDeepSfm,
      title={Distributed Global Structure-from-Motion with a Deep Front-End}, 
      author={Ayush Baid and John Lambert and Travis Driver and Akshay Krishnan and Hayk Stepanyan and Frank Dellaert},
      year={2023},
      eprint={2311.18801},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Citing the open-source Python implementation:

@misc{GTSFM,
    author = {Ayush Baid and Travis Driver and Fan Jiang and Akshay Krishnan and John Lambert
       and Ren Liu and Aditya Singh and Neha Upadhyay and Aishwarya Venkataramanan
       and Sushmita Warrier and Jon Womack and Jing Wu and Xiaolong Wu and Frank Dellaert},
    title = { {GTSFM}: Georgia Tech Structure from Motion},
    howpublished={\url{https://github.com/borglab/gtsfm}},
    year = {2021}
}

Note: authors are listed in alphabetical order (by last name).