visual-slam-roadmap

Visual-SLAM Developer Roadmap - 2026

Visual-SLAM is a special case of 'Simultaneous Localization and Mapping' in which a camera is used to gather exteroceptive sensory data.

Below is a set of topics you need to understand for Visual-SLAM, ranging from absolute-beginner level to getting ready to work as a Visual-SLAM engineer/researcher.

---

Visual-SLAM is often portrayed as a rather difficult topic - many think good C++ programming skills and a deep understanding of mathematics are necessary.

On the other hand, there are not many courses provided for beginners, especially in non-English languages.

I made this roadmap to share my thoughts and experience on studying Visual-SLAM, and hopefully beginners can get a grasp of where to start.

---

The purpose of these roadmaps is to give you an idea about the general overview of Visual-SLAM, and to guide you if you are confused about where to start.

Acknowledge that SLAM has a relatively high entry barrier - it's not because it requires understanding difficult mathematics, but because it requires equipping yourself with various types of skills. Don't feel overwhelmed - you don't need to learn everything if you are just getting started. Instead, enjoy the journey itself and progress topic by topic. The result will be very rewarding.

---

Table of Contents

Level	Topic	Focus
1	Beginner	Math, programming, and camera/image fundamentals
2	Getting Familiar	Geometry, optimization, and the anatomy of a SLAM system
3	Monocular SLAM	Classical monocular SLAM — feature-based, direct, semi-direct, SfM, dynamic scenes
4	RGB-D SLAM	Dense tracking and volumetric/surfel fusion with depth sensors
5	Deep Learning + SLAM	Learned frontends, differentiable backends, end-to-end systems, foundation-model & neural SLAM, scene understanding
6	VIO / VINS	Fusing cameras with IMUs — filtering vs optimization
7	Stereo SLAM	Metric scale and depth from stereo pairs
8	Collaborative SLAM	Multi-robot mapping, inter-robot loop closure, map merging
9	LiDAR & Visual-LiDAR	LiDAR odometry and tight camera–LiDAR–IMU fusion
10	Event Camera SLAM	Asynchronous vision for HDR and high-speed motion
11	World Models & Spatial AI	From SLAM maps to learned world representations

---

Level 1: Beginner

Programming

• C++: Pointer, OOP
• Python
• Bash/Linux: Basic terminal usage

Mathematics

• Basic Probability & Statistics: Gaussian distribution, Bayes' theorem
• Basic Linear Algebra: Vectors & Matrices, Determinant, Dot & Cross product, Rank, Inverse matrix, Transpose matrix, SVD, Eigenvalues/Eigenvectors
• Logarithm & Exponential
• Basic Calculus: Differentiation, Taylor expansion

Projective Geometry

• Pinhole camera model → Image projection
• Camera calibration: Intrinsic/Extrinsic parameters, Lens distortion
• Rigid body motion: Euler/Quaternion/Rotation Matrix, Projective space & Vanishing point, Homogeneous transformation
• Epipolar geometry → Essential & Fundamental matrix
• Triangulation
• Camera models beyond pinhole: Fisheye (Kannala-Brandt), double-sphere, omnidirectional; rolling-shutter awareness

Camera Device

• Lens, Sensor, Resolution/ISO/Aperture

Image Data

• Color image, Resolution, Grayscale image
• Thresholding, Gaussian blur
• Corner detector: Harris corner
• Edge detector: Sobel & Canny Edge
• Stereo vision, RGB-D, Disparity, Depth

---

Level 2: Getting Familiar with SLAM

Programming (core)

• C++: OOP, Modern C++, Data structures & Algorithms, Compilers, CMake/Makefile/Ninja, Design patterns, OpenCV C++
• C
• Git/GitHub
• OpenCV (opencv-python)
• Python: Deep learning, Graph plots, System scripts
• Bash/Linux: ssh, CLI text editor/Vim/tmux
• Math libraries: Eigen, Ceres-solver/GTSAM/g2o
• C++/Python interop: PyBind11, nanobind
• ROS/ROS2
• Docker

Programming (optional — engineering skills for SLAM jobs)

• Concurrency: SIMD-SSE/AVX/Neon, OpenMP, CUDA
• Edge deployment: TensorRT/ONNX export of learned frontends, Jetson benchmarking
• Mobile: Android (Java/Kotlin), iOS (Objective-C/Swift)
• C#: Unity AR, Microsoft HoloLens
• CI/CD: GitHub Actions
• Simulation: Gazebo, Isaac Sim

Image Processing

• Keypoints → Detector/Descriptor
  • SIFT, FAST, ORB, AKAZE
  • Deep features: R2D2, SuperPoint
• Image pyramid, oFAST, rBRIEF

Local Feature Matching

• Brute-Force, FLANN, Kd-Tree
• LSH, Multi-probe LSH, HBST
• SuperGlue

Global Feature Matching

• Bag of Visual Words, NetVLAD
• Deep image retrieval, Hierarchical localization

Feature Tracking

• Optical flow, KLT Tracker

Multiple View Geometry

• 2D-2D correspondence: Essential/Fundamental, Homography
• 2D-3D correspondence: P3P, PnP, SVD
• 3D-3D correspondence: ICP

Outlier Rejection

• RANSAC, PROSAC, M-Estimator, MAXCON, Convex relaxation
• Robust pose-graph optimization: Switchable constraints, Dynamic covariance scaling (DCS), Pairwise consistency maximization (PCM)

Certifiably Optimal Algorithms

System	Author/Year	Key Concepts
SE-Sync	Rosen 2019	Certifiable pose graph optimization via SDP + Riemannian opt (arXiv 2016, IJRR 2019)
TEASER++	Yang 2020	Point cloud registration, 90%+ outlier robust, TLS + Max Clique (T-RO/RSS 2020)
GNC	Yang 2020	Graduated Non-Convexity, continuation from convex → robust cost
QUASAR	Yang 2019	Certifiably optimal rotation search (Wahba problem with outliers), quaternion QCQP + SDP relaxation

Least Squares Optimization

• Reprojection error, Bundle adjustment
• Non-linear optimization, Lie algebra
• Lie groups: SO(3), SE(3)
• Gauss-Newton, Levenberg-Marquardt
• Pose graph optimization
• Schur complement / Sparsity

Motion Model

• Proprioceptive sensor: IMU, Wheel
• Odometry (pose)

Observation Model

• Exteroceptive sensor: Camera, LiDAR
• Landmark (Map)
• Joint optimization, MLE & MAP

Factor Graph Optimization

• Factor graph: SLAM as a bipartite graph of variables (poses, landmarks) and factors (measurements)
• MAP inference as sparse nonlinear least squares; variable elimination and the Bayes tree
• Incremental smoothing: iSAM / iSAM2
• Marginalization and fixed-lag smoothing
• Reference: Dellaert & Kaess, Factor Graphs for Robot Perception (2017); GTSAM tutorials

Gaussian Belief Propagation

System	Author/Year	Key Concepts
FutureMapping 1	Davison 2018	Computational structure of Spatial AI, GBP for SLAM
FutureMapping 2	Davison 2019	GBP as core Spatial AI primitive, visual intro to GBP
BA on Graph Processor	Ortiz 2020	Bundle Adjustment on Graphcore IPU, tile-based parallelism
DANCeRS	Patwardhan 2025	GBP-based distributed consensus in robot swarms

Mapping

• Point cloud, Occupancy grid mapping, TSDF, Surfel, Voxel map

Sensors

• Camera device: Wide/telecentric lens, Lens MTF, CCD/CMOS, Rolling/Global shutter, Exposure/ISO, Stereo vision, RGB-D, Structured light, Active IR/ToF
• LiDAR → Visual-LiDAR fusion
• IMU → VIO
• RADAR → Sensor fusion, Extended Kalman filter
• Sonar
• Multi-sensor calibration: Camera-IMU, Camera-LiDAR (Kalibr)

Evaluation

• Metrics: ATE (Absolute Trajectory Error), RPE (Relative Pose Error)
• Consistency: NEES (Normalized Estimation Error Squared)
• Datasets: KITTI, TUM RGB-D, EuRoC, TartanAir, TUM-VI, 4Seasons, Hilti SLAM Challenge, Newer College, Project Aria
• Tools: evo (trajectory evaluation)

---

Level 3: Monocular Visual-SLAM

Key Concepts

• VO vs SLAM — VO is local (no loop closure), SLAM includes global map + loop closure
• Scale ambiguity — Fundamental limitation of monocular SLAM; absolute scale is classically unrecoverable from geometry alone (learned metric-depth priors such as Metric3D or MASt3R can supply approximate scale)
• Covisibility graph — Shared map point visibility between keyframes; core data structure in ORB-SLAM
• Visual Place Recognition (VPR) — Recognizing previously visited places for loop closure
• Self-supervised depth — Learning monocular depth without ground truth (Monodepth2, Godard 2019)

Feature-based SLAM

System	Author/Year	Key Concepts
Visual Odometry	Nistér 2004	Five-point essential matrix solver, RANSAC, Triangulation, VO (local-only, no loop closure)
MonoSLAM	Davison 2007	First real-time monocular SLAM, EKF-based, single camera, sparse 3D map, probabilistic feature initialization
PTAM	Klein & Murray 2007	FAST feature, Tracking, Frontend/Backend separation, Parallel threads, Keyframe, Mapping, Bundle adjustment, Manual initialization
Visual-SLAM why filter?	Strasdat 2012	Bundle adjustment, Scale-aware BA, Motion-only BA
ORB-SLAM	Mur-Artal 2015	ORB keypoint, Automatic initialization (Homography vs Fundamental selection), Tracking thread, Local (covisibility-based) BA + global BA on loop closure, Local mapping, Large-scale, Loop closure, Bag of Visual Words, Global optimization, Covisibility graph, Map point management (culling, merging)
Pop-up SLAM	Yang 2016	Line/Plane features
PL-SLAM	Pumarola 2017	Point/Line features
ORB-SLAM2	Mur-Artal 2017	→ Stereo SLAM, → RGB-D SLAM
CubeSLAM	Yang 2019	Monocular 3D cuboid detection + SLAM, 9-DoF object representation
OpenVSLAM	Sumikura 2019	ORB-based SLAM framework, perspective/fisheye/equirectangular camera models, map save/load + localization mode
Stella-VSLAM	Community 2021	OpenVSLAM successor, license reboot (→ also in Level 7)
UcoSLAM	Muñoz-Salinas 2019	Fiducial markers
DeepFusion	Laidlow 2019	Dense monocular reconstruction, semi-dense MVS + CNN depth/gradient predictions, probabilistic fusion with learned uncertainties
ORB-SLAM3	Campos 2020	Monocular + Stereo + VIO, Multi-map, IMU integration
DXSLAM	Li 2020	Deep features for SLAM
PyCuVSLAM	NVIDIA 2025	Python + CUDA GPU-accelerated VSLAM toolkit (cuVSLAM wrapper; stereo/multi-camera VIO)

Direct SLAM

System	Author/Year	Key Concepts
DTAM	Newcombe 2011	Dense mapping, Keyframe mapping, GPGPU
LSD-SLAM	Engel 2014	Photometric error minimization, High gradient pixels/edges, Large scale, Loop closure, Pose graph optimization
DSO	Engel 2016	Photometric bundle adjustment, Sliding window BA, No loop closure/global optimization
LDSO	Gao 2018	DSO + Loop closure (BoW-based), addresses DSO's main weakness
CNN-SLAM	Tateno 2017	Depth from LSD-SLAM + deep depth, Semantic label
DVSO	Yang 2018	Deep single image depth estimation, StackNet
D3VO	Yang 2020	Deep single image depth estimation, Deep pose, Deep aleatoric uncertainty

Semi-direct (Hybrid)

System	Author/Year	Key Concepts
SVO	Forster 2014	FAST feature detection, sparse direct image alignment, depth filters
SVO2	Forster 2017	Multi-camera/Fisheye, Probabilistic depth estimation, Direct method convergence, Sparse method, Bundle adjustment
Stereo DSO	Wang 2017	→ Stereo SLAM
VI-DSO	von Stumberg 2018	→ VIO/VINS

SfM Tools

System	Author/Year	Key Concepts
COLMAP	Schönberger 2016	De-facto standard incremental SfM + MVS pipeline (C++/CUDA, pycolmap bindings)
GLOMAP	Pan 2024	Global SfM revisited — COLMAP-compatible, much faster mapping
InstantSfM	Zhong 2025	GPU-native sparse-aware SfM pipeline, large speedups over COLMAP

Dynamic Environment SLAM

System	Author/Year	Key Concepts
DynaSLAM	Bescós 2018	Mask R-CNN dynamic-object removal + background inpainting, ORB-SLAM2-based
DS-SLAM	Yu 2018	Semantic segmentation (SegNet) + motion consistency check
MaskFusion	Rünz 2018	RGB-D recognition, tracking and reconstruction of multiple moving objects
MID-Fusion	Xu 2019	Octree-based object-level multi-instance dynamic RGB-D SLAM
VDO-SLAM	Zhang 2020	Dynamic object-aware SLAM, joint camera + object motion estimation
DynaSLAM II	Bescós 2021	Tightly-coupled multi-object tracking and SLAM
MonST3R	Zhang 2024	DUSt3R-family pointmap estimation in the presence of motion

---

Level 4: RGB-D Visual-SLAM

Key Concepts

• Depth from sensor — Structured light vs Active IR (ToF); metric scale for free, but range/material limitations
• Frame-to-model tracking — Aligning each frame against the accumulated model (ICP) instead of frame-to-frame
• TSDF vs Surfel maps — Volumetric signed-distance fusion (KinectFusion) vs point-based surfel fusion (ElasticFusion)

RGB-D Camera Devices

• Intel RealSense D series
• Orbbec Femto series (Azure Kinect successor), Orbbec Astra
• Luxonis OAK-D
• Legacy (discontinued): Microsoft Kinect v1/v2, Azure Kinect DK, Occipital Structure Core

GPGPU Programming

• CUDA, OpenGL GLSL

Systems

System	Author/Year	Key Concepts
ICP	Besl & McKay 1992	Iterative Closest Point, closest-point correspondence, closed-form rigid transform, local convergence (needs initialization), foundation of 3D-3D registration
DTAM	Newcombe 2011	→ see Level 3 Direct SLAM
KinectFusion	Newcombe 2011	GPGPU, Tracking (project depth → 3D, surface normal, coarse-to-fine ICP), Mapping (volumetric integration, TSDF), Robust to small scene changes, Cannot model deformation, Map growth cubic, Room-size only
Double Window Optimisation	Strasdat 2011	Inner window (local BA) + outer window (pose graph), covisibility graph, constant-time optimization
Kintinuous	Whelan 2012	Volume shift, Geometric, Photometric, dBoW+SURF, Optimization, Loop closure
RGBD-SLAM-V2	Endres 2013	Tracking (color image, visual features, depth image, point cloud, transformation), Mapping (OctoMap 2013)
SLAM++	Salas-Moreno 2013	Object-oriented SLAM
DVO	Kerl 2013	Keyframe, Depth, Direct method, Optimization, Loop closure
RTAB-Map	Labbé 2014	Loop closure, Map merge, Multi-session memory management
MRS-Map	Stückler 2014	Multi-resolution surfel maps in an octree, shape + color statistics per surfel, noise-aware RGB-D registration, real-time on CPU
ElasticFusion	Whelan 2015	Active: frame-to-model tracking (photometric + geometric), joint optimization, fused surfel-based model reconstruction · Inactive: local loop closure (model-to-model local surface, submodel separation), global loop closure (randomised fern encoding, non-rigid space deformation)
DynamicFusion	Newcombe 2015	6D motion field, Deformable scene
ORB-SLAM2 (RGB-D mode)	Mur-Artal 2017	Bundle adjustment, Sparse reconstruction (→ also in Level 3)
BundleFusion	Dai 2016	Local-to-global optimization, Sparse RGB feature, Coarse global pose estimation, Fine pose refinement (geometric + photometric)
SemanticFusion	McCormac 2016	Deep Learning CNN, Deep Semantic SLAM
InfiniTAM v3	Prisacariu 2017	Tracking (scene raycast, depth image, RGB image), Relocalization (random ferns), Mapping (TSDF reconstruction, voxel hashing, surfel reconstruction)
Fusion++	McCormac & Clark 2018	Deep Learning CNN, Mask-RCNN instance segmentation, Object-level SLAM, No prior, Object-level TSDF reconstruction
PointFusion / DenseFusion	Xu 2018 / Wang 2019	RGB-D object 6-DoF pose estimation, point cloud + image feature fusion (object frontend for object-level SLAM)
BAD SLAM	Schöps 2019	Direct RGB-D bundle adjustment, surfel map, real-time GPU BA, ETH3D benchmark
RTAB-Map (RGB-D / LiDAR)	Labbé 2019	Multi-sensor RGB-D/LiDAR support, light-source detection (2016)
MoreFusion	Wada 2020	DL instance segmentation, Object-level volumetric fusion, Volumetric pose prediction, 3D scene reconstruction, Collision-based refinement, Semantic SLAM, Object pose estimation, CAD object fitting
NodeSLAM	Sucar 2020	Occupancy VAE, Object-level SLAM (→ also in Level 5 Latent Representation)
DSP-SLAM	Wang (UCL) 2021	DeepSDF shape prior + ORB-SLAM2, object-level dense reconstruction (mono/stereo/LiDAR)

---

Level 5: Applying Deep Learning

Key Concepts

• Learned vs hand-crafted — Replacing individual classical modules (features, depth, matching) with networks vs end-to-end learning
• Differentiability — Making classical optimization (RANSAC, BA) differentiable so it can be trained through
• Foundation models — Large pretrained models (CLIP, SAM, DUSt3R-family) as reusable perception backbones

Level 5 is organized into five pillars: A. Frontend — learned perception components replacing hand-crafted modules B. Backend — learned/certifiable optimization replacing classical solvers C. Systems — end-to-end deep VO/SLAM pipelines D. Scene Understanding — semantic, language, and relational reasoning on SLAM maps E. Foundation-Model & Neural SLAM — pointmap transformers, NeRF- and 3DGS-based dense SLAM systems

A. Deep Frontend — Perception

Feature Detection & Matching

System	Author/Year	Key Concepts
NetVLAD	Arandjelović 2016	VLAD, place recognition
SuperPoint	DeTone 2017	Homographic Adaptation, Self-supervised, VGG encoder + detector/descriptor heads
HardNet	Mishchuk 2017	Learned local descriptor
R2D2	Revaud 2019	Repeatable + Reliable detector/descriptor, explicit repeatability/reliability maps
KeyNet	Barroso-Laguna 2019	Learned keypoint detector
HF-Net	Sarlin 2019	Global feature, Local feature, Visual localization
SuperGlue	Sarlin 2020	Self/Cross-attention GNN, Sinkhorn optimal assignment, dustbin for outliers
DISK	Tyszkiewicz 2020	Policy gradient (RL) training, match success/failure as reward
Patch NetVLAD	Hausler 2021	Multi-scale patch-level VLAD
LoFTR	Sun 2021	Detector-free, Transformer coarse-to-fine dense matching
LightGlue	Lindenberger 2023	Adaptive depth/width, 5-10× faster than SuperGlue
XFeat	Potje 2024	0.3M params, 1400 FPS (RTX 4090), 64-dim descriptor, embedded-friendly
RoMa	Edstedt 2024	DINOv2 foundation feature + coarse-to-fine dense matching
DeDoDe	Edstedt 2024	Joint detect-and-describe in one stage
RoMa v2	Edstedt 2025	Harder-better-faster-denser dense feature matching

Depth Estimation

System	Author/Year	Key Concepts
MonoDepth	Godard 2016	Left-Right photometric consistency, self-supervised
MiDaS	Ranftl 2020	Multi-dataset mixing, scale-and-shift invariant loss, relative depth
DPT	Ranftl 2021	Dense Prediction Transformer (ViT backbone), global context
ZoeDepth	Bhat 2023	Zero-shot metric depth, Metric Bins Module
Metric3D	Yin 2023	Camera intrinsic-conditioned metric depth, Canonical Camera Space
Depth Anything	Yang 2024	62M images, foundation model for monocular depth
Depth Anything V2	Yang 2024	Improved with synthetic data, better edge preservation
Depth Anything 3	Lin 2025	Any-view geometry from arbitrary inputs, depth-ray prediction target, single plain transformer (DINOv2), teacher-student training
Marigold	Ke 2024	Stable Diffusion for depth, fine detail, uncertainty via sampling
Align3R	Lu 2025	Video temporal consistency, DUSt3R-based, CVPR 2025 Highlight
Masked Depth Modeling (LingBot-Depth)	Tan 2026	Fixes RGB-D failures on glass/mirrors/metal

Optical Flow & Scene Flow

System	Author/Year	Key Concepts
FlowNet	Dosovitskiy 2015	First end-to-end deep optical flow (SimpleNet / CorrNet)
FlowNet 2.0	Ilg 2017	Stacked networks, classical-level accuracy
PWC-Net	Sun 2018	Pyramid-Warping-Cost volume, coarse-to-fine, 8.4M params
FlowNet3D	Liu 2019	Point cloud scene flow, PointNet++ based
RAFT	Teed 2020	All-Pairs Correlation + iterative ConvGRU update, ECCV Best Paper
RAFT-3D	Teed 2021	Scene flow (3D motion) from RAFT
FlowFormer	Huang 2022	Transformer on cost volume tokens, global context
SEA-RAFT	Wang 2024	Efficient RAFT variant for real-time

Camera Pose Regression & Relocalization

System	Author/Year	Key Concepts
PoseNet	Kendall 2015	CNN-based 6-DoF pose regression (APR), GoogLeNet backbone
DSAC	Brachmann 2017	Differentiable RANSAC, Scene Coordinate Regression (SCR)
DSAC++	Brachmann 2018	Self-supervision, RGB-D support
CNN Pose Regression Limitations	Sattler 2019	Pose regression ≈ image retrieval performance
LM-Reloc	von Stumberg 2020	Deep direct relocalization
DSAC*	Brachmann 2021	Visual relocalization from RGB/RGB-D, improved learning stability (TPAMI)
ACE	Brachmann 2023	Accelerated Coordinate Encoding, 5-min training per scene
ACE Zero	Brachmann 2024	Zero-shot SCR, no pre-built 3D map needed
ACE-G	Bruns 2025	Generalizable SCR via query pretraining, new scenes without fine-tuning
ACE-SLAM	Alzugaray 2025	Neural implicit real-time SLAM, network weights = map
hloc	Sarlin 2019	Toolbox implementing HF-Net's hierarchical localization: coarse (NetVLAD) → fine (SuperGlue)

Object Detection & Segmentation for SLAM

System	Author/Year	Key Concepts
YOLO (v1→v11)	Redmon 2016→2024	Real-time object detection, Ultralytics ecosystem
DETR	Carion 2020	Transformer detection, anchor-free, no NMS
RT-DETR	Zhao (Baidu) 2023	Real-time DETR, YOLO-speed + Transformer quality
RF-DETR	Robinson 2025	Weight-sharing NAS over DETRs, accuracy-latency Pareto tuning, first real-time detector past 60 AP on COCO
SAM	Kirillov 2023	Segment Anything, prompt-based, Foundation Model
SAM 2	Meta 2024	Video segmentation, Memory Attention, temporal consistency
SAM 3	Carion 2025	Promptable concept segmentation (noun-phrase / exemplar prompts), presence head, detector + memory-based video tracker
Grounding DINO	Liu 2023	Text-prompted detection → SAM pipeline (Grounded SAM)
Open-YOLO 3D	Boudjoghra 2024	2D open-vocab detection → 3D instance seg, 16× faster

B. Deep Backend — Optimization

Differentiable Bundle Adjustment

System	Author/Year	Key Concepts
BA-Net	Tang 2019	FPN + differentiable LM layer, end-to-end SfM (ICLR)
DROID-SLAM	Teed 2021	Dense optical flow + differentiable dense BA, all-pixels reprojection
DPVO	Teed 2023	Patch-based DROID-SLAM, 30+ FPS real-time
Theseus	Pineda (Meta) 2022	Differentiable nonlinear optimization library (PyTorch)
Lietorch	Teed 2021	Lie group operations for PyTorch (SE(3)/SO(3))

C. End-to-End Deep VO / SLAM Systems

Self-supervised & Learned VO

System	Author/Year	Key Concepts
DeepVO	Wang 2017	Supervised learning
SfM-Learner	Zhou 2017	Unsupervised, deep depth + deep pose
DeMoN	Ummenhofer 2017	Depth + Motion from two frames, encoder-decoder
UndeepVO	Li 2018	Stereo self-supervised, absolute scale recovery
DeepTAM	Zhou 2018	Deep tracking and mapping, cost volume based
DeepV2D	Teed 2018	Iterative depth from video, differentiable geometry layers
Depth from Videos in the Wild	Gordon 2019	Unconstrained video depth, learned camera intrinsics
Neural Ray Surfaces	Vasiljevic 2020	Learned ray surface model, non-pinhole cameras
GradSLAM	Murthy 2020	Differentiable SLAM framework (PyTorch, supports multiple SLAM backends)
DeepSLAM	Li 2020	TrackingNet, MappingNet, LoopNet
MonoRec	Wimbauer 2021	Self-supervised monocular 3D reconstruction, moving objects
TANDEM	Koestler 2021	Real-time tracking + dense mapping via MVS depth, DSO-based

Learning-based SLAM Systems

System	Author/Year	Key Concepts
DROID-SLAM	Teed 2021	Differentiable BA, dense optical flow, end-to-end learned
TartanVO	Wang 2021	Generalizable visual odometry
DPV-SLAM	Lipson 2024	DPVO + loop closure, full SLAM (ECCV 2024)
MAC-VO	Qiu 2024	Learning-based VO, metric-aware
VoT	Yugay 2025	Visual Odometry with Transformers (later retitled FVO)

Latent Representation SLAM

System	Author/Year	Key Concepts
CodeSLAM	Bloesch 2018	Depth as 128-dim latent code, photometric BA on codes + poses
SceneCode	Zhi 2019	Depth + semantic in single latent code, cross-modal constraints
DeepFactors	Czarnowski 2020	Probabilistic depth codes + factor graph, GPU 30+ FPS
NodeSLAM	Sucar 2020	Object-level DeepSDF codes, occupancy VAE per object
CodeMapping	Matsuki 2021	Sparse SLAM + learned dense mapping, hybrid approach

Neural Rendering (reference)

NeRF/3DGS-based SLAM systems → see Pillar E below

System	Author/Year	Key Concepts
NeRF	Mildenhall 2020	Neural Radiance Fields, novel view synthesis (foundational)
DIFIX3D+	Wu 2025	Single-step diffusion for 3D reconstruction artifact removal (post-processing)

D. Scene Understanding

Benchmarks & Foundations

System	Author/Year	Key Concepts
EFM3D	Straub (Meta) 2024	Egocentric Foundation Model 3D benchmark, depth/surface/semantic from ego-video

3D Scene Graph

System	Author/Year	Key Concepts
Kimera / 3D Dynamic Scene Graph	Rosinol 2020	Kimera-VIO, Kimera-Mesher, Kimera-PGMO, Kimera-Semantics, Kimera-DSG (stereo/mono visual-inertial pipeline)
Hydra	Hughes (MIT SPARK) 2022	Real-time hierarchical Scene Graph (mesh→objects→places→rooms→buildings)
Hydra-Multi	Chang 2023	Distributed multi-robot 3D Scene Graph
Clio	Maggio (MIT SPARK) 2024	Open-set task-driven Scene Graph, CLIP embeddings per node
Khronos	Schmid (MIT SPARK) 2024	Spatio-temporal Scene Graph, dynamic object history tracking
ConceptGraphs	Gu 2023	Open-vocabulary 3D Scene Graph, SAM + CLIP + LLM relations

---

Semantic / Language-Grounded SLAM

System	Author/Year	Key Concepts
ConceptFusion	Jatavallabhula (MIT) 2023	CLIP features fused into 3D map, open-vocabulary language queries
LERF	Kerr 2023	Language Embedded Radiance Fields, DINO multi-scale, NeRF + CLIP
OpenScene	Peng (ETH) 2023	Language features back-projected to 3D point clouds
SpatialLM	Mao 2025	Point cloud → LLM, structured indoor modeling as Python scripts

Also see: LEGS, OpenGS-SLAM (Pillar E above); Open-YOLO 3D (Level 5 Object Detection)

E. Foundation-Model & Neural-Representation SLAM

Foundation-Model SLAM

System	Author/Year	Key Concepts
DUSt3R	Wang 2024	Pointmap regression from image pairs, no calibration needed
MASt3R	Leroy 2024	DUSt3R + local feature matching
MASt3R-SLAM	Murai 2024	Real-time dense SLAM from MASt3R priors
VGGT	Wang (Meta) 2025	Feed-forward inference of poses, depths, pointmaps, tracks from N views (CVPR 2025 Best Paper)
VGGT-SLAM	Maggio 2025	Dense RGB SLAM optimized on the SL(4) manifold, VGGT frontend
VGGT-SLAM 2.0	Maggio 2026	Real-time dense feed-forward scene reconstruction
VGGT-Geo	Qin 2026	Probabilistic geometric fusion of VGGT priors for dense indoor SLAM
IGGT	Li 2025	Instance-grounded geometry transformer — unified 3D reconstruction + instance-level understanding
AMB3R	Wang 2025	Accurate feed-forward metric-scale 3D reconstruction with backend, SfM/SLAM support
MASt3R-Fusion	Zhou 2025	MASt3R feed-forward visual model + IMU + GNSS fusion

NeRF-based

System	Author/Year	Key Concepts
iMAP	Sucar 2021	First NeRF-SLAM, single MLP, real-time tracking/mapping
BARF	Lin 2021	Bundle-Adjusting NeRF, coarse-to-fine positional encoding, joint pose+NeRF opt (not full SLAM — pose+NeRF co-optimization)
NICE-SLAM	Zhu & Peng 2022	Hierarchical feature grid (coarse/mid/fine), scalable
Co-SLAM	Wang 2023	Hash grid (Instant-NGP) + coordinate encoding, 5-10× faster than NICE-SLAM
ESLAM	Johari 2023	Tri-plane representation, O(N²) vs O(N³) memory
Point-SLAM	Sandström 2023	Neural point cloud based
NeRF-SLAM	Rosinol 2023	NeRF + classical SLAM pipeline
NICER-SLAM	Zhu 2024	RGB-only NeRF-SLAM (no depth sensor), monocular depth integration
vMAP	Kong 2023	Object-level NeRF-SLAM, per-object neural fields
GO-SLAM	Zhang 2023	Global optimization + NeRF-SLAM, loop closure + global BA

3DGS-based

System	Author/Year	Key Concepts
SplaTAM	Keetha 2024	Among the first 3DGS SLAM systems (concurrent with GS-SLAM, MonoGS), RGB-D, silhouette-guided densification
MonoGS	Matsuki 2024	First monocular 3DGS SLAM (CVPR 2024 highlight), direct rasterization-based tracking, analytic camera Jacobians
GS-ICP SLAM	Ha 2024	Gaussian-to-Gaussian ICP (Mahalanobis distance), geometric tracking
Photo-SLAM	Huang 2024	Explicit geometry + implicit appearance (MLP color), anti-aliasing
RTG-SLAM	Peng 2024	Real-time focus, adaptive Gaussian budget, Jetson Orin 25 FPS
EGG-Fusion	Pan 2025	Geometry-aware Gaussian surfel fusion on the fly, information-filter-based, real-time
Online 3DGS Modeling	Lee 2025	Online 3D Gaussian Splatting modeling with novel view selection
ActiveSplat	Li 2025	Active mapping with 3DGS + Voronoi-based path planning
OpenGS-SLAM	Yang 2025	Open-set dense semantic 3DGS SLAM, object-level scene understanding
LEGS	Yu 2024	Language Embedded Gaussian Splats, real-time language-queryable 3D

---

Level 6: VIO / VINS

Key Concepts

• Tightly-coupled vs Loosely-coupled — Joint vs separate optimization of visual and inertial measurements
• Filter-based vs Optimization-based — EKF approaches vs nonlinear optimization (BA)
• IMU preintegration — Integrating IMU measurements between keyframes (Lupton 2012; on-manifold formulation: Forster 2015)
• IMU noise model — Bias, random walk, Allan variance
• Observability — 4 unobservable DoF in VIO (3-DoF global translation + yaw); scale becomes additionally unobservable under constant-acceleration motion
• Deployed VIO — Commercial XR stacks (Meta Quest, ARKit/ARCore) are the highest-volume deployed VIO systems — worth studying as case studies

Foundations

Resource	Author/Year	Key Concepts
Introduction to Inertial Navigation	Woodman 2007	IMU fundamentals, coordinate frames, error sources — essential prerequisite
IMU Preintegration on Manifold	Forster 2015	On-manifold preintegration, bias correction without re-integration
Quaternion kinematics for error-state KF	Solà 2017	Quaternion math, error-state formulation

Filter-based

System	Author/Year	Key Concepts
MSCKF	Mourikis 2007	Multi-State Constraint KF, efficient VIO without landmarks in state
ROVIO	Bloesch 2015	Robocentric VIO, direct photometric tracking + EKF
OpenVINS	Geneva 2020	Open-source MSCKF, modular, extensible

Optimization-based

System	Author/Year	Key Concepts
OKVIS	Leutenegger 2015	Keyframe-based, tightly-coupled, sliding window optimization
VINS-Mono	Qin 2018	Tightly-coupled, relocalization, loop closure, pose graph optimization
VI-DSO	von Stumberg 2018	Direct sparse VIO, dynamic marginalization, photometric error
VINS-Fusion	Qin 2019	Stereo + GPS fusion extension
maplab	Schneider 2018	Multi-session visual-inertial mapping framework
Kimera-VIO	Rosinol 2020	Fast VIO frontend for Kimera pipeline, structureless vision factors
Basalt	Usenko 2020	Non-linear factor recovery (NFR) of marginalization priors, visual-inertial odometry + mapping
ORB-SLAM3	Campos 2020	VIO mode, multi-map, IMU initialization
DM-VIO	von Stumberg 2022	Direct (DSO-based) monocular VIO, delayed marginalization, pose-graph BA for IMU initialization
OKVIS2	Leutenegger 2022	Multi-session, improved marginalization
AirVO	Xu 2023	Point-line VIO, illumination-robust
OKVIS2-X	Boche & Leutenegger 2025	Multi-sensor SLAM (Visual+Inertial+Depth+LiDAR+GNSS), dense volumetric occupancy maps, submapping for large-scale (9km+), EuRoC/Hilti22 SOTA

---

Level 7: Stereo SLAM

Key Concepts

• Stereo rectification — Epipolar alignment for efficient disparity search
• Disparity vs Depth — d = f·B/Z, baseline determines depth range/accuracy
• Scale observability — Stereo provides metric scale (unlike monocular)

Systems

System	Author/Year	Key Concepts
S-PTAM	Pire 2017	Stereo PTAM, ROS-compatible, real-time
ORB-SLAM2 (stereo)	Mur-Artal 2017	Stereo + RGB-D modes, loop closure, relocalization
StereoMSCKF	Sun 2018	MSCKF with stereo, efficient for resource-constrained platforms
RTAB-Map	Labbé 2019	Multi-sensor (stereo/RGB-D/LiDAR), memory management, large-scale (→ also in Level 4)
ORB-SLAM3 (stereo)	Campos 2020	Multi-map, Atlas, stereo + IMU
Stella-VSLAM	Community 2021	OpenVSLAM successor, stereo support (→ also in Level 3)
Stereo DSO	Wang 2017	Direct sparse stereo odometry, large-scale (DSO extension)

---

Level 8: Collaborative / Multi-Robot SLAM

Key Concepts

• Centralized vs Decentralized — Single server vs peer-to-peer map merging
• Inter-robot loop closure — Place recognition across robots with different viewpoints
• Communication constraints — Bandwidth-limited map sharing, sparse descriptors
• Map merging — Aligning submaps from different robots into a global map

Systems

System	Author/Year	Key Concepts
C2TAM	Riazuelo 2014	Cloud-based collaborative monocular SLAM
CCM-SLAM	[Sc