zed-ros2-wrapper

This package enables the use of ZED cameras with ROS 2, providing access to a variety of data types, including:

• Color and grayscale images, both rectified and unrectified
• Depth data
• Colored 3D point clouds
• Position and mapping, with optional GNSS data fusion
• Sensor data
• Detected objects
• Human skeleton data
• And more...

More information

Installation

Prerequisites

• Ubuntu 20.04 (Focal Fossa), Ubuntu 22.04 (Jammy Jellyfish), Ubuntu 24.04 (Noble Numbat), or Ubuntu 26.04 (Resolute Raccoon)
• ZED SDK v5.2 (to support older versions please check the releases)
• CUDA dependency
• A supported ROS 2 distribution (see Supported ROS 2 distributions below):
  • Humble Hawksbill on Ubuntu 22.04: LTS [EOL May 2027]
  • Jazzy Jalisco on Ubuntu 24.04: LTS [EOL May 2029]
  • Lyrical on Ubuntu 26.04: LTS
  • Foxy Fitzroy on Ubuntu 20.04: backward compatibility only [EOL reached]
  • Rolling Ridley, for developers

Supported ROS 2 distributions

• :white_check_mark: All the current LTS distributions are officially supported: ROS 2 Humble Hawksbill (Ubuntu 22.04), Jazzy Jalisco (Ubuntu 24.04), and Lyrical (Ubuntu 26.04).
• :warning: Foxy Fitzroy (Ubuntu 20.04) is still supported for backward compatibility, with limitations when newer packages are not available for it (Foxy reached its End Of Life).
• :hammer_and_wrench: Rolling Ridley is supported for developers, but we cannot always guarantee that breaking changes introduced in Rolling are supported immediately.
• :grey_question: Non-LTS distributions (e.g. Iron, Kilted) are not officially tested, but they could work.

Build the package

The zed_ros2_wrapper is a colcon package.

:pushpin: Note: If you haven’t set up your colcon workspace yet, please follow this short tutorial.

To install the zed_ros2_wrapper, open a bash terminal, clone the package from GitHub, and build it:

mkdir -p ~/ros2_ws/src/ # create your workspace if it does not exist
cd ~/ros2_ws/src/ #use your current ros2 workspace folder
git clone https://github.com/stereolabs/zed-ros2-wrapper.git
cd ..
sudo apt update
rosdep update
rosdep install --from-paths src --ignore-src -r -y # install dependencies
colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release --parallel-workers $(nproc) # build the workspace
echo source $(pwd)/install/local_setup.bash >> ~/.bashrc # automatically source the installation in every new bash (optional)
source ~/.bashrc

:pushpin: Note: the dependency zed_msgs is no longer installed as a submodule of this package; it is available through apt as a binary package with ROS 2 Humble. When working with ROS 2 Foxy, or other distributions, you can install it from the sources from the zed-ros2-interfaces repository.

:pushpin: Note: If rosdep is missing, you can install it with:

sudo apt-get install python3-rosdep python3-rosinstall-generator python3-vcstool python3-rosinstall build-essential

:pushpin: Note: The zed_debug package is intended for internal development only. If you don’t need it, you can skip building this package by adding --packages-skip zed_debug to your colcon command.

:pushpin: Note: When using the ZED ROS 2 Wrapper on an NVIDIA Jetson with JP6, you may get the following error when building the package for the first time

CMake Error at /usr/share/cmake-3.22/Modules/FindCUDA.cmake:859 (message):
  Specify CUDA_TOOLKIT_ROOT_DIR
Call Stack (most recent call first):
 /usr/local/zed/zed-config.cmake:72 (find_package)
 CMakeLists.txt:81 (find_package)

You can fix the problem by installing the missing nvidia-jetpack packages:

sudo apt install nvidia-jetpack nvidia-jetpack-dev

:pushpin: Note: The option --symlink-install is very important, it allows the use of symlinks instead of copying files to the ROS 2 folders during the installation, where possible. Each package in ROS 2 must be installed, and all the files used by the nodes must be copied into the installation folders. Using symlinks allows you to modify them in your workspace, reflecting the modification during the next executions without issuing a new colcon build command. This is true only for all the files that don't need to be compiled (Python scripts, configurations, etc.).

:pushpin: Note: If you are using a different console interface like zsh, you have to change the source command as follows: echo source $(pwd)/install/local_setup.zsh >> ~/.zshrc and source ~/.zshrc.

Starting the ZED node

:pushpin: Note: we recommend reading this ROS 2 tuning guide to improve the ROS 2 experience with ZED.

To start the ZED node, open a bash terminal and use the CLI command ros2 launch:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model>

Replace <camera_model> with the model of the camera that you are using: 'zed', 'zedm', 'zed2', 'zed2i', 'zedx', 'zedxm', 'zedxnano', 'zedxhdrmini', 'zedxhdr', 'zedxhdrmax', 'virtual','zedxonegs','zedxone4k','zedxonehdr'.

The zed_camera.launch.py is a Python launch script that automatically starts the ZED node using "manual composition". The parameters for the indicated camera model are loaded from the relative "YAML files." A Robot State Publisher node is started to publish the camera static links and joints loaded from the URDF model associated with the camera model.

:pushpin: Note: You can set your configurations by modifying the parameters in the files common_stereo.yaml, zed.yaml zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, zedxm.yaml, common_mono.yaml, zedxonegs.yaml, zedxone4k.yaml, zedxonehdr.yaml available in the folder zed_wrapper/config.

You can get the list of all the available launch parameters by using the -s launch option:

ros2 launch zed_wrapper zed_camera.launch.py -s
ros2 launch zed_display_rviz2 display_zed_cam.launch.py -s

For full descriptions of each parameter, follow the complete guide here.

RViz visualization

To start a pre-configured RViz environment and visualize the data of all ZED cameras, we provide in the zed-ros2-examples repository. You'll see more advanced examples and visualizations that demonstrate depth, point clouds, odometry, object detection, etc.

You can also quickly check that your depth data is correctly retrieved in RViz with rviz2 -d ./zed_wrapper/config/rviz2/<your camera model>.rviz. RViz subscribes to numerous ROS topics, which can potentially impact the performance of your application compared to when it runs without RViz.

Simulation mode

In simulation mode the node does not open a camera: it connects to a simulated ZED that streams images and IMU data to the ZED SDK exactly like a real camera would. The rest of the pipeline (depth, positional tracking, object detection, mapping, ...) runs unchanged.

:pushpin: Note: This feature is incompatible with the older first-generation ZED cameras.

Launch a standalone ZED ROS 2 node with simulated ZED data as input by using the following command:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx sim_mode:=true use_sim_time:=true

The ZED X One monocular cameras are supported as well:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedxonegs sim_mode:=true use_sim_time:=true

Two simulated ZED X One cameras paired as a virtual stereo camera are supported too. Unlike a real setup, no serial number or camera ID is required because the simulator streams the already paired stereo images:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=virtual sim_mode:=true use_sim_time:=true

Launch options:

• [Mandatory] sim_mode: start the ZED node in simulation mode if true.
• [Mandatory] camera_model: the model of the simulated camera. It must match the model of the simulated device, otherwise camera features can be silently disabled. Simulated stereo models: zedx, zedxm, zedxnano, zedm, zed2i, virtual. Simulated monocular models: zedxonegs, zedxone4k, zedxonehdr.
• [Recommended] use_sim_time: force the node to wait for valid messages on the topic /clock, and so use the simulation clock as the time reference. Required to keep the ROS 2 graph synchronized with the simulation timeline.
• [Optional] sim_address: the address of the simulation server. The default is 127.0.0.1, valid when the node runs on the same machine as the simulator.
• [Optional] sim_port: the port of the simulation server. Default: 30000. It must match the Streaming Port field of the ZED Camera Helper (or ZED Camera One Helper) Action Graph node.

Resolution and framerate

The streamed resolution and framerate are decided by the simulator (the Resolution and FPS fields of the ZED Camera Helper Action Graph node), not by general.grab_resolution and general.grab_frame_rate: the values configured in the simulator are read back from the stream and reported in the node log.

general.grab_frame_rate still bounds the maximum accepted value of general.pub_frame_rate and general.grab_compute_capping_fps, so set it to the simulated FPS (the extension default is 30) if you need to limit the publishing rate. Leaving general.pub_frame_rate at 0 publishes at the simulated framerate, whatever it is.

Multi-camera simulations

Each simulated camera needs its own even Streaming Port in the simulator (30000, 30002, 30004, ...). Start one node per camera with a distinct camera_name and the matching sim_port:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx camera_name:=zed_front \
  sim_mode:=true use_sim_time:=true sim_port:=30000
ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx camera_name:=zed_rear \
  sim_mode:=true use_sim_time:=true sim_port:=30002

What is not available in simulation

Because the image source is the simulator and not a real sensor:

• The camera controls (exposure, gain, white balance, etc.) are not applied.
• The CMOS temperature is not published.
• The barometer and the magnetometer are not simulated, so the imu/mag and atm_press topics carry no data. The IMU is streamed and published normally.

Visualization

You can also start a preconfigured instance of rviz2 to visualize all the information available in the simulation by using the command:

ros2 launch zed_display_rviz2 display_zed_cam.launch.py camera_model:=zedx \
  sim_mode:=true use_sim_time:=true

The display_zed_cam.launch.py launch file includes the zed_camera.launch.py launch file and forwards sim_mode, sim_address, sim_port and use_sim_time to it.

Here's an example of rviz2 running with the simulated information obtained by placing the ZED camera on a shelf of a simulated warehouse:

Troubleshooting

• The node retries the connection while the simulation is not streaming yet, then gives up with a message reporting the configured simulation.sim_address / simulation.sim_port. Check that the simulation is playing and that the port matches the Action Graph node.
• With use_sim_time:=true the node blocks on Waiting for a valid simulation time on the '/clock' topic... until the simulation publishes /clock. Add an Isaac Read Simulation Time node feeding a ROS2 Publish Clock node to the Action Graph.
• Set debug.debug_sim: true in the configuration file to enable the simulation debug messages.

Supported simulation environments:

• NVIDIA Isaac Sim

More features

Point Cloud Transport

The ZED ROS 2 Wrapper supports Point Cloud Transport to publish point clouds using different compression methods.

This feature is available only if the package point_cloud_transport is installed in your ROS 2 environment; otherwise, it will be disabled automatically.

To install the packages, use the command:

sudo apt install ros-$ROS_DISTRO-point-cloud-transport ros-$ROS_DISTRO-point-cloud-transport-plugins

:pushpin: Note: We removed the point_cloud_transport package as a required dependency of the ZED ROS 2 Wrapper to avoid forcing its installation in all the environments where the ZED ROS2 Wrapper is used. If you want to use this feature, you must install the package manually.

SVO recording

SVO recording can be started and stopped while the ZED node is running using the service start_svo_recording and the service stop_svo_recording. More information

Object Detection

:pushpin: Note: This feature is incompatible with the ZED X One and the older first-generation ZED cameras.

Object Detection can be enabled automatically when the node starts by setting the parameter object_detection/od_enabled to true in the file common_stereo.yaml. The Object Detection can be enabled/disabled manually by calling the services enable_obj_det.

You can find a detailed explanation of the Object Detection module in the ZED ROS 2 documentation.

Custom Object Detection with YOLO-like ONNX model file

:pushpin: Note: This feature is incompatible with the ZED X One and the older first-generation ZED cameras.

Object Detection inference can be performed using a custom inference engine in YOLO-like ONNX format.

You can generate your ONNX model by using Ultralytics YOLO tools.

Install Ultralytics YOLO tools:

python -m pip install ultralytics

If you have already installed the ultralytics package, we recommend updating it to the latest version:

pip install -U ultralytics

Export an ONNX file from a YOLO model (more info here), for example:

yolo export model=yolo11n.pt format=onnx simplify=True dynamic=False imgsz=640

For a custom-trained YOLO model, the weight file can be changed, for example:

yolo export model=yolov8l_custom_model.pt format=onnx simplify=True dynamic=False imgsz=512

Please refer to the Ultralytics documentation for details.

Modify the common_stereo.yaml parameters to match your configuration:

• Set object_detection.model to CUSTOM_YOLOLIKE_BOX_OBJECTS

Modify the custom_object_detection.yaml parameters to match your configuration.

:pushpin: Note: The first time the custom model is used, the ZED SDK optimizes it to get the best performance from the GPU installed on the host. Please wait for the optimization to complete. When using Docker, we recommend using a shared volume to store the optimized file on the host and perform the optimization only once.

Console log while optimization is running:

[zed_wrapper-3] [INFO] [1729184874.634985183] [zed.zed_node]: === Starting Object Detection ===
[zed_wrapper-3] [2024-10-17 17:07:55 UTC][ZED][INFO] Please wait while the AI model is being optimized for your graphics card
[zed_wrapper-3]  This operation will be run only once and may take a few minutes 

You can find a detailed explanation of the Custom Object Detection module in the ZED ROS 2 documentation.

Body Tracking

:pushpin: Note: This feature is incompatible with the ZED X One and the older first-generation ZED cameras.

The Body Tracking can be enabled automatically when the node starts by setting the parameter body_tracking/bt_enabled to true in the file common_stereo.yaml.

Spatial Mapping

:pushpin: Note: This feature is incompatible with the ZED X One camera.

The Spatial Mapping can be enabled automatically when the node starts setting the parameter mapping/mapping_enabled to true in the file common_stereo.yaml. The Spatial Mapping can be enabled/disabled manually by calling the service enable_mapping.

GNSS fusion

:pushpin: Note: This feature is incompatible with the ZED X One camera.

The ZED ROS 2 Wrapper can subscribe to a NavSatFix topic and fuse GNSS data information with Positional Tracking information to obtain a precise robot localization referred to Earth coordinates. To enable GNSS fusion, set the parameter gnss_fusion.gnss_fusion_enabled to true. You must set the correct gnss_frame parameter when launching the node, e.g., gnss_frame:='gnss_link'. The services toLL and fromLL can be used to convert Latitude/Longitude coordinates to robot map coordinates.

2D mode

:pushpin: Note: This feature is incompatible with the ZED X One camera.

For robots moving on a planar surface, activating the "2D mode" (parameter pos_tracking/two_d_mode in common_stereo.yaml) is possible. The value of the coordinate Z for odometry and pose will have a fixed value (parameter pos_tracking/fixed_z_value in common_stereo.yaml). Roll, Pitch, and the relative velocities will be fixed to zero.

NVIDIA® Isaac ROS integration

The ZED ROS 2 Wrapper is compatible with the NVIDIA® Isaac ROS framework, which provides a set of tools and libraries for building robotics applications on NVIDIA® platforms.

CUDA zero-copy transport (Isaac ROS 5, ROS 2 Lyrical)

Starting with v5.6.0, on ROS 2 Lyrical and newer, the ZED ROS 2 Wrapper supports the rosidl::Buffer message fields: the images, depth maps, confidence maps, and point clouds stay in GPU memory, and are shared without copies with the subscribers on the same host and GPU that accept the cuda buffer backend. All the other subscribers automatically receive standard CPU data, with no change on their side.

The transport is enabled by default with the general.buffer_backend parameter (cuda or cpu), and requires:

• the cuda_buffer package when the wrapper is built, and the cuda_buffer_backend plugin when the node runs
• the rmw_fastrtps_cpp middleware (default), or rmw_zenoh_cpp v0.10.5 or newer

Read the CUDA zero-copy transport guide to learn how it works, how to set it up, and how to receive the ZED data in GPU memory in your own nodes.

NITROS (Isaac ROS 3, ROS 2 Humble)

With Isaac ROS 3, the ZED ROS 2 Wrapper leverages NITROS (NVIDIA® Isaac Transport for ROS), a technology to enable data streaming through NVIDIA-accelerated ROS graphs.

Read the full Isaac ROS integration guide and learn how to set up your development environment to use the ZED ROS 2 Wrapper with Isaac ROS.

Examples and Tutorials

Examples and tutorials are provided to better understand how to use the ZED wrapper and how to integrate it into the ROS 2 framework. See the zed-ros2-examples repository

RVIZ2 visualization examples

• Example launch files to start a preconfigured instance of RViz displaying all the ZED Wrapper node information: zed_display_rviz2
• ROS 2 plugin for ZED2 to visualize the results of the Object Detection and Body Tracking modules (bounding boxes and skeletons): rviz-plugin-zed-od

Tutorials

A series of tutorials is provided to better understand how to use the ZED nodes in the ROS 2 environment :

• Video subscribing: zed_video_tutorial - In this tutorial, you will learn how to write a simple node that subscribes to messages of type sensor_msgs/Image to retrieve the Left and Right rectified images published by the ZED node.
• Depth subscribing: zed_depth_tutorial - In this tutorial, you will learn how to write a simple node that subscribes to messages of type sensor_msgs/Image to retrieve the depth images published by the ZED node and to get the measured distance at the center of the image.
• Pose/Odometry subscribing: zed_pose_tutorial - In this tutorial, you will learn how to write a simple node that subscribes to messages of type geometry_msgs/PoseStamped and nav_msgs/Odometry to retrieve the position and the odometry of the camera while moving in the world.
• ROS 2 Composition + BGRA2BGR conversion: zed_rgb_convert - In this tutorial, you will learn how to use the concept of "ROS 2 Composition" and "Intra Process Communication" to write a ROS 2 component that gets a 4 channel BGRA image as input and re-publishes it as 3 channels BGR image.
• ROS 2 Multi-Camera: zed_multi_camera - In this tutorial, you will learn how to use the provided launch file to start a multi-camera robot configuration.
• ROS 2 Multi-Camera + Intra Process Communication: `zed_ipc - In this tutorial, you will learn how to use the provided launch file to start a multi-camera configuration, and load a new processing node in the same process to leverage Intra Process Communication with ROS 2composition.
• Robot integration: zed_robot_integration - In this tutorial, you will learn how to add one or more ZED cameras to a robot configuration.

Examples

How to use the ZED ROS 2 nodes alongside other ROS 2 packages or advanced features.

• zed_aruco_localization: Use localized ArUco tag as a reference for localization.
• zed_depth_to_laserscan: Convert ZED Depth maps into virtual Laser Scans using

Update the local repository

To update the repository to the latest release, use the following command that will retrieve the latest commits of zed-ros2-wrapper and of all the submodules:

git checkout master # if you are not on the main branch  
git pull 

Clean the cache of your colcon workspace before compiling with the colcon build command to be sure that everything will work as expected:

cd <ros2_workspace_root> # replace with your workspace folder, for example ~/ros2_ws/src/
rm -r install
rm -r build
rm -r log
colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release --parallel-workers $(nproc)

Known issues

• Nothing to report yet.

If you find a bug or want to request a new feature, please open an issue on the GitHub issues page.