tensorRT_Pro-Vision

Melody-ZhouMIT活跃维护

基于 tensorRT_Pro 扩展的 TensorRT C++ 推理库，集成 YOLO、RT-DETR、PPOCR、车道线检测、深度估计等多类视觉模型，提供高性能部署方案。

This repository is forked from shouxieai/tensorRT_Pro, extending it to support a wide range of vision models with high-performance TensorRT C++ inference.

v1.0.0发布于 2026-06-25 06:46 UTC416star74fork同步于 2026-11-10 12:00 UTC

TensorRT C++推理视觉模型部署目标检测车道线检测

对比 GitHub bytetrack clrernet clrnet depth-anything laneatt

怎么上手

由 README 提炼，执行前请确认

原文

git clone https://github.com/Melody-Zhou/tensorRT_Pro-Vision.git
cd tensorRT_Pro-Vision
# 修改 Makefile 中的 TensorRT / CUDA / cuDNN / OpenCV 路径
make -j$(nproc)
cd workspace && ./pro <task_name>

硬件要求按 README 说明

README 未说明显存要求；需 NVIDIA GPU 并依赖 CUDA、cuDNN、TensorRT、OpenCV，编译时按显卡改 cuda_arch（如 A30 用 compute_80/sm_80），平台为 Linux。

需要 CUDA

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

一套 TensorRT C++ 代码覆盖 20+ 视觉任务，做端到端 GPU 推理部署

基于 tensorRT_Pro 扩展的 TensorRT C++ 推理库，集成 YOLO、RT-DETR、PPOCR、车道线检测、深度估计等多类视觉模型，提供高性能部署方案。

核心能力

用任务名切换 20+ 模型，共用单个可执行文件 pro
resize、normalize、decode、NMS 全用 CUDA kernel 在 GPU 上跑
条件编译兼容 TensorRT 8.x 与 10.x，不依赖 protobuf
支持动态 Batch 与 FP32/FP16/INT8，内置 Entropy 与 MinMax 校准器
新增任务只需实现 preprocess、worker 和后处理 kernel

容易踩的坑

编译前必须手动改 Makefile 或 CMakeLists.txt 里的 TensorRT/CUDA/cuDNN/OpenCV 路径和 cuda_arch；ONNX 模型要自行导出或从百度网盘下载后放进 workspace/；部分任务需先用 workspace/*_build.sh 生成 engine，其余首次运行才自动构建。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

一套 TensorRT C++ 代码，覆盖 检测、分类、实例分割、语义分割、姿态、OBB、深度、OCR、车道线与多目标跟踪；预处理、后处理、NMS 全部 GPU 加速。

从 shouxieai/tensorRT_Pro fork 并重构而来，面向多任务、低延迟的 C++ 部署场景。

Quick Start

git clone https://github.com/Melody-Zhou/tensorRT_Pro-Vision.git
cd tensorRT_Pro-Vision
# 修改 Makefile 中的 TensorRT / CUDA / cuDNN / OpenCV 路径
make -j$(nproc)
# 按任务文档导出 ONNX 并放入 workspace/，然后：
cd workspace && ./pro <task_name>

部分任务会在首次运行时自动从 ONNX 构建 TensorRT engine，其余任务通过 workspace/*_build.sh 生成；完整的依赖版本、Makefile/CMake 配置与各任务模型导出步骤见环境配置和任务文档。

Environment

该项目依赖于 CUDA、cuDNN、TensorRT、OpenCV 库，请在 Makefile 或 CMakeLists.txt 中手动指定路径配置。

依赖	TensorRT 8.x 推荐	TensorRT 10.x 推荐
CUDA	>= 10.2	>= 12.0
cuDNN	>= 8.x	>= 9.x
TensorRT	>= 8.4	>= 10.0
OpenCV	>= 4.x	>= 4.x

修改 Makefile 中的库文件路径与 cuda_arch：

# ===== TensorRT 8.x =====
lean_tensor_rt := /opt/TensorRT-8.6.1.6
lean_cudnn     := /home/zhouwenguang/lean/cudnn-8.5.0.96
lean_cuda      := /usr/local/cuda-11.4
lean_opencv    := /home/zhouwenguang/lean/opencv-4.6.0

# ===== 或者 TensorRT 10.x =====
# lean_tensor_rt := /home/zhouwenguang/lean/TensorRT-10.16.1
# lean_cudnn     := /home/zhouwenguang/lean/cudnn-9.18.0
# lean_cuda      := /usr/local/cuda-12.8
# lean_opencv    := /home/zhouwenguang/lean/opencv-4.6.0

# 根据显卡修改，例如 A30：compute_80 / sm_80
cuda_arch := -gencode=arch=compute_80,code=sm_80

编译：

make -j$(nproc)

修改 CMakeLists.txt 中的库文件路径
编译：

mkdir build && cd build
cmake .. && make -j$(nproc)

What's New

2026/10：新增 YOLO26-Depth 单目米制深度估计，支持动态 batch、align_corners 后处理与 Spectral/metric 可视化。
2026/6/26：TensorRT 10.x 全面支持；移除 vendored ONNX parser 与 protobuf 依赖；修复 getMaxBatchSize() 警告；仓库更名为 tensorRT_Pro-Vision。

完整历史更新记录（2023–2026）请查看 v1.0.0 README。

Model Downloads

ONNX 测试模型包：Baidu Drive（提取码 onnx）

下载后放入 workspace/，执行 ./pro <task_name>；如未找到所需模型，请按任务文档自行导出。

Why this repo

20+ 模型，一个可执行文件：workspace/pro 通过任务名切换模型，无需为每个任务维护独立工程。
全链路 GPU：resize、normalize、decode、NMS、depth resize 均为 CUDA kernel，避免 CPU↔GPU 往返。
TensorRT 8.x / 10.x 自适应：通过 NV_TENSORRT_MAJOR 条件编译兼容两代 API，不依赖 protobuf。
动态 Batch + 三种精度：FP32 / FP16 / INT8，内置 Entropy 与 MinMax 校准器。
模块化扩展：新增任务只需实现 preprocess()、worker() 和后处理 kernel。

Supported Tasks

类别	已支持模型
检测	YOLOv3 · YOLOv5 · YOLOv6 · YOLOv7 · YOLOv8 · YOLOv9 · YOLOv10 · YOLO11 · YOLOv12 · YOLOv13 · YOLO26 · RT-DETR · RF-DETR
分类	YOLOv8-Cls · YOLO11-Cls · YOLO26-Cls
分割	YOLOv8-Seg · YOLO11-Seg · YOLO26-Seg · YOLO26-Sem
姿态	YOLOv8-Pose · YOLO11-Pose · YOLO26-Pose · RTMO
旋转框	YOLOv8-OBB · YOLO11-OBB · YOLO26-OBB
深度	Depth-Anything V1/V2 · YOLO26-Depth
OCR	PP-OCRv4（det / cls / rec）
车道线	LaneATT · CLRNet · CLRerNet
跟踪	ByteTrack
自定义插件	LayerNorm · DCNv2 · HSigmoid · HSwish · ScatterND

任务文档：按模型系列分组，点击系列标题展开；每个模型仍可单独折叠。

YOLO 经典 · YOLOv8 · YOLOv9–v11 · YOLOv12/v13 · YOLO26 · DETR · 深度估计 · OCR · 车道线 · 姿态与跟踪 · 自定义插件

下载 YOLOv3

git clone https://github.com/ultralytics/yolov3.git

修改代码, 保证动态 batch

# ========== export.py ==========

# yolov3/export.py第160行
# output_names = ['output0', 'output1'] if isinstance(model, SegmentationModel) else ['output0']
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#         elif isinstance(model, DetectionModel):
#             dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(model, SegmentationModel) else ['output']
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1,25200,85)

导出 onnx 模型

cd yolov3
python export.py --weights=yolov3.pt --dynamic --simplify --include=onnx --opset=11

复制模型并执行

cp yolov3/yolov3.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision

# 修改代码在 src/application/app_yolo.cpp: app_yolo 函数中, 使用 V3 的方式即可运行
# test(Yolo::Type::V3, TRT::Mode::FP32, "yolov3");

make yolo -j64

下载 YOLOX

git clone https://github.com/Megvii-BaseDetection/YOLOX.git

导出 onnx 模型

cd YOLOX
export PYTHONPATH=$PYTHONPATH:.
python tools/export_onnx.py -c yolox_s.pth -f exps/default/yolox_s.py --output-name=yolox_s.onnx --dynamic --decode_in_inference

复制模型并执行

cp YOLOX/yolox_s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision

# 修改代码在 src/application/app_yolo.cpp: app_yolo 函数中, 使用 X 的方式即可运行
# test(Yolo::Type::X, TRT::Mode::FP32, "yolox_s");

make yolo -j64

下载 YOLOv5

git clone https://github.com/ultralytics/yolov5.git

修改代码, 保证动态 batch

# ========== export.py ==========

# yolov5/export.py第160行
# output_names = ['output0', 'output1'] if isinstance(model, SegmentationModel) else ['output0']
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#         elif isinstance(model, DetectionModel):
#             dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(model, SegmentationModel) else ['output']
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 1: 'anchors'}  # shape(1,25200,85)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1,25200,85)

导出 onnx 模型

cd yolov5
python export.py --weights=yolov5s.pt --dynamic --simplify --include=onnx --opset=11

复制模型并执行

cp yolov5/yolov5s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision

# 修改代码在 src/application/app_yolo.cpp: app_yolo 函数中, 使用 V5 的方式即可运行
# test(Yolo::Type::V5, TRT::Mode::FP32, "yolov5s");

make yolo -j64

下载 YOLOv6

git clone https://github.com/meituan/YOLOv6.git

修改代码, 保证动态 batch，并去除 anchor 维度

# ========== export_onnx.py ==========

# YOLOv6/deploy/ONNX/export_onnx.py第84行
# output_axes = {
#     'outputs': {0: 'batch'},
# }
# 修改为：

output_axes = {
    'output': {0: 'batch'},
}

# YOLOv6/deploy/ONNX/export_onnx.py第106行
# torch.onnx.export(model, img, f, verbose=False, opset_version=13,
#                     training=torch.onnx.TrainingMode.EVAL,
#                     do_constant_folding=True,
#                     input_names=['images'],
#                     output_names=['num_dets', 'det_boxes', 'det_scores', 'det_classes']
#                     if args.end2end else ['outputs'],
#                     dynamic_axes=dynamic_axes)
# 修改为：

torch.onnx.export(model, img, f, verbose=False, opset_version=13,
                    training=torch.onnx.TrainingMode.EVAL,
                    do_constant_folding=True,
                    input_names=['images'],
                    output_names=['num_dets', 'det_boxes', 'det_scores', 'det_classes']
                    if args.end2end else ['output'],
                    dynamic_axes=dynamic_axes)

# 根据不同的 head 去除 anchor 维度
# ========== effidehead_distill_ns.py ==========
# YOLOv6/yolov6/models/heads/effidehead_distill_ns.py第141行
# return torch.cat(
#     [
#         pred_bboxes,
#         torch.ones((b, pred_bboxes.shape[1], 1), device=pred_bboxes.device, dtype=pred_bboxes.dtype),
#         cls_score_list
#     ],
#     axis=-1)
# 修改为：
return torch.cat(
    [
        pred_bboxes,
        cls_score_list
    ],
    axis=-1)

# ========== effidehead_fuseab.py ==========
# YOLOv6/yolov6/models/heads/effidehead_fuseab.py第191行
# return torch.cat(
#     [
#         pred_bboxes,
#         torch.ones((b, pred_bboxes.shape[1], 1), device=pred_bboxes.device, dtype=pred_bboxes.dtype),
#         cls_score_list
#     ],
#     axis=-1)
# 修改为：
return torch.cat(
    [
        pred_bboxes,
        cls_score_list
    ],
    axis=-1)

# ========== effidehead_lite.py ==========
# YOLOv6/yolov6/models/heads/effidehead_lite.py第123行
# return torch.cat(
#     [
#         pred_bboxes,
#         torch.ones((b, pred_bboxes.shape[1], 1), device=pred_bboxes.device, dtype=pred_bboxes.dtype),
#         cls_score_list
#     ],
#     axis=-1)
# 修改为：
return torch.cat(
    [
        pred_bboxes,
        cls_score_list
    ],
    axis=-1)

导出 onnx 模型

cd YOLOv6
python deploy/ONNX/export_onnx.py --weights yolov6s.pt --img 640 --dynamic-batch --simplify

复制模型并执行

cp YOLOv6/yolov6s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision

# 修改代码在 src/application/app_yolo.cpp: app_yolo 函数中, 使用 V6 的方式即可运行
# test(Yolo::Type::V6, TRT::Mode::FP32, "yolov6s");

make yolo -j64

下载 YOLOv7

git clone https://github.com/WongKinYiu/yolov7.git

导出 onnx 模型

python export.py --dynamic-batch --grid --simplify --weights=yolov7.pt

复制模型并执行

cp yolov7/yolov7.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision

# 修改代码在 src/application/app_yolo.cpp: app_yolo 函数中, 使用 V7 的方式即可运行
# test(Yolo::Type::V7, TRT::Mode::FP32, "yolov7");

make yolo -j64

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第72行，forward函数
# return y if self.export else (y, x)
# 修改为：

return y.permute(0, 2, 1) if self.export else (y, x)

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov8s.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov8s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo -j64

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    dynamic['output'] = {0: 'batch'}
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov8s-cls.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov8s-cls.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_cls -j64

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第106行，forward函数
# return (torch.cat([x, mc], 1), p) if self.export else (torch.cat([x[0], mc], 1), (x[1], mc, p))
# 修改为：

return (torch.cat([x, mc], 1).permute(0, 2, 1), p) if self.export else (torch.cat([x[0], mc], 1), (x[1], mc, p))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov8s-seg.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov8s-seg.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_seg -j64

下载 YOLOv8

glit clone https://github.com/ultralytics/ultralytics.git
cd ultralytics
git checkout tags/v8.1.0 -b v8.1.0

修改代码, 保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第141行，forward函数
# return torch.cat([x, angle], 1) if self.export else (torch.cat([x[0], angle], 1), (x[1], angle))
# 修改为：

return torch.cat([x, angle], 1).permute(0, 2, 1) if self.export else (torch.cat([x[0], angle], 1), (x[1], angle))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第353行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov8s-obb.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov8s-obb.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_obb -j64

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第130行，forward函数
# return torch.cat([x, pred_kpt], 1) if self.export else (torch.cat([x[0], pred_kpt], 1), (x[1], kpt))
# 修改为：

return torch.cat([x, pred_kpt], 1).permute(0, 2, 1) if self.export else (torch.cat([x[0], pred_kpt], 1), (x[1], kpt))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    dynamic['output'] = {0: 'batch'}
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov8s-pose.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov8s-pose.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_pose -j64

说明

本项目的 YOLOv9 部署实现并不是官方原版，而是采用的集成到 ultralytics 的 YOLOv9

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第75行，forward函数
# return y if self.export else (y, x)
# 修改为：

return y.permute(0, 2, 1) if self.export else (y, x)

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第365行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型, 在 ultralytics-main 新建导出文件 export.py 内容如下：

# ========== export.py ==========
from ultralytics import YOLO

model = YOLO("yolov9c.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolov9c.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo -j64

前置条件

tensorRT >= 8.5

下载 YOLOv10

git clone https://github.com/THU-MIG/yolov10

修改代码, 保证动态 batch

# ========== exporter.py ==========

# yolov10-main/ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型，在 yolov10-main 新建导出文件 export.py 内容如下

from ultralytics import YOLO

model = YOLO("yolov10s.pt")

success = model.export(format="onnx", dynamic=True, simplify=True, opset=13)

cd yolov10-main
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp yolov10-main/yolov10s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
# 取消 build.sh 中 yolov10 engine 生成的注释
bash build.sh

执行

make yolo -j64

下载 YOLO11

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第68行，forward函数
# return y if self.export else (y, x)
# 修改为：

return y.permute(0, 2, 1) if self.export else (y, x)

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第400行
# output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
# 修改为：

output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch"}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo11s.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolo11s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo -j64

下载 YOLO11

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第400行
# output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
# 修改为：

output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch"}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo11s-cls.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolo11s-cls.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_cls -j64

下载 YOLO11

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第186行，forward函数
# return (torch.cat([x, mc], 1), p) if self.export else (torch.cat([x[0], mc], 1), (x[1], mc, p))
# 修改为：

return (torch.cat([x, mc], 1).permute(0, 2, 1), p) if self.export else (torch.cat([x[0], mc], 1), (x[1], mc, p))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第400行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo11s-seg.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolo11s-seg.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_seg -j64

下载 YOLO11

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第212行，forward函数
# return torch.cat([x, angle], 1) if self.export else (torch.cat([x[0], angle], 1), (x[1], angle))
# 修改为：

return torch.cat([x, angle], 1).permute(0, 2, 1) if self.export else (torch.cat([x[0], angle], 1), (x[1], angle))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第400行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo11s-obb.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolo11s-obb.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_obb -j64

下载 YOLO11

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第239行，forward函数
# return torch.cat([x, pred_kpt], 1) if self.export else (torch.cat([x[0], pred_kpt], 1), (x[1], kpt))
# 修改为：

return torch.cat([x, pred_kpt], 1).permute(0, 2, 1) if self.export else (torch.cat([x[0], pred_kpt], 1), (x[1], kpt))

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第400行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    dynamic['output'] = {0: 'batch'}
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo11s-pose.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

复制模型并执行

cp ultralytics/yolo11s-pose.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_pose -j64

下载 YOLOv12

git clone https://github.com/sunsmarterjie/yolov12

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第74行，forward函数
# return y if self.export else (y, x)
# 修改为：

return y.permute(0, 2, 1) if self.export else (y, x)

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第499行
# output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
# 修改为：

output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch"}  # shape(1, 84, 8400)

导出 onnx 模型，在 yolov12 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO('yolov12s.pt')

model.export(format="onnx", dynamic=True)

cd yolov12
python export.py

复制模型并执行

cp yolov12/yolov12s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo -j64

环境搭建

git clone https://github.com/iMoonLab/yolov13.git
cd yolov13
wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.3/flash_attn-2.7.3+cu11torch2.2cxx11abiFALSE-cp311-cp311-linux_x86_64.whl
conda create -n yolov13 python=3.11
conda activate yolov13
pip install -r requirements.txt
pip install -e .

修改代码，保证动态 batch

# ========== head.py ==========

# ultralytics/nn/modules/head.py第74行，forward函数
# return y if self.export else (y, x)
# 修改为：

return y.permute(0, 2, 1) if self.export else (y, x)

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第499行
# output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
# 修改为：

output_names = ["output0", "output1"] if isinstance(self.model, SegmentationModel) else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch"}  # shape(1, 84, 8400)

导出 onnx 模型，在 yolov13 新建导出文件 export.py 内容如下：

from ultralytics import YOLO
model = YOLO('yolov13s.pt')
model.export(format="onnx", dynamic=True)

cd yolov13
conda activate yolov13
python export.py

复制模型并执行

cp yolov13/yolov13s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo -j64

前置条件

tensorRT >= 8.6

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/ultralytics/engine/exporter.py第687行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
#     if self.args.nms:  # only batch size is dynamic with NMS
#         dynamic["output0"].pop(2)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output"] = {0: "batch"}  # shape(1, 84, 8400)
    if self.args.nms:  # only batch size is dynamic with NMS
        dynamic["output0"].pop(2)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

model = YOLO("yolo26s.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp ultralytics/yolo26s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
# 取消 build.sh 中 yolo26 engine 生成的注释
bash build.sh

执行

make yolo -j64

Note：YOLO26 和 YOLOv10 类似，都是 anchor-free 的模型，无 NMS 后处理，所以部署流程完全可以参考 YOLOv10，甚至用同一套推理代码

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码，保证动态 batch

# ========== exporter.py ==========

# ultralytics/ultralytics/engine/exporter.py第687行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
#     if self.args.nms:  # only batch size is dynamic with NMS
#         dynamic["output0"].pop(2)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    dynamic['output'] = {0: 'batch'}
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
    if self.args.nms:  # only batch size is dynamic with NMS
        dynamic["output0"].pop(2)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下：

from ultralytics import YOLO

model = YOLO("yolo26s-cls.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics
python export.py

复制模型并执行

cp ultralytics/yolo26s-cls.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision
make yolo_cls -j64

前置条件

tensorRT >= 8.6

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/ultralytics/engine/exporter.py第687行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
#     if self.args.nms:  # only batch size is dynamic with NMS
#         dynamic["output0"].pop(2)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
    if self.args.nms:  # only batch size is dynamic with NMS
        dynamic["output0"].pop(2)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

model = YOLO("yolo26s-seg.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp ultralytics/yolo26s-seg.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
# 取消 build.sh 中 yolo26-seg engine 生成的注释
bash build.sh

执行

make yolo_seg -j64

前置条件

tensorRT >= 8.6

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/ultralytics/engine/exporter.py第687行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
#     if self.args.nms:  # only batch size is dynamic with NMS
#         dynamic["output0"].pop(2)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    dynamic["output"] = {0: "batch"}
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
    if self.args.nms:  # only batch size is dynamic with NMS
        dynamic["output0"].pop(2)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

model = YOLO("yolo26s-obb.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp ultralytics/yolo26s-obb.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
# 取消 build.sh 中 yolo26-obb engine 生成的注释
bash build.sh

执行

make yolo_obb -j64

前置条件

tensorRT >= 8.6

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/ultralytics/engine/exporter.py第687行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output0"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
#         dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
#     if self.args.nms:  # only batch size is dynamic with NMS
#         dynamic["output0"].pop(2)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    dynamic["output"] = {0: "batch"}
    if isinstance(self.model, SegmentationModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 116, 8400)
        dynamic["output1"] = {0: "batch", 2: "mask_height", 3: "mask_width"}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic["output0"] = {0: "batch", 2: "anchors"}  # shape(1, 84, 8400)
    if self.args.nms:  # only batch size is dynamic with NMS
        dynamic["output0"].pop(2)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

model = YOLO("yolo26s-pose.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp ultralytics/yolo26s-pose.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
# 取消 build.sh 中 yolo26-pose engine 生成的注释
bash build.sh

执行

make yolo_pose -j64

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/engine/exporter.py 第 638 行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output1"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

# Load a model
model = YOLO("yolo26s-sem.pt")  # load an official model

# Export the model
model.export(format="onnx", imgsz=(512, 1024), dynamic=True, simplify=True)

cd ultralytics
python export.py

复制模型并执行

cp ultralytics/yolo26s-sem.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
make yolo_sem -j64

前置条件

tensorRT >= 8.6

下载 YOLO26

git clone https://github.com/ultralytics/ultralytics.git

确认导出动态 batch

# ========== exporter.py ==========

# ultralytics/engine/exporter.py 第 1152 行
# output_names = ["output0", "output1"] if self.model.task == "segment" else ["output1"]
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {"images": {0: "batch", 2: "height", 3: "width"}}  # shape(1,3,640,640)
#     ...
#     elif isinstance(self.model, DepthModel):
#         dynamic["output0"] = {0: "batch", 2: "height", 3: "width"}  # shape(1,1,640,640) dense map, not anchors
# 修改为：

output_names = ["output0", "output1"] if self.model.task == "segment" else ["output"]
dynamic = self.args.dynamic
if dynamic:
    dynamic = {"images": {0: "batch"}}  # shape(1,3,640,640)
    ...
    elif isinstance(self.model, DepthModel):
        dynamic["output"] = {0: "batch"}  # shape(1,1,640,640) dense map, not anchors

导出 onnx 模型，在 ultralytics 新建导出文件 export.py 内容如下

from ultralytics import YOLO

# Load a model
model = YOLO("yolo26s-depth.pt")  # load an official model

# Export the model
model.export(format="onnx", imgsz=(768, 768), dynamic=True, simplify=True)

cd ultralytics
python export.py

复制模型并执行

cp ultralytics/yolo26s-depth.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
make yolo_depth -j64

首次运行会自动由 TRT::compile 生成 yolo26s-depth.FP16.trtmodel（768 输入、456 层，首次编译约需数分钟，之后直接复用），无需 trtexec 脚本。结果保存在 yolo26s-depth_FP16_result/：

*_depth.jpg：Spectral_r 视差着色图（近处暖色、远处冷色）
*_metric.jpg：固定 0~20m 量程的 inferno 米制深度图
*_overlay.jpg：jet 深度热力图与原图叠加
*_depth.tiff：float32 米制深度原始数据，便于数值验证

如需单图或视频演示，取消 src/application/app_yolo_depth.cpp 中 test_single_image() / test_video() 的注释即可：单图输出 Result-YoloDepth.jpg（原图 | 深度 | 叠加）和 Result-YoloDepth.tiff，视频输出 result_yolo_depth.mp4。

注意：YOLO26-Depth 的预处理是直接拉伸到 768x768（scale_fill=True），不是 letterbox，也没有 114 灰边填充；归一化仅为 /255。ONNX 输出已经是米制深度，C++ 端只需做 align_corners=True 的 bilinear 缩放到原图分辨率。

前置条件

tensorRT >= 8.6

下载 YOLOv8

git clone https://github.com/ultralytics/ultralytics.git

修改代码, 保证动态 batch

# ========== exporter.py ==========

# ultralytics/engine/exporter.py第323行
# output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output0']
# dynamic = self.args.dynamic
# if dynamic:
#     dynamic = {'images': {0: 'batch', 2: 'height', 3: 'width'}}  # shape(1,3,640,640)
#     if isinstance(self.model, SegmentationModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
#         dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
#     elif isinstance(self.model, DetectionModel):
#         dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 84, 8400)
# 修改为：

output_names = ['output0', 'output1'] if isinstance(self.model, SegmentationModel) else ['output']
dynamic = self.args.dynamic
if dynamic:
    dynamic = {'images': {0: 'batch'}}  # shape(1,3,640,640)
    if isinstance(self.model, SegmentationModel):
        dynamic['output0'] = {0: 'batch', 2: 'anchors'}  # shape(1, 116, 8400)
        dynamic['output1'] = {0: 'batch', 2: 'mask_height', 3: 'mask_width'}  # shape(1,32,160,160)
    elif isinstance(self.model, DetectionModel):
        dynamic['output'] = {0: 'batch'}  # shape(1, 84, 8400)

导出 onnx 模型，在 ultralytics-main 新建导出文件 export.py 内容如下（可能会由于 torch 版本问题导出失败, 具体可参考 #6144）

from ultralytics import RTDETR

model = RTDETR("rtdetr-l.pt")

success = model.export(format="onnx", dynamic=True, simplify=True)

cd ultralytics-main
python export.py

engine 生成

Engine 生成：利用 trtexec 工具生成 engine

cp ultralytics/yolov8s.onnx tensorRT_Pro-Vision/workspace
cd tensorRT_Pro-Vision/workspace
bash build.sh

执行

make rtdetr -j64

前置条件

tensorRT >= 8.6

下载 RF-DETR

git clone https://github.com/roboflow/rf-detr.git

修改代码，将后处理塞进 ONNX 计算图

step 1. 在 rf-detr/src/rfdetr/export/_onnx 文件夹下新建 post_process.py，内容如下：

# ------------------------------------------------------------------------
# RF-DETR
# Copyright (c) 2025 Roboflow. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 [see LICENSE for details]
# ------------------------------------------------------------------------
"""Lightweight wrapper that embeds post-processing into the ONNX graph for deployment."""

import torch
from torch import nn
class ExportPostProcessor(nn.Module):
    """Wraps the raw LWDETR model so that sigmoid + argmax + merge are part of the ONNX graph.

    The wrapped model's ``forward()`` produces a single tensor that is ready for threshold filtering
    at the caller side — no additional CUDA kernels are needed for post-processing.

    Input:
        images: (B, 3, H, W) float32, already resized and normalized.

    Output:
        output: (B, 300, 6) where the last dim is ``[cx, cy, w, h, confidence, class_id]``.
    """

    def __init__(self, model: nn.Module) -> None:
        super().__init__()
        self.model = model

    def forward(self, images: torch.Tensor) -> torch.Tensor:
        outputs = self.model(images)

        # forward_export() returns a tuple (boxes, logits, ...); standard forward() returns a dict.
        if isinstance(outputs, tuple):
            pred_boxes = outputs[0]
            pred_logits = outputs[1]
        else:
            pred_boxes = outputs["pred_boxes"]
            pred_logits = outputs["pred_logits"]

        # Sigmoid → per-query argmax to select the best class.
        probs = pred_logits.sigmoid()           # (B, 300, 91)
        scores, labels = probs.max(dim=-1)       # (B, 300), (B, 300)
        labels = labels.float().unsqueeze(-1)    # (B, 300, 1)
        scores = scores.unsqueeze(-1)            # (B, 30

tensorRT_Pro-Vision

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

v1.0.0 - 自定义 ONNX Parser 稳定版

已支持模型

技术栈

用户点评

README 原文

Quick Start

Environment

What's New

Model Downloads

Why this repo

Supported Tasks

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

v1.0.0 - 自定义 ONNX Parser 稳定版

已支持模型

技术栈

用户点评

README 原文

Quick Start

Environment

What's New

Model Downloads

Why this repo

Supported Tasks

相关项目