mmdetection

open-mmlabApache-2.0长期未更新

基于 PyTorch 的目标检测工具箱与基准库，集成了 Faster R-CNN、DETR、Grounding DINO 等主流模型，方便训练、评测和复现检测与分割算法。

OpenMMLab Detection Toolbox and Benchmark

v3.3.0发布于 2024-01-05 06:24 UTC33,032star +6 / 7 天9,815fork同步于 2026-11-10 11:21 UTC

目标检测实例分割全景分割基准库 PyTorch

对比 GitHub 官网 cascade-rcnn convnext detr fast-rcnn faster-rcnn

怎么上手

由 README 提炼，执行前请确认

原文

pip install -U openmim
mim install mmdet

硬件要求按 README 说明

README 未说明具体显存要求；主分支需 PyTorch 1.8+，基础 bbox 与 mask 算子运行在 GPU 上。

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

基于 PyTorch 的开源目标检测工具箱，模块化组合即可搭出定制检测框架

基于 PyTorch 的目标检测工具箱与基准库，集成了 Faster R-CNN、DETR、Grounding DINO 等主流模型，方便训练、评测和复现检测与分割算法。

核心能力

拆解检测框架为组件，按需组合搭建自定义流程
开箱支持目标检测、实例分割、全景分割与半监督检测
bbox 与 mask 基础算子全部跑在 GPU 上
提供模型库与基准结果，含 RTMDet 等 SOTA 模型
发布 MM-Grounding-DINO 开放词表检测预训练权重

容易踩的坑

依赖 MMEngine 和 MMCV，需按版本对应关系安装；安装细节需查阅官方 Installation 文档；从 2.x 升级需参考 migration 指南。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

Highlight

v3.2.0 was released in 12/10/2023:

1. Detection Transformer SOTA Model Collection (1) Supported four updated and stronger SOTA Transformer models: DDQ, CO-DETR, AlignDETR, and H-DINO. (2) Based on CO-DETR, MMDet released a model with a COCO performance of 64.1 mAP. (3) Algorithms such as DINO support AMP/Checkpoint/FrozenBN, which can effectively reduce memory usage.

2. Comprehensive Performance Comparison between CNN and Transformer RF100 consists of a dataset collection of 100 real-world datasets, including 7 domains. It can be used to assess the performance differences of Transformer models like DINO and CNN-based algorithms under different scenarios and data volumes. Users can utilize this benchmark to quickly evaluate the robustness of their algorithms in various scenarios.

3. Support for GLIP and Grounding DINO fine-tuning, the only algorithm library that supports Grounding DINO fine-tuning The Grounding DINO algorithm in MMDet is the only library that supports fine-tuning. Its performance is one point higher than the official version, and of course, GLIP also outperforms the official version. We also provide a detailed process for training and evaluating Grounding DINO on custom datasets. Everyone is welcome to give it a try.

Model	Backbone	Style	COCO mAP	Official COCO mAP
Grounding DINO-T	Swin-T	Zero-shot	48.5	48.4
Grounding DINO-T	Swin-T	Finetune	58.1(+0.9)	57.2
Grounding DINO-B	Swin-B	Zero-shot	56.9	56.7
Grounding DINO-B	Swin-B	Finetune	59.7
Grounding DINO-R50	R50	Scratch	48.9(+0.8)	48.1

4. Support for the open-vocabulary detection algorithm Detic and multi-dataset joint training. 5. Training detection models using FSDP and DeepSpeed.

ID	AMP	GC of Backbone	GC of Encoder	FSDP	Peak Mem (GB)	Iter Time (s)
1					49 (A100)	0.9
2	√				39 (A100)	1.2
3		√			33 (A100)	1.1
4	√	√			25 (A100)	1.3
5		√	√		18	2.2
6	√	√	√		13	1.6
7		√	√	√	14	2.9
8	√	√	√	√	8.5	2.4

6. Support for the V3Det dataset, a large-scale detection dataset with over 13,000 categories.

亮点

v3.2.0 版本已经在 2023.10.12 发布：

1. 检测 Transformer SOTA 模型大合集 (1) 支持了 DDQ、CO-DETR、AlignDETR 和 H-DINO 4 个更新更强的 SOTA Transformer 模型 (2) 基于 CO-DETR, MMDet 中发布了 COCO 性能为 64.1 mAP 的模型 (3) DINO 等算法支持 AMP/Checkpoint/FrozenBN，可以有效降低显存

**2. [提供了全面的 CNN 和 T

v3.1.0MMDetection v3.1.0 Release2023-06-30 · 2023-06-30 07:41 UTC
查看更新说明
Highlights
- Supports tracking algorithms including multi-object tracking (MOT) algorithms SORT, DeepSORT, StrongSORT, OCSORT, ByteTrack, QDTrack, and video instance segmentation (VIS) algorithm MaskTrackRCNN, Mask2Former-VIS.
- Support ViTDet
- Supports inference and evaluation of multimodal algorithms GLIP and XDecoder, and also supports datasets such as COCO semantic segmentation, COCO Caption, ADE20k general segmentation, and RefCOCO. GLIP fine-tuning will be supported in the future.
- Provides a gradio demo for image type tasks of MMDetection, making it easy for users to experience.
Exciting Features

GLIP inference and evaluation

s multimodal vision algorithms continue to evolve, MMDetection has also supported such algorithms. This section demonstrates how to use the demo and eval scripts corresponding to multimodal algorithms using the GLIP algorithm and model as the example. Moreover, MMDetection integrated a gradio_demo project, which allows developers to quickly play with all image input tasks in MMDetection on their local devices. Check the document for more details.

Preparation

Please first make sure that you have the correct dependencies installed:
```
# if source
pip install -r requirements/multimodal.txt

# if wheel
mim install mmdet[multimodal]
```
MMDetection has already implemented GLIP algorithms and provided the weights, you can download directly from urls:
```
cd mmdetection
wget https://download.openmmlab.com/mmdetection/v3.0/glip/glip_tiny_a_mmdet-b3654169.pth
```
Inference

Once the model is successfully downloaded, you can use the demo/image_demo.py script to run the inference.
```
python demo/image_demo.py demo/demo.jpg glip_tiny_a_mmdet-b3654169.pth --texts bench
```
Demo result will be similar to this:

If users would like to detect multiple targets, please declare them in the format of xx . xx . after the --texts.
```
python demo/image_demo.py demo/demo.jpg glip_tiny_a_mmdet-b3654169.pth --texts 'bench . car .'
```
And the result will be like this one:

You can also use a sentence as the input prompt for the --texts field, for example:
```
python demo/image_demo.py demo/demo.jpg glip_tiny_a_mmdet-b3654169.pth --texts 'There are a lot of cars here.'
```
The result will be similar to this:

Evaluation

The GLIP implementation in MMDetection does not have any performance degradation, our benchmark is as follows:

Model official mAP mmdet mAP
glip_A_Swin_T_O365.yaml 42.9 43.0
glip_Swin_T_O365.yaml 44.9 44.9
glip_Swin_L.yaml 51.4 51.3

Users can use the test script we provided to run evaluation as well. Here is a basic example:
```
# 1 gpu
python tools/test.py configs/glip/glip_atss_swin-t_fpn_dyhead_pretrain_obj365.py glip_tiny_a_mmdet-b3654169.pth

# 8 GPU
./tools/dist_test.sh configs/glip/glip_atss_swin-t_fpn_dyhead_pretrain_obj365.py glip_tiny_a_mmdet-b3654
```
v3.0.0MMDetection releases v3.0.02023-04-06 · 2023-04-06 07:26 UTC
查看更新说明
v3.0.0 (6/4/2023)

We have released the official version of MMDetection v3.0.0

Highlights
- Support Semi-automatic annotation Base Label-Studio (#10039)
- Support EfficientDet in projects (#9810)
New Features
- File I/O migration and reconstruction (#9709)
- Release DINO Swin-L 36e model (#9927)
Bug Fixes
- Fix benchmark script (#9865)
- Fix the crop method of PolygonMasks (#9858)
- Fix Albu augmentation with the mask shape (#9918)
- Fix RTMDetIns prior generator device error (#9964)
- Fix img_shape in data pipeline (#9966)
- Fix cityscapes import error (#9984)
- Fix solov2_r50_fpn_ms-3x_coco.py config error (#10030)
- Fix Conditional DETR AP and Log (#9889)
- Fix accepting an unexpected argument local-rank in PyTorch 2.0 (#10050)
- Fix common/ms_3x_coco-instance.py config error (#10056)
- Fix compute flops error (#10051)
- Delete data_root in CocoOccludedSeparatedMetric to fix bug (#9969)
- Unifying metafile.yml (#9849)
Improvements
- Added BoxInst r101 config (#9967)
- Added config migration guide (#9960)
- Added more social networking links (#10021)
- Added RTMDet config introduce (#10042)
- Added visualization docs (#9938, #10058)
- Refined data_prepare docs (#9935)
- Added support for setting the cache_size_limit parameter of dynamo in PyTorch 2.0 (#10054)
- Updated coco_metric.py (#10033)
- Update type hint (#10040)
Contributors

A total of 19 developers contributed to this release.

Thanks @IRONICBo, @vansin, @RangeKing, @Ghlerrix, @okotaku, @JosonChan1998, @zgzhengSEU, @bobo0810, @yechenzhi, @Zheng-LinXiao, @LYMDLUT, @yarkable, @xiejiajiannb, @chhluo, @BIGWangYuDong, @RangiLyu, @zwhus, @hhaAndroid, @ZwwWayne
v2.28.2MMDetection V2.28.2 Release2023-02-28 · 2023-02-28 02:24 UTC
查看更新说明
New Features and Improvements
- Add Twitter, Discord, Medium and YouTube link (#9774)
- Update customize_runtime.md (#9797)
Bug Fixes
- Fix WIDERFace SSD loss for Nan problem (#9734)
- Fix missing API documentation in Readthedoc (#9729)
- Fix the configuration file and log path of CenterNet (#9791)
New Contributors
- @Ginray made their first contribution in https://github.com/open-mmlab/mmdetection/pull/9791
- @co63oc made their first contribution in https://github.com/open-mmlab/mmdetection/pull/9797
Contributors

A total of 4 developers contributed to this release. Thanks @co63oc, @Ginray, @vansin, @RangiLyu

Full Changelog: https://github.com/open-mmlab/mmdetection/compare/v2.28.1...v2.28.2

Model	official mAP	mmdet mAP
glip_A_Swin_T_O365.yaml	42.9	43.0
glip_Swin_T_O365.yaml	44.9	44.9
glip_Swin_L.yaml	51.4	51.3

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

English | 简体中文

Introduction

MMDetection is an open source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.8+.

Modular Design

We decompose the detection framework into different components and one can easily construct a customized object detection framework by combining different modules.
Support of multiple tasks out of box

The toolbox directly supports multiple detection tasks such as object detection, instance segmentation, panoptic segmentation, and semi-supervised object detection.
High efficiency

All basic bbox and mask operations run on GPUs. The training speed is faster than or comparable to other codebases, including Detectron2, maskrcnn-benchmark and SimpleDet.
State of the art

The toolbox stems from the codebase developed by the MMDet team, who won COCO Detection Challenge in 2018, and we keep pushing it forward. The newly released RTMDet also obtains new state-of-the-art results on real-time instance segmentation and rotated object detection tasks and the best parameter-accuracy trade-off on object detection.

Apart from MMDetection, we also released MMEngine for model training and MMCV for computer vision research, which are heavily depended on by this toolbox.

What's New

💎 We have released the pre-trained weights for MM-Grounding-DINO Swin-B and Swin-L, welcome to try and give feedback.

Highlight

v3.3.0 was released in 5/1/2024:

MM-Grounding-DINO: An Open and Comprehensive Pipeline for Unified Object Grounding and Detection

Grounding DINO is a grounding pre-training model that unifies 2d open vocabulary object detection and phrase grounding, with wide applications. However, its training part has not been open sourced. Therefore, we propose MM-Grounding-DINO, which not only serves as an open source replication version of Grounding DINO, but also achieves significant performance improvement based on reconstructed data types, exploring different dataset combinations and initialization strategies. Moreover, we conduct evaluations from multiple dimensions, including OOD, REC, Phrase Grounding, OVD, and Fine-tune, to fully excavate the advantages and disadvantages of Grounding pre-training, hoping to provide inspiration for future work.

code: mm_grounding_dino/README.md

We are excited to announce our latest work on real-time object recognition tasks, RTMDet, a family of fully convolutional single-stage detectors. RTMDet not only achieves the best parameter-accuracy trade-off on object detection from tiny to extra-large model sizes but also obtains new state-of-the-art performance on instance segmentation and rotated object detection tasks. Details can be found in the technical report. Pre-trained models are here.

Task	Dataset	AP	FPS(TRT FP16 BS1 3090)
Object Detection	COCO	52.8	322
Instance Segmentation	COCO	44.6	188
Rotated Object Detection	DOTA	78.9(single-scale)/81.3(multi-scale)	121

Installation

Please refer to Installation for installation instructions.

Getting Started

Please see Overview for the general introduction of MMDetection.

For detailed user guides and advanced guides, please refer to our documentation:

We also provide object detection colab tutorial and instance segmentation colab tutorial .

To migrate from MMDetection 2.x, please refer to migration.

Overview of Benchmark and Model Zoo

Results and models are available in the model zoo.

Some other methods are also supported in projects using MMDetection.

FAQ

Please refer to FAQ for frequently asked questions.

Contributing

We appreciate all contributions to improve MMDetection. Ongoing projects can be found in out GitHub Projects. Welcome community users to participate in these projects. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMDetection is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new detectors.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

@article{mmdetection,
  title   = {{MMDetection}: Open MMLab Detection Toolbox and Benchmark},
  author  = {Chen, Kai and Wang, Jiaqi and Pang, Jiangmiao and Cao, Yuhang and
             Xiong, Yu and Li, Xiaoxiao and Sun, Shuyang and Feng, Wansen and
             Liu, Ziwei and Xu, Jiarui and Zhang, Zheng and Cheng, Dazhi and
             Zhu, Chenchen and Cheng, Tianheng and Zhao, Qijie and Li, Buyu and
             Lu, Xin and Zhu, Rui and Wu, Yue and Dai, Jifeng and Wang, Jingdong
             and Shi, Jianping and Ouyang, Wanli and Loy, Chen Change and Lin, Dahua},
  journal= {arXiv preprint arXiv:1906.07155},
  year={2019}
}

License

This project is released under the Apache 2.0 license.

Projects in OpenMMLab

MMEngine: OpenMMLab foundational library for training deep learning models.
MMCV: OpenMMLab foundational library for computer vision.
MMPreTrain: OpenMMLab pre-training toolbox and benchmark.
MMagic: OpenMMLab Advanced, Generative and Intelligent Creation toolbox.
MMDetection: OpenMMLab detection toolbox and benchmark.
MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
MMRotate: OpenMMLab rotated object detection toolbox and benchmark.
MMYOLO: OpenMMLab YOLO series toolbox and benchmark.
MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.
MMPose: OpenMMLab pose estimation toolbox and benchmark.
MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.
MMSelfSup: OpenMMLab self-supervised learning toolbox and benchmark.
MMRazor: OpenMMLab model compression toolbox and benchmark.
MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.
MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
MMTracking: OpenMMLab video perception toolbox and benchmark.
MMFlow: OpenMMLab optical flow toolbox and benchmark.
MMEditing: OpenMMLab image and video editing toolbox.
MMGeneration: OpenMMLab image and video generative models toolbox.
MMDeploy: OpenMMLab model deployment framework.
MIM: MIM installs OpenMMLab packages.
MMEval: A unified evaluation library for multiple machine learning libraries.
Playground: A central hub for gathering and showcasing amazing projects built upon OpenMMLab.

mmdetection

怎么上手

项目说明