awesome-segment-anything

Awesome Segment Anything

Segment Anything has led to a new breakthrough in the field of Computer Vision (CV), and this repository will continue to track and summarize the research progress of Segment Anything in various fields, including Papers/Projects, etc.

If you find this repository helpful, please consider Stars ⭐ or Sharing ⬆️. Thanks.

🏅Recognition

This repository has been featured on Shypd.Ai, the largest open index of AI tools worldwide (over 80,000 tools). It is categorized under Academic Research, an emerging section that currently hosts fewer than 1,800 high-quality AI resources.

News

• 2026.4: Added RankSEG (NeurIPS 2025 & JMLR 2023), EfficientSAM (CVPR 2024), EdgeSAM, Semantic-SAM, MedSAM-2, SurgicalSAM (AAAI 2024) and more.
• 2025.11.19: Added SAM 3 - Meta's latest breakthrough in visual segmentation with text-based prompting and concept-level understanding.
• 2024.9: Added SAM 2.1 - Enhanced version with improved video segmentation and training code release.
• 2024.8.16: Added Segment Anything 2 (SAM 2) and SaLIP - First unified model for real-time object segmentation in images and videos.
• 2023.8.29: Updated with recent works and expanded derivative papers section.
• 2023.5.20: Updated document structure and added robotics-related article. Happy 520 Day!
• 2023.5.4: Added SEEM - Segment Everything Everywhere All at Once.
• 2023.4.18: Added Inpainting Anything and SAM-Track for video segmentation.
• 2023.4.12: Initial release tracking SAM papers and projects.

Contents

• Basemodel Papers
• Derivative Papers
  • Analysis and Expansion of SAM
  • Medical Image Segmentation
  • Inpainting
  • Camouflaged Object Detection
  • Video Frame Interpolation
  • Low Level Vision
  • Image Matting
  • Robotic
  • Bioinformatics
  • 3D
  • Remote Sensing
  • Tracking
  • Audio-visual Localization and Segmentation
  • Adversarial Attacks
  • LLM and SAM
  • Autonomous Driving
• Derivative Projects
  • Image Segmentation task
  • Video Segmentation task
  • Medical image Segmentation task
  • Inpainting task
  • 3D task
  • Image Generation task
  • Remote Sensing task
  • Moving Object Detection task
  • OCR task
• front-end framework
  • JS SDK for SAM

Papers/Projects

Basemodel Papers

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
CLIP		arXiv	Colab	Code	OpenAI	Contrastive Language-Image Pre-Training.
OWL-ViT		ECCV2022	-	Code	Google	A open-vocabulary object detector.
OvSeg		CVPR2023	Project	Code	META	Segment an image into semantic regions according to text descriptions.
Painter		CVPR2023	-	Code	BAAI	A Generalist Painter for In-Context Visual Learning.
Grounding DINO		arXiv	Colab &Huggingface	Code	IDEA	A stronger open-set object detector
Segment Anything		arXiv	Project page	Code	Meta	A stronger Large model which can be used to generate masks for all objects in an image.
SegGPT		arXiv	Project page	Code	BAAI	Segmenting Everything In Context based on Painter.
Segment Everything Everywhere All at Once (SEEM)		arXiv	Project Page	Code	Microsoft	Semantic Segmentation with various prompt types.
Segment Anything 2 (SAM 2)		Paper	Project Page	Code	Meta	A foundation model towards solving promptable visual segmentation in images and videos.
SAM 3	-	arXiv	Project Page	-	Meta	Segment Anything with Concepts: doubles accuracy on promptable concept segmentation in images and videos.

Derivative Papers

Analysis and Expansion of SAM

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
CLIP_Surgery		arXiv	Demo	Code	HKUST	This work about SAM based on CLIP's explainability to achieve text to mask without manual points.
GenSAM		arXiv	Project Page	Code	QMUL	This work relaxes the requirement for instance-specific prompts in SAM.
Segment Anything Is Not Always Perfect		arXiv	-	-	Samsung	This paper analyzes and discusses the benefits and limitations of SAM.
PerSAM		arXiv	Project Page	Code	-	Segment Anything with specific concepts.
Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching		arXiv	-	Code	-	One shot semantic segmentation by integrating an all-purpose feature extraction model and a class-agnostic segmentation model.
Segment Anything in High Quality		NeurIPS2023	Project Page	Code	ETH Zürich & HKUST	HQ-SAM: improve segmentation quality of SAM using learnable High-Quality Output Token.
Detect Any Shadow: Segment Anything for Video Shadow Detection		arXiv	-	Code	University of Science and Technology of China	Use SAM to detect initial frames then use an LSTM network for subsequent frames.
Fast Segment Anything		arXiv	Project Page	Code	-	Reformulate the architecture and improve the speed of SAM.
MobileSAM (Faster Segment Anything)		arXiv	Project Page	Code	Kyung Hee University	Make SAM mobile-friendly by replacing the heavyweight image encoder with a lightweight one.
FoodSAM (Any Food Segmentation)		arXiv	Project Page	Code	UCAS	Semantic, instance, panoptic, interactive segmentation on food image.
DefectSAM		arXiv	-	Code	ZJU, Westlake, UESTC, etc.	Infrared thermal images, defect detection.
SlimSAM		arXiv	-	Code	NUS	0.1% Data Makes Segment Anything Slim.
EfficientSAM		CVPR2024	Project Page	Code	Meta	Lightweight SAM models via masked image pretraining (SAMI), achieving competitive performance with greatly reduced complexity.
EdgeSAM		arXiv	Project Page	Code	MMLab@NTU	Prompt-in-the-loop distillation for on-device SAM deployment; 40× faster than SAM, first to run >30 FPS on iPhone 14.
Semantic-SAM		arXiv	Project Page	Code	Microsoft	Universal image segmentation model that segments and recognizes anything at any desired granularity.
RankSEG		NeurIPS 2025 / JMLR 2023	Docs / Demo	Code / Colab	CUHK	RankSEG is a plug-and-play post-processing module that improves segmentation results during inference. It works with segmentation models (SAM, DeepLab, SegFormer, UPerNet, etc.) without any retraining or fine-tuning.

Medical Image Segmentation

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Segment Anything Model (SAM) for Digital Pathology		arXiv	-	-	-	SAM + Tumor segmentation/Tissue segmentation/Cell nuclei segmentation.
Segment Anything in Medical Images		arXiv	-	Code	-	A step-by-step tutorial with a small dataset to help you quickly utilize SAM.
SAM Fails to Segment Anything?		arXiv	-	Code	-	SAM-adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More.
Segment Anything Model for Medical Image Analysis: an Experimental Study		arXiv	-	-	-	Thorough experiments evaluating how SAM performs on 19 medical image datasets.
Medical-SAM-Adapter		arXiv	-	Code	-	A project to finetune SAM using Adaption for the Medical Imaging.
SAM-Med2D		arXiv	-	Code	Sichuan University & Shanghai AI Laboratory	The most comprehensive studies on applying SAM to medical 2D images.
ScribblePrompt-SAM		arXiv	Project Page	Code	MIT & MGH	Fine-tuned SAM on 65 biomedical imaging datasets with scribble, click, and bounding box inputs.
SaLIP	-	arXiv	-	Code	-	Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero-shot Medical Image Segmentation.
MedSAM-2	-	arXiv	-	Code	-	Medical SAM 2: treats all 2D and 3D medical segmentation tasks as video object tracking, enabling universal auto-tracking segmentation.
SurgicalSAM		AAAI2024	-	Code	University of Adelaide	Efficient class-promptable surgical instrument segmentation by fine-tuning SAM with surgical-specific information, eliminating the need for explicit point/box prompts.

Bioimage Analysis

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Segment Anything for Microscopy		bioRxiv	Demo	Code	University of Göttingen, Germany	Segment Anything for Microscopy implements automatic and interactive annotation for microscopy data. It is built on top of Segment Anything and specializes it for microscopy and other bio-imaging data. Its core components are: The micro_sam tools for interactive data annotation with napari.The micro_sam library to apply Segment Anything to 2d and 3d data or fine-tune it on your data.The micro_sam models that are fine-tuned on publicly available microscopy data. Our goal is to build fast and interactive annotation tools for microscopy data.

Inpainting

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Inpaint Anything		arXiv	-	Code	USTC & EIT	SAM + Inpainting, which is able to remove the object smoothly.
SAM + Stable Diffusion for Text-to-Image Inpainting		-	Project	Code	comet	Grounding DINO + SAM + Stable Diffusion.

Camouflaged Object Detection

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
SAMCOD	-	arXiv	-	Code	-	SAM + Camouflaged object detection (COD) task.

Video Frame Interpolation

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame Interpolation		arXiv	Project Page & Interactive Demo	Code	Shanghai AI Laboratory & Snap Inc.	Editable video frame interpolation with SAM.

Low Level Vision

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Segment Anything in Video Super-resolution		arXiv	-	-	-	The first step to use SAM for low-level vision.
SAM-IQA		arXiv	-	Code	Megvii	The first to introduce the SAM in IQA and demonstrate its strong generalization ability in this domain.

Image Matting

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Matte Anything		arXiv	-	Code	HUST Vision Lab	An interactive natural image matting system with excellent performance for both opaque and transparent objects.
Matting Anything		arXiv	Project page	Code	SHI Labs	Leverage feature maps from SAM and adopts a Mask-to-Matte module to predict the alpha matte.

Robotic

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Instruct2Act		arXiv	-	Code	OpenGVLab	A SAM application in the Robotic field.

Bioinformatics

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
IAMSAM		bioRxiv	-	Code	Portrai Inc.	A SAM application for the analysis of Spatial Transcriptomics.

3D

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Point-SAM		arXiv	Page	Code	UCSD	An open-world 3D native promptable point-cloud segmentation method.
SAMPro3D		arXiv	Page	Code	CUHKSZ, MSRA	A novel method to segment any 3D indoor scenes by applying the SAM to 2D frames, without need any training, tuning, distillation or 3D pretrained networks.
Seal		arXiv	Page	Code	-	A framework capable of leveraging 2D vision foundation models for self-supervised learning on large-scale 3D point clouds.
TomoSAM		arXiv	Video Tutorial	Code	-	An extension of 3D Slicer using the SAM to aid the segmentation of 3D data from tomography or other imaging techniques.
SegmentAnythingin3D		arXiv	Project	Code	-	A novel framework to Segment Anything in 3D, named SA3D.

Remote Sensing

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
RSPrompter		arXiv	Project Page	Code	Beihang University	An automated instance segmentation approach for remote sensing images based on the SAM.
SAM-CD		arXiv	-	Code	PLA Information Engineering University	A sample-efficient change detection framework that employs SAM as the visual encoder.
SAM-Road: Segment Anything Model for Road Network Graph Extraction		arXiv	-	Code	Carnegie Mellon University	A simple and fast method applying SAM for vectorized large-scale road network graph extraction. It reaches state-of-the-art accuracy while being 40 times faster.

Tracking

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Follow Anything		arXiv	Page	Code	MIT, Harvard University	An open-vocabulary and multimodal model to detect, track, and follow any objects in real-time.
Track-Anything	Video	arXiv	-	Code	-	An interactive video object tracking and segmentation tool based on SAM and XMem.
SAM-Track	Video	arXiv	-	Code	Zhejiang University	A framework called Segment And Track Anything (SAMTrack) that allows users to precisely and effectively segment and track any object in a video.

Audio-visual Localization and Segmentation

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
AV-SAM		arXiv	-	Code	CMU	A simple yet effective audio-visual localization and segmentation framework based on the SAM.

Adversarial Attacks

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
Attack-SAM	-	arXiv	-	-	KAIST	The first work of conduct a comprehensive investigation on how to attack SAM with adversarial examples.

Multimedia Forensics

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
SAFIRE: Segment Any Forged Image Region		arXiv	-	Code	KAIST	Extends SAM's point prompting capability to image forensics, enabling precise source-aware segmentation for forgery localization.

LLM and SAM

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
LISA		CVPR2024	Project Page	Code	CUHK	Large Language Instructed Segmentation Assistant: inherits LLM language generation while producing segmentation masks from complex implicit text queries.
Sa2VA		arXiv	Project Page	Code	-	Marries SAM 2 with LLaVA for dense grounded understanding of images and videos; unifies text, image, and video into a shared LLM token space.

Autonomous Driving

Title	Presentation	Paper page	Project page	Code base	Affiliation	Description
AD-SAM	-	arXiv	-	-	-	Fine-tuned SAM with dual-encoder and deformable decoder for semantic segmentation in autonomous driving scenes.
SAM4UDASS	-	arXiv	-	-	-	Incorporates SAM into self-training UDA methods for refining pseudo-labels in unsupervised domain adaptive semantic segmentation for intelligent vehicles.

Derivative Projects

Image Segmentation task

Title	Presentation	Project page	Code base	Affiliation	Description
Grounded Segment Anything		Colab & Huggingface	Code	-	Combining Grounding DINO and Segment Anything.
GroundedSAM Anomaly Detection		-	Code	-	Grounding DINO + SAM to segment any anomaly.
Semantic Segment Anything		-	Code	Fudan	A dense category annotation engine.
Magic Copy		-	Code	-	Magic Copy is a Chrome extension that uses SAM.
YOLO-World + EfficientViT SAM		🤗 HuggingFace Space	Code	-	Efficient open-vocabulary object detection and segmentation with YOLO-World + EfficientViT SAM.
Segment Anything with Clip		🤗 HuggingFace Space	Code	-	SAM + CLIP.
SAM-Clip		-	Code	-	SAM + CLIP.
Prompt Segment Anything		-	Code	-	SAM + Zero-shot Instance Segmentation.
RefSAM	-	-	Code	-	Evaluating the basic performance of SAM on the Referring Image segmentation task.
SAM-RBox		-	Code	-	An implementation of SAM for generating rotated bounding boxes with MMRotate.
Open Vocabulary Segment Anything		-	Code	-	An interesting demo by combining OWL-ViT of Google and SAM.
SegDrawer		-	Code	-	Simple static web-based mask drawer, supporting semantic drawing with SAM.
AnyLabeling		YoutubeDemo	Code	-	SAM + Labelme + LabelImg + Auto-labeling.
ISAT with segment anything		YoutubeDemo BiliBili Demo	Code	-	Labeling tool by SAM, supports SAM, sam-hq, MobileSAM, EdgeSAM etc.
Annotation Anything Pipeline		-	Code	-	GPT + SAM.
Roboflow Annotate		App	Blog	Roboflow	SAM-assisted labeling for training computer vision models.
SALT		-	Code	-	A tool that adds a basic interface for image labeling and saves the generated masks in COCO format.
SAM U Specify		-	Code	-	Use SAM and CLIP model to segment unique instances you want.
SAM web UI		App	Code	-	A web interface for the SAM.
Finetune Anything		-	Code	-	A class-aware one-stage tool for training fine-tuning models based on SAM.
NanoSAM		-	Code	NVIDIA	A distilled Segment Anything (SAM) model capable of running real-time with NVIDIA TensorRT.
Segment-Anything-UI		-	Code	-	A PySide6 based annotation tool for Segment Anything.
Segment-Anything-2-UI		-	Code	-	A PySide6 based annotation tool for Segment Anything 2. Video tracking with multiple objects.

Video Segmentation task

Title	Presentation	Project page	Code base	Affiliation	Description
MetaSeg		HuggingFace	Code	-	SAM + Video.
SAM-Track	Video	YoutubeDemo	Code	Zhejiang University	This project, which is based on SAM and DeAOT, focuses on segmenting and tracking objects in videos.

Medical image Segmentation task

Title	Presentation	Project page	Code base	Affiliation	Description
SAM in Napari	Video	-	Code	-	Segment anything with Napari integration of SAM.
SAM Medical Imaging		-	Code	-	SAM for Medical Imaging.

Inpainting task

Title	Presentation	Project page	Code base	Affiliation	Description
SegAnythingPro		-	Code	-	SAM + Inpainting/Replacing.

3D task

Title	Presentation	Project page	Code base	Affiliation	Description
3D-Box		-	Code	-	SAM is extended to 3D perception by combining it with VoxelNeXt.
Anything 3D Novel View		-	Code	-	SAM + Zero 1-to-3.
Any 3D Face		-	Code	-	SAM + HRN.
Segment Anything 3D		-	Code	Pointcept	Extending Segment Anything to 3D perception by transferring the segmentation information of 2D images to 3D space.

Image Generation task

Title	Presentation	Project page	Code base	Affiliation	Description
Edit Anything		-	Code	-	Edit and Generate Anything in an image.
Image Edit Anything		-	Code	-	Stable Diffusion + SAM.
SAM for Stable Diffusion Webui		-	Code	-	Stable Diffusion + SAM.

Remote Sensing task

Title	Presentation	Project page	Code base	Affiliation	Description
Earth Observation Tools		Colab	Code	-	SAM + Remote Sensing.

Moving Object Detection task

Title	Presentation	Project page	Code base	Affiliation	Description
Moving Object Detection		-	Code	-	SAM + Moving Object Detection.

OCR task

Title	Presentation	Project page	Code base	Affiliation	Description
OCR-SAM		Blog	Code	-	Optical Character Recognition with SAM.

front-end framework

SAMJS

Title	Presentation	Project page	Code base	Affiliation	Description
SAMJS		demo	Code	-	JS SDK for SAM, Support remote sensing data segmentation and vectorization.

Acknowledgement

Some of the presentations in this repository are borrowed from the original author, and we are very thankful for their contribution.

License

This project is released under the MIT license. Please see the LICENSE file for more information.