๐ Explore โข ๐ฅ Latest Updates โข ๐ก Applications โข ๐ ๏ธ Tools โข ๐ Learn
|
|
# Clone repository
git clone https://github.com/umitkacar/SAM-Foundation-Models.git
cd SAM-Foundation-Models
# Install development dependencies
pip install -e ".[dev]"
# Set up pre-commit hooks
pre-commit install
# Run tests
pytest -n auto --cov=src tests/
# Run all quality checks
make checkFull documentation: See DEVELOPMENT.md for complete setup guide
|
SAM 2 processes images |
Requires 3x fewer |
Real-time on mobile |
|
Best Paper |
Real-time video |
Medical imaging |
Click to expand/collapse
- ๐ฅ Official Models & Latest Updates
- ๐ฌ SAM 2 & Video Segmentation
- โก Optimization & Mobile Deployment
- ๐ Academic Research & Surveys
- ๐ก Domain-Specific Applications
- ๐ Training & Fine-Tuning
- ๐ ๏ธ Production Deployment & Tools
- ๐ง SAM Extensions & Variants
- ๐ป Implementation Libraries
- ๐ Datasets & Benchmarks
- ๐ Educational Resources
- ๐ฏ Key Insights & Trends
๐ SAM 2.1 - Latest Release (September 2024)
| Feature | Performance | Status |
|---|---|---|
| ๐ Speed | 6x faster than SAM 1 | โ Released |
| ๐ฏ Efficiency | 3x fewer interactions | โ Stable |
| ๐ฌ Video FPS | 44 FPS real-time | โ Production |
| ๐ License | Apache 2.0 | โ Open Source |
| ๐ Recognition | ICLR 2025 Best Paper | โ Awarded |
# Install SAM 2
pip install segment-anything-2
# Quick Start
from sam2 import SAM2Model
model = SAM2Model.from_pretrained("facebook/sam2-hiera-large")Links:
- ๐ฆ GitHub Repository
- ๐ Meta AI Blog
- ๐ Research Paper - ArXiv:2408.00714
- ๐ค HuggingFace Hub - 182K+ downloads/month
- โ๏ธ AWS SageMaker JumpStart - Feb 2025
๐จ Original SAM (April 2023)
- ๐ Official Website
- ๐ฆ GitHub Repository
- ๐ Paper - ArXiv:2304.02643
- ๐ฏ Papers with Code
๐ Integration Platforms
| Platform | Features | Link |
|---|---|---|
| ๐ค Ultralytics | Production-ready YOLO integration | Docs |
| ๐ค HuggingFace | Model hub, Transformers support | Hub |
| โ๏ธ SageMaker | AWS deployment ready | JumpStart |
๐ SAM2Long
|
๐ฅ Grounded-SAM-2
|
๐ค AL-Ref-SAM2
|
๐ฅ Surgical SAM 2
|
- ๐ฏ Segment-and-Track-Anything
- โก RAP-SAM - Real-Time All-Purpose on Video
| Model | Speed | Size | Device | Highlights |
|---|---|---|---|---|
| ๐ EdgeSAM | 30+ FPS | - | iPhone 14 | First mobile SAM @ 30+ FPS |
| โก EfficientSAM | 10-20 img/s | 9.8M | Edge | Best accuracy-efficiency trade-off |
| ๐ฑ MobileSAM | 40x | 60x smaller | Mobile | Lightweight variant |
| ๐ FastSAM | ~100 img/s | 68M | GPU | Maximum throughput |
๐ Performance Comparison: EfficientSAM vs FastSAM
| Metric | EfficientSAM-Ti | EfficientSAM-S | FastSAM |
|---|---|---|---|
| COCO AP | 45.0 (+4.1) | - (+6.5) | 37.0 |
| LVIS AP | - (+5.3) | - (+7.8) | - |
| Params | 9.8M | - | 68M |
| Speed | 10-20 img/s | 10-20 img/s | ~100 img/s |
Winner: ๐ EfficientSAM for accuracy, ๐ FastSAM for speed
Click to see optimization tools & papers
- ๐ ArXiv: On Efficient Variants of SAM (2024)
- ๐ Awesome-Efficient-Segment-Anything
| Model | Link | Special Feature |
|---|---|---|
| ๐ EdgeSAM | GitHub โข Paper | CNN-based, iPhone ready |
| โก EfficientSAM | GitHub โข Site | Best AP/params ratio |
| ๐ฑ MobileSAM | GitHub | 60x compression |
| ๐ FastSAM | GitHub โข Docs | Prompt-free |
| ๐ฌ TinySAM | GitHub | Ultra-compact |
| ๐ HQ-SAM | GitHub | NeurIPS 2023, Quality++ |
๐ Must-Read Surveys
ArXiv:2306.06211 โข Updated: Oct 2024 โข ๐ Most Comprehensive
๐
Coverage: April 2023 - September 2024
๐ฏ Topics: SAM & SAM 2, Prompt Engineering
๐ Papers: 200+ analyzed
โญ Rating: โญโญโญโญโญ
ArXiv:2305.08196 โข Foundation model analysis
๐ฏ Topics: Computer Vision Applications
๐ฌ Depth: Technical Deep Dive
๐ Applications: Multiple Domains
ArXiv:2507.22792 โข 2025 โข Video-specific review
๐ฌ Focus: Video Object Segmentation & Tracking
โฐ Timeline: Past โ Present โ Future
๐ฏ Comprehensive VOST analysis
๐๏ธ Curated Collections
| Repository | Description | Stars |
|---|---|---|
| ๐ Awesome-Segment-Anything | First comprehensive survey | |
| ๐ฌ Awesome-SAM2 | SAM 2 specific resources | |
| ๐ฅ SAM4MIS | Medical imaging collection |
- ๐ 2024 Paper List - Continuously updated
๐ฉบ SAM4MISBenchmarks:
|
๐ง SAM-Med3DGeneral-purpose 3D Medical Segmentation Dataset:
|
๐ฌ More Medical Applications
| Paper | Venue | Application | Performance |
|---|---|---|---|
| Interactive 3D Medical Segmentation | ArXiv 2024 | Zero-shot 3D CT | State-of-art |
| SAM 2 for 3D Medical Imaging | JMIR AI 2025 | Abdominal CT scans | Promising |
| ProtoSAM-3D | PubMed | Volumetric imaging | Interactive |
| AutoProSAM | WACV 2025 | Multi-organ 3D | Automated |
| SAM-Med2D Analysis | BMC 2024 | 2D Medical images | Improved |
- ๐ Review: SAM for Medical Imaging - Comprehensive 2025 review
๐พ Agricultural Applications
๐ฐ๏ธ SAMGeo
SciPy 2024 Presentation
- ๐ Automated remote sensing segmentation
- ๐ฆ Open-source geospatial package
- ๐ฏ User-friendly API
๐ฑ SAM for Crop Mapping
MDPI Remote Sensing 2024
# Automated sample generation
โ
Sentinel-2 imagery (10m resolution)
โ
Landsat-8 support (30m resolution)
โ
Automatic quality filtering
โ
Sample cleaning pipelineArXiv 2023 | Zero-shot Performance Evaluation
- ๐ฏ Crop-type mapping
- ๐ Precision agriculture
- ๐ Zero-shot capabilities
- ๐พ Multi-spectral challenges
๐บ๏ธ ESRI ArcGIS Integration
- ๐๏ธ Urban planning
- ๐ฒ Environmental monitoring
- ๐ง Water body extraction
- ๐๏ธ Infrastructure mapping
๐ค Robotics & AV Applications
๐ SAMUNet
2025 | Shape-aware 3D Object Detection
- ๐ฏ Pillar-based detection
- ๐ Autonomous driving optimized
- ๐ Enhanced 3D understanding
- ๐ 7 SOTA Point Cloud Models
- ๐ฏ LiDAR-based detection
- ๐ค Robotic perception
- ๐ 3D scene understanding
Applications:
โ
3D Object Detection
โ
Semantic Segmentation
โ
Instance Segmentation
โ
Panoptic Segmentation
๐ Defect Detection & QC
MDPI Sensors 2025
Application: Oil & Gas Pipeline Inspection
Method: Ultrasonic B-scan Analysis
Models: SAM 1 (ViT-Base) + SAM 2 (Hiera-Base+)
Performance: F1-Score 0.940
Defect Type: Lack of Fusion (LOF)MDPI Processes 2025 | YOLO11 + SAM
- ๐ฏ Micro-vibration motor QC
- ๐ค YOLO11 detection + SAM segmentation
- ๐ Quantitative severity assessment
- โ 90%+ accuracy
- โก Real-time capable
Benefits:
โ
Automated inspection
โ
Cost-effective solution
โ
Real-time analysis
โ
Quantitative metrics
๐ Other Applications
| Domain | Project | Description |
|---|---|---|
| ๐ Depth | Depth-Anything-V2 | Monocular depth estimation |
| ๐ฎ 3D Gaussian | gaussian-grouping | 3D Gaussian splatting |
| ๐๏ธ 3D Recon | garfield | 3D reconstruction |
| ๐ท Mesh | MeshAnything | 3D mesh generation |
| ๐ฌ 4D | SA4D | 4D scene understanding |
| ๐ OCR | OCR-SAM | Text recognition |
| ๐ Food | FOOD-SAM | Food segmentation |
| ๐ธ Deblur | SAM-Deblur | Image deblurring |
๐ง Top Fine-Tuning Repositories
| Repository | Method | Domain | Updated |
|---|---|---|---|
| finetune-SAM | LoRA + Full | ๐ฅ Medical | โ 2024 |
| SAM-fine-tune ๐ | LoRA | ๐ General | โ Active |
| lora_sam | LoRA + ๐ค | ๐ฏ Vision | โ 2024 |
| SAMed | LoRA | ๐ฅ Medical | โ Stable |
| med-sam-brain | PEFT + LoRA | ๐ง Brain Tumor | โ 2024 |
๐ Tutorials & Learning Resources
| Resource | Level | Topics |
|---|---|---|
| Labellerr: SAM + LoRA | ๐ข Beginner | One-shot learning, ship segmentation |
| Encord: Fine-Tune Guide | ๐ก Intermediate | Complete pipeline, best practices |
| Medium: PEFT for Segmentation | ๐ก Intermediate | Parameter-efficient methods |
- ๐ฌ Conv-LoRA - OpenReview
- Lightweight convolutional parameters
- Combined with LoRA
- Enhanced performance
from transformers import SamModel, SamProcessor
from peft import LoraConfig, get_peft_model
# Load base model
model = SamModel.from_pretrained("facebook/sam-vit-base")
# Configure LoRA
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["qkv"],
lora_dropout=0.05,
)
# Apply LoRA
model = get_peft_model(model, lora_config)
# Fine-tune on your data
# ... training code ...
โจ CVATComputer Vision Annotation Tool SAM 2 Integration (2024):
Resources:
|
๐ท๏ธ Label StudioML Data Labeling Platform Features:
Resources: |
๐ CVAT vs Label Studio Comparison
| Feature | CVAT | Label Studio |
|---|---|---|
| Best For | Video annotation, beginners | Enterprise, multi-modal |
| SAM Integration | โ SAM 2 native | โ Via HuggingFace |
| Video Tools | โญโญโญโญโญ | โญโญโญ |
| Enterprise | โญโญโญ | โญโญโญโญโญ |
| Ease of Use | โญโญโญโญโญ | โญโญโญโญ |
| ML Pipeline | โญโญโญ | โญโญโญโญโญ |
| Pricing | Free + Enterprise | Free + Enterprise |
๐ง Other Annotation Tools
- ๐ฅ๏ธ AnyLabeling - Desktop labeling with SAM
- ๐ง SALT - Segment Anything Labelling Tool
๐ Deployment Options
# Install transformers
pip install transformers
# Use SAM with transformers
from transformers import SamModel, SamProcessor
model = SamModel.from_pretrained("facebook/sam-vit-huge")
processor = SamProcessor.from_pretrained("facebook/sam-vit-huge")Resources:
- ๐ Official Docs
- ๐ฏ Models Hub
- ๐ Jozu MLOps: HF to Production
| Tool | Format | Features |
|---|---|---|
| SAM2ONNX | ONNX | SAM 2 converter |
| sam_onnx_full_export | ONNX | Complete export |
| sam4onnx | ONNX | Optimization |
| samexporter | Multi | Multi-format |
๐ฅ Popular Extensions
The Ultimate Combo:
๐ฏ Grounding DINO (Detection)
โ
๐จ SAM (Segmentation)
โ
๐ผ๏ธ Stable Diffusion (Generation)
โ
โจ Detect โ Segment โ Generate ANYTHING
Features:
- โ Text-to-detection
- โ Automatic segmentation
- โ Image inpainting
- โ HuggingFace integration
๐ก๏ธ RobustSAM
CVPR 2024 | Adversarial Robustness
- ๐ Robust to adversarial attacks
- ๐ฏ Enhanced generalization
- ๐ Better performance on corrupted images
๐จ Personalize-SAM
DreamBooth Integration
- ๐ญ Personalized segmentation
- ๐ค Transformers Tutorial
- ๐ vs Mask R-CNN
๐งฉ Full Pipeline Solutions
| Project | Description | Key Feature |
|---|---|---|
| SEEM | Segment Everything Everywhere | Universal segmentation |
| Full-SAM | Complete pipeline | End-to-end solution |
| AUTODISTILL | Auto labeling | Dataset generation |
| GroundingDINO | Language grounding | Text-guided detection |
| RAM | Recognize Anything | Image tagging |
| CLIP | Vision-Language | Foundation model |
โ๏ธ C++ Implementations
| Repository | Framework | Platform |
|---|---|---|
| segment-anything-cpp-wrapper | Pure C++ | Cross-platform |
| sam-cpp-macos | Extended | macOS |
| sam.cpp | GGML | Multi-platform |
| SegmentAnything-OnnxRunner | ONNX | Cross-platform |
| SAM-ONNX-AX650-CPP | QT + Lama | GUI + Inpaint |
๐ท Other Languages
- ๐ SamSharp - C# implementation
- โก sam_onnx_rust - Rust ONNX
- ๐ฏ Libtorch-MobileSAM-Example - C++ PyTorch
๐ฑ Mobile Runtimes
- ๐ฑ EdgeSAM-MNN - EdgeSAM on MNN
- ๐ mnn-segment-anything - Mobile Neural Network
| SA-1B (Images) | SA-V (Videos) |
|---|---|
|
๐ฆ SA-1B Dataset Images: 11 Million
Masks: 1+ Billion
Type: Open world images
License: Licensed & privacy-respecting
Status: โ
Released 2023Superlatives:
|
๐ฌ SA-V Dataset Videos: 51,000
Countries: 47
Masks: 600,000+
Resolution: 240p โ 4K
Duration: 4s โ 138s
Status: โ
Released 2024Features:
|
๐ SAM 2 Evaluation Benchmarks
| Benchmark | Task | Metrics |
|---|---|---|
| DAVIS | Video object segmentation | J&F score |
| MOSE | Multi-object segmentation | J&F score |
| LVOS | Long-term video segmentation | Success rate |
| YouTube-VOS | Large-scale video | J&F score |
| COCO | Instance segmentation | AP, AR |
| LVIS | Large vocabulary segmentation | AP, APr |
SAM 2 Performance:
โ
3x fewer interactions
โ
Better accuracy
โ
6x faster inference
โ
SOTA on all benchmarks
๐ Must-Read Tutorials
| Resource | Level | Topics | Link |
|---|---|---|---|
| ๐ฏ Encord Ultimate Guide | ๐ข All | Architecture, Training, Applications | Read |
| ๐ค Roboflow Breakdown | ๐ข Beginner | Concepts, Use Cases | Read |
| ๐ ๏ธ Roboflow How-to | ๐ก Intermediate | Practical Implementation | Read |
| ๐ท๏ธ V7 Labs Guide | ๐ข All | Complete Overview | Read |
| ๐จโ๐ป LabelVisor Hands-on | ๐ก Intermediate | Effortless Segmentation | Read |
๐ Optimization Deep Dives
- ๐ EfficientSAM vs SAM - Detailed comparison
- ๐ Fast Faster SAM - Speed optimization
- ๐ฑ Faster SAM for Mobile - Mobile deployment
- ๐ SAM and Friends - Model ecosystem
๐บ YouTube Channels
| Channel | Focus | Subscriber Count |
|---|---|---|
| Rob Mulla | ML Tutorials, SAM Applications | Data Science |
| ArjanCodes | Software Engineering, Clean Code | Python & AI |
timeline
title SAM Evolution 2024-2025
2024-07 : SAM 2 Release
: Video Segmentation
2024-09 : SAM 2.1 Update
: ICLR 2025 Award
2024-12 : Medical Breakthrough
: 26+ Tasks, 15+ Benchmarks
2025-02 : AWS Integration
: SageMaker JumpStart
| Metric | Value | Model |
|---|---|---|
| โก Speed Improvement | 6x faster | SAM 2 vs SAM 1 |
| ๐ฏ Efficiency Gain | 3x fewer interactions | SAM 2 |
| ๐ฑ Mobile FPS | 30+ FPS | EdgeSAM (iPhone 14) |
| ๐ฌ Video FPS | 44 FPS | SAM 2 real-time |
| ๐ฅ Surgical FPS | 86 FPS | Surgical SAM 2 |
| ๐ญ Industrial Accuracy | F1: 0.940 | Weld Defect Detection |
| ๐ Medical Tasks | 26+ tasks | SAM4MIS |
๐ Emerging Research Directions
1. ๐ Training-Free Adaptation
โโ LoRA, Conv-LoRA, Prompt Tuning
2. ๐ฌ Video Understanding
โโ SAM 2, SAM2Long, Temporal Consistency
3. ๐ฅ Medical Imaging
โโ 3D Segmentation, Multi-modal Fusion
4. ๐ฑ Edge Deployment
โโ Quantization, Pruning, Distillation
5. ๐ Multi-Modal Integration
โโ Audio-Visual, Language-Vision
6. ๐ฏ Zero-Shot Transfer
โโ Cross-domain, Few-shot Learning
7. ๐ฎ 3D/4D Understanding
โโ Point Clouds, Temporal Dynamics
| Domain | Growth | Key Applications |
|---|---|---|
| ๐ฅ Medical | โญโญโญโญโญ | 3D imaging, Surgery |
| ๐ Autonomous | โญโญโญโญ | LiDAR, Perception |
| ๐ฐ๏ธ Remote Sensing | โญโญโญโญ | Agriculture, Mapping |
| ๐ญ Industrial | โญโญโญโญโญ | QC, Defect Detection |
| ๐จ Creative | โญโญโญ | Editing, Generation |
๐ Click to explore 2024 projects
| Project | Description | Stars |
|---|---|---|
| ReplaceAnything | ๐ญ Replace objects in images | |
| Depth-Anything | ๐ Monocular depth estimation | |
| OMG-Seg | ๐ Open-world segmentation | |
| OVSAM | ๐ Open-vocabulary SAM |
๐ Essential 2023 projects
- Matting-Anything - Professional image matting
- Inpaint-Anything - Intelligent inpainting
- EditAnything - Flexible image editing
- Painter - Generative models
- Prompt-Segment-Anything - Advanced prompting
- Semantic-Segment-Anything - Semantic seg
- Segment-Any-Anomaly - Anomaly detection
- GroundedSAM-Anomaly - Zero-shot AD
- Caption-Anything - Image captioning
- Count-Anything - Object counting
- detect-anyshadow - Shadow detection
- ShowAnything - Visualization
- Anything-3D - 3D reconstruction
We welcome contributions! Please ensure:
- โ Resources are from reputable sources
- โ Links are active and high-quality
- โ Descriptions are accurate and concise
- โ Proper categorization
- โ Include relevant badges/stars
- โ Add performance metrics when available
๐ BibTeX Citations
@article{kirillov2023segment,
title={Segment Anything},
author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
journal={arXiv:2304.02643},
year={2023}
}@article{ravi2024sam2,
title={SAM 2: Segment Anything in Images and Videos},
author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
journal={arXiv:2408.00714},
year={2024}
}
|
|
Latest Release: v1.0.0 | Status: โ Production Ready | Last Updated: November 2025
Last Updated: January 2025 ๐๏ธ Maintainer: Community-driven ๐ฅ License: Collection of resources with individual licenses ๐
Disclaimer: This is a curated collection. Each project has its own license.
Made with โค๏ธ by the SAM Community