S4M: 4-points to Segment Anything

Prompting SAM like a Sonographer

Adrien Meyer, Lorenzo Arboit, Giuseppe Massimiani, Shih-Min Yin, Didier Mutter, Nicolas Padoy

This article will be presented at IPCAI 2026, Nagoya, Japan

🚨 NEW: We added an interative demo and a checkpoint pretrained on a large scale ultrasound dataset!

S4M prompts SAM using only four points: the endpoints of the major and minor axes, inspired by sonographic measurements. We refer to these as the major/minor points. A Canvas auxiliary task helps the model learn the relationships between these points and encode shape. S4M is designed for medical image segmentation, with examples on ultrasound and endoscopic surgery data.

Install

Click to expand Install

This guide you to install S4M in a conda env

Clone the repo

git clone https://github.com/CAMMA-public/S4M
cd S4M

Create a conda environment and activate it. (Tested with cuda-11.8 & gcc-12)

conda create --name S4M python=3.10.16 -y
conda activate S4M

Install the OpenMMLab suite and other dependencies. Building mmcv may take a few minutes.

pip install setuptools==65.0.0
pip install --no-cache-dir --force-reinstall "numpy==2.1.2"

pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu118
pip install -U openmim
mim install mmengine
mim install "mmcv==2.1.0" --no-build-isolation
mim install mmdet
mim install mmpretrain
pip install tensorboard

pip install scikit-learn scikit-image

Download the pretrained checkpoints, endoscapes (endoscopic surgery) and mmotu (ultrasound)

UltraS4M is pretrained on the US-43d dataset from .

wget -O ./endoscapes_majmin.pth "https://s3.unistra.fr/camma_public/github/S4M/endoscapes_majmin.pth"
wget -O ./mmotu_majmin.pth "https://s3.unistra.fr/camma_public/github/S4M/mmotu_majmin.pth"
wget -O ./UltraS4M_majmin.pth "https://s3.unistra.fr/camma_public/github/S4M/UltraS4M_majmin.pth"

Interactive demo

Click to expand Interactive demo

The interactive demo lets you segment one image from user-provided prompts in a GUI, without a ground-truth annotation file.

export PYTHONPATH=$PYTHONPATH:.
export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1

python demo_s4m_interactive.py

The first four clicks define two pairs of major/minor points. The demo draws the two lines, assigns the longer pair as the major axis and the shorter pair as the minor axis, then runs S4M. After the first mask, select a correction type with the Positive/Negative checkboxes, click correction points, and use Segment / Update to run S4M again.

All CLI options are optional:

python demo_s4m_interactive.py \
  --image sample_dataset/sample_images/test.MMOTU_2d__00003.png \
  --config S4M/configs/S4M/mmotu_majmin.py \
  --checkpoint UltraS4M_majmin.pth \
  --device cpu

Use a custom image:

python demo_s4m_interactive.py \
  --image /path/to/image.png

Use another config/checkpoint pair, for example Endoscapes:

python demo_s4m_interactive.py \
  --image /path/to/image.png \
  --config S4M/configs/S4M/endoscapes_majmin.py \
  --checkpoint endoscapes_majmin.pth

Run on GPU when CUDA is available:

python demo_s4m_interactive.py \
  --device cuda:0

Options:

• --image: image path to open in the demo.
• --config: S4M config used to build the model and image transforms.
• --checkpoint: model checkpoint to load.
• --device: inference device, for example cpu or cuda:0.

Testing the model (ultrasound)

Click to expand Testing

To test on sample MMOTU datasets, using model.num_mask_refinements=0 extra points (ie only the 4 major/minor points)

export PYTHONPATH=$PYTHONPATH:.
export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1
mim test mmdet S4M/configs/S4M/mmotu_majmin.py --checkpoint mmotu_majmin.pth --cfg-options model.num_mask_refinements=0 --work-dir ./work_dir/example --show-dir ./show_dir

in the work_dir, you will find a "show_dir" with the predictions.

To test on sample MMOTU datasets, using model.num_mask_refinements=3, meaning 3 extra points are added to the 4 major/minor points as successive positive (green) or negative (red) correction points.

mim test mmdet S4M/configs/S4M/mmotu_majmin.py --checkpoint mmotu_majmin.pth --cfg-options model.num_mask_refinements=3 --work-dir ./work_dir/example --show-dir ./show_dir

Custom training

Click to expand Custom training

Let's download a small custom dataset in ./data, "balloons"

python S4M/tools/dl_balloons.py

Let's finetune S4M. We modify the training config inline. You can also create a custom config file instead.

export PYTHONPATH=$PYTHONPATH:.
export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1

mim train mmdet S4M/configs/S4M/endoscapes_majmin.py \
  --work-dir ./work_dir/balloon_train \
  --cfg-options \
    load_from=endoscapes_majmin.pth \
    resume=False \
    train_dataloader.batch_size=2 \
    train_dataloader.dataset.data_root=data/balloon \
    train_dataloader.dataset.ann_file=train/annotation_coco_rle.json \
    train_dataloader.dataset.data_prefix.img=train \
    train_dataloader.dataset.metainfo.classes="('background','balloon')" \
    val_dataloader.dataset.data_root=data/balloon \
    val_dataloader.dataset.ann_file=val/annotation_coco_rle.json \
    val_dataloader.dataset.data_prefix.img=val \
    val_dataloader.dataset.metainfo.classes="('background','balloon')" \
    test_dataloader.dataset.data_root=data/balloon \
    test_dataloader.dataset.ann_file=val/annotation_coco_rle.json \
    test_dataloader.dataset.data_prefix.img=val \
    test_dataloader.dataset.metainfo.classes="('background','balloon')" \
    val_evaluator.0.ann_file=data/balloon/val/annotation_coco_rle.json \
    val_evaluator.1.ann_file=data/balloon/val/annotation_coco_rle.json \
    test_evaluator.0.ann_file=data/balloon/val/annotation_coco_rle.json \
    test_evaluator.1.ann_file=data/balloon/val/annotation_coco_rle.json \
    optim_wrapper.optimizer.lr=1e-5 \
    train_cfg.max_iters=200 \
    train_cfg.val_interval=100 \
    param_scheduler.0.end=100 \
    param_scheduler.1.end=200 \
    param_scheduler.1.milestones="[100,150]"

Let's test the model!

CKPT=$(cat work_dir/balloon_train/last_checkpoint)

mim test mmdet S4M/configs/S4M/endoscapes_majmin.py \
  --checkpoint "$CKPT" \
  --cfg-options \
    test_dataloader.dataset.data_root=data/balloon \
    test_dataloader.dataset.ann_file=val/annotation_coco_rle.json \
    test_dataloader.dataset.data_prefix.img=val \
    test_dataloader.dataset.metainfo.classes="('background','balloon')" \
    test_evaluator.0.ann_file=data/balloon/val/annotation_coco_rle.json \
    test_evaluator.1.ann_file=data/balloon/val/annotation_coco_rle.json \
    model.num_mask_refinements=0 \
  --work-dir ./work_dir/balloon_test \
  --show-dir ./show_dir

Citation

If you find our work helpful for your research, please consider starring the repository ⭐ and citing our paper using the following BibTeX entry:

@article{meyer2026s4m,
  title={S4M: 4-points to segment anything},
  author={Meyer, Adrien and Arboit, Lorenzo and Massimiani, Giuseppe and Yin, Shih-Min and Mutter, Didier and Padoy, Nicolas},
  journal={International Journal of Computer Assisted Radiology and Surgery},
  pages={1--9},
  year={2026},
  publisher={Springer}
}

---