Some images are stuck in DBSCAN

[xiaoyao9184]

watermark-anything/notebooks/inference_utils.py

Line 150 in 45d56c2

db = DBSCAN(eps=epsilon, min_samples=min_samples).fit(valid_pred)

Conducting multi-region scans on single-region watermarked images can be quite slow, which is understandable because separation is not possible.

However, some images seem to cause the DBSCAN algorithm to get stuck indefinitely. like this

Since I can't understand why the DBSCAN algorithm is getting stuck, I vaguely noticed that high-resolution images seem not to have this issue. Perhaps I should adjust the epsilon and min_samples parameters based on the image size like this scikit-learn/scikit-learn#17650 ?

[pierrefdz]

Hi, sorry for the delay of answer.

• Would you be able to send your code?
• I have the same issue when trying to extract one message from more than 40% of image pixels. I think it comes from the fact that DBSCAN clustering must run on too many pixels. The issue is more complicated than I thought and may be tricky to solve. I'll try to have a look at it in the following days. If you find something please tell me :)

[xiaoyao9184]

This is a Jupyter Notebook file, to be placed in the notebooks folder for execution.

%cd ..
!curl https://raw.githubusercontent.com/xiaoyao9184/docker-wam/refs/heads/main/label/test_images/image.webp > assets/images/image.webp

import os
import torch
import torch.nn.functional as F

from PIL import Image
from notebooks.inference_utils import (
    load_model_from_checkpoint,
    default_transform,
    multiwm_dbscan
)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model from the specified checkpoint
exp_dir = "checkpoints"
json_path = os.path.join(exp_dir, "params.json")
ckpt_path = os.path.join(exp_dir, 'wam_mit.pth') 
wam = load_model_from_checkpoint(json_path, ckpt_path).to(device).eval()

# Load and preprocess the image
img = Image.open('assets/images/image.webp').convert("RGB")
img_pt = default_transform(img).unsqueeze(0).to(device)

# Detect the watermark in the multi-watermarked image
preds = wam.detect(img_pt)["preds"]  # [1, 33, 256, 256]
mask_preds = F.sigmoid(preds[:, 0, :, :])  # [1, 256, 256], predicted mask
mask_preds_res = F.interpolate(mask_preds.unsqueeze(1), size=(img_pt.shape[-2], img_pt.shape[-1]), mode="bilinear", align_corners=False)  # [1, 1, H, W]
bit_preds = preds[:, 1:, :, :]  # [1, 32, 256, 256], predicted bits

# DBSCAN parameters for detection
epsilon = 1 # min distance between decoded messages in a cluster
min_samples = 500 # min number of pixels in a 256x256 image to form a cluster

# stuck here
centroids, positions = multiwm_dbscan(bit_preds, mask_preds, epsilon = epsilon, min_samples = min_samples)