Sat-SlideMix Augmentation

[isaaccorley]

Adds Sat-SlideMix augmentation from the paper Data Augmentation Approaches for Satellite Imagery

I've made this into a Kornia augmentation so that it will work with both images and masks. This has some nice benefits like automatically sampling from a range of percentages to shift and random sampling which dim and direction to shift.

Example usage:

import requests
import torch
import torchvision.transforms.functional as TF
from torchgeo.transforms import SatSlideMix
import kornia.augmentation as K
from PIL import Image
from IPython.display import display


image_url = "https://project.inria.fr/aerialimagelabeling/files/2011/12/kit1.jpg"
mask_url = "https://project.inria.fr/aerialimagelabeling/files/2011/12/kit2.jpg"
image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB").resize((512, 512))
mask = Image.open(requests.get(mask_url, stream=True).raw).convert("L").resize((512, 512))
image = TF.pil_to_tensor(image).unsqueeze(0)
mask = TF.pil_to_tensor(mask).unsqueeze(0)
print(image.shape, mask.shape)

aug = K.AugmentationSequential(SatSlideMix(gamma=2, beta=(0.0, 1.0), p=1.0), data_keys=None)

batch = dict(image=image.clone().repeat(2, 1, 1, 1)/255.0, mask=mask.clone().repeat(2, 1, 1, 1)/255.0)
print(batch["image"].shape, batch["mask"].shape)  # image shape: (2, 3, 512, 512)  mask shape: (2, 1, 512, 512)
batch = aug(batch)
print(batch["image"].shape, batch["mask"].shape)  # image shape: (4, 3, 512, 512)  mask shape: (4, 1, 512, 512)

for i in range(batch["image"].shape[0]):
    display(TF.to_pil_image(batch["image"][i]))
    display(TF.to_pil_image(batch["mask"][i]))

Sample 1

Sample 2

Sample 3

Sample 4

[Copilot]

Pull Request Overview

This PR adds a new augmentation method, Sat-SlideMix, which circularly shifts satellite images to generate additional training samples as described in the referenced paper.

• Introduces the sat_slidemix function in augmentations.py
• Implements input validation and image rolling for augmentation

[adamjstewart]

Can we implement this in a way that is compatible with Kornia?

[isaaccorley]

Can we implement this in a way that is compatible with Kornia?

Working on it

[burakekim]

Do we refer to the paper in the docstring?

[isaaccorley]

Do we refer to the paper in the docstring?

Not yes, still converting it to work with kornia images and masks. Will add it though.

[adamjstewart]

I'm trusting you that the implementation is correct, only reviewing for typos.

[isaaccorley]

cc @burakekim

[adamjstewart]

I assume the inverse transform works too?

[isaaccorley]

I assume the inverse transform works too?

I have not implemented this. Would need to think about this a bit more tbh. I'd prefer to leave that for another PR in the event that someone actually needs the inverse transform.