Request for More Flexible RCF Implementation (Variable Image Sizes & Multiple Kernel Sizes)

[cullen-molitor]

I would like to request two enhancements to the RCF class in torchgeo.models:

1. Flexible Handling of Image Sizes in Empirical Mode

   • Currently, RCF with mode="empirical" requires images to be resized to a uniform height and width. The RCF process effectively collapses spatial information so maintaining a fixed image shape is not necessary. It would be great if the class could natively handle varying image shapes!
   • This was previously discussed here
   • Suggested code edits below

2. Ability to Specify Multiple Kernel Sizes

   • Right now, the constructor accepts only a single kernel_size (e.g., 3), and all features are generated using that one size. I would like the ability to pass two kernel sizes, where the model automatically splits the desired number of features. For example, if I request 4,000 features and kernel_size=[3, 6], it could allocate the first 2,000 features to use kernel_size=3 and the next 2,000 to use kernel_size=6.
   • This is how the publicly available MOSAIKS features are extracted
   • This one might be more complicated to implement but I wanted to put it out there in case there was interest!

Suggested edits for point 1 (untested, but based on Calebs previous response):

if mode == 'empirical':
    assert dataset is not None
    num_channels, height, width = dataset[0]['image'].shape
    assert num_channels == in_channels
    patches = np.zeros(
        (num_patches, num_channels, kernel_size, kernel_size), dtype=np.float32
    )
    idxs = torch.randint(
        0, len(dataset), (num_patches,), generator=generator
    ).numpy()
    

    for i in range(num_patches):
        img = dataset[idxs[i]]["image"]
        c, h, w = img.shape

        # Make sure the kernel fits within the current image
        if h < kernel_size or w < kernel_size:
            raise ValueError(
                f"Image at index {idxs[i].item()} is too small for kernel size {kernel_size}."
            )

        # Randomly choose a valid top-left corner for the patch
        y = torch.randint(0, h - kernel_size + 1, (1,), generator=generator).item()
        x = torch.randint(0, w - kernel_size + 1, (1,), generator=generator).item()

        patches[i] = img[:, y : y + kernel_size, x : x + kernel_size]

    patches = self._normalize(patches)
    self.weights = torch.tensor(patches)

Thanks!

[adamjstewart]

If you want to open a PR I'm sure @calebrob6 would be happy to review!