Augmentation behavior intended?

[manuelknott]

Hi,

eraser_transform erases patches in the right image only. The valid mask for those patches is not set to 0 which is fine because only the left image GT gets supervised. However, eraser_transform is called before spatial_transform which includes 50% chance of horizontal flip where left and right images are swapped. Thus, erased patches can end up in the left image.

In my view, validity masks should be 0 for erased patches in the left image.

I would like to clarify if this is the intended behavior and if so, what was the idea behind this.

Thank you!

[lahavlipson]

Great point: this augmentation is not designed to emulate the single-image occlusions that show up in stereo matching; rather, we borrow it as-is from single-image algorithms, since our model also probably uses monocular cues

[manuelknott]

Thank you for your reply. I want to make sure I fully understand the topic, so I have a few follow-up questions:
I understood that the augmentation was originally designed for mono-depth, but it is now applied in a stereo setup: randomly on either the left or right image.

Was your intention to apply the augmentation only to the right image, so that the model relies on monocular cues to predict the left disparity? I think this is a great way to use the augmentation in this context.

If that’s the case, would you consider it a bug that the augmentation also appears on the left image (without validity masking) due to the random horizontal flip? In that scenario, the model would need to predict the left disparity solely from monocular cues in the right image - which is an unintuitive task in my view. But maybe I am missing something here.