Question on left–right ambiguity in nnU-Net for small symmetric structures (pituitary region)

[mubeeyax]

Dear Fabian,

I hope this message finds you well. I wanted to start by saying how much I admire your work on nnU-Net—it has been an incredibly powerful and enabling framework for someone like me coming from a medical background with a growing interest in AI-based segmentation. I am very grateful for the openness and robustness of the method.

I am reaching out to ask for your insight regarding a challenge I am encountering with left–right discrimination when segmenting small, symmetric anatomical structures using nnU-Net.

My project focuses on MRI-based segmentation of pituitary adenoma (pituitary gland tumour at the base of the brain) and surrounding structures, including the carotid arteries and cavernous sinus. I noticed that nnU-Net, when trained in the standard way, struggles to consistently distinguish left from right for these particular structures—even after disabling mirroring during training and disabling test-time augmentation (TTA) during inference.

Some colleagues mentioned that simply turning off mirroring worked well for them, but in their cases the target structures were much larger. In my dataset, the carotid arteries and cavernous sinus are relatively small and highly symmetric, and this seems to make the problem more pronounced.

The approach that appears to work for us is adding an extra input channel encoding a left–right coordinate (essentially a normalized spatial coordinate). With this additional channel, the left/right confusion seems to be resolved quite reliably. My current understanding—please correct me if this is misguided—is that because nnU-Net uses patch-based training, the network often only “sees” small local regions at a time. For small, symmetric structures, these local patches may not contain enough global spatial context to infer whether a structure is left or right, even when mirroring is disabled.

At the moment, our working solution is:

• no mirroring during training,

• no TTA during inference,

• and an additional left–right coordinate channel as a form of preprocessing.

This seems to solve the issue, but before fully committing to this approach (and explaining it in a publication), I wanted to ask whether:

1. You or others in the nnU-Net community have encountered similar left–right ambiguity for small symmetric structures.

2. There is a more “canonical” way within nnU-Net to handle this, such as recommended preprocessing, configuration choices, or even post-processing strategies that I may have overlooked.

I should also mention—so you have the right expectations—that I am not a computer science expert by training. My background is clinical/medical, and while I am learning as much as I can, I would deeply appreciate any explanation in relatively lay terms if possible.

Thank you very much for your time and for creating such an impactful tool. Any guidance or thoughts you might be willing to share would mean a great deal to me.

With sincere appreciation,
Mubaraq Yakubu