IndexError: index-1 is out of bounds for dimension 0 with size 0

[Wangxu930123]

When I did the neural style transfer experiment, I added SSN as normalization layers to the CovLayer and UpsampleCovLayer of TransformerNet, and trained the TransformerNet with COCO2014 train-set .When the iteration went to 1400 times, it stopped suddenly and appeared ” IndexError: index-1 is out of bounds for dimension 0 with size 0”. With IN, neither BN, GN nor SN encountered this problem. This same problem happened when I used SSN as normalization layers to train the ResNet-50 on Cifar-100 for image classification.
Have you ever met the above problem? Can you tell me how to solve it? Thank you very much.

[MengTianjian]

You can try to manually set the grad of fixed norms to None, it should solve this problem.