Thanks for publishing this quite interesting work. I believe it is customary to use either bias or BatchNorm, but not both at once. the nn.Conv2d layers not have bias=False added to their constructor args, as otherwise they would default to bias=True?