-
Notifications
You must be signed in to change notification settings - Fork 48
Open
Description
The group_sparsity regularizer of the Stacked2dCore the convolution layers are referred to as conv, but that does not generalize to other conv layer types like ds_conv for instance.
neuralpredictors/neuralpredictors/layers/cores/conv2d.py
Lines 247 to 257 in 7f1b303
| def group_sparsity(self): | |
| """ | |
| Sparsity regularization on the filters of all the conv2d layers except the first one. | |
| """ | |
| ret = 0 | |
| for feature in self.features[1:]: | |
| ret = ret + feature.conv.weight.pow(2).sum(3, keepdim=True).sum(2, keepdim=True).sqrt().mean() | |
| return ret / ((self.num_layers - 1) if self.num_layers > 1 else 1) | |
| def regularizer(self): | |
| return self.group_sparsity() * self.gamma_hidden + self.gamma_input * self.laplace() |
Metadata
Metadata
Labels
No labels