Skip to content

group_sparsity regularization results in an error with different conv types #147

@mohammadbashiri

Description

@mohammadbashiri

The group_sparsity regularizer of the Stacked2dCore the convolution layers are referred to as conv, but that does not generalize to other conv layer types like ds_conv for instance.

def group_sparsity(self):
"""
Sparsity regularization on the filters of all the conv2d layers except the first one.
"""
ret = 0
for feature in self.features[1:]:
ret = ret + feature.conv.weight.pow(2).sum(3, keepdim=True).sum(2, keepdim=True).sqrt().mean()
return ret / ((self.num_layers - 1) if self.num_layers > 1 else 1)
def regularizer(self):
return self.group_sparsity() * self.gamma_hidden + self.gamma_input * self.laplace()

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions