Skip to content

ACER raises an error when GaussianHeadWithFixedCovariance is used #144

New issue
New issue
Open
#145
Open
ACER raises an error when GaussianHeadWithFixedCovariance is used#144
#145
Assignees
muupan
Labels
bugSomething isn't working
@muupan

Description

@muupan
muupan
opened on May 26, 2021

Reported in #143

ACER assumes that all the parameters of a distribution (defined by get_params_of_distribution) require grad so that the algorithm can compute the gradient wrt the parameters.

pfrl/pfrl/agents/acer.py

Lines 172 to 180 in 44bf2e4

def get_params_of_distribution(distrib):
if isinstance(distrib, torch.distributions.Independent):
return get_params_of_distribution(distrib.base_dist)
elif isinstance(distrib, torch.distributions.Categorical):
return (distrib._param,)
elif isinstance(distrib, torch.distributions.Normal):
return distrib.loc, distrib.scale
else:
raise NotImplementedError("{} is not supported by ACER".format(type(distrib)))

pfrl/pfrl/agents/acer.py

Lines 218 to 221 in 44bf2e4

distrib_params = get_params_of_distribution(distrib)
for param in distrib_params:
assert param.shape[0] == 1
assert param.requires_grad

However, GaussianHeadWithFixedCovariance (

pfrl/pfrl/policies/gaussian_policy.py

Line 96 in 44bf2e4

class GaussianHeadWithFixedCovariance(nn.Module):
) is used, the scale parameter of the torch.distributions.Normal distribution does not require grad, resulting in an assertion error.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions