Skip to content

Will radam be affacted by weight decay? #59

Open
@CoinCheung

Description

@CoinCheung

Hi,

It is said that naive adam will make performance bad if weight decay is added. Thus people invented adamW to make adam compatible with weight decay. Now I have a question, does radam work well if I use it together with weight decay ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions