Skip to content

在训练的stage2,请问代码是如何将attention冻住的 #1727

Description

@liu-zongxi

根据文章描述,在训练好BEiT后,将冻住VFFN和attention,只训练LFFN,但好像在您提供的代码里我没有发现在进行textonlymlm的时候有冻住任何层,可以请教一下是在代码哪里实现的吗

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions