Skip to content

Bug report: shape of pre_attn #3

@Elliot-QxZhang

Description

@Elliot-QxZhang

Thank you for your great work. However, when I tried to train on TCGA-BRCA, I got an error:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (131072x1024 and 768x768)

After debugging, I believe the error is in the mmssl.py (line 28)

pre_params = {
                "input_dim": self.patch_embedding_dim,
                "hidden_dim": self.patch_embedding_dim,
}

I believe the input_dim should be set to 1024 instead of the same as hidden_dim 768. I reset it and it worked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions