-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Open
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation
Description
1. 遇到问题的章节 / Affected Chapter
2.1.6多头注意力
2. 具体问题描述 / Problem Description
矩阵内积是什么呢?应该是矩阵乘法么? 向量才有内积,矩阵应该是乘法吧
3. 问题重现材料 / Reproduction Materials
2.1.6多头注意力:
但上述实现时空复杂度均较高,我们可以通过矩阵运算巧妙地实现并行的多头计算,其核心逻辑在于使用三个组合矩阵来代替了n个参数矩阵的组合,也就是矩阵内积再拼接其实等同于拼接矩阵再内积。具体实现可以参考下列代码:
确认事项 / Verification
- 此问题未在过往Issue中被报告过 / This issue hasn't been reported before
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation