Skip to content

Commit d0ca80e

Browse files
committed
fix queries projection having bias
1 parent 9ac9d69 commit d0ca80e

2 files changed

Lines changed: 2 additions & 2 deletions

File tree

enformer_pytorch/finetune.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ def __init__(
150150
self.scale = dim_head ** -0.5
151151
self.heads = heads
152152
inner_dim = heads * dim_head
153-
self.to_queries = nn.Linear(enformer_hidden_dim, inner_dim)
153+
self.to_queries = nn.Linear(enformer_hidden_dim, inner_dim, bias = False)
154154

155155
self.null_key = nn.Parameter(torch.randn(inner_dim))
156156
self.null_value = nn.Parameter(torch.randn(inner_dim))

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
name = 'enformer-pytorch',
55
packages = find_packages(exclude=[]),
66
include_package_data = True,
7-
version = '0.1.18',
7+
version = '0.1.19',
88
license='MIT',
99
description = 'Enformer - Pytorch',
1010
author = 'Phil Wang',

0 commit comments

Comments
 (0)