Skip to content

Commit d2dd38b

Browse files
committed
Fix the parameterization for the 20b model
Signed-off-by: John St. John <jstjohn@nvidia.com>
1 parent 4709e95 commit d2dd38b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

bionemo-recipes/recipes/evo2_megatron/src/bionemo/evo2/models/evo2_provider.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -692,7 +692,7 @@ class Hyena20bModelProvider(HyenaModelProvider):
692692
rotary_base: int = 1_000_000
693693
seq_len_interpolation_factor: float = 128
694694
hyena_medium_conv_len: int = 128
695-
hyena_short_conv_len: int = 3
695+
hyena_short_conv_len: int = 7
696696
add_attn_proj_bias: bool = True
697697

698698

0 commit comments

Comments
 (0)