Search results

An explanation on why ALiBi works

Status: Open.

#22 In ofirpress/attention_with_linear_biases;

· lucaskup opened on Nov 12, 2025

Using Alibi with pytorch transformerEncoder with casual mask and padding mask

questionFurther information is requestedFurther information is requested

Status: Open.

#21 In ofirpress/attention_with_linear_biases;

· anikamurarka opened on Sep 24, 2025

implementation detail about alibi_mask

questionFurther information is requestedFurther information is requested

Status: Open.

#18 In ofirpress/attention_with_linear_biases;

· bugm opened on Nov 29, 2023

For a value of 12 I'm seeing a jump in the plotting of values, e.g. 0.7 shown below.

Status: Open.

#17 In ofirpress/attention_with_linear_biases;

· razodactyl opened on Aug 16, 2023

Modifying ALiBi for Encoder-Attention or Cross-Attention

questionFurther information is requestedFurther information is requested

Status: Open.

#5 In ofirpress/attention_with_linear_biases;

· ofirpress opened on Nov 18, 2021