|
Can someone provide a detailed technical explanation of this concept in deep learning and transformer systems? |
Answered by
Ultron09
Aug 27, 2026
Replies: 1 comment
|
Repetition penalty scales down the logits of tokens that have already appeared in the generated context: logit_i = logit_i / theta (if logit_i > 0) else logit_i * theta (for theta > 1). This discourages degenerate repetitive loops in open-ended autoregressive decoding. |
0 replies
Answer selected by
Ultron09
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Repetition penalty scales down the logits of tokens that have already appeared in the generated context: logit_i = logit_i / theta (if logit_i > 0) else logit_i * theta (for theta > 1). This discourages degenerate repetitive loops in open-ended autoregressive decoding.