Open
Description
For 405B the sampling parameter config sets the max output tokens to be 20k.
However, given the reference output distribution with max output length of 1.7k, I don't think we should set this parameter in the sampler that high.
@nvzhihanj @arjunsuresh @mrmhodak
Metadata
Assignees
Labels
No labels