deduplicate and order suppression tokens in apply not add#1923

rjames-0 wants to merge 1 commit intoOpenNMT:masterOpenNMT/CTranslate2:masterfrom

corticph:batch-with-token-suppressioncorticph/CTranslate2:batch-with-token-suppressionCopy head branch name to clipboard