Add a dedicated padding token to beam search to avoid padding with the start sentence token #10386
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e start sentence token
LoicDagnas
requested review from
jaeyounkim,
lehougoogle,
rachellj218 and
saberkun
as code owners
saberkun
requested review from
frederick0329
and removed request for
jaeyounkim,
rachellj218 and
lehougoogle
|
Conflicts need to resolve. |
saberkun
requested review from
ByzanTine
and removed request for
saberkun and
frederick0329
Chen-Zhe
approved these changes
Mar 5, 2024
Chen-Zhe
reviewed
Mar 5, 2024
| @@ -409,7 +412,7 @@ def _create_initial_state(self, initial_ids, initial_cache, batch_size): | |||
| alive_seq = expand_to_beam_size(initial_ids, self.beam_size) | |||
| alive_seq = tf.expand_dims(alive_seq, axis=2) | |||
| if self.padded_decode: | |||
| alive_seq = tf.tile(alive_seq, [1, 1, self.max_decode_length + 1]) | |||
| alive_seq = tf.pad(alive_seq, [[0, 0], [0, 0], [0, self.max_decode_length]], constant_values=self.pad_id) | |||
There was a problem hiding this comment.
We need to remove the extra spaces in front of this line to conform to our Python styling
There was a problem hiding this comment.
I think need to resolve the conflict as well, will create a new PR with all above changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
So far, the beam search module pads its predictions using the initial token ids, it is more modular and easy to use to have the capacity to specify the padding token. By default this padding token should be zero and not the initial token id.
Type of change
For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.
Note: Please delete options that are not relevant.
Tests
I have added a naive unittest which failed before my modification and passes now.
Checklist