You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 18, 2024. It is now read-only.
Check that you are up-to-date with the master branch of keras-preprocessing. You can update with: pip install git+git://github.com/keras-team/keras-preprocessing.git --upgrade --no-deps
Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).
Describe the problem.
from tensorflow.keras.preprocessing.text import Tokenizer
text = ['abc def']
tokenizer = Tokenizer(char_level=True, split='')
tokenizer.fit_on_texts(text)
sequence = tokenizer.texts_to_sequences(text)
text_after = tokenizer.sequences_to_texts(sequence)
print(text_after)
>>> ['a b c d e f']
notice that text_after and text are different, additional SPACEs are inserted
Describe the expected behavior.
text_after should be same as text
I believe this line is where the problem is, replacing: