diff --git a/notebooks/search/tokenization.ipynb b/notebooks/search/tokenization.ipynb index 95fb92b0..809f1b60 100644 --- a/notebooks/search/tokenization.ipynb +++ b/notebooks/search/tokenization.ipynb @@ -201,7 +201,6 @@ "source": [ "We can observe:\n", "- There are special tokens `[CLS]` and `[SEP]` to model the the beginning and end of the text. These two extra tokens will become relevant below.\n", - "- All tokens are lower-cased.\n", "- Punctuations are they own tokens.\n", "- Compounds words are split into two tokens, for example `hitmen` becomes `hit` and `##men`.\n", "\n",