Code

Tokenization notebook#173

Code

Merged

maxjakob merged 9 commits intoelastic:mainelastic/elasticsearch-labs:mainfrom maxjakob:tokenization-nbmaxjakob/elasticsearch-labs:tokenization-nbCopy head branch name to clipboardJan 26, 2024

+330

Commits

Commits on Jan 25, 2024

• Tokenization notebook

  maxjakob

  committed
  0ca0e68

  Copy full SHA for 0ca0e68

• remove comment

  maxjakob

  committed
  3158306

  Copy full SHA for 3158306

• exclude special tokens before decoding

  maxjakob

  committed
  e24a7b5

  Copy full SHA for e24a7b5

• improve descriptions

  maxjakob

  committed
  41741c3

  Copy full SHA for 41741c3

• add overlap; remove text file

  maxjakob

  committed
  f22766a

  Copy full SHA for f22766a

Commits on Jan 26, 2024

• run nbtest through Makefile

  maxjakob

  committed
  4f6dfaa

  Copy full SHA for 4f6dfaa

• reformulate recommendation

  maxjakob

  committed
  c956be6

  Copy full SHA for c956be6

• remove paragraph I wanted to omit

  maxjakob

  committed
  5c8313b

  Copy full SHA for 5c8313b

• move to document-chunking folder

  maxjakob

  committed
  f53bbbf

  Copy full SHA for f53bbbf