Skip to content

Commit c0cb1a2

Browse files
committed
Add comment about tokenizer functions to apiignore.json5
1 parent cbaa01c commit c0cb1a2

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

apiignore.json5

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@
2121
"pdb.alias_out_safe",
2222
"pdb.alias_recv",
2323
"pdb.alias_send",
24+
25+
// These functions are implementation details of tokenizers that the ORM
26+
// doesn't need to interact with
2427
"pdb.chinese_compatible_in",
2528
"pdb.chinese_compatible_out",
2629
"pdb.chinese_compatible_recv",

0 commit comments

Comments
 (0)