Summary
In the C runtime (src/runtime/c, master branch), the parser fails on words whose lexical sequence has a case-variant twin in the sequence table — deterministically, per word. With the German WordNet grammar (ParseGer, compiled with the master-branch gf):
parse -cat=N "Schule" -> Unexpected token: "Schule" (also: Brot, Stadt)
parse -cat=N "Birne" -> pear_1_N (also: Haus, ...)
lookupMorpho("Schule") returns all the school_*_N senses and l -table (UseN school_1_N) shows a perfectly normal inflection table — only parse fails, and it fails instantly: no lexeme is ever predicted at that position.
Tested at master 1c086bed2 on Debian 12 (WSL2), grammar compiled with the main-branch compiler, reproduced through both the Python binding (Concr.parse) and pgf_parse directly.
Root cause
pgf_parsing_lookahead (src/runtime/c/pgf/parser.c) is a "find all prefixes" binary search over concr->sequences that fills state->lexicon_idx (the bottom-up lexical predictions). It assumes a strict total order: on an exact match (cmp == 0) it registers exactly one sequence and recurses with length bounds (max = len-1 on the left, min = len+1 on the right) that exclude any other sequence of the same length.
With case-insensitive matching, a capitalized lexical sequence and its lowercase twin (KS "Schule" / KS "schule", adjacent in the sorted table) compare equal — a two-element tie. Only one of the twins is registered, and which one the bisection lands on is pure array-index parity. German noun lemmas are indexed on the capitalized sequence. For words where the search happens to land on the lowercase sequence — this is the case for Schule, Brot and Stadt — the noun is never predicted, so the word cannot be parsed; for others (Birne, Haus) the search lands on the capitalized sequence and everything works.
The behavior is input-independent: a given word always fails or always works, which makes it look like a lexicon gap rather than a parser bug.
Fix
On an exact match, register the entire contiguous run of sequences that are equal under the comparator (scan left/right from the match while pgf_symbols_cmp == 0), then recurse strictly outside that run. ~40 lines, no behavior change for unambiguous matches; lookupMorpho and linearization are untouched. Verified against an unmodified master build (before/after): all affected words parse correctly afterwards, existing parses unchanged.
Summary
In the C runtime (
src/runtime/c, master branch), the parser fails on words whose lexical sequence has a case-variant twin in the sequence table — deterministically, per word. With the German WordNet grammar (ParseGer, compiled with the master-branchgf):lookupMorpho("Schule")returns all theschool_*_Nsenses andl -table (UseN school_1_N)shows a perfectly normal inflection table — onlyparsefails, and it fails instantly: no lexeme is ever predicted at that position.Tested at master
1c086bed2on Debian 12 (WSL2), grammar compiled with the main-branch compiler, reproduced through both the Python binding (Concr.parse) andpgf_parsedirectly.Root cause
pgf_parsing_lookahead(src/runtime/c/pgf/parser.c) is a "find all prefixes" binary search overconcr->sequencesthat fillsstate->lexicon_idx(the bottom-up lexical predictions). It assumes a strict total order: on an exact match (cmp == 0) it registers exactly one sequence and recurses with length bounds (max = len-1on the left,min = len+1on the right) that exclude any other sequence of the same length.With case-insensitive matching, a capitalized lexical sequence and its lowercase twin (
KS "Schule"/KS "schule", adjacent in the sorted table) compare equal — a two-element tie. Only one of the twins is registered, and which one the bisection lands on is pure array-index parity. German noun lemmas are indexed on the capitalized sequence. For words where the search happens to land on the lowercase sequence — this is the case for Schule, Brot and Stadt — the noun is never predicted, so the word cannot be parsed; for others (Birne, Haus) the search lands on the capitalized sequence and everything works.The behavior is input-independent: a given word always fails or always works, which makes it look like a lexicon gap rather than a parser bug.
Fix
On an exact match, register the entire contiguous run of sequences that are equal under the comparator (scan left/right from the match while
pgf_symbols_cmp == 0), then recurse strictly outside that run. ~40 lines, no behavior change for unambiguous matches;lookupMorphoand linearization are untouched. Verified against an unmodified master build (before/after): all affected words parse correctly afterwards, existing parses unchanged.