- Include a map of rule name to ruleId in the module.
- Implicit space skipping
- Error handling
- NonterminalNodes should keep track of the rule
- When iteration contains a sequence, the children are flattened into the iter node.
- Basic parameterized rules
- Parameterized rules with >3 params
- Parameters that aren't terminals
- Memoization for parameterized rules
- Support direct left recursion.
- Separate API for creating the Wasm module from the WasmMatcher interface.
- Implement a proper CLI.
Cleanups:
- Handle left recursion detection at grammar parse time.
- Handle non-memoization of inline rules at grammar parse time
- Move to a failureOffset in memo entries
- Add assertions for any known input size limitations.
Optimizations:
- Avoid unnecessary dispatch in generalized rules
- Avoid duplicate lifted rules.
- Compressed (32-bit) header for Nonterminal nodes in common case
- Compressed (inline 32-bit) repr for Terminal nodes
- Proper preallocated nodes (incl. failurePos) for common cases
- The input is assumed to be no bigger than 64k.
- For the memo table, we assume that there are no more than 256 rules in the grammar.
- How to deal with matchLength in lookahead. In regular Ohm, lookahead does bind things. But that is hard to square with the current CST representation, that stores only the matchLength. Because somehow the things inside a lookahead must consume nothing — but if you have
&("a" "b"), the only way to make them consume nothing (in the current representation) is to rewrite the matchLength of the two terminal nodes.- Could we introduce a pseudo-node for lookahead? It could get transparently unpacked when walking the tree.