Uncaught exception in get_normalizer_issues

[DavidKorczynski]

Hello,

We have been trying to fuzz parso to help catch any uncaught exceptions that may occur. We found that the following program generates an uncaught exception:

import sys
import atheris
import parso

def TestOneInput(data):
  fdp = atheris.FuzzedDataProvider(data)
  try:
    grammar = parso.load_grammar()
    module = parso.parse(fdp.ConsumeUnicodeNoSurrogates(sys.maxsize))
    if module != None:
        grammar._get_normalizer_issues(module)
  except RecursionError:
    # Not interesting
    pass

data = (b"\xff\x5c\x0a\x23")
TestOneInput(data)

The Atheris module used is https://pypi.org/project/atheris/ FuzzedDataProvider is used to convert the raw bytes in data into appropriate types.

If we write out the value generated by fdp.ConsumeUnicodeNoSurrogates in the above example, we can create the following program to trigger the issue:

import parso

grammar = parso.load_grammar()
module = parso.parse("\\\n#")
if module is not None:
  grammar._get_normalizer_issues(module

The traceback is:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 42, in visit
    children = node.children
AttributeError: 'EndMarker' object has no attribute 'children'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./reproducer.py", line 6, in <module>
    grammar._get_normalizer_issues(module)
  File "/usr/local/lib/python3.8/site-packages/parso/grammar.py", line 203, in _get_normalizer_issues
    normalizer.walk(node)
  File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 36, in walk
    value = self.visit(node)
  File "/usr/local/lib/python3.8/site-packages/parso/python/errors.py", line 408, in visit
    return super().visit(node)
  File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 47, in visit
    return ''.join(self.visit(child) for child in children)
  File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 47, in <genexpr>
    return ''.join(self.visit(child) for child in children)
  File "/usr/local/lib/python3.8/site-packages/parso/python/errors.py", line 408, in visit
    return super().visit(node)
  File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 44, in visit
    return self.visit_leaf(node)
  File "/usr/local/lib/python3.8/site-packages/parso/python/pep8.py", line 350, in visit_leaf
    self._visit_part(part, part.create_spacing_part(), leaf)
  File "/usr/local/lib/python3.8/site-packages/parso/python/pep8.py", line 441, in _visit_part
    if len(indentation) > len(n.indentation):
AttributeError: 'NoneType' object has no attribute 'indentation'

This was found by way of OSS-Fuzz and the set up here: https://github.com/google/oss-fuzz/tree/master/projects/parso If you find this issue helpful then it would be great to have maintainer emails in the project.yaml to receive notifications of bug reports, which contain all of the details similar to what I posted above.