Validation errors are not shown correctly during autograding #1933
Open
Description
nbgrader_version=0.9.3 and older
When autograding a notebook with metadata errors, the error messages are not helpful and/or misleading.
Here is an example notebook with a duplicate grade_id
:
from nbformat.v4 import new_notebook, new_markdown_cell
from nbgrader.preprocessors import CheckCellMetadata
from nbgrader.nbgraderformat import MetadataValidator
nb = new_notebook()
cell = new_markdown_cell(
metadata=dict(
nbgrader=dict(
grade_id="my_id",
grade=False,
solution=False,
locked=True,
schema_version=3
)
)
)
nb.cells.append(
cell
)
nb.cells.append(
cell
)
Now when the MetadataValidator
is used to validate the notebook there is a descriptive error message that gives the correct information about the error:
MetadataValidator().validate_nb(nb)
Output:
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
---> 27 MetadataValidator().validate_nb(nb)
File /opt/conda/lib/python3.11/site-packages/nbgrader/nbgraderformat/v3.py:110, in MetadataValidatorV3.validate_nb(self, nb)
108 grade_id = cell.metadata['nbgrader']['grade_id']
109 if grade_id in ids:
--> 110 raise ValidationError("Duplicate grade id: {}".format(grade_id))
111 ids.add(grade_id)
ValidationError: Duplicate grade id: my_id
When the CheckCellMetadata
preprocessor is used the error is undescriptive:
CheckCellMetadata().preprocess(nb, {})
Output:
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
---> 28 CheckCellMetadata().preprocess(nb, {})
File /opt/conda/lib/python3.11/site-packages/nbgrader/preprocessors/checkcellmetadata.py:18, in CheckCellMetadata.preprocess(self, nb, resources)
16 msg = "Notebook failed to validate; the nbgrader metadata may be corrupted."
17 self.log.error(msg)
---> 18 raise ValidationError(msg)
20 return nb, resources
ValidationError: Notebook failed to validate; the nbgrader metadata may be corrupted.
When autograding a notebook like this I actually get this misleading error originating from here: https://github.com/jupyter/nbgrader/blob/main/nbgrader/converters/base.py#L421
AutogradeApp | ERROR] One or more notebooks in the assignment use an old version
of the nbgrader metadata format. Please **back up your class files
directory** and then update the metadata using:
nbgrader update .
This might be related to #1629.
Metadata
Assignees
Labels
No labels