feature: Raise `PluginError` on expected failed code execution

[ianhi]

Is your feature request related to a problem? Please describe.

I would like to use the docs build execution as a way to check that the code in the docs is staying up to date with the underlying python library. For this to work the docs build must fail when the code raises an exception. That is, a code block containing raise ValueError should stop the docs build with a non-zero returncode.

Describe the solution you'd like

When a code block raises a value error unexpectedly (i.e. not indicated by the returncode option) throw a PluginError

I think this would amount to (maybe optionally) adding a line to this block:

markdown-exec/src/markdown_exec/formatters/base.py

Lines 145 to 155 in 36db197

	except ExecutionError as error:
	identifier = id or extra.get("title", "")
	identifier = identifier and f"'{identifier}' "
	exit_message = "errors" if error.returncode is None else f"unexpected code {error.returncode}"
	log_message = (
	f"Execution of {language} code block {identifier}exited with {exit_message}\n\n"
	f"Code block is:\n\n{_format_log_details(source_input)}\n\n"
	f"Output is:\n\n{_format_log_details(str(error), strip_fences=True)}\n"
	)
	logger.warning(log_message)
	return markdown.convert(str(error))

Describe alternatives you've considered

I don't think there is an alternative method for this plugin.

Additional context

I think this could be configurable, even opt in. And it should still be possible to have errors, in the same way the shell code works.

I'm happy to make a PR implementing this if that's helpful

[ianhi]

Confusingly raising a pluginerror is not stopping the mkdocs build as the docs seem to imply it should be doing 🤔 .

 except ExecutionError as error:
        identifier = id or extra.get("title", "")
        identifier = identifier and f"'{identifier}' "
        exit_message = "errors" if error.returncode is None else f"unexpected code {error.returncode}"
        log_message = (
            f"Execution of {language} code block {identifier}exited with {exit_message}\n\n"
            f"Code block is:\n\n{_format_log_details(source_input)}\n\n"
            f"Output is:\n\n{_format_log_details(str(error), strip_fences=True)}\n"
        )

# added lines below
        print('Definitely here')
        raise PluginError(log_message)

will execute the print, but the PluginError doesn't seem to go anywhere or stop the execution.

[pawamoy]

Hi @ianhi, thanks for the request.

Aren't strict builds enough? In strict mode, a Python code block raising an exception will make the build fail.

[pawamoy]

Plugin errors are only understood by MkDocs. Here execution happens during Markdown conversion, in between MkDocs hooks.

[ianhi]

Thanks for the quick response @pawamoy ! Indeed strict mode gets what I want, but It's not quite what I need, because then my build fails on warnings from other packages as well. Which I should of course fix those, but it makes me liable to failures based on warnings that may not be the most important in the future.

[pawamoy]

I see, thanks. Then we could consider adding an option to emit error logs instead of warnings in case of error (exception) 🙂 This way you wouldn't need strict mode to have your builds fail.

[ianhi]

Ah yes, that seems ideal!