Add and standardize a control sequence for non-rendering lines in code blocks

[tgross35]

Is your feature request related to a problem? Please describe.
There are handful of tools that help with the formatting and checking of .. code-block:: python, but sometimes context is needed to make those work. My idea is to add a simple character sequence to indicate that specific lines of code should not be rendered.

The sphinx doctest module allows for some things like this, but requires code that is quite verbose and only supports pycon style. The proposal here would be simpler to use, easier to read, and more terse without breaking anything.

Describe the solution you'd like

I would propose using a control character like ##!. The only thing necessary from sphinx would be to hide comment lines starting with those characters. Essentially just ensure that this:

.. code-block:: python

    ##! from typing import Optional, TYPE_CHECKING
    ##! if TYPE_CHECKING:
    ##! from mymodule.something import CoolClass
    def  fn(x: Optional[int]) -> CoolClass:
        return CoolClass(x)
    ##! assert fn(5) == CoolClass(5)

Displays identical to

.. code-block:: python

    def  fn(x: Optional[int]) -> CoolClass:
        return CoolClass(x)

Some example use cases and benefits:

• Using asserts, like shown, to validate code. A super simple tool could strip the ##! from every line and just run the code. This sort of thing is doable with doctest, but this way is simpler and allow for python (not pycon) formatting.
• The ##! could be stripped and the code passed to a linter (flake8/pylint) or static type checker (mypy)
• A code formatter could take this code block as-is (unstripped) and format it, or optionally strip and re-add the ##! if it's smart enough
• Nothing RST is broken, syntax highlighters will still display properly
• If using an outdated version of sphinx that doesn't properly hide the comments, it's not the end of the world and things will still compile
• Super easy for sphinx to implement, just remove lines that .startswith('##!') from the rendering. Could potentially add a conf.py flag to disable it (I'd propose being enabled by default)
• Code execution can be easily verified and copied by users since it's python style. With doctest, either the users copy the whole thing (with sphinx-copybutton) or can copy a few lines where the >>> and ... need to be removed manually - this avoids that problem. any part of the code is copyable

Describe alternatives you've considered

Pycon-style doctests, but that is not a good solution for many cases as mentioned above. https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

Additional context

The way this works is inspired by rustdoc https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html#hiding-portions-of-the-example

For control characters - it has to start with # to be valid python, ## is pretty commonly used in comments so that's no good, #! is used by shebang, so ##! it is. Happy medium that perfectly hints it's an executable comment.

Thanks! I'd be happy to submit a PR if this gets positive feedback.

[AA-Turner]

Hi Trevor,

I don't really understand the motivating examples. How would having comments in the code-block allow you to run the asserts?

A

[tgross35]

Hi Adam,

it wouldn’t run exactly as written - the goal is simply so that any tool could strip those characters and then be left with 100% valid Python, and a working example.

The ##! would just be a directive to sphinx to not show those lines - that’s all that sphinx would need to do. Then a very thin wrapper (plug-in or external tool) could delete those characters (not lines) and use the remaining code as valid Python.

This is similar to the current way doctest work. But instead of using testsetup/testcleanup, code that would usually go into testsetup/testcleanup just goes into the same block as the displayed code, but gets marked with ##!

I’ll write an example script that illustrates what I mean

[tgross35]

Ok, say you have the following somewhere in your .rst:

.. code-block:: python 

    ##! from __future__ import annotations
    ##! from typing import Optional
    class ColorRed:
        color = 'RED'
        
        def colorize(self, x: Optional[str]) -> ColorRed:
            if x.lower() == 'red':
                return ColorRed()

    ##! assert ColorRed().colorize('RED').color == 'RED'

And the RST parser correctly pulls the code block into python:

code_block_raw= """##! from __future__ import annotations
##! from typing import Optional

class ColorRed:
    color = 'RED'
    
    def colorize(self, x: Optional[str]) -> ColorRed:
        if x.lower() == 'red':
            return ColorRed()

##! assert ColorRed().colorize('RED').color == 'RED'
"""

Sphinx part

Instead of sending that to rendering, sphinx just needs to run a quick modifier to remove those commented lines:

code_block_displayable = "\n".join(
    line for line in code_block_raw.splitlines() if not line.startswith('##!')
)

Result, to be rendered:

class ColorRed:
    color = 'RED'
    
    def colorize(self, x: Optional[str]) -> ColorRed:
        if x.lower() == 'red':
            return ColorRed()

Tool part

Now if you're a tool or plugin that wants access to fully runnable code, you can easily just remove the control sequence.

lines = []
for line in code_block_raw.splitlines():
    if line.startswith('##!'):
        lines.append(line.strip('##! '))
    else:
        lines.append(line)

code_block_runnable= "\n".join(lines)

and get the following result:

from __future__ import annotations
from typing import Optional
class ColorRed:
    color = 'RED'
    
    def colorize(self, x: Optional[str]) -> ColorRed:
        if x.lower() == 'red':
            return ColorRed()

assert ColorRed().colorize('RED').color == 'RED'

Then do whatever is applicable with it

# These functions are simplifications of the potential API
flake8(code_block_runnable) # verify code block lints
mypy(code_block_runnable) # validate types
exec(code_block_runnable) # verify the code executes without error
# Allow pytest runner to validate this code
pytest_doctest.register_test(f"def file_123_test():\n    {code_block_runnable}")
# Maybe a plugin that adds a play button
sphinx_runnable_code_plugin.register_popup(code_block_runnable)

[AA-Turner]

Ahh, I see. This would need to be implemented in external tools to be useful.

Firstly (minor) on spelling: tilde (~) has precedenct in Sphinx for "supress", so perhaps ##~ over ##!?

I am though wary of an xkcd 927 situation if we specify this magic prefix and then no-one uses it, or other tools have their own similar construct. If there is precedent in other lightweight markup formats (asciidoc, POD, JavaDoc, etc) for a specific style, it would be better to try and follow that rather than inventing something ourselves.

@chrisjsewell -- does markdown (commonmark?) have such a constuct that you know of?

A

[tgross35]

Completely understood about the support and all! One of the reasons I’m asking is because I recently became the maintainer of the (currently not working) flake8-rst package, and would kind of like to grow that into a useable standard.

I also does a bit of rust work, these easy nonprinting lines are something I miss the most from their (well thought out imo) docs standard - figured I might as well start at the top and see if there’s interest.

[AA-Turner]

flake8-rst

Have you seen sphinx-lint? I wonder if there's any crossover.

I think it's a useful idea, and we might even be able to use the same magic string for doctests, but I am hesitant to add something we'll have to support basically forever if it turns out that everyone else uses some different formulation for the same problem.

A

[tgross35]

I was not familiar with sphinx-lint, but it seems like maybe that’s meant to validate the entire rst file? Rather than checking for validity of the code blocks within it

[tk0miya]

You can modify the content of code blocks via your own extension. So it's better to implement such a filter as an extension. I think it's difficult to make a standard syntax and keep it maintained. I suppose we'll need to invent new standards for other languages if we introduce this feature to Sphinx.

[AA-Turner]

So it's better to implement such a filter as an extension.

I think Trevor's question is if Sphinx will as the core application bless such a syntax. The risk of leaving it to extensions is that two extensions use two different conflicting magic strings, and you are then out of options in your documentation. I share your concerns on doing it right, but if it is done at all I think it should be in the core.

A