DOC: Cannot upgrade to Sphinx 4 and later

[neutrinoceros]

Bug report

Bug summary

In #4409, we did some experimentations around upgrading sphinx, and found that the docs build broke starting with version 4.x.
See #4409 (comment)

The reason seem to be that https://github.com/ngoldbaum/RunNotebook is not compatible with modern versions of sphinx. Since it is unmaintained, we should consider migrating to https://github.com/spatialaudio/nbsphinx/

[jzuhone]

+1 to switching to nbsphinx

[matthewturk]

If we are going to migrate, I would suggest we consider migrating to something that is a bit more forward compatible -- like MyST. It has an 'onboard from ReST' tool and it would make the integration of notebooks, etc, much easier. (Up to and including running examples inline either via pyodide or thebe.) I'd be happy to pursue this, particularly as I've been working on it for the yt-4.0 paper as well.

[neutrinoceros]

Seems cool. I for one have no strong opinions on this but I'd welcome any initiative to migrate docs from rst to markdown if it can reliably be automated.

[matthewturk]

I'll take it on

[neutrinoceros]

This has become more urgent now that we have dropped Python 3.8 and are running 3.10 on the server that builds documentation (Sphinx 4 is not compatible with Python 3.10).
#4663 is aimed at fixing this.

[Xarthisius]

Just to update wrt #4663 and things that still have to happen:

1. remove all invocations of notebook:: (mostly done by DOC: updating documentation to use Sphinx-7.x #4663)
2. replace notebook-cell:: with something else.