Switch to Docutils HTML5 writer instead of the HTML4 one

[qookei]

• I have searched the issues (including closed ones) and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.
• I am willing to lend a hand to help implement this feature.

Feature Request

Pelican has used the Docutils HTML4 writer for about 11 years at this point, and I think it's time to put it to rest. Since 0.13, Docutils includes an HTML5 writer called html5_polyglot, which generates (X)HTML5, with correct semantic tags for article sections, etc.

As far as I can tell at a glance, all that is needed is just swapping out the import. I have done so in a custom plugin, by subclassing pelican.readers.RstReader and changing the writer_class (along with the translator) and it appears to work just fine.

Note that this is a breaking change in some aspects. For example, themes that add styles for div.section will break as now those are replaced with actual section tags (they still have their ids though).

Also worth of note is that while issue #959 is similar, it's not related as at that time Docutils did not know about HTML5.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your participation and understanding.

[justinmayer]

I apologize for the delay in responding to this issue, which has been raised before in #2549. Given that some years have passed since that issue was closed, perhaps it is time to revisit this topic, particularly because according to the Docutils release notes, it seems html5 will soon become the default writer.

It occurs to me that maybe the Docutils writer could also be a configurable setting?

@qookei: Given the concerns raised in #2549 regarding whether Docutils produces valid HTML5, can you run the output through some validators and confirm that the current released version of Docutils can produce valid HTML5?

[justinmayer]

@qookei: Any thoughts on my comment above?

[qookei]

Oh apologies, I looked at your response on my phone, but forgot to respond later.

With the skeleton generated by pelican-quickstart, the following plugin to generate HTML5 output: https://gist.github.com/qookei/c302b95ff716257b424e877f3dceb502 (more or less a copy of the existing HTML4 code in Pelican), and a simple RST file, the W3C markup validator reports the following for the post HTML page:

This seems fine to me, and the reported warnings can be fixed by changing the theme's templates (HTML generated with the theme I wrote validates with no messages).

EDIT: The above warnings are also reported with the default HTML4 writer.
I also decided to run it through a larger RST file, and that doesn't show any extra warnings or errors either.

[ceyusa]

I've just change

--- a/pelican/readers.py
+++ b/pelican/readers.py
@@ -11,7 +11,7 @@ import docutils
 import docutils.core
 import docutils.io
 from docutils.parsers.rst.languages import get_language as get_docutils_lang
-from docutils.writers.html4css1 import HTMLTranslator, Writer
+from docutils.writers.html5_polyglot import HTMLTranslator, Writer
 
 from pelican import rstdirectives  # NOQA
 from pelican.cache import FileStampDataCacher

And my blog was built correctly (some random https://validator.w3.org/) obviously it needs a lot of more work got get it merged (a lot of tests need to be updated)

[justinmayer]

If someone would like to submit a pull request that implements the needed changes for a switch to the Docutils HTML5 writer, I would be happy to include it in a subsequent major Pelican release.

[nkr0]

@justinmayer PR #3432