RuntimeError: Cannot understand pandocversion=3.1.8

[greyltc]

Not too sure what's going on here. Something important seems to have died with RuntimeError: Cannot understand pandocversion=3.1.8

[user@box wikmd]$ virtualenv venv
created virtual environment CPython3.11.8.final.0-64 in 175ms
  creator CPython3Posix(dest=/tmp/wikmd/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/grey/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.0.2, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[user@box wikmd]$ source venv/bin/activate
(venv) [user@box wikmd]$ python -m pip install .
Processing /tmp/wikmd
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting Flask==3.0.2 (from wikmd==1.9.0)
  Using cached flask-3.0.2-py3-none-any.whl.metadata (3.6 kB)
Collecting GitPython==3.1.42 (from wikmd==1.9.0)
  Using cached GitPython-3.1.42-py3-none-any.whl.metadata (12 kB)
Collecting Markdown==3.5.2 (from wikmd==1.9.0)
  Using cached Markdown-3.5.2-py3-none-any.whl.metadata (7.0 kB)
Collecting PyYAML==6.0.1 (from wikmd==1.9.0)
  Using cached PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting Werkzeug==3.0.1 (from wikmd==1.9.0)
  Using cached werkzeug-3.0.1-py3-none-any.whl.metadata (4.1 kB)
Collecting Whoosh==2.7.4 (from wikmd==1.9.0)
  Using cached Whoosh-2.7.4-py2.py3-none-any.whl.metadata (3.1 kB)
Collecting beautifulsoup4==4.12.3 (from wikmd==1.9.0)
  Using cached beautifulsoup4-4.12.3-py3-none-any.whl.metadata (3.8 kB)
Collecting pandoc-eqnos==2.5.0 (from wikmd==1.9.0)
  Using cached pandoc_eqnos-2.5.0-py3-none-any.whl.metadata (757 bytes)
Collecting pandoc-fignos==2.4.0 (from wikmd==1.9.0)
  Using cached pandoc_fignos-2.4.0-py3-none-any.whl.metadata (754 bytes)
Collecting pandoc-secnos==2.2.2 (from wikmd==1.9.0)
  Using cached pandoc_secnos-2.2.2-py3-none-any.whl.metadata (757 bytes)
Collecting pandoc-tablenos==2.3.0 (from wikmd==1.9.0)
  Using cached pandoc_tablenos-2.3.0-py3-none-any.whl.metadata (757 bytes)
Collecting pandoc-xnos==2.5.0 (from wikmd==1.9.0)
  Using cached pandoc_xnos-2.5.0-py3-none-any.whl.metadata (729 bytes)
Collecting pandocfilters==1.5.1 (from wikmd==1.9.0)
  Using cached pandocfilters-1.5.1-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting pypandoc==1.13 (from wikmd==1.9.0)
  Using cached pypandoc-1.13-py3-none-any.whl.metadata (16 kB)
Collecting requests==2.31.0 (from wikmd==1.9.0)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting lxml==5.1.0 (from wikmd==1.9.0)
  Using cached lxml-5.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.5 kB)
Collecting watchdog==2.1.9 (from wikmd==1.9.0)
  Using cached watchdog-2.1.9-py3-none-manylinux2014_x86_64.whl.metadata (33 kB)
Collecting cachelib==0.12.0 (from wikmd==1.9.0)
  Using cached cachelib-0.12.0-py3-none-any.whl.metadata (2.0 kB)
Collecting soupsieve>1.2 (from beautifulsoup4==4.12.3->wikmd==1.9.0)
  Using cached soupsieve-2.5-py3-none-any.whl.metadata (4.7 kB)
Collecting Jinja2>=3.1.2 (from Flask==3.0.2->wikmd==1.9.0)
  Using cached Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
Collecting itsdangerous>=2.1.2 (from Flask==3.0.2->wikmd==1.9.0)
  Using cached itsdangerous-2.2.0-py3-none-any.whl.metadata (1.9 kB)
Collecting click>=8.1.3 (from Flask==3.0.2->wikmd==1.9.0)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting blinker>=1.6.2 (from Flask==3.0.2->wikmd==1.9.0)
  Using cached blinker-1.7.0-py3-none-any.whl.metadata (1.9 kB)
Collecting gitdb<5,>=4.0.1 (from GitPython==3.1.42->wikmd==1.9.0)
  Using cached gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)
Collecting psutil<6,>=4.1.0 (from pandoc-xnos==2.5.0->wikmd==1.9.0)
  Using cached psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Collecting charset-normalizer<4,>=2 (from requests==2.31.0->wikmd==1.9.0)
  Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests==2.31.0->wikmd==1.9.0)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests==2.31.0->wikmd==1.9.0)
  Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests==2.31.0->wikmd==1.9.0)
  Using cached certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from Werkzeug==3.0.1->wikmd==1.9.0)
  Using cached MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython==3.1.42->wikmd==1.9.0)
  Using cached smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)
Using cached beautifulsoup4-4.12.3-py3-none-any.whl (147 kB)
Using cached cachelib-0.12.0-py3-none-any.whl (20 kB)
Using cached flask-3.0.2-py3-none-any.whl (101 kB)
Using cached GitPython-3.1.42-py3-none-any.whl (195 kB)
Using cached lxml-5.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.1 MB)
Using cached Markdown-3.5.2-py3-none-any.whl (103 kB)
Using cached pandoc_eqnos-2.5.0-py3-none-any.whl (20 kB)
Using cached pandoc_fignos-2.4.0-py3-none-any.whl (21 kB)
Using cached pandoc_secnos-2.2.2-py3-none-any.whl (18 kB)
Using cached pandoc_tablenos-2.3.0-py3-none-any.whl (21 kB)
Using cached pandoc_xnos-2.5.0-py3-none-any.whl (31 kB)
Using cached pandocfilters-1.5.1-py2.py3-none-any.whl (8.7 kB)
Using cached pypandoc-1.13-py3-none-any.whl (21 kB)
Using cached PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (757 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached watchdog-2.1.9-py3-none-manylinux2014_x86_64.whl (78 kB)
Using cached werkzeug-3.0.1-py3-none-any.whl (226 kB)
Using cached Whoosh-2.7.4-py2.py3-none-any.whl (468 kB)
Using cached blinker-1.7.0-py3-none-any.whl (13 kB)
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached gitdb-4.0.11-py3-none-any.whl (62 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Using cached Jinja2-3.1.3-py3-none-any.whl (133 kB)
Using cached MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Using cached psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB)
Using cached soupsieve-2.5-py3-none-any.whl (36 kB)
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Using cached smmap-5.0.1-py3-none-any.whl (24 kB)
Building wheels for collected packages: wikmd
  Building wheel for wikmd (pyproject.toml) ... done
  Created wheel for wikmd: filename=wikmd-1.9.0-py3-none-any.whl size=9599313 sha256=971b3e4f389c7a4048c042ddde3d3bd237b921027dd3eb64b363ce3c892ef1c1
  Stored in directory: /tmp/pip-ephem-wheel-cache-itruuwiq/wheels/25/7e/55/0d93f15975185eab2cb6daae3577ae99f9022925c637ca040a
Successfully built wikmd
Installing collected packages: Whoosh, watchdog, urllib3, soupsieve, smmap, PyYAML, pypandoc, psutil, pandocfilters, MarkupSafe, Markdown, lxml, itsdangerous, idna, click, charset-normalizer, certifi, cachelib, blinker, Werkzeug, requests, pandoc-xnos, Jinja2, gitdb, beautifulsoup4, pandoc-tablenos, pandoc-secnos, pandoc-fignos, pandoc-eqnos, GitPython, Flask, wikmd
Successfully installed Flask-3.0.2 GitPython-3.1.42 Jinja2-3.1.3 Markdown-3.5.2 MarkupSafe-2.1.5 PyYAML-6.0.1 Werkzeug-3.0.1 Whoosh-2.7.4 beautifulsoup4-4.12.3 blinker-1.7.0 cachelib-0.12.0 certifi-2024.2.2 charset-normalizer-3.3.2 click-8.1.7 gitdb-4.0.11 idna-3.7 itsdangerous-2.2.0 lxml-5.1.0 pandoc-eqnos-2.5.0 pandoc-fignos-2.4.0 pandoc-secnos-2.2.2 pandoc-tablenos-2.3.0 pandoc-xnos-2.5.0 pandocfilters-1.5.1 psutil-5.9.8 pypandoc-1.13 requests-2.31.0 smmap-5.0.1 soupsieve-2.5 urllib3-2.2.1 watchdog-2.1.9 wikmd-1.9.0
(venv) [user@box wikmd]$ python -m wikmd.wiki
DrawIO integration
Alerts system
Mermaid integration
embed-pages
[2024-04-26 15:14:34,710] INFO in image_manager: Checking if webp is available for image optimization ...
1.4.0
libsharpyuv: 0.4.0
WebP Encoder version: 1.4.0
WebP Mux version: 1.4.0
[2024-04-26 15:14:34,728] INFO in wiki: Starting Wikmd with wiki directory /tmp/wikmd/wiki
[2024-04-26 15:14:34,729] INFO in wiki: Wiki directory doesn't exists, copy template
[2024-04-26 15:14:34,729] INFO in wiki: Creating upload folder >>> wiki/img
[2024-04-26 15:14:34,731] INFO in git_manager: Creating a new local repo >>> wiki ...
[2024-04-26 15:14:34,738] INFO in git_manager: Creating 'main' branch ...
[2024-04-26 15:14:34,755] INFO in git_manager: New git commit >>> First init commit
[2024-04-26 15:14:34,762] INFO in wiki: Search index creation...
[2024-04-26 15:14:34,825] INFO in wiki: Spawning search indexer watchdog
 * Serving Flask app 'wiki'
 * Debug mode: on
[2024-04-26 15:14:44,097] INFO in wiki: Showing HTML page >>> 'homepage'
[2024-04-26 15:14:44,098] INFO in wiki: Converting to HTML with pandoc >>> 'homepage' ...
[2024-04-26 15:14:45,202] ERROR in wiki: Conversion to HTML failed >>> Pandoc died with exitcode "83" during conversion: Traceback (most recent call last):
  File "/tmp/wikmd/venv/bin/pandoc-xnos", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/tmp/wikmd/venv/lib/python3.11/site-packages/pandocxnos/main.py", line 38, in main
    m.main(stdin, stdout)
  File "/tmp/wikmd/venv/lib/python3.11/site-packages/pandoc_fignos.py", line 608, in main
    PANDOCVERSION = pandocxnos.init(args.pandocversion, doc)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/wikmd/venv/lib/python3.11/site-packages/pandocxnos/core.py", line 224, in init
    _PANDOCVERSION = _get_pandoc_version(pandocversion, doc)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/wikmd/venv/lib/python3.11/site-packages/pandocxnos/core.py", line 179, in _get_pandoc_version
    raise RuntimeError(msg)
RuntimeError: Cannot understand pandocversion=3.1.8
Error running filter pandoc-xnos:
Filter returned error status 1

Could it be that this project depends on an outdated version of pandoc?

[Linbreux]

Hi @greyltc

pandoc-xnos seems to not be in active development anymore. There is a solution. So you could give that a try. It's also possible to remove pandoc-xnos filters out of your wiki.py in case you don't need it.
here:

wikmd/src/wikmd/wiki.py

Line 309 in 33e1c0b

filters=['pandoc-xnos'])

and here:

wikmd/src/wikmd/wiki.py

Line 197 in 33e1c0b

format='md', extra_args=["--mathjax"], filters=['pandoc-xnos'])

An other option would be to downgrade pandoc to version 2...

Maybe we could find an alternative or we could write this functionality as our own plugin.

[marriotb]

The patch that you've identified above from the pandoc-xnos page works for me (Python 3.12 on MacOS 14.5, installed with pip).

Within your virtual environment, run the command:

python -m pip install --force-reinstall git+https://github.com/tomduck/pandoc-xnos@284474574f51888be75603e7d1df667a0890504d#egg=pandoc-xnos

Perhaps it could be added to the installation instructions until a more thorough fix is available.
B.

[johann-petrak]

Could the usage of an abandoned package which does not work with current pandoc versions not get removed from the code?

[Linbreux]

@johann-petrak, yes that would indeed be better. But we need a replacement for it or write our own plugin. Until then, I'd like to keep it

[johann-petrak]

ok, but I have to admit I do not understand why it should be kept ... removing this from the code until there is an alternate solution is just a small fix and would prevent all new users for having to 1) encounter the problem and either give up immediately or 2) figure out what is going on and 3) manually edit the source code (something not many non-developer people would want to do)
I already did this so it does not even impact me any more, but I think having this in the code will discourage quite a few people to continue using wikmd.

[Linbreux]

@johann-petrak How did you run it? since we integrated pypandoc-binary this issue should not happen anymore.

This way it should work:
https://linbreux.github.io/wikmd/installation/regular_install.html

[johann-petrak]

I did:

• clone the repo
• create a conda env
• pip install -e .
• run from command line using "wikmd"

Encountered the error, removed the mentions of pandoc-xnos from the wiki.py file and after that it worked.

[Linbreux]

Did you already have pandoc version 3 installed? Version 2 should still work

[johann-petrak]

Yes, there was/is a system-wide installation of pandoc version 3.1.3

[bijusk]

My confusion stemmed from the Windows-Section of the installation page - because it says pandoc should be installed, so I specifically installed it. This way I got the wrong version and had to manually deinstall it before wikimd worked for me.
Maybe it is enough to clear things up in the installations-page.

[Linbreux]

@bijusk Good that you got it up and running! Could you create a PR with the current installation method on windows? I don't actively develop on Windows, nor do I have access to a Windows machine. 😄