Markdown extensions to support Math, Tables, etc. not working

[mgc8]

Checklist

• I am running the latest version. Installing Isso from GitHub from the master branch does not fix my issue
• I have checked the troubleshooting guide
• I have searched the open issues, but my issue has not already been reported

What is not working?

The documentation here makes note of a number of extensions that can be added to the default Markdown editor, such as "math", "footnotes", "highlight", "tables" etc. However, even after adding them to the configuration, there does not seem to be any change, and no errors in the logs either.

How can one reproduce this issue?

Isso configuration file contains the following [markup] section:

[markup]
options = autolink, fenced-code, no-intra-emphasis, strikethrough, superscript, math, highlight, tables, underline
flags = hard-wrap
allowed-elements =
allowed-attributes =

The "default" ones -- "autolink, fenced-code, no-intra-emphasis, strikethrough, superscript" work correctly and as expected, adding and removing them from the line produces the expected results. However, "math, highlight, tables, underline" produce no effect whatsoever. A demo can be seen on this page, the last comment:
https://mihnea.net/hello-world/#isso-11

Is there something I'm missing from the installation? The Misaka library appears to be present under isso/lib/python3.9/site-packages/misaka and is at version 2.1.1.

[ix5]

Hi, thanks for reporting your issue.

Could you please provide a working example of your config, the text you typed in and the rendered text you expected to see?

(Related: #751)

[mgc8]

Hi @ix5, thank you for looking into this.

Here is the configuration in use:

[general]
dbpath = <path>/example.com-comments.db
name = example.com
host =
    http://example.com/
    https://example.com/
max-age = 15m
notify = smtp
reply-notifications = true
log-file = <path-to-isso>/sites/example.com.log
gravatar = true
gravatar-url = https://www.gravatar.com/avatar/{}?d=monsterid&s=55&r=x
latest-enabled = false

[admin]
enabled = false
password = please_choose_a_strong_password

[moderation]
enabled = true
approve-if-email-previously-approved = false
purge-after = 30d

[server]
listen = http://localhost:1234
public-endpoint = https://example.com/isso
reload = false
profile = false
trusted-proxies = 127.0.0.1
samesite = Strict

[smtp]
username =
password =
host = localhost
port = 25
security = none
to = webmaster
from = "example.com Comments" <webmaster@example.com>
timeout = 10

[guard]
enabled = true
ratelimit = 2
direct-reply = 3
reply-to-self = false
require-author = true
require-email = true

[markup]
options = autolink, fenced-code, no-intra-emphasis, strikethrough, superscript, math, highlight, tables, underline
flags = hard-wrap
allowed-elements =
allowed-attributes =

[hash]
salt = <salt>
algorithm = pbkdf2

[rss]
base =
limit = 100

You can see the examples in the link I posted, but sure I can add them here as well for clarity:

1. This code in the comment _underline_ **bold** ~~strikethrough~~ *emphasis* ==highlight==
   => produces this HTML: underline <strong>bold</strong> <del>strikethrough</del> <em>emphasis</em> highlight

You can see that 'bold/strike/emphasis' have tags added, but the other two 'underline/highlight' don't, despite the extensions being specified in the config

2. The 'math' extension should parse LaTeX-style code, right? This does not do anything at all (I took the example from https://github.blog/2022-05-19-math-support-in-markdown/ )
   When $a \ne 0$, there are two solutions to $(ax^2 + bx + c = 0)$ and they are $$ x = {-b \pm \sqrt{b^2-4ac} \over 2a} $$

3. Finally, the 'tables' extension should parse code like the following, but doesn't appear to do anything either:

| Tables   |      Are      |  Cool |
|----------|:-------------:|------:|
| col 1 is |  left-aligned | $1600 |
| col 2 is |    centered   |   $12 |
| col 3 is | right-aligned |    $1 |

Hope this helps tracking things down, if I can provide more information just let me know.

[joseph-vidal-rosset]

Hello. Thanks for isso, that's a very nice comments engine.
@ix5 , I ask exactly the same question that was @mgc8's. Is math option works with isso? Do we need to install something more in order to get equations inside comments? (Same questions for tables and all options that does not work as it was remarked in this thread.) Thanks for your help.

[joseph-vidal-rosset]

Markdown on Github explains that it works ```math
For example
The Cauchy-Schwarz Inequality

$$\left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right)$$

but also with '$$' :

$$ \left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right) $$

Therefore it should works with isso too.

[ix5]

Hi @joseph-vidal-rosset, sorry for the late response. Isso's markdown parser is misaka, which needs some configuration to allow more advanced syntax.

As for LaTeX: You need something to actually render in the browser. misaka simply ignores/doesn't modify anything between $ / $$ and leaves it up to the browser to style it. This differs from GitHub who "magically" do the right thing for you.

https://docs.mathjax.org/en/v2.7-latest/start.html#mathjax-cdn suggests e.g.:

<script type="text/javascript" async
  src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/MathJax.js?config=TeX-MML-AM_CHTML">
</script>

We as the Isso project would do well to make the handling of markdown parsing more intuitive and automatic as well as align it with GHFM. I have tried documenting the current behavior here: docs: Configure how comments are rendered.

[ix5]

As for underline not working: You need to allow the <u> element in [markup] allowed-elements:

[markup]
options = [...], underline
[...]
allowed-elements = u

As for your table not displaying: That's unfortunately a similar issue, you need to allow the tr HTML tag in allowed-elements. Another contributor is working on simplifying this a bit, see #1007

We plan on doing the right thing for you when you enable a markdown extension automatically (e.g. allow all table-related elements when you use the tables extension).

[mgc8]

As for underline not working: You need to allow the <u> element in [markup] allowed-elements:
(...)
As for your table not displaying: That's unfortunately a similar issue, you need to allow the tr HTML tag in allowed-elements. Another contributor is working on simplifying this a bit, see #1007

Hello @ix5 , thank you for the additional information, we finally have a direction to explore on fixing this.

Using your suggestion, I was able to get "underline" to work via the "u" tag, but nothing else. "highlight" still does nothing (what would be the tag for that?), while the "table" also does absolutely nothing. Is this the correct syntax for the allowed-elements option?

allowed-elements = u, tr

I tried adding all the table-related tags there as well, with no difference. Really not sure why tr specifically had been blocked before, but in any case -- what else is needed to get this to work?

We plan on doing the right thing for you when you enable a markdown extension automatically (e.g. allow all table-related elements when you use the tables extension).

That would be great indeed, but until that's in place it would be great to be able to get these extensions working even manually, since right now the documentation you point to at docs: Configure how comments are rendered is a bit misleading since most extensions don't do anything...

[mgc8]

Hi @joseph-vidal-rosset, sorry for the late response. Isso's markdown parser is misaka, which needs some configuration to allow more advanced syntax.
As for LaTeX: You need something to actually render in the browser. misaka simply ignores/doesn't modify anything between $ / $$ and leaves it up to the browser to style it. This differs from GitHub who "magically" do the right thing for you.

Hmm, ok, but that's a third-party JavaScript library with no connection to misaka discussed here. Why is there a math extension specifically, if it does absolutely nothing? The documentation you linked to specifically says:
"Parse inline LaTeX-style math blocks, such as inline $equations$ or display $$equations$$ >> Extension name: math"

What kind of "parsing" is meant here?

I don't think it's a problem if this functionality is not actually supported, but in that case we should fix the documentation so that people don't waste time troubleshooting something that was never intended to work...

[ix5]

"highlight" still does nothing (what would be the tag for that?)

That would be the <mark> element

while the "table" also does absolutely nothing

For me, [markup] options = no-intra-emphasis, strikethrough, underline, highlight, tables, math with allowed-elements = u, tr, mark works with the following table:

| Tables   |      Are      |  Cool |
|----------|:-------------:|------:|
| col 1 is |  left-aligned | $1600 |
| col 2 is |    centered   |   $12 |
| col 3 is | right-aligned |    $1 |

As well as with _underline_ **bold** ~~strikethrough~~ *emphasis* ==highlight==

since right now the documentation you point to at docs: Configure how comments are rendered is a bit misleading since most extensions don't do anything...

Yes, the issue is known. We have to decide just how much we want to overhaul the whole system without breaking setups of existing users.

Hmm, ok, but that's a third-party JavaScript library with no connection to misaka discussed here. Why is there a math extension specifically, if it does absolutely nothing? The documentation you linked to specifically says:
"Parse inline LaTeX-style math blocks, such as inline $equations$ or display $$equations$$ >> Extension name: math"

That's due to the underlying misaka library choices: https://github.com/FSX/misaka/issues/71#issuecomment-516033877

we should fix the documentation so that people don't waste time troubleshooting something that was never intended to work...

Feel free to submit a PR

[mgc8]

Hi @ix5 ,

Thank you for the clarifications, this was very helpful to finally fix and understand the behaviour here!

"highlight" still does nothing (what would be the tag for that?)
That would be the <mark> element

Done, works fine, thanks!

while the "table" also does absolutely nothing
For me, [markup] options = no-intra-emphasis, strikethrough, underline, highlight, tables, math with allowed-elements = u, tr, mark works with the following table:
(...)

Interestingly, I had to track this for a bit. It seemed to work fine in a new comment, but not in the existing one(s). What eventually turned out to be the problem was that the rendering did not kick in if there was any text right next to the table, it would keep it as 'code' then. Adding a line break made it render fine!

As well as with _underline_ **bold** ~~strikethrough~~ *emphasis* ==highlight==

Yes, all of these work now after adding the missing tags under allowed-elements.

That's due to the underlying misaka library choices: FSX/misaka#71 (comment)

This was pretty unclear to me until spending some time to read the information in that comment. Basically, the math extension doesn't actually do anything, but it marks the LaTeX-style code so that it is skipped by misaka, allowing other (external, third-party, etc.) libraries to parse and render it later. Now I understand.

we should fix the documentation so that people don't waste time troubleshooting something that was never intended to work...

Feel free to submit a PR

Ok, done -- #1017 -- I've added the above few notes, hope it's ok!

[joseph-vidal-rosset]

I am sorry, but for me the situation stays the same: neither ==highlight== nor table nor math formula are rendered in isso comments...

[mgc8]

I am sorry, but for me the situation stays the same: neither ==highlight== nor table nor math formula are rendered in isso comments...

Maybe if you paste the contents of your [markup] section in the configuration file would help trace what's happening... You do have the allowed-elements = u, tr, mark line, right? And you've restarted uwsgi or the specific WSGI tool you use to run isso?