Ergonomic way to enhance ToC within Markdown: insert TOC slice, exclude headings

[ISSOtm]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Description

Docusaurus currently allows manually altering (and even outright replacing) the ToC in Docs, as per #3915 (comment), but this is not documented.

This issue is as much a question on whether this is something we can expect to rely on, as a request to document it if the answer is "yes".

Has this been requested on Canny?

No response

Motivation

This is useful for documentation that is generated from other sources (in this case, a man page): while the HTML can be injected, the ToC does not follow suit (and I wouldn't expect it to. Or is that preferable?).

API design

Here is a doc which largely consists of externally-generated HTML, for which we additionally generate the ToC via a script.

# rgbds(5) — object file format documentation

import generated from '!!raw-loader!./rgbds.5.html';

<div className="manual-text" dangerouslySetInnerHTML={{ __html: generated }} />

export const toc = [
{
	"value": "DESCRIPTION",
	"id": "DESCRIPTION",
	"level": 2,
},
{
	"value": "FILE STRUCTURE",
	"id": "FILE_STRUCTURE",
	"level": 2,
},
{
	"value": "Header",
	"id": "Header",
	"level": 3,
},
{
	"value": "Source file info",
	"id": "Source_file_info",
	"level": 3,
},
{
	"value": "Symbols",
	"id": "Symbols",
	"level": 3,
},
{
	"value": "Sections",
	"id": "Sections",
	"level": 3,
},
{
	"value": "Assertions",
	"id": "Assertions",
	"level": 3,
},
{
	"value": "RPN EXPRESSIONS",
	"id": "RPN_EXPRESSIONS",
	"level": 3,
},
{
	"value": "SEE ALSO",
	"id": "SEE_ALSO",
	"level": 2,
},
{
	"value": "HISTORY",
	"id": "HISTORY",
	"level": 2,
},
];

Have you tried building it?

If the existing behavior is to be accepted as official, then nothing needs to be built; otherwise, what replacement API is deemed better will need to be discussed first.

Self-service

• I'd be willing to contribute this feature to Docusaurus myself.

[Josh-Cena]

Hi, since this feature is already in place, is it essentially a documentation request?

The TOC shape is most likely stable, and it's kind of documented here: https://docusaurus.io/docs/next/markdown-features/inline-toc#custom-table-of-contents However, up till now we haven't figured out an ergonomic way to let you use it. For example, #3915 is very likely to be solved by letting you code a TOC yourself.

[ISSOtm]

This is first a question about whether modifying the ToC like in the example is intended to be part of the API, or is just exposed internals.

If the answer is yes, then this is a documentation request. If it's no, this is a request for an alternative.

This is not intended to supersede #3915, as is explained in the comment that I originally linked to, though it can be used as a stopgap in the meantime.

[slorber]

This is first a question about whether modifying the ToC like in the example is intended to be part of the API, or is just exposed internals.

This is internal implementation detail that is IMHO stable enough to use externally.

We could document it and make it officially a public API, but it probably requires some upfront thinking to be sure that the current API is the best way to solve your use-case

[ISSOtm]

It fits exactly my use case, since this document has the entirety of its content auto-generated.

However, if only a section were to be as such, then I'd want to be able to splice the ToC accordingly, more similarly to #3915. I think this could be arranged by having a file—autogenned.md—use this mechanism to manually specify the entirety of its own ToC, and then including autogenned.md performing the splice like requested in that PR.

Hopefully I didn't sound too confused?

[slorber]

If we add this to our doc, we should probably dogfood this on our own doc, explaining the constraints (like elements requiring a unique id for linking to work). We could showcase manual construction of a toc for generated HTML (like your case).

We should probably add this to this page (to be renamed as just "TOC"?): https://docusaurus.io/docs/markdown-features/inline-toc

Do you want to submit a PR?

I'm not sure what you mean by autogenned.md but I'd rather avoid showcasing the "import mdx partial" case of #3915, as it's not really a feature but an actual bug/limitation that we really want to solve.

[Josh-Cena]

@slorber What about we invent a "TOC enhancement" syntax ourselves and parse that in the TOC generating remark plugin?

I don't have a clear plan yet, but it would be something like:

import Content, {partialToc} from './_partial.md';

# A Markdown page

## Actual Heading 1

[[insert toc]]
- Inserted Heading 1
- Inserted Heading 2
  - Subheading 1

## Actual Heading 2

<Content />

[[import toc: partialToc]]

Which will generate a TOC:

export const toc = [
  {title: 'Actual Heading 1'},
  {title: 'Inserted Heading 1'},
  {title: 'Inserted Heading 2', children: [{title: 'Subheading 1'}]},
  {title: 'Actual Heading 2'},
  ...partialToc,
];

However, importing partials are very difficult. We either have to actually read the imported file and parse that as well, or we just let the user import that and we spread it into the final TOC. In the former case, we need to actually keep track of the imported component => MD file path mapping and read & parse more files, and means sacrificed performance; in the latter case, it means we can't be sure how the partial TOC should be spread. For example, for the example above, if ./_partial.md actually looks like:

### Imported Subheading 1

### Imported Subheading 2

Then the final TOC should have the two "Imported Subheading"'s as children of "Actual Heading 2" instead of spreading it to the root of TOC.

In hindsight it may be better if we have started off designing our server-side TOC structure as a flat list instead of a recursive tree. It's going to be rendered on client-side as a list anyways and we hardly take advantage of the tree structure. For now we recursively render each TOC level, but we could well use tocItems.map(i => <TOCItem item={i} />)...

[slorber]

@Josh-Cena I'm not sure it's really related to this issue, as here the content is in HTML and not a mdx partial import.

---

@slorber What about we invent a "TOC enhancement" syntax ourselves and parse that in the TOC generating remark plugin?

That's something I thought about but do we really want to invent a syntax that will only serve temporarily? I'm already not a fan of inventing a non-std md syntax 😅

The end goal is that the TOC works for imported files automatically, without asking the user to use any new fancy syntax.

---

However, importing partials are very difficult. We either have to actually read the imported file and parse that as well

I don't think we should do that, it duplicates work to each doc importing a partial and also adds more weight to each page, as the shared toc ends up being duplicated in each doc

We'd rather favor composition and have a remark plugin that compose the toc from current doc and partials without inlining the partial tocs into the document, like your spread example, but handled automatically by the remark plugin, not hand-written by the user

In hindsight it may be better if we have started off designing our server-side TOC structure as a flat list instead of a recursive tree

Good point, it will probably be needed to flatten that structure

So we are maybe not ready to make this "manual toc" feature an official public API 😅

[Josh-Cena]

I'm proposing this custom TOC syntax in place of handwriting the entire TOC structure which contains a lot of boilerplate. If we have that, we don't need to document the export toc = ... syntax at all. Also, currently it's all or nothing: you write everything or you let Remark do it all, there's no progressive enhancement. Such an API is not ready to be documented, and we can't extend this API to allow enhancement either, we have to do it from scratch.

The end goal is that the TOC works for imported files automatically, without asking the user to use any new fancy syntax.

As I said, it means we have to "understand" that <Content /> is an MDX partial, not a JSX component, which isn't easy. I've tried a while back to implement a Remark plugin and call Babel to parse the JSX for me, but overall it's a very painful thing to let Remark understand JSX. It also means we have to insert an extra {toc} named import back into the import Content from './_partial.mdx' statement, and that can be intractable.

Then there's the question of (a) user wanting to hide some headings from the TOC and (b) user wanting to insert extra headings that would otherwise not be visible to Remark. We would need a way to let users handwrite & insert part of the TOC, hence the proposal for the [[Inserted TOC]] and [[Imported TOC]] syntax. Even if [[Imported TOC]] can be heavylifted by us, [[Inserted TOC]] is there to stay.

Glad I made the flattened list point through :P Going to see what we can do about it

[Josh-Cena]

This will be triaged as a feature request and we will figure out an ergonomic way to tweak the TOC structure.

Just for reference, VuePress has this [[toc]] syntax: https://v2.vuepress.vuejs.org/guide/markdown.html#table-of-contents It's not anywhere close to the functionality we are discussing here, but the syntax is similar

[ISSOtm]

Fine by me, then. I'll keep relying on toc until then.