How can I make a repeated parser depend on the previous amount of repetitions?

[Lilith-In-Starlight]

I'm making a markdown parser. I want to make a parser for titled sections, where the title is a header # Like This

I want the headers to be structured such that a # Section will fail to parse if we are currently parsing a ## Section body, but a ### Section will succeed.

So, I have a "markdown body" parser. This parser contains a "section" parser, which contains another "markdown body" parser, but what that second body parser can do depends on what the section parser has done

I have absolutely no idea how to make this work with chumsky

[zesterer]

You can use .repeated() in combination with .exactly(_) for this:

let title = ... ;
let h = |n| just('#').repeated().exactly(n).then(title);

let h1 = h(1); // # Section
let h2 = h(2); // ## Section
let h3 = h(3); // ### Section

Then you could combine them like so:

let section3 = h3.then(paragraph);

// A h2 header, then any number of section3s or a paragraph
let section2 = h2.then(choice((section3, paragraph)).repeated().collect());

// A h1 header, then any number of section2s or a paragraph
let section1 = h1.then(choice((section2, paragraph)).repeated().collect());

I guess this all depends on how much structure you want to enforce at the level of the parser.

[Lilith-In-Starlight]

This way of doing it is a little bit inconvenient, since, for example, I'd like to allow section2 to be the first level, or to have a section 4 directly inside a section 2. It seems like this would require a lot of repetition. Is there no other way?

I've also got a bunch of other elements which can be inside paragraph (like html tags) and those should respect section hierarchy as well.

[zesterer]

Ah, I see what you mean.

I think this all really depends on exactly how much structure you want to be enforcing in the parser. You could go for a loose design that just generates a vector of sections, or you could go stricter and enforce some sort of hierarchy.

I usually start off by defining my AST types and then figuring out how to write a parser that emits it (the structure of the parser will generally follow that of the AST). Perhaps if you could show what AST you're looking to produce, it might be possible to give a little more assistance here.

[Lilith-In-Starlight]

The relevant types are:

• Html type, whose body contains a vec of Ast
• Section type, whose title and body contain a vec of Ast, also contains depth such that a ### will be parsed with a depth of 3
• Text, a unit struct for a &'a str
• Ast, an enum containing all of these

A Section with depth 3 should not be able to contain a tag that contains a section with depth 2. This should not parse at all

A section with depth 3 immediately before a section with depth 2, separated only by text and well-formed html, should parse as two separate sections

A section with depth 3 immediately before a section with depth 4, separated only by text and well-formed html, should parse as a section containing another section

[Lilith-In-Starlight]

I feel like this should be doable with parsers that contain state or context but I have no idea how that would work

[Lilith-In-Starlight]

I find it important that the hierarchy of sections is present in the Ast, because it's reflected in the output Html (each section is in a <section> tag, instead of only using <h1> and <h2>)

[muja]

@Lilith-In-Starlight wouldn't it be possible to put this in a linting pass directly after parsing? it's definitely a matter of preference but your case may get convoluted quickly and parsers explode they get big, increasing compile times (among other issues)

you can probably do it but you're also losing the ability to react accordingly and flexibly in case there is section / header mismatch (e.g. the parsing step does not allow for warnings, while your custom pass may)

[Lilith-In-Starlight]

It is not possible, because sections don't have delimiters. Once a section starts, the only thing that may end it is a section failing to parse inside it but not outside of it

[muja]

I see what you mean but couldn't you collect them into a Vec<Section> where the section level is in a property? Then you could create the hierarchy after parsing (and optionally filter for valid child elements)