Can We use Quarto Yet?

[zkamvar]

As of 2021-09-02, not quite because there is no native support for remote theming in the way we use {varnish}: quarto-dev/quarto-cli#103

Here are my thoughts at the moment (though they may evolve):

The strength of the new lesson infrastructure is the fact that all of our components are clearly separated, which means that it is not infeasable to consider multiple entry points to our lesson

Quarto is a drop-in replacement for RMarkdown for folks who just want to be able to write technical documents and present them without any bells and whistles (though, to be fair, there are a lot of bells and whistles). The problem for us is that we need a little more control of the output so that we can adapt it to our own templates.

[cderv]

If I may chime in, I believe Quarto will probably gain in the future some extension mechanism to create custom output formats (similarly to rmarkdown or bookdown). This could help you when this time come.

Your use case that we already discussed once would be a greate opportunity to do the right thing and add missing feature from R Markdown website that lead you to pkgdown. In my opinion...

[jjallaire]

I'd be surprised if we couldn't get something to work well for your use case even given current constraints. Could you let us know exactly the gaps and we can reason about how to bridge them?

[cderv]

Taking the opportunity to just putting here some notes I have from previous discussion on R Markdown website and this project:

• sandpaper uses pkgdown to create a website for the flexibility it provides regarding templating system. One can create a template package to use which contains default configuration + additional / new template + parameters to configure per project.

• varnish (https://github.com/carpentries/varnish) is the pkgdown template project that is used. It contains html template with placeholder filled by parameters in pkgdown.yml, + some styles specific to the project.

I also believe we could get something to work as Pandoc has all the feature need (templating mechanism) and Quarto brings some nice new features (theming using scss file). I was wondering if everything is there yet... 🤔

Happy to know more about the differences - Project may have evolved since our last talk. 😄

[zkamvar]

Thank you both for being so responsive! I definitely think that Quarto has a future with this infrastructure!

Chris is absolutely correct in how we are currently set up and I absolutely agree that we could set up pandoc templates for this (a la some hybrid between {rticles} and {pkgdown}).

I have more thoughts, but I am currently at the end of my day and out of steam. Be back on Tuesday!

[jjallaire]

Okay, great. Look forward to discussing further. I reviewed your requirements document (https://carpentries.org/blog/2020/08/lesson-template-design/) and am pretty sure you could pretty close to the ideal system you describe w/ Quarto today.

One thing you might not have seen (that addresses the dependencies problem mentioned in the doc) is the ability to freeze computational output (https://quarto.org/docs/projects/code-execution.html#freeze). The idea here would be that individual lessons can have whatever dependencies they need but that doesn't impose installation of those dependencies on anyone that wants to render the site. Consequently, you could have a site that uses R & Python (+ packages for both) but that can actually be rendered with only an installation of Quarto (no R or Python required b/c the computations are cached in the _freeze directory).

[zkamvar]

I have more thoughts, but I am currently at the end of my day and out of steam. Be back on Tuesday!

well, I guess, s/Tue/Wedne/.

In any case, I think we landed on a very similar implementation and I don't think it should be too hard for us to adapt in the future 😁

I really like how you've made it easy to freeze and redirect generated content and I'm really impressed at how easy it was to install. This is key for us because we are really focused on making the process of creating, contributing, and curating lessons straightforward for newcomers.

What I've developed so far is similar in the sense that it defaults to auto-freezing, but it differs because it follows a more {blogdown}-like approach where we cache the markdown files generated via knitr and have a text database recording the source, md5sum, and output (which would be similar to the output of quarto render --debug, I think). Because our lessons are developed by several contributors with their own unique setups (though we are giving the option to use {renv}), these files are not tracked by git locally. Instead, using the {pkgdown} method, they are added to an orphan branch on the remote Git server.

We use these sources to not only re-render websites weekly (to update styling), but also to give maintainers an easier time auditing what would change in a given pull request where packages update because markdown is easily diff-able (for a slightly contrived example, zkamvar/test-with-renv#4 updated a single package, which generated zkamvar/test-with-renv@acd8000).

That all being said, I'll be really glad for the day that I don't have to tell people to install a whole computer language (or four, as we have in our current template) in order to make a lesson on needlepoint.

[zkamvar]

(also, sorry for the long-winded response)

[jjallaire]

Yes, the _freeze dir sounds similar to what you store on the orphan branch. We store the md5sum and the markdown + other file output (and the source isn't tracked separately from what is checked in on the same branch).

Hopefully we can make quarto work well for you next generation of this infrastructure. Once you dig into it pls do let us know wha the gap are and hopefully we can close them.

[zkamvar]

In some initial testing, there are still some quirks that we must overcome in order to use quarto. Namely there is a clash between some of our sections and those provided by quarto:

https://quarto.org/docs/authoring/cross-references.html#theorems-and-proofs

This causes a problem with rendering our documents in quarto because as soon as we attempt to render any document that uses the ::: solution blocks, we get this error:

Error running filter /opt/quarto/share/filters/crossref/crossref.lua:
/opt/quarto/share/filters/crossref/crossref.lua:2009: attempt to get length of a nil value (field 'content')
stack traceback:
	/opt/quarto/share/filters/crossref/crossref.lua:1103: in function </opt/quarto/share/filters/crossref/crossref.lua:1097>

The reason this error happens is because the theorem() filter assumes that this "solution" is a solution to a proof/theorem/mathy thing instead of a dropdown solution in a Carpentries lesson and is trying to format the heading as the solution title and expects an explanatory paragraph so it would be perfectly fine to write something like:

::: solution

### Heading

text text

```
code
```

:::

This would produce a solution block that would look like this

::: {.solution .proof}
[*Heading*. ]{.proof-title}text text

```
code
```
:::

Which, as an aside, is not what we want...

But, because Carpentries lessons often have solutions that are just code examples, we end up with something that looks like this, which causes Line 119 to bork for a reason that I'm not entirely certain.

::: solution

### Heading

```
code
```

:::

[zkamvar]

The solution to #161 (comment) is to add a lua filter to modify solution blocks that acts before quarto filters

quarto-dev/quarto-cli#1854 (comment)