Allow Sphinx to present multiple content fields

[ktbartholomew]

Problem

Rather than having a big monolithic body attribute with all of a page's content, it would grant us a lot of flexibility to be able to split this content into different fields. The current use case for this is in rackerlabs/docs-core-infra-user-guide, where most pages include a ToC of a custom set of child pages at the end of the body. Sphinx doesn't interpret this as the ToC, which it generates on its own by parsing the sections on the current page. The user-defined ToC is then just part of the body, and there's not a lot we can do with that on the front-end (without hacking the DOM with JavaScript).

Proposal

Using the container directive in Sphinx, we can split a document into multiple discrete parts, each of which has an HTML id attribute.

In deconst/preparer-sphinx, we can then extract these containers and set each one as a property in the JSON payload that gets submitted to content-service. These properties should be in addition to the body property, in order to make the use of containers an option for writers and not a requirement. The body, with all child container removed, should also be set as a reserved "body" container, so its content can be used standalone and without duplicating content on the page.

Example

The following ReST:

This heading creates a section and begins the body
==================================================

The main content goes here, without a container directive. When run through the
preparer, the main content will appear in both :code:`envelope.body` (child
containers and all) and :code:`envelope.sections.body` (child containers
removed)

.. container:: :name: toc

    This content is an a section named "toc". In the JSON payload, it will be
    accessible at :code:`envelope.sections.toc`.

    Containers **cannot** contain sections (insert headdesk here), so a
    workaround is required if a container needs to include a heading element:

    .. raw:: html

        <h2>Contents</h2>

    .. toctree:: :maxdepth: 3

        cloud_config/index

...results in HTML roughly like this:

<div class="section" id="this-heading-creates-a-section-and-begins-the-body">
    <!-- body HTML -->
    <div class="container" id="toc">
        <!-- toc HTML -->
    </div>
</div>

preparer-sphinx will extract the child container(s) and create separate JSON properties for each:

{
    "body": "<div class=\"section\" id=\"this-heading-creates-a-section-and-begins-the-body\"><!-- body HTML --><div class=\"container\" id=\"toc\"><!-- toc HTML --></div></div>",
    "sections": {
        "body": "<div class=\"section\" id=\"this-heading-creates-a-section-and-begins-the-body\"><!-- body HTML --></div>",
        "toc": "<!-- toc HTML -->"
    }
}

This allows templates to simply use the body attribute if that's all they need, or to construct more complex layouts by outputting each section individually.

cc @smashwilson

[smashwilson]

👍 This is a great idea. It'll be really nice to have, and not just for the TOC.

Stream-of-consciousness, non-blocking thoughts:

Is there an equivalent way to pull selected content out of a Jekyll page? We don't need it immediately, but I do try to have rough feature parity between the preparers, at least as much as it makes sense to.

The ability to have arbitrary keys within the metadata envelope will also make it more difficult to validate the schema of envelopes that are submitted to the content service. Maybe we can shunt it into a known sub-element ({{{ envelope.sections.toc }}})?

[ktbartholomew]

@smashwilson:

---
title: This is sample frontmatter with multi-line HTML
sections:
    one: |
        <div id="one">
            <p>This is the content for section one.</p>
        </div>
    two: |
        <div id="two">
            <p>This is the content for section two.</p>
        </div>
    three: |
        # The Joys of Markdown

        **Markdown** makes writing HTML fun!

In the preparer, we could parse the Front Matter sections as Markdown to simplify authors' lives, and the part following the Front Matter would be treated as the "body" property, as it is currently.