Skip to content

Intermediate transform stage between read and write phases #12698

Open
@bpatath

Description

@bpatath

Is your feature request related to a problem? Please describe.

Yes. I am writing a Sphinx extension and here is a simplified version of my issue:

  1. I want some directive in file A to generate some things in the environment.
    .. create_thing:: foo
    .. create_thing:: bar
    
  2. I want some directive in file B to display a list all of these things.
    .. list_things::
    
  3. I want some role in file C to be able to reference the things rendered in B.
    A link to :thing:`bar` in file B.
    

The first step is typically done in parallel in the read phase, where I can fill the environment, either directly in directive's run, or using special nodes and Transform.

The third step is typically done in parallel in the write phase, where I can just look for a target document/id in my environment and replace the role node with a reference node, either using the resolved event, or a PostTransform.

But I can't find a way to implement the second step. The reading phase is too soon, because we can't have a list of things to display in B, until after A has been read. But writing phase is too late, because we would need to store the target id of things displayed in B before C is being written.

Describe the solution you'd like

I initially implemented step 2 and 3 as PostTransforms with increasing priority, as I thought that a higher priority PostTransform was going to run on all doctrees before lower priorities PostTransforms are applied. Since the point is that the writing phase can be parallelized, I understand why this is not the case.

The two ideas that come to mind are:

  • Another registry of PostTransforms that are being all being applied before the writing phase. These could also be applied in order of (priority then document) instead of (document then priority). But this would mean a new name, documentation, internal changes, maintenance cost, etc.
  • Or a new event that would allow an extension to run a transform on all updated doctrees, again before the writing phase. Extensions would receive the list of all updated doctrees and would be able to run transforms in the correct order.

For the record, I also did try to define an event handler for env-check-consistency which is actually in the middle of the two phases. But the list of updated doctrees is not available, as it is a local variable of builder functions, and not a property of the builder. And if I try to create such a list myself and retrieve the doctree, there is no easy API to do so. BuildEnvironment does define a get_doctree method, but it does not include access to the read cache, and for some reason, my changes wouldn't persist to the writing phase anyway.

Describe alternatives you've considered
Could not see an alternative, but maybe I am missing something in how Sphinx works... ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions