Multi-phase build

[ryantownsend]

I'm currently integrating compression dictionaries into an 11ty project (with a view to open-sourcing a plugin). In order to complete this, I need to create a multi-phase build:

1. MD5 the source directory to define the fingerprint for our dictionary (to be used in Liquid/etc to render the file path)
2. Build our HTML
3. Use the built HTML as an input to generating our compression dictionary
   (we can't use our source as that could leak internal code)
4. Compress our HTML files using the newly generated dictionary, outputting additional .dcz files
5. Build our edge functions (from .ts.liquid) to handle proxying
   (this depends on the dictionary existing in step 4 so we can generate a SHA-256 of its contents)

There are a few alternatives that world work:

• In step 2, we could build the whole project as normal and then simply rebuild the edge function in step 5
• In step 2, we could build everything except the edge function, then build just the edge function in step 5

Are there any APIs within 11ty that could help me achieve any of these flows?

Sidenote: it would be far easier if Netlify allowed files to be imported into edge functions and we didn't need to compile them into the JavaScript (something they offer for Serverless functions as included_files)... that would remove the need for this entire complexity, but given it's a popular host for 11ty projects, I need to figure out this workaround.

[Ryuno-Ki]

Hm, I would think that 1. could be done as part of before event. Alternatively you could look into using virtual templates here.
Then 2. runs through as normal.
What do you plan for 3.? Something like parse5 would be similar to what Eleventy uses (at least in WebC land). Cheerio is higher level but may be easier to handle (feels like jQuery for Node.js).
I'm personally interested in 4. as I have taken part in js13kgames competitions in the past 🤓
I haven't used any Edge functions myself so I cannot comment on 5.

[ryantownsend]

Sorry, I’m not so much looking for recommendations on technology – it’s working already, albeit with some constraints, namely the edge function being one-build out-of-date when running locally.

What I really need is specifics on how to build subsets of files in 11ty (step 2 and 5).

[Ryuno-Ki]

Ah, okay. Misinterpreted „any of these flows”.

Would a preprocessor help to select the candidates? This way, Eleventy would „see” only what it needs to see and you could control the behaviour using environment variables.

[ryantownsend]

I don't think preprocessors help I'm afraid as I need to generate the dictionary and edge function after 11ty has completed the build (of at least the HTML).

The current workflow basically works like this:

1. 11ty builds all the files (HTML, edge function)
2. Dictionary 1 gets built based on the HTML
3. The edge function is already built so has no knowledge of the Dictionary 1
4. ...change another file...
5. 11ty builds all the files (HTML, edge function)
6. The Dictionary 2 gets built based on the HTML
7. The edge function is already built so has knowledge of Dictionary 1 but not Dictionary 2

So if in step 3/7 I could trigger an 11ty function basically saying "Rebuild all files under the ./_site/edge-functions directory", then the edge function would be rebuilt with knowledge the new dictionary files.