Prevent webpack compilers from running on every page compile during development

[colbyfayock]

Currently our webpack compilers run on every page load which slows down performance a bit during development.

This doesn't impact production as it's statically compiled.

These plugins work by running at compile time to provide features like an RSS feed, OG images, and a sitemap

https://github.com/colbyfayock/next-wordpress-starter/tree/main/plugins

Ideally we can only run this when it will actulaly make a difference to avoid the performance hit during development

https://github.com/colbyfayock/next-wordpress-starter/blob/main/plugins/plugin-compiler.js#L56-L64

[petercr]

Hey Colby 👋 is this still an issue that needs to be worked on?
It seems like it's from a while ago 😅

Anyways let me know 👌

[colbyfayock]

yup still a problem!

[petercr]

Okay thanks. I just wanted to check in first and make sure before I spent any time on the issue.
But I guess now I'm free to jump in and see what the heck is going on 🤪

I'll let you know what I come up with 👍

[colbyfayock]

aweosme sounds great. im also pretty open to new solutions around the entire plugin chain here. @GuilleAngulo and i worked on it a while back as something that we were hoping would be easy to fit in new capabilities without having to rely on package.json-based scripts which wouldn't actually fall into the build process and may potentially get skipped? (not sure if that's a valid concern)

i'd also like to figure out a way to enhance code reuse. we have the util.js with a bunch of request capabilities in there. we did that because we weren't able to easily import the existing functions with the require vs import

[petercr]

So I finally got to give this code a good look over 👀 and think about it.
After looking through some Webpack docs, I came across this method: recordsPath

Here's the info on it:

Use this option to generate a JSON file containing webpack "records" – pieces of data used to store module identifiers across multiple builds. You can use this file to track how modules change between builds.

Hopefully that could give us a reference point to check the builds against, allowing for the build to be skipped if needed.

I'm going to play around with it and see where that gets me 👍🏻

[colbyfayock]

ohhhh thats super interesting. only thought, would that override other's recordsPath or is it per-plugin?

[petercr]

Hmm I don't know the answer to that, but I'll look into it 👀

Also I haven't quite figured out how to get it working yet 😂
It seems to run the command, but no records file is being generated.

[colbyfayock]

this might be worth looking into as well https://nextjs.org/blog/next-12-2#swc-plugins-experimental

haven't played with it, so not sure if it'll actually be useful for this use case, but worth thinking about and looking into

[petercr]

Hey Colby 👋 thanks for the link 👍
I checked it out, and found a list of the official swc plugins.
Link to the list of SWC plugins

Here's the list:

• emotion
• jest
• relay
• styled-components
• styled-jsx
• transform-imports

I'm not sure if any of those will be of help with this particular issue.
I was thinking maybe if we tried to implement some kind of caching for the webpack build, then it would only run if changes were made?

Perhaps even something like next-bundle-analyzer. And then we could query that data instead of displaying it to the user, of course 😉
Or even just some kind of regular cache plugin for Webpack.
Then hopefully it wouldn't run the build again if the input had not changed.

Let me know what you think 😄

[colbyfayock]

ah okay so you can't make a custom one? i guess that's where my mind was going, a custom plugin, kinda like we were doing for webpack, but maybe that's not the right route

caching sounds like a great strategy! especially if that's a common approach

totally open to seeing what you come up with, just hoping to avoid slowing down dev with the processing each time

if going the caching route, wonder if there's an apollo caching mechanism to take advantage of too 🤷 but idk how the requests work whether they're all sahred in the same process or not

looking forward to seeing what you come up with

[colbyfayock]

thought just come to mind, should we by default disable these from running on development? that's common isnt it?

a caching solution would totally be nice, but wondering if that would make more sense

we could add an npm script to manually trigger them to test and an environment variable / flag to enable / disable

only other thought is not all of them should be disabled though. Search for instance should always be available, so the problem persists there, as it's used clientside to make search requests

[petercr]

Nice well I think that would be an easy way to fix the problem! 😂
I mean sure I think it's common to disable running extra stuff in development.
You know how us devs love our fast refresh! 🚀

We could disable the ones we don't want running on the dev server and keep the ones we do want.
On that note, we could also declare some kind of variable like hasRun to hold the states of the plugins.
And that way we could only run each plugin 1 time on the dev server. So they're available.
Or only run the ones we want to run.

Maybe that's a good solution?
Let me know what you think 👌