What is the recommended way to share state extracted by a loader with plugin(s) within a compilation lifecycle?

[mjames-c]

similar question to this but for loaders. we currently use the following pattern:

// loader
module.exports = function(source) {
  // ... extra metadata ...
  // patch module with extracted metadata
  this._module.buildInfo.yourMetadata = metadata;
};

// plugin
class MyPlugin {
  extractLoaderProvidedMetadata() {
    for (const chunk of compilation.chunks) {
      for (const module of compilation.chunkGraph.getChunkModules() {
        metadata.push(module.buildInfo?.yourMetadata)
      }
    }
  }
}

whilst it may seem inefficient to have to iterate built modules to extract the metadata the reason we implemented it this way is because it takes advantage of webpack's file cache -- the metadata is included in the pack files webpack writes. however, given that this._module is deprecated how should we be doing this?

[alexander-akait]

Why do you want to store something in build info? Ideally loaders should not know nothing about other loaders and plugins, they can be run in parallels mode in future (we are working on it)

[alexander-akait]

@mjames-c Just interesting why do you need loader? Looks like make sense to build just a plugin, not loader

[AdrianBannister]

Hey @alexander-akait, the loader @mjames-c mentioned also does source transformations. For example, it transforms const banana = featureFlag() → const banana = featureFlag('banana').

The loader is a wrapper around @babel/core where we pass in a couple of plugins (including the feature flags plugin). We could potentially move the metadata collection outside the loader to the Webpack parser hooks by adding the metadata to parser.state.module.buildInfo (or parser.state.current.buildInfo?).

However, I suspect the reason this was originally implemented as a loader is because we use thread-loader, so the work can be parallelised across threads.

[alexander-akait]

The loader is a wrapper around @babel/core where we pass in a couple of plugins (including the feature flags plugin). We could potentially move the metadata collection outside the loader to the Webpack parser hooks by adding the metadata to parser.state.module.buildInfo (or parser.state.current.buildInfo?).

It will be faster and more valid, because using babel and webpack together you will get two rounds of parsing (babel, then acorn under the hood of webpack), but if babel still require for your build pipeline (you make more transformation using babel or have more babel plugin) using loader and _module is a solution too

thread-loader has limitations about _compilation, _compiler and _module, because we are serializing data to move them into other processes and it will be very expensive serialize whole compilation/compiler/module, that is why they are private and do not recommended to use, legacy API, (there is that we are supporting for thread-loader - https://github.com/webpack-contrib/thread-loader/blob/master/src/index.js#L13)

Some questions:

• Does transformer (i.e. do babel plugin) return metadata from plugin?
• Where is this metadata used later and for what purpose?

Just to provide detailed answer and provide better solution.

[AdrianBannister]

but if babel still require for your build pipeline (you make more transformation using babel or have more babel plugin) using loader and _module is a solution too

Unfortunately we have quite a few babel plugins so we'll probably have to keep it for now (although it is slow and we would like to kill it).

thread-loader has limitations about _compilation, _compiler and _module

True, we've worked around this by adding the meta data to the AdditionalData parameter in the loader callback, and then we have a loader outside of thread-loader that takes that data and adds it to the module. It's ugly to have a loader that doesn't actually effect the source, but it was the only way we could access the AdditionalData after the loaders are done executing.

Does transformer (i.e. do babel plugin) return metadata from plugin?

Only the feature flag babel plugin returns meta data, all the other babel plugins just transforms the source.

Where is this metadata used later and for what purpose?

It's used in a processAssets hook where we write the feature flag data to a custom manifest that our servers use to serve the page

[alexander-akait]

True, we've worked around this by adding the meta data to the AdditionalData parameter in the loader callback, and then we have a loader outside of thread-loader that takes that data and adds it to the module. It's ugly to have a loader that doesn't actually effect the source, but it was the only way we could access the AdditionalData after the loaders are done executing.

That is a good solution too 👍

Any problems with storing such things there?