optimization.concatenateModules leading to nondeterministic build output

[pgoldberg]

I recently upgraded to webpack 5, and I'm noticing that our builds no longer have deterministic outputs. I've narrowed down the issue to optimization.concatenateModules – when I set it to false, I get the same output on every build, although I'd like to keep concatenateModules on. With it on, the output just differs slightly. It looks like the modules are being concatenated in different orders? For example:

-        Et = new pt(Ct),
-        kt = new ht(Ct);
+        Et = new ht(Ct),
+        kt = new pt(Ct);

Any idea what might be causing this? Not sure if it's a bug in the ModuleConcatenationPlugin, but given that I haven't seen any other issues about this or any mention of this problem online, I figure maybe there's something in our configuration that could be causing this?

[alexander-akait]

Do you have ModuleConcatenationPlugin in plugins or use optimization.concatenateModules to setup it?

[pgoldberg]

Just did it with the minimal repro and it does seem like the issue goes away – looks like it must be a race condition. After spending some time debugging with the test repo, it seems like it's a race condition with NormalModuleFactory.create for inner_dep2/index.js finishing before inner_dep/index.js here:

webpack/lib/Compilation.js

Lines 1973 to 2033 in 2738eeb

	factory.create(
	{
	contextInfo: {
	issuer: originModule ? originModule.nameForCondition() : "",
	issuerLayer: originModule ? originModule.layer : null,
	compiler: this.compiler.name,
	...contextInfo
	},
	resolveOptions: originModule ? originModule.resolveOptions : undefined,
	context: context
	? context
	: originModule
	? originModule.context
	: this.compiler.context,
	dependencies: dependencies
	},
	(err, result) => {
	if (result) {
	// TODO webpack 6: remove
	// For backward-compat
	if (result.module === undefined && result instanceof Module) {
	result = {
	module: result
	};
	}
	if (!factoryResult) {
	const {
	fileDependencies,
	contextDependencies,
	missingDependencies
	} = result;
	if (fileDependencies) {
	this.fileDependencies.addAll(fileDependencies);
	}
	if (contextDependencies) {
	this.contextDependencies.addAll(contextDependencies);
	}
	if (missingDependencies) {
	this.missingDependencies.addAll(missingDependencies);
	}
	}
	}
	if (err) {
	const notFoundError = new ModuleNotFoundError(
	originModule,
	err,
	dependencies.map(d => d.loc).filter(Boolean)[0]
	);
	return callback(notFoundError, factoryResult ? result : undefined);
	}
	if (!result) {
	return callback();
	}
	if (currentProfile !== undefined) {
	currentProfile.markFactoryEnd();
	}
	callback(null, factoryResult ? result : result.module);
	}
	);

That's the first place I see the order become incorrect

[alexander-akait]

So let's open an issue fir this, sounds like a bug

[pgoldberg]

Filed #15866

[vankop]

factorizing in different order is ok since modules itself are independent, but this should not lead to different concat results

[vankop]

probably this happens because as a non-determenistic result of compilation.modules ( Set )