Dedupe regression with browserify v8

[jgoz]

I'm logging the issue here, but the error is due to changes in browserify 8.0.0.

This might be best explained with an example:

• Module A depends on module B
• Other modules depend on module B', which is identical to B but a separate copy exists for whatever reason
• B is deduped by browserify and points to B'
• B gets routed to the A bundle, assigned ID of 200
• B' gets routed to the common bundle, assigned ID of 100
• Page includes common bundle then A bundle

A bundle in browserify v7:

200:[function(require,module,exports){
module.exports=require(100)
},{"dup":100}]}

A bundle in browserify v8:

200:[function(require,module,exports){
arguments[4][100][0].apply(exports,arguments)
},{"dup":100}]}

This results in an exception Uncaught TypeError: Cannot read property '0' of undefined because ID 100 is not defined in the current bundle.

This worked in v7 because it used require, which tries to resolve the module with previously defined requires from other bundles.

I realize this is an edge case and that if npm is correctly deduping dependencies, this situation should not occur. And the v8 behaviour is more correct in theory because B may have different dependencies from B'. However, assuming that any module is defined in the current bundle is dangerous when factor-bundle is involved.

[terinjokes]

Here we end up deduping with npm, rather than relying on Browserify (since in previous versions, we ended up with suboptimal bundles). I can put in the README this as a suggestion.

I can dig into the factoring to see if there's anywhere that we can detect duplicates and handle properly. Mind creating a failing test case for this?

[jgoz]

Here we end up deduping with npm, rather than relying on Browserify (since in previous versions, we ended up with suboptimal bundles). I can put in the README this as a suggestion.

I agree, that's by far the best way to do it. Putting it in the README is a good idea.

I can dig into the factoring to see if there's anywhere that we can detect duplicates and handle properly. Mind creating a failing test case for this?

From what I could tell, it was hard to reliably detect this scenario due to row ordering, but I will come up with a minimal test case later today.

[terinjokes]

Mentioning @substack to see if he hasn't any ideas.

[malte-wessel]

I'm also experiencing this issue. Is there a workaround to fix it?

[zoubin]

+1

This worked in v7 because it used require, which tries to resolve the module with previously defined requires from other bundles.

By the way, what's the problem with require?

I haven't found more details except browserify/browserify@3249915

[Avnerus]

+1
I 'm also experiencing this issue and running npm dedupe doesn't seem to help.
Only reverting the aforementioned commit fixes the problem.

[jesstelford]

I'm running into this issue now. Anyone got any ideas on how to fix it other than reverting the commit in browserify?

[Avnerus]

you could also override the browserify.prototype._dedupe function in your user code, but that's also not a very legitimate solution..

[d0b1010r]

This is regularly breaking the builds in our setup.

[terinjokes]

@substack can you take a look at this?

[darrennolan]

Also running into this today.

@terinjokes - Can you please elaborate on

Here we end up deduping with npm, rather than relying on Browserify (since in previous versions, we ended up with suboptimal bundles). I can put in the README this as a suggestion.

How do you go about doing this? I didn't see anything in the docs just yet about deduping flags, either to browserify or factor-bundle.

I'm just using cli for our builds, only deduping methods I saw were of the API.

Edit:
Just further to this, I was previously mucking about with force-dedupe-git-modules in the past. This has resolved my issue. (Problem I think stems from one of my top level bundles is also partially included in another top level bundle. Using git for these dependencies doesn't always work great as they're always checked out again, even if the same tag).

But for the moment, this appears to be working for us.

[tellnes]

I've written a plugin which will probably solve this. It removes duplicates files completly from the pipeline right before browserify's dedupe.

https://github.com/tellnes/prundupify
https://www.npmjs.com/package/prundupify

[magalhini]

@tellnes I can't thank you enough, in this world, for your brilliant, lovely, life-saving prundupify plugin. I've been banging my head on this issue for almost 4 days now, and your solution was the only one that magically made everything work again. Since what it does is pretty low-level for me, I'm struggling to understand exactly what's going on.

If you could shine some light as to what it's exactly doing and how it's magically fixing this issue, I'd be very very grateful. Not that I am not already, trust me, very much so 😄