CODEWITHSUNDEEP
node.jsnpm
jdm
jdm

Reputation: 10070

System-wide deduplication of identical node modules

I'm working on a Javascript project, and as it so happens one of my dependencies pulls in puppeteer, which in turn downloads a whole copy of Chromium into my node_modules. My larger project is split into multiple Javascript packages, so I end up with multiple identical copies of Chromium among other stuff.

Is there a way to deduplicate these packages system wide? Note, npm dedupe seems to do something completely different to what I want.

I imagine there would be a module repository in my home directory which contains every package I need (in every version needed), and then in the local node_modules directories would contain only symlinks to the repository. This seems like an incredibly obvious optimisation, but I can't find any way to do it in npm. If not in npm, is it maybe possible in yarn?

As an added complication, this should also work on Windows (where symbolic link support has historically been not so good).

Upvotes: 0

Views: 396

Answers (1)

jdm
jdm

Reputation: 10070

It seems the following command does what I want:

npm config set link -g

Then delete node_modules, and do npm install again. It should be much smaller now.

The documentation says:

If true, then local installs will link if there is a suitable globally installed package.

Note that this means that local installs can cause things to be installed into the global space at the same time. The link is only done if one of the two conditions are met:

  • The package is not already installed globally, or
  • the globally installed version is identical to the version that is being installed locally.

I am not sure if this has any negative side-effects - for example clobbering the global namespace with commands I don't want. For now, it seems to work fine.

Upvotes: 1

Related Questions