reduce-node-modules-bloat.md

How you can help reduce node_modules bloat

This recent reddit thread reveals discontent among the web development community about the sheer volume of stuff in a typical node_modules dir. 140MB in this case!

Is it a design flaw in npm?

Opinions in the thread varied from "I'm surprised npm even works" to "everything is fine". I'm not going to offer an opinion, just these two observations:

1. node_modules dirs typically do contain lots of stuff that doesn't need to be there.
2. The latest version mitigates overall size by flattening the dependency tree, but some of the bloat is beyond npm's control.

The beyond npm's control part is where we come in, and it has to do with how npm modules are developed and published. Specifically, two common oversights on the part of module owners contribute to node_modules bloat:

First: faulty package.json configurations

Let's review the difference between dependencies and devDependencies inside package.json. The former is what the module needs to work as advertized. The latter is what people need to contribute to its development.

If you npm install thing, everything in its dependencies gets downloaded into your node_modules dir, while everything in its devDependencies is ignored. After all, if you wanted to contribute, you wouldn't npm install thing, you'd git clone thing and then npm install.

Unfortunately, module owners sometimes forget to pay attention to this, so things like gulp or mocha find their way into dependencies, forcing everyone in the world to download them on install.

Fortunately, "Move X to devDependencies" is an easy pull request. Better yet, it fixes a small problem permanently, for everyone. It's also a great way to get into OSS development and building rapport with module owners!

Second: missing or incomplete .npmignore files

By default, everything in your project gets uploaded to npm, and subsequently downloaded to users' machines on install. npm blacklists .git dirs and a few other obvious things, but it can't read you mind.

You have to tell it to ignore your tests/ dir and/or *.test.js files, for example, or that 3MB of test data, or the PhantomJS binary. All of that stuff becomes dead weight in node_modules dirs, planet-wide.

A simple .npmignore file in the project root—which works just like .gitignore—can fix this, but module owners sometimes forget, or don't know it's an option. I didn't know it either until someone pointed it out to me.

# npmignore file
tests/
*.test.js
phantomjs/

This is another opportunity for a simple PR which will forever fix the issue for everyone on the planet.

Finally: be nice to OSS developers!

I'm emphasizing PRs instead of telling module owners to fix their stuff, because OSS is a community effort. Yes, owners should fix their stuff, but there's a worrying lack of gratitude in the wider community toward OSS developers. These are people who create useful things in their spare time, and then give it away for free.

I'd personally like to keep developing and using OSS for the foreseeable future, but that means being respectful of others' time and contributions, and striving to be helpful in whatever ways I can. Ultimately it means assuming some responsibility for the things I don't like.

Thanks for this. I'd forgotten about .npmignore.

I'm researching the topic of node_modules bloat because my team is doing some AWS development using Lambda functions in a node.js environment.

Have you checked out modclean by any chance? Looks like something that can take pruning to the next level.

Also, npm and yarn have dedupe commands, non?