This recent reddit thread reveals discontent among the web development community about the sheer volume of stuff in a typical node_modules dir. 140MB in this case!
Opinions in the thread varied from "I'm surprised npm even works" to "everything is fine". I'm not going to offer an opinion, just these two observations:
node_modulesdirs typically do contain lots of stuff that doesn't need to be there.- The latest version mitigates overall size by flattening the dependency tree, but some of the bloat is beyond npm's control.
The beyond npm's control part is where we come in, and it has to do with how npm modules are developed and published. Specifically, two common oversights on the part of module owners contribute to node_modules bloat:
Let's review the difference between dependencies and devDependencies inside package.json. The former is what the module needs to work as advertized. The latter is what people need to contribute to its development.
If you npm install thing, everything in its dependencies gets downloaded into your node_modules dir, while everything in its devDependencies is ignored. After all, if you wanted to contribute, you wouldn't npm install thing, you'd git clone thing and then npm install.
Unfortunately, module owners sometimes forget to pay attention to this, so things like gulp or mocha find their way into dependencies, forcing everyone in the world to download them on install.
Fortunately, "Move X to devDependencies" is an easy pull request. Better yet, it fixes a small problem permanently, for everyone. It's also a great way to get into OSS development and building rapport with module owners!
By default, everything in your project gets uploaded to npm, and subsequently downloaded to users' machines on install. npm blacklists .git dirs and a few other obvious things, but it can't read you mind.
You have to tell it to ignore your tests/ dir and/or *.test.js files, for example, or that 3MB of test data, or the PhantomJS binary. All of that stuff becomes dead weight in node_modules dirs, planet-wide.
A simple .npmignore file in the project root—which works just like .gitignore—can fix this, but module owners sometimes forget, or don't know it's an option. I didn't know it either until someone pointed it out to me.
# npmignore file
tests/
*.test.js
phantomjs/
This is another opportunity for a simple PR which will forever fix the issue for everyone on the planet.
I'm emphasizing PRs instead of telling module owners to fix their stuff, because OSS is a community effort. Yes, owners should fix their stuff, but there's a worrying lack of gratitude in the wider community toward OSS developers. These are people who create useful things in their spare time, and then give it away for free.
I'd personally like to keep developing and using OSS for the foreseeable future, but that means being respectful of others' time and contributions, and striving to be helpful in whatever ways I can. Ultimately it means assuming some responsibility for the things I don't like.
We need to do something about this collectively. Perhaps make a checklist & send it to every module owner. Collaboratively check the top 1000 modules. Think it will be trivially possible considering node land has significant community strength. Perhaps another solution to this is a new standard for the packages (we should be able to generate most of the following):
This will allow npm to clone the 'releases at specified version' with
--depth 1flag, downloading only the very minimal requirement. Perhaps overall this may reduce the file size significantly.Currently with this size; its practically impossible to use npm in developing countries with spotty internet. Hope we can do something about this.