3 undocumented breaking changes we encountered while upgarding npm to v5.0.1 in homebrew

issue.md

I'm opening this issue because:

• npm is crashing.
• npm is producing an incorrect install.
• npm is doing something I don't understand.
• Other (see below for feature requests):

supporting information:

• npm -v prints: 5.0.1
• node -v prints: v8.0.0
• npm config get registry prints: https://registry.npmjs.org/
• Windows, OS X/macOS, or Linux?: macOS
• Network issues: no

While upgrading npm from npm@4 to npm@5 in the node Homebrew formula over at Homebrew/homebrew-core#14085 we encountered 3 undocumented breaking changes in npm@5 of which the last 2 are more or less in a grey area. I decided to report them here anyway, so that you could make your own assessment if they are considered to be real bugs or just an unsupported usage of npm. I would completely understand if you label them as a wontfix.

1. installing a package with broken file system permission fails.

This issue was already reported in npm/npm#16896 and I joined the discussion over there with more details.

(npm install [email protected] will fail with EACCES becasue npm@5 doesn't normalize permissions anymore and the 0.0.1 tarball is lacking +x on its lib folder)

2. npm@5 destroys node_modules folder installed by yarn

Refs: Homebrew/homebrew-core#14125

By mistake the grafana formula previously executed after a complete installation of all its dependencies with yarn install also a npm install grunt-cli (grunt-cli was already installed by yarn, but the issue also reproduces if you install something new using npm). While using npm@4 the redundant npm install grunt-cli command just did nothing as expected, but when run with npm@5, npm@5 removes all other dependencies from node_modules except grunt-cli and its dependencies breaking the previously (with yarn) installed modules completly.

How to reproduce:

1. Install yarn: npm install -g yarn
2. Download and extract grafana from https://github.com/grafana/grafana/archive/v4.3.0.tar.gz somewhere
3. Install all dependencies with yarn install
4. Install grunt-cli using npm@5: npm install grunt-cli

Expected result: all dependencies installed with yarn are still there inside node_modules

Actual result: all dependencies installed with yarn were removed and replaced by the grunt-cli only installation

This was fixed in Homebrew/homebrew-core#14125 by dropping the redundant npm install grunt-cli call.

See also the following brew logs containing the full verbose npm log:

• for npm@5 removed 929 packages and updated 16 packages in 6.735s: https://gist.github.com/ilovezfs/5e1280cc34b2dcc2f95c8f416da98a5c#file-gistfile1-txt-L70
• for npm@4 (no yarn deps removed): https://gist.github.com/ilovezfs/c1f95d7e5d50dd4ed0cfc9acbfff2c18#file-gistfile1-txt-L91

3. make install can't be used to install npm to a prefix anymore

Refs: Homebrew/homebrew-core#14085 (comment)

Until now (for npm <=4) we installed npm by the the hacky way of doing a configure --prefix=libexec/npm followed by a make install. Because make install is only an alias for node bin/npm-cli.js install -g -f ${NPMOPTS} and the breaking npm install <folder> change in npm@5, make install would now only create a symlink to our temporary build path (which gets removed by brew automatically after installation) instead of copying npm over to the specified prefix as npm@4 used to do. This has broken the way we had installed npm until now, and yes I know what we did was never a recommended way of installing npm. Nonetheless, the npm@5 make install behaviour doesn't match what you would expect from a typical make install call in the unix world anymore. For now we have worked around this by using a temporary version of npm extracted from the npm registry tarball to install said registry tarball to our prefix in libexec.