Caching NPM proxy using Nginx

nginx.conf

user www-data;
worker_processes 4;

error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
}

http {
  include /etc/nginx/mime.types;

  access_log off;

  default_type application/octet-stream;

  sendfile on;
  tcp_nodelay on;
  tcp_nopush off;

  reset_timedout_connection on;

  server_tokens off;

  # Cache 10G worth of packages for up to 1 month
  proxy_cache_path /var/lib/nginx/npm levels=1:2 keys_zone=npm:16m inactive=1M max_size=10G;

  # Multiple server definitions makes nginx retry
  upstream registry_npm {
    server registry.npmjs.org;
    server registry.npmjs.org;
    keepalive 16;
  }

  gzip on;
  gzip_types application/json text/css text/javascript;
  gzip_proxied any;
  gzip_vary on;

  server {
    listen 80 default_server;
    server_name npm.example.com;

    root /var/www;

    proxy_cache npm;
    proxy_cache_key $uri;
    proxy_cache_lock on;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;

    proxy_http_version 1.1;
    proxy_pass_request_headers off;
    proxy_set_header Host registry.npmjs.org;

    location / {
      proxy_cache_valid any 5m;

      add_header X-Cache $upstream_cache_status;

      proxy_pass http://registry_npm;
    }

    location ~ ^/.+/-/.+ {
      proxy_cache_valid any 1M;

      add_header X-Cache $upstream_cache_status;

      proxy_pass http://registry_npm;
    }
  }
}

Im getting a 400 with this configuration.

Docker image: node4
npm version: 2.14.12
nginx: 1.11.12

Error: npm info retry will retry, error on last attempt: Error: tunneling socket could not be established, cause=Parse Error

npm logs

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'install',
1 verbose cli   'lodash' ]
2 info using [email protected]
3 info using [email protected]
4 verbose config Skipping project config: /root/.npmrc. (matches userconfig)
5 verbose install initial load of /root/package.json
6 verbose readDependencies loading dependencies from /root/package.json
7 silly cache add args [ 'lodash', null ]
8 verbose cache add spec lodash
9 silly cache add parsed spec Result {
9 silly cache add   raw: 'lodash',
9 silly cache add   scope: null,
9 silly cache add   name: 'lodash',
9 silly cache add   rawSpec: '',
9 silly cache add   spec: 'latest',
9 silly cache add   type: 'tag' }
10 silly addNamed lodash@latest
11 verbose addNamed "latest" is being treated as a dist-tag for lodash
12 info addNameTag [ 'lodash', 'latest' ]
13 silly mapToRegistry name lodash
14 silly mapToRegistry using default registry
15 silly mapToRegistry registry https://registry.npmjs.org/
16 silly mapToRegistry uri https://registry.npmjs.org/lodash
17 verbose addNameTag registry:https://registry.npmjs.org/lodash not in flight; fetching
18 verbose request uri https://registry.npmjs.org/lodash
19 verbose request no auth needed
20 info attempt registry request try #1 at 4:23:13 PM
21 verbose request id c85b7e2ec0404227
22 http request GET https://registry.npmjs.org/lodash
23 info retry will retry, error on last attempt: Error: tunneling socket could not be established, cause=Parse Error
24 info attempt registry request try #2 at 4:23:23 PM
25 http request GET https://registry.npmjs.org/lodash
26 info retry will retry, error on last attempt: Error: tunneling socket could not be established, cause=Parse Error
27 info attempt registry request try #3 at 4:24:23 PM
28 http request GET https://registry.npmjs.org/lodash
29 verbose stack Error: tunneling socket could not be established, cause=Parse Error
29 verbose stack     at ClientRequest.onError (/usr/local/lib/node_modules/npm/node_modules/request/node_modules/tunnel-agent/index.js:174:17)
29 verbose stack     at ClientRequest.g (events.js:260:16)
29 verbose stack     at emitOne (events.js:77:13)
29 verbose stack     at ClientRequest.emit (events.js:169:7)
29 verbose stack     at Socket.socketOnData (_http_client.js:307:9)
29 verbose stack     at emitOne (events.js:77:13)
29 verbose stack     at Socket.emit (events.js:169:7)
29 verbose stack     at readableAddChunk (_stream_readable.js:146:16)
29 verbose stack     at Socket.Readable.push (_stream_readable.js:110:10)
29 verbose stack     at TCP.onread (net.js:523:20)
30 verbose cwd /root
31 error Linux 4.9.12-moby
32 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "lodash"
33 error node v4.3.2
34 error npm  v2.14.12
35 error code ECONNRESET
36 error network tunneling socket could not be established, cause=Parse Error
36 error network This is most likely not a problem with npm itself
36 error network and is related to network connectivity.
36 error network In most cases you are behind a proxy or have bad network settings.
36 error network
36 error network If you are behind a proxy, please make sure that the
36 error network 'proxy' config is set properly.  See: 'npm help config'
37 verbose exit [ 1, true ]

NGINX logs

172.25.0.3 - - [06/Apr/2017:16:18:53 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"
172.25.0.3 - - [06/Apr/2017:16:19:03 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"
172.25.0.3 - - [06/Apr/2017:16:20:03 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"
172.25.0.3 - - [06/Apr/2017:16:23:13 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"
172.25.0.3 - - [06/Apr/2017:16:23:23 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"
172.25.0.3 - - [06/Apr/2017:16:24:23 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 400 166 "-" "-" "-"

@Puneeth-n did you ever figure out a solution for this issue?

I needed to add proxy_ignore_headers Set-Cookie; as well. https://www.nginx.com/blog/nginx-caching-guide/#Frequently-Asked-Questions-(FAQ)

it works, thank you!

Would you extend caching to like a month? Turning this into a full mirror?

Some endpoints would need shorter caching periods. I wonder if npm itself sends caching headers that can be used.

Another idea would be to extend caching only for the .tgz files.