How to use a MITM Proxy with any node app

mitmproxy.md

mitmproxy

How-to man-in-the-middle your own application's traffic for debugging. Below is a quick step-by-step guide to putting mitmproxy between your hard-to-debug application's HTTP(s) network traffic and its destinations. Allowing you to spy on all requests, as well as modify and replay them. Theres a nice blog post with more screenshots and some explanation behind mitm-ing available here: https://earthly.dev/blog/mitmproxy/

1. Install dependencies

1. global-agent - Global node proxy configuration via environment variables (npm) Optional - only necessary for proxying local node scripts/apps that don't have an explicit proxy option. a. $ npm i -g global-agent
2. mitmproxy - Popular opensource python man-in-the-middle proxy with web interface (homepage)
   a. $ yay mitmproxy (Arch Linux)
   b. $ brew install mitmproxy (MacOS)
   c. $ sudo apt install mitmproxy (Ubuntu + Debian)

2. Start the mitmproxy web interface

This will open the proxy itself (:8080) and the web interface (:8081)

If you've got other things running on either of those default ports, they can be adjusted with the --web-port and --listen-port flags.

Once you're sure the selected ports are available, execute the mitmweb binary. This command will start the proxy on localhost:8080 and the web interface on localhost:8123 and not auto open the browser:

$ mitmweb --web-port 8123 --no-web-open-browser

You can then open the mitmweb web interface at localhost:8123 in your browser and view all HTTP requests going in and out of the target process once you tell the process you're trying to monitor about the proxy.

3. Setup your process to use the HTTP Proxy

3a. Set env variables to define your proxy settings

By default, many applications generally, and node applications specifically, support the HTTP_PROXY environment variable.

So you can simply export HTTP_PROXY=http://127.0.0.1:8080 before executing your command, or prepend the env var to your command, like HTTP_PROXY=http://127.0.0.1:8080 node index.js.

However, if they don't support http proxies out of the box, we can use the npm package global-agent. This will monkey patch the node built-in http.request module. To define a proxy location to be used by global-agent, you have to use this environment variable:

$ export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:8080

To apply global-agent settings to your node process, you can import their package in your code, or if you don't have access to the code, you can use node's preload option like so:

$ node -r 'global-agent/bootstrap' your-script.js

For more information on how to use global-agent, check out their 'usage' docs.

3b. Set the proxy option in the Playwright launch config

To setup Playwright to use your mitmproxy, you can define a proxy config in the launch methods options:

const browser = await chromium.launch({
  proxy: {
    server: 'http://localhost:8080',
  }
});

4. Example proxy setup with global-agent

For example, to listen to the vercel cli login process:

$ export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:8080
$ node -r 'global-agent/bootstrap' ~/.npm-global/bin/vercel login

To listen to a jest process, for example, including passing some non-mitmproxy related environment variables:

$ export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:8080
$ NODE_ENV=development TZ=Europe/Berlin NODE_CONFIG_DIR=$(pwd)/config node -r 'global-agent/bootstrap'  ./node_modules/.bin/jest --testMatch **/*integration-spec.js --maxWorkers=2 --rootDir ./src/modules/public-api

From here you can view all the details of HTTP requests, including modifying and replaying them in the accompanying mitmweb web interface (localhost:8081 by default). The mitmproxy docs have a nice tutorial for more in-depth use. Including a demo sending arbitrarily high numbers to Apple's "Game Center" leaderboards 😂.

If you need to proxy HTTPS traffic, it also supports that, however setup is a bit more complicated including installing their CA certificates to your system/browser. More details can be found here.

HTTPS - Certificate Authority

In order to proxy HTTPS requests to external services, you're going to have to add the mitm ca-cert to your systems trusted certificates. The intstructions vary by OS, but this is the general flow for linux OSes. More info here: https://docs.mitmproxy.org/stable/concepts-certificates/

1. After installing mitmproxy, you must run it at least once - type mitmproxy in a terminal session and quit. This will create the necessary certificate files at ~/.mitmproxy.

2. Extract the certificate to .crt format:
   openssl x509 -in ~/.mitmproxy/mitmproxy-ca-cert.pem -inform PEM -out ca.crt

3. Copy the certificate to your systems certificate store
   a. Arch Linux: Copy ca.crt to /usr/local/share/ca-certificates and /etc/ssl/certs and run sudo update-ca-trust. Then trust the certificate - sudo trust anchor ca.crt
   b. Mac OS: sudo security add-trusted-cert -d -p ssl -p basic -k /Library/Keychains/System.keychain ~/.mitmproxy/mitmproxy-ca-cert.pem
   c. For information on other platforms, please checkout their docs.

4. Run the mitmproxy again

Troubleshooting

• I also had to update my locally installed version of the python package cryptography.

$ pip install "cryptography>35.0.0"

• When trying to proxy https traffic, you may have to tell node to use the openssl CA store. This can be done with the --use-openssl-ca flag. So for example:

$ export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:8080
$ NODE_ENV=development HTTP_PROXY=http://127.0.0.1:8080 ENFORCE_HTTPS=false AWS_ENV=local npx nodemon --exec 'node -r 'global-agent/bootstrap' --use-openssl-ca index.js'

4 node js

NODE_EXTRA_CA_CERTS=~/.mitmproxy/mitmproxy-ca-cert.pem