Certificate Pinning Bypassing: Setup with Frida, mitmproxy and Android Emulator with a writable file system

00-android-bypass-certificate-pinning-and-mitm-attack-setup.md

Android Bypass Certificate Pinning and MitM Attack Setup

This is a gist used in the following blog posts:

• How to MitM Attack the API of an Android App
• How to Bypass Certificate Pinning with Frida on an Android App

01-mitmproxy-setup.md

MitMProxy Setup

Install mitmproxy

In order to intercept the traffic we will need a proxy interceptor tool, and in this tutorial we will use the mitmproxy CLI interface from within a docker container, but feel free to install it by using any other method listed in their docs.

Download the mitmproxy docker image:

sudo docker pull mitmproxy/mitmproxy:6.0.2

Test it works with:

sudo docker run --rm -it mitmproxy/mitmproxy:6.0.2 mitmproxy --version

The output should look like this:

Mitmproxy: 6.0.2
Python:    3.8.5
OpenSSL:   OpenSSL 1.1.1i  8 Dec 2020
Platform:  Linux-5.4.0-71-generic-x86_64-with

Start the mitmproxy

Before we add the mitmproxy certificate to the emulator’s system trusted store we need to first start mitmproxy, so that its certificate is created at ~/.mitmproxy.

To start mitmproxy we also need to provide the IP address where it will be listening to, and we will use our WiFI IP address because it will be later easy to proxy the emulator through it. Find the WiFi IP Address The emulator will need to reach the proxy via the wifi network where mitimproxy will be listening on port 8080.

To find the wifi ip address:

ip address | grep -i wlp -

You should see something like this:

3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.0.08/24 brd 192.168.0.255 scope global dynamic noprefixroute wlp3s0

The IP address 192.168.0.08 will be necessary to start the mitmproxy and the emulator, thus you should replace it with your own one in any subsequent command you find it being used.

To Run mitmproxy listening on the WiFi network open a new terminal window or tab and execute:

sudo docker run --rm -it -v ~/.mitmproxy:/home/mitmproxy/.mitmproxy -p 192.168.0.08:8080:8080 mitmproxy/mitmproxy:6.0.2 mitmproxy --showhost --view-filter "approov"

NOTE: We use the --showhost option to be able to see the domain instead of just the IP address for each http request in the mitmproxy console output. The --view-filter options are to limit the output to requests containing the word approov in the URL, otherwise the output will be very noisy, due to all the other HTTP requests being done by the Android OS and other apps installed.

The proxy is now listening on port 8080 for the IP address of your WiFi network.

02-frida-setup.md

Frida Setup

Install Frida with Python

Create the Python virtual env with:

python3 -m venv frida-venv

Activate the virtual env:

source frida-venv/bin/activate

Now that we are inside the virtual env, it is time to update it:

pip3 install -U setuptools

Next, install the the Frida tools package with:

pip3 install frida-tools

Finally, test that Frida is correctly installed:

frida --version

03-android29-emulator-setup.md

Android 29 Emulator Setup

Environment

To use some of the tools bundled with Android studio we need to add them to the $PATH environment variable and set the $JAVA_HOME variable.

Java Home Var

If your $JAVA_HOME is not set then you need to set it to the path on your machine. In this example I will use the path for the Java installation packaged inside Android Studio, that you may have installed in /opt or at /usr/local:

# export JAVA_HOME=/usr/local/android-studio/jre/jre
export JAVA_HOME=/opt/android-studio/jre/jre

If you have installed Android Studio as a Snap package then it will be located at /snap/android-studio:

export JAVA_HOME=/snap/android-studio/current/android-studio/jre

Adb Path

Check that adb is in the path:

adb

If it says that the command is not found then add it to the path with:

export PATH=~/Android/Sdk/platform-tools:$PATH

Try it out with:

adb help

Avd Manager Path

Check that the avdmanager is in the path:

avdmanager

If it says that the command is not found then add it to the path with:

export PATH=~/Android/Sdk/tools/bin:$PATH

Try it out by listing the available targets:

avdmanager list target

If you are in a recent version of Android Studio you may get an error:

Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
...

This error can be fixed by installing the Android Command Line Tools:

# At the time of this download the latest version was `7583922` but at the time you read this a newer one may already exist.
# Please check their download page at https://developer.android.com/studio#command-tools
curl -o tools.zip https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip

# Installation steps
mkdir -p ~/Android/Sdk/cmdline-tools/
unzip tools.zip -d ~/Android/Sdk/cmdline-tools
rm -rf tools.zip
mv ~/Android/Sdk/cmdline-tools/cmdline-tools ~/Android/Sdk/cmdline-tools/latest

# To make it permanent you want to add this line to your shell file `~/.bashrc`, `~/.zshrc`, etc. 
# If you add it to your shell file then you need to reload your current shell session with `. ~/.bashrc`.
# For this tutorial you just need to execute it in your termainal
export PATH=~/Android/Sdk/cmdline-tools/latest/bin:$PATH

Emulator Path

Check the command is installed:

emulator

If it says that the command is not found then add it to the path with:

export PATH=~/Android/Sdk/emulator:$PATH

Test it by listing your current emulators:

emulator -list-avds

Install Android API 29

If not already present in your Android installation, you need to add it.

Start by installing the platform tools with:

sdkmanager "platform-tools" "platforms;android-29"

Next, install the system image with:

sdkmanager "system-images;android-29;google_apis;x86"

Finally, accept all package licenses with:

sdkmanager --licenses

Create the Emulator AVD for Android API 29

Let’s create a Pixel AVD for Android API 29 with:

avdmanager create avd --name pixel-android-api-29 --package "system-images;android-29;google_apis;x86" --device "pixel"

Add the physical keyboard support:

echo "hw.keyboard=yes" >> ~/.android/avd/pixel-android-api-29.avd/config.ini

Start the Emulator with a Writable File System

To be able to add the mitmproxy certificate as a trusted certificate in the emulator we need to first make its file system writable.

First, disconnect any mobile device you may have connected to your computer.

Next, close any running instance of any emulator you may have running.

Now, let’s start the emulator for Android 29 in writable mode:

emulator -avd pixel-android-api-29 -writable-system &> /dev/null &

NOTE: the bit &> /dev/null will discard all output, including errors, and & will run the command in the background so that we get the shell back. If the emulator doesn’t start or otherwise misbehaves try to remove &> /dev/null to see the errors being reported.

Wait for the emulator to complete the boot process and then restart adb as root:

adb wait-for-device && adb root

The output should look like:

restarting adbd as root

In Android 29 we need to disable verification of the filesystem before we remount it as writable:

adb shell avbctl disable-verification

The output should look like this:

Successfully disabled verification. Reboot the device for changes to take effect.

Reboot for changes to take effect:

adb reboot && adb wait-for-device

After the device have completed the reboot we need to change again adb to root:

adb root

Now we need to enable the writable file system in the emulator:

adb remount

output should look like:

# ... some omitted output

remount succeeded

04-add-the-mitmproxy-certificate-to-the-android-emulator.md

Add the mitmproxy Certificate to the Android Emulator

For mitmproxy to be able to intercept the traffic coming from the Android emulator is necessary that we add it’s certificate to the trusted store, and we will follow their docs instructions.

Create a hash of the certificate to use as the filename:

FILENAME=$(openssl x509 -inform PEM -subject_hash_old -in ~/.mitmproxy/mitmproxy-ca-cert.cer | head -1).0

Copy the certificate to a new file that uses the hash filename computed in the previous step:

cp ~/.mitmproxy/mitmproxy-ca-cert.cer $FILENAME

Push the certificate to the system trusted store of the emulator:

adb push $FILENAME /system/etc/security/cacerts

Give the certificate the correct permissions:

adb shell "chmod 664 /system/etc/security/cacerts/$FILENAME"

Reboot the emulator for changes to take effect:

adb reboot && adb wait-for-device

After the boot is completed you can move to the next step.

05-android-frida-server-setup.md

Android Frida Server Setup

Install the Frida Server in the Android Emulator

Get the Android architecture:

adb shell getprop ro.product.cpu.abi

The output should look like:

x86

Now that we know the architecture is x86 we can use it to download the Frida server:

version=$(frida --version) && curl -Lo frida-server.xz https://github.com/frida/frida/releases/download/$version/frida-server-$version-android-x86.xz

After the download it’s finish we need to decompress it with:

xz -d frida-server.xz

Next, we will push the frida-server into the Android device with:

adb push frida-server /data/local/tmp

Give it executable permissions:

adb shell "chmod +x /data/local/tmp/frida-server"

Start the Frida Server in the Android Emulator

Now, open another shell in your computer to start the frida-server inside the Android device or emulator.

Switch adb to the root user with:

adb root

Start the Frida server in the background with:

adb shell "/data/local/tmp/frida-server&" &

Now, check the frida-server is running on the device:

frida-ps -U

The output should be a process list:

 PID  Name
----  ---------------------------------------------------
5310  adbd
1687  [email protected]
1790  [email protected]
...

May I ask, why you do adb shell avbctl disable-verification? After some time reading blogs, is it true if I say:

AVB starts with read-only portions of /system. But we need to remount it as writable. If AVB found that /system is not readonly, it will not loads & execute /system. So this must be disable ( adb shell avbctl disable-verification ) in order to remount it as writable.

If my memory doesn't betray me this is the reason.

Wow ! Thank you so much, you literally saved me from wasting hundreds of hours pulling my hair out !
You totally nailed it ! <3

Thanks :)

It really took me a lot of hours across several days to pull all this out, and maybe some hair pulled out :)

@approovm Step about setting the proxy on the emulator could be added. Here's my fork

This gist is used as a preparation to follow the two blog posts linked on the description, and in the blog post you can find the remaining instructions.

For example to start the emulator with the proxy enabled the article says:

Close the current open emulator and start it again, but this time in writable mode and with a proxy set to your WiFi IP address (replace 192.168.0.08 with the same IP you used to start mitmproxy) on port 8080:

emulator -avd pixel-android-api-29 -writable-system -http-proxy http://192.168.0.08:8080 &> /dev/null &

Oh @Exadra37 I kinda used this gist only so I thought this could be valuable addition. Nevertheless if someone is stuck they can see my comment and hopefully it helps 🙂

I think it's important to point out why we use -writable-system

Keep in mind: You always have to start the emulator using the -writable-system option if you want to use your certificate. Otherwise Android will load a “clean” system image.

From mitm docs