Installing and running Android Emulator on Amazon AWS EC2 (Ubuntu 16.04 / m5.xlarge)

Android-Emulator-on-AWS-EC2.txt

Update (2022): https://gist.github.com/atyachin/2f7c6054c4cd6945397165a23623987d


Steps for installing the Android Emulator from EC2 console:
-----------------------------------------------------------
sudo apt install default-jdk
wget https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip
unzip sdk-tools-linux-4333796.zip -d android-sdk
sudo mv android-sdk /opt/
export ANDROID_SDK_ROOT=/opt/android-sdk
echo "export ANDROID_SDK_ROOT=/opt/android-sdk" >> ~/.bashrc
echo "export PATH=$PATH:$ANDROID_SDK_ROOT/tools" >> ~/.bashrc
re-login
cd /opt/android-sdk/tools/bin
/opt/android-sdk/tools/bin/sdkmanager --update
/opt/android-sdk/tools/bin/sdkmanager --licenses
/opt/android-sdk/tools/bin/sdkmanager "system-images;android-25;google_apis;armeabi-v7a"
/opt/android-sdk/tools/bin/sdkmanager "emulator"
/opt/android-sdk/tools/bin/sdkmanager "platform-tools"
touch /home/ubuntu/.android/repositories.cfg
mkdir /opt/android-sdk/platforms
/opt/android-sdk/tools/bin/avdmanager -v create avd -f -n MyAVD -k "system-images;android-25;google_apis;armeabi-v7a" -p "/opt/android-sdk/avd"

Optional: eanbling/disabling certain HW features 
-------------------------------------------------
echo "hw.audioInput=no" >> /opt/android-sdk/avd/config.ini
echo "hw.audioOutput=no" >> /opt/android-sdk/avd/config.ini
echo "hw.cpu.ncore=2" >> /opt/android-sdk/avd/config.ini
echo "hw.camera.back=none" >> /opt/android-sdk/avd/config.ini
echo "hw.camera.front=none" >> /opt/android-sdk/avd/config.ini
echo "hw.gsmModem=no" >> /opt/android-sdk/avd/config.ini
echo "hw.gps=no" >> /opt/android-sdk/avd/config.ini
echo "hw.accelerometer=no" >> /opt/android-sdk/avd/config.ini
echo "hw.battery=no" >> /opt/android-sdk/avd/config.ini
echo "hw.trackBall=no" >> /opt/android-sdk/avd/config.ini
echo "hw.dPad=no" >> /opt/android-sdk/avd/config.ini
echo "hw.sensors.proximity=no" >> /opt/android-sdk/avd/config.ini
echo "hw.sensors.magnetic_field=no" >> /opt/android-sdk/avd/config.ini
echo "hw.sensors.orientation=no" >> /opt/android-sdk/avd/config.ini
echo "hw.sensors.temperature=no" >> /opt/android-sdk/avd/config.ini


Running the Emulator:
---------------------
/opt/android-sdk/emulator/emulator -ports 5554,5555 -avd MyAVD -no-window -no-audio -gpu swiftshader_indirect -show-kernel


Fixing 100% CPU: Disabling the "Ok Google" Hotword detection:
-------------------------------------------------------------
./adb shell "su root pm disable com.google.android.googlequicksearchbox"
Note: run after Emulator loaded


Notes:
------
1. We run ARM emulation on x86 instance.
2. We must use ARM ABI, since x86 requires KVM and EC2 instance doesn't support HW virtualization. 
3. We must disable GUI (-no-window) and audio (-no-audio) when running the emulator.
4. We must specify "-gpu swiftshader_indirect" when running the emulator to prevent a boot loop. 
5. First startup takes VERY LONG time. Use "-show-kernel" to see kernel messages and in secondary console window use "/opt/android-sdk/platform-tools/adb logcat" to see the system log while the emulator starts.

@atyachin Thanks I successfully launched emulator on ubuntu machine now I am trying to execute some automation scripts on emulator with appium 1.15 server however I am getting error process system is not responding app is launched successfully but while entering data it is giving error
do you have an any idea ?

sudo apt install default-jdk -> do not run this in Ubuntu 18 as that will install JDK 11 which android tools will complain about.
Instead - sudo apt install openjdk-8-jdk openjdk-8-jre

To accept all licenses -> yes | ./sdkmanager --licenses
To wait till emulator comes up - adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done;'

@atyachin Have you tried this ARM EC2 type https://aws.amazon.com/about-aws/whats-new/2018/11/introducing-amazon-ec2-a1-instances/ ?

@atyachin what you mean "re-login" line 10

@atyachin

I have one doubt,
It is slightly trivial but I want to know when have my emulator finished starting successfully, there is some logs i am getting continuously
after executing line 42 and its not failed, its going continuously

@atyachin on our ec2 instance, we are not able to reach http://dl.google.com/ to get image from repository. what is the link to download system image
"system-images;android-25;google_apis;armeabi-v7a" manually. My plan is to deploy image on artifactory and wget on ec2 instance

/sdkmanager --list is not showing emulator in the available package

/opt/android-sdk/tools/bin/sdkmanager "emulator" gives
Warning: Failed to find package emulator

I am trying it on EC2 A1.metal instance

I am trying the same steps on EC2 instance(Platform details:Linux/UNIX, Instance type:m4.large) from jenkins - execute shell:
Able to run create avd, java_home, android_sdk_root, android_home is all set.

On running /opt/android-sdk/emulator/emulator -ports 5554,5555 -avd MyAVD -no-window -no-audio -gpu auto -show-kernel
I get the following:
emulator: ERROR: AdbHostServer.cpp:102: Unable to connect to adb daemon on port: 5037
emulator: WARNING: encryption is off
Failed to open libEGL_translator: [libGL.so.1: cannot open shared object file: No such file or directory]
Failed to init_egl_dispatch
emulator: ERROR: OpenGLES initialization failed!
emulator: ERROR: OpenGLES emulation library could not be initialized!
emulator: ERROR: OpenGLES emulation failed to initialize. Please consider the following troubleshooting steps:

1. Make sure your GPU drivers are up to date.

2. Erase and re-download the emulator ($ANDROID_SDK_ROOT/emulator).

3. Try software rendering: Go to Extended Controls > Settings > Advanced tab and change "OpenGL ES renderer (requires restart)" to "Swiftshader".

Or, run emulator from command line with "-gpu swiftshader_indirect". 4. Please file an issue to https://issuetracker.google.com/issues?q=componentid:192727 and provide your complete CPU/GPU info plus OS and display setup.

crashhandler_die: fatal: OpenGLES emulation failed to initialize. Please consider the following troubleshooting steps:

1. Make sure your GPU drivers are up to date.

2. Erase and re-download the emulator ($ANDROID_SDK_ROOT/emulator).

3. Try software rendering: Go to Extended Controls > Settings > Advanced tab and change "OpenGL ES renderer (requires restart)" to "Swiftshader".

Or, run emulator from command line with "-gpu swiftshader_indirect". 4. Please file an issue to https://issuetracker.google.com/issues?q=componentid:192727 and provide your complete CPU/GPU info plus OS and display setup.

Segmentation fault (core dumped)
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Some help around this?

hello!

did you use this with android 28?
is it possible to do it at all?
the new sdk downloads are the command line tools and the corresponding system-image would be system-images;android-28;google_apis;x86_64 which does not have the ARM ABI option

thanks in advance

hello
while running emulator by this command
/opt/android-sdk/emulator/emulator -ports 5554,5555 -avd MyAVD -no-window -no-audio -gpu swiftshader_indirect -show-kernel
getting following error
PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value [/opt/android-sdk/]!

I am getting the same error as @dimple-doshi

Using

echo no | android-sdk-linux/tools/bin/avdmanager create avd --force --name testAVD --package "system-images;android-25;google_apis;armeabi-v7a"
...
android-sdk-linux/emulator/emulator  -ports 5554,5555 -avd testAVD -no-audio -no-window -gpu swiftshader_indirect -show-kernel &

Getting

adb: no emulators found
 Waiting for emulator to start
 emulator: WARNING: encryption is off
 emulator: ERROR: Could not load OpenGLES emulation library [libOpenglRender]: libGL.so.1: cannot open shared object file: No such file or directory
 emulator: ERROR: Retrying in program directory/lib64...
 emulator: ERROR: Could not load OpenGLES emulation library [<path to>/android-sdk-linux/emulator/qemu/linux-x86_64/lib64/libOpenglRender]: <path to>/android-sdk-linux/emulator/qemu/linux-x86_64/lib64/libOpenglRender.so: cannot open shared object file: No such file or directory (2nd try)
 emulator: ERROR: OpenGLES emulation failed to initialize. Please consider the following troubleshooting steps:
 1. Make sure your GPU drivers are up to date.
 2. Erase and re-download the emulator ($ANDROID_SDK_ROOT/emulator).
 3. Try software rendering: Go to Extended Controls > Settings > Advanced tab and change "OpenGL ES renderer (requires restart)" to "Swiftshader".
 Or, run emulator from command line with "-gpu swiftshader_indirect". 4. Please file an issue to https://issuetracker.google.com/issues?q=componentid:192727 and provide your complete CPU/GPU info plus OS and display setup.
 crashhandler_die: fatal: OpenGLES emulation failed to initialize. Please consider the following troubleshooting steps:
 1. Make sure your GPU drivers are up to date.
 2. Erase and re-download the emulator ($ANDROID_SDK_ROOT/emulator).
 3. Try software rendering: Go to Extended Controls > Settings > Advanced tab and change "OpenGL ES renderer (requires restart)" to "Swiftshader".
 Or, run emulator from command line with "-gpu swiftshader_indirect". 4. Please file an issue to https://issuetracker.google.com/issues?q=componentid:192727 and provide your complete CPU/GPU info plus OS and display setup.

What am I doing wrong here?

emulator: ERROR: Not enough disk space to run AVD 'MyAVD'. Exiting...

@atyachin Thanks very much for this document, it helped me get an emulator running, and I get 1 from adb wait-for-device shell getprop sys.boot_completed, which I think means that the emulator has completed booting and is ready.

But when I try to install an APK using adb install file_name.apk the command line gets stuck at Performing Streamed Install. I'm also not able to install the app using Gradle (I have built the app on the EC2 instance using Gradle).

Does anyone have an idea why this could be happening?

@kukabi
adb shell getprop sys.boot_completed returns 1 only indicates that the system has been booted but the services are just starting to load...
If you adb install right at that moment, there's a chance that necessary components for Package Manager are NOT ready yet. The result could be a dead-lock that causes your command adb install to get stuck. Eventually the Android WatchDog may restart your emulator.

Watchdog: *** WATCHDOG KILLING SYSTEM PROCESS: Blocked in handler on PackageManager (PackageManager)

I suggest that you open another console and run adb logcat to see what's going on with the system.

@nhphong thanks a lot, I will test in a bit and update here.

@nhphong Actually it worked this time, but it took 10 minutes to install the APK. I'm running adb install from another window, and the last time I didn't wait thinking the process was frozen. You may find the logcat output here. Do you know if there's a way to shorten this time?

Though I'm guessing it's because it's an armeabi-v7a. I remember it was extremely slow to run one locally too.

@nhphong So I got it all working on an EC2 instance (Ubuntu 18.04) including some passing instrumented tests but managed to get the WATCHDOG KILLING SYSTEM PROCESS error on a Docker instance (again, Ubuntu 18.04) on my laptop. Emulator restarted and I got adb: failed to install. What is a good way to be certain that the emulator is ready for adb install?

@kukabi
Use this to check when the boot animation finished

adb shell getprop init.svc.bootanim

For example:

adb wait-for-device shell 'while [[ $(getprop init.svc.bootanim) != stopped ]]; do sleep 1; done;'

Note that the above approach does not work if you start the emulator with -no-boot-anim option (for example, when you run a headless emulator on the memory)

Thanks for this gist.

2. We must use ARM ABI, since x86 requires KVM and EC2 instance doesn't support HW virtualization.

This is no longer the case- You can use x86 virtualization on any of the .metal EC2 instances, which provide direct HW access.

Here's the release announcement + case study: https://aws.amazon.com/jp/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/

Emulator on this solution is extremely slow do someone have experience with Amazon EC2 A1 Instances ?

We ended up GitHub actions with a Mac OS X virtual machine and we're very happy. Here's the test workflow.

Emulator on this solution is extremely slow do someone have experience with Amazon EC2 A1 Instances ?

@kukabi the point is to use Amazon infra

Is it possible to run multiple emulators > 20 in this infrastructure @atyachin ?

Hi，All
I want to boot Android with UEFI in an ARM virtual machine,is there any feasible way?

I keep getting "Process system isn't responding". I was using t2.xlarge Ubuntu x86 instance

Update howto:
https://gist.github.com/atyachin/2f7c6054c4cd6945397165a23623987d

Having an issue while running the emulator:

ERROR: Insufficient RAM free for launching emulator.
Please free up memory (close other programs and files),
or decrease the AVD RAM size and try again.

I have already tried everything regarding the ram size modifications in the config files to changing the storage setting on the Ec2 but it won’t work.