agoda-com / docker-emulator-android Goto Github PK

Hey, I tried to use this docker image but faced several things related mainly to performance. I would be grateful for your help or advice.

Mainly using api28 image, built from the latest emulator_api_28_x86 branch

• Emulator boot time. I use different types of cloud instances(on GCE), but the average boot time from logs ~ 120000 ms. Sometimes it follows with emulator: VERBOSE: System.cpp:1486: Timed out with running command |/opt/android-sdk-linux/platform-tools/adb -s emulator-5554 shell pm revoke com.google.android.googlequicksearchbox android.permission.RECORD_AUDIO |

Most of the time it also resolves in "System UI not responding" message on the emulator screen which results in failed tests. How can I fix it?

• Are there any scalable examples besides k8s farm? If I just want to create 10-20 emulators of 1 API image and connect CI container to run tests on them? I use simple docker-compose with a scale, but emulator start time is still questionable.

• Do you have any CPU\Memory recommendations for host instances? According to api28 config, one emulator should use 2 cores and 3Gb of memory of the host machine.

When running, it returns

fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /root/.android/avd/x86/userdata.img

Failed to build the docker image. What could be the reason?
os: Ubuntu 16.04
command: make build
error:

terminate called after throwing an instance of 'std::out_of_range'
what(): basic_string::substr: __pos (which is 38) > this->size() (which is 37)
Aborted (core dumped)
The command '/bin/sh -c dpkg --add-architecture i386 && echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic main restricted universe multiverse" > /etc/apt/sources.list && echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-updates main restricted universe multiverse" >> /etc/apt/sources.list && echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-security main restricted universe multiverse" >> /etc/apt/sources.list && echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-backports main restricted universe multiverse" >> /etc/apt/sources.list && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -yq software-properties-common libstdc++6:i386 zlib1g:i386 libncurses5:i386 locales ca-certificates apt-transport-https curl unzip redir iproute2 openjdk-8-jdk xvfb x11vnc fluxbox nano libpulse0 telnet expect --no-install-recommends && locale-gen en_US.UTF-8 && curl -L $LINK_ANDROID_SDK > /tmp/android-sdk-linux.zip && unzip -q /tmp/android-sdk-linux.zip -d /opt/android-sdk-linux/ && rm /tmp/android-sdk-linux.zip && yes | sdkmanager --no_https --licenses && sdkmanager emulator tools platform-tools "platforms;android-28" "system-images;android-28;google_apis;x86" --verbose | uniq && echo no | avdmanager create avd -n "x86" --package "system-images;android-28;google_apis;x86" --tag google_apis && DEBIAN_FRONTEND=noninteractive apt-get purge -yq unzip openjdk-8-jdk && qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.qcow2 && mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.img && qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.qcow2 && mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.img && qemu-img convert -O qcow2 -c /root/.android/avd/x86.avd/userdata.img /root/.android/avd/x86.avd/userdata.qcow2 && mv /root/.android/avd/x86.avd/userdata.qcow2 /root/.android/avd/x86.avd/userdata.img && (qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.qcow2 && mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.img || true) && apt-get -yq autoremove && apt-get clean && apt-get autoclean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*' returned a non-zero code: 134
Makefile:21: recipe for target 'build' failed
make[1]: *** [build] Error 134
make[1]: Leaving directory '/home/cloud/src/docker-emulator-android/docker-emulator-android/build/android-28'
Makefile:21: recipe for target 'build' failed
make: *** [build] Error 2

adb_usb.ini is no longer used by adb, so I guess it can be removed from container

https://android-review.googlesource.com/c/platform/system/core/+/115816

I modified a Makefile so it includes only android-28:

VERSIONS ?= android-28

Then I ran 'make build' and now I am having this:

Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  openjdk-8-jdk* unzip*
  0 upgraded, 0 newly installed, 2 to remove and 5 not upgraded.
  After this operation, 2,134 kB disk space will be freed.
  (Reading database ... 26126 files and directories currently installed.)
  Removing openjdk-8-jdk:amd64 (8u212-b03-0ubuntu1.18.04.1) ...
  Removing unzip (6.0-21ubuntu1) ...
  Processing triggers for mime-support (3.60ubuntu1) ...
  (Reading database ... 26091 files and directories currently installed.)
  Purging configuration files for unzip (6.0-21ubuntu1) ...
  WARNING: Image format was not specified for '/root/.android/avd/x86.avd/userdata.img' and probing guessed raw.
           Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
                    Specify the 'raw' format explicitly to remove the restrictions.
                    Image resized.
                    e2fsck 1.44.1 (24-Mar-2018)
                    ext2fs_open2: Bad magic number in super-block
                    e2fsck: Superblock invalid, trying backup blocks...
                    e2fsck: Bad magic number in super-block while trying to open /root/.android/avd/x86.avd/userdata.img
                    The superblock could not be read or does not describe a valid ext2/ext3/ext4
                    filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
                    filesystem (and not swap or ufs or something else), then the superblock
                    is corrupt, and you might try running e2fsck with an alternate superblock:
                        e2fsck -b 8193 <device>
                         or
                             e2fsck -b 32768 <device>

                             The command '/bin/sh -c dpkg --add-architecture i386 &&     echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic main restricted universe multiverse" > /etc/apt/sources.list &&     echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-updates main restricted universe multiverse" >> /etc/apt/sources.list &&     echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-security main restricted universe multiverse" >> /etc/apt/sources.list &&     echo "deb mirror://mirrors.ubuntu.com/mirrors.txt bionic-backports main restricted universe multiverse" >> /etc/apt/sources.list &&     apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get install -yq software-properties-common libstdc++6:i386 zlib1g:i386 libncurses5:i386                         locales ca-certificates apt-transport-https curl unzip redir iproute2                         openjdk-8-jdk xvfb x11vnc fluxbox nano libpulse0 telnet expect                        --no-install-recommends &&     locale-gen en_US.UTF-8 &&     curl -L $LINK_ANDROID_SDK > /tmp/android-sdk-linux.zip &&     unzip -q /tmp/android-sdk-linux.zip -d /opt/android-sdk-linux/ &&     rm /tmp/android-sdk-linux.zip &&     yes | sdkmanager --no_https --licenses &&     yes | sdkmanager emulator tools platform-tools "platforms;android-28" "system-images;android-28;google_apis;x86" --verbose | uniq &&     echo no | avdmanager create avd -n "x86" --package "system-images;android-28;google_apis;x86" --tag google_apis &&     DEBIAN_FRONTEND=noninteractive apt-get purge -yq unzip openjdk-8-jdk &&     qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.qcow2 &&     mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/system.img &&     qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.qcow2 &&     mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/userdata.img &&     qemu-img resize /root/.android/avd/x86.avd/userdata.img 2G &&     e2fsck -fy /root/.android/avd/x86.avd/userdata.img &&     resize2fs /root/.android/avd/x86.avd/userdata.img &&     qemu-img convert -O qcow2 -c /root/.android/avd/x86.avd/userdata.img /root/.android/avd/x86.avd/userdata.qcow2 &&     mv /root/.android/avd/x86.avd/userdata.qcow2 /root/.android/avd/x86.avd/userdata.img &&     (qemu-img convert -O qcow2 -c /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.img /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.qcow2 &&     mv /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.qcow2 /opt/android-sdk-linux/system-images/android-28/google_apis/x86/vendor.img || true) &&     apt-get -yq autoremove &&     apt-get clean &&     apt-get autoclean &&     rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*' returned a non-zero code: 8
                             make[1]: *** [build] Error 8
                             make[1]: Leaving directory `/opt/docker-emulator-android/build/android-28'
                             make: *** [build] Error 2

What might be a problem?

I faced with the problem that command
sdkmanager emulator tools platform-tools "platforms;{{ platform }}" "system-images;{{ platform }};google_apis;x86" --verbose
requires to accept license from January 16, 2019, but yes command had not passed in the pipe before.

Problem fixed after I had added it to the script. Should I prepare PR?

Hey guys, don't know if someone is still maintaining this repository, but if so, I would like to have a more detailed instructions on how to connect with openSTF.

thks for any ideas, and congrats on this awesome project.

Following the Readme.md I grab the docker-registy-url with
docker info | grep -i registry
and got https://index.docker.io/v1/

This I used for

export DOCKER_USER=<my user>
export DOCKER_PASS=<my password>
make PROXY=docker-registry-url/ build tag login push

But I run into

make PROXY=https://index.docker.io/v1/ build tag login push
for version in android-16 android-17 android-18 android-19 android-21 android-22 android-23 android-24 android-25 android-26 android-27 android-28; do \
	mkdir -p ./build/$version ; \
	sed "s/{{ platform }}/$version/g" templates/Dockerfile > build/$version/Dockerfile ; \
	sed "s/{{ platform }}/$version/g" templates/config.ini > build/$version/config.ini ; \
	sed "s/{{ platform }}/$version/g" templates/start.sh > build/$version/start.sh ; \
	sed "s/{{ platform }}/$version/g" templates/Makefile > build/$version/Makefile ; \
	sed "s/{{ platform }}/$version/g" templates/snapshot.sh > build/$version/snapshot.sh ; \
	sed "s/{{ platform }}/$version/g" templates/snapshot.expect > build/$version/snapshot.expect ; \
	sed "s/{{ platform }}/$version/g" templates/take_snapshot.sh > build/$version/take_snapshot.sh ; \
	cp base/* ./build/$version ; \
done
for version in android-16 android-17 android-18 android-19 android-21 android-22 android-23 android-24 android-25 android-26 android-27 android-28; do \
	make -C build/$version build; \
done
make[1]: Entering directory '/home/hannes/git/docker-emulator-android/build/android-16'
docker build \
	--build-arg VCS_REF=`git rev-parse --short HEAD` \
	--build-arg IMAGE_VERSION=1.0.0 \
	-t https://index.docker.io/v1/agoda/docker-emulator-android-16:latest /home/hannes/git/docker-emulator-android/build/android-16
invalid argument "https://index.docker.io/v1/agoda/docker-emulator-android-16:latest" for "-t, --tag" flag: invalid reference format

Great project!

For a research project I need to run multiple android emulators in the cloud. So far, I tried to set it up on

• AWS EC2
  • using this container on an Ubuntu host
  • did not work because the emulator needs hardware acceleration

     emulator: CPU Acceleration: DISABLED
     emulator: CPU Acceleration status: Could not open /dev/kvm : Is a directory
     emulator: ERROR: x86 emulation currently requires hardware acceleration!
     Please ensure KVM is properly installed and usable.
     CPU acceleration status: Could not open /dev/kvm : Is a directory
    

• paperspace
• I installed Android studio on a Windows host, since they have nested virtualization as an option if you contact support
• it was very unstable and the machine always hangs itself up completely when I start the emulator.

This thread about nested virtualization suggests that I should try google cloud or azure.

Before looking into it, I would love to hear whether/how others run this docker in the cloud.

It looks like emulator won't start when adb is renamed to _adb

Dismissing the error won't resume emulator booting sequence.
There's a setting to provide custom adb path, but it won't accept _adb.

Due to this issue, emulator never comes online in the android-farm.

Hello,

I'm not an Android developer.
I tried to modify Dockerfile to build up a ARM emulator, but I failed.
Could you help me to release an ARM version?

Thanks a lot.

after restart container emulator not worked
screen with logs from container

Right now anyone can merge changes without any peer feedback. I'd like to suggest to setup GH in a way that people are required to have at least 1 peer review (ideally 2).

Also, some form of CI that tests that docker images are created and are indeed working would be nice. Ideally with comparisons of image size and startup performance.

Thanks! Let me know if I can help you on how to achieve that with GH :)

I am getting the following error while trying to run docker command
Error:
Error response from daemon: pull access denied for agoda-com/docker-emulator-android-23, repository does not exist or may require 'docker login'.