Robert Foss
September 02, 2016
Share this post:
Reading time:
Note: This guide only applies to aosp version 6.x.x, and has never been tested with anything else. It is unlikely to work with aosp 7.x.x, and will definitely not work with >8.x.x.
Developing Linux for Android on Qemu allows you to do some things that are not necessarily possible using the stock emulator. For my purposes I need access to a GPU and be able to modify the driver, which is where Virgilrenderer and Qemu comes in handy.
The guide below helps you compile Android and run it on top of Qemu with Mesa/Virgilrenderer supplying a virtual GPU. Because of this, the following guide is aimed at Linux hosts.
This guide is based on Rob Herrings fantastic guide, but has been slightly streamlined and had physical hardware support stripped out.
These dependencies were available on Ubuntu 16.04, some alternative packages might be needed for other distributions.
sudo apt install autoconf gcc-aarch64-linux-gnu libaio-dev libbluetooth-dev libbrlapi-dev libbz2-dev libcap-dev libcap-ng-dev libcurl4-gnutls-dev libepoxy-dev libfdt-dev libgbm-dev libgles2-mesa-dev libglib2.0-dev libibverbs-dev libjpeg8-dev liblzo2-dev libncurses5-dev libnuma-dev librbd-dev librdmacm-dev libsasl2-dev libsdl1.2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh2-1-dev libtool libusb-1.0-0 libusb-1.0-0-dev libvde-dev libvdeplug-dev libvte-2.90-dev libxen-dev valgrind xfslibs-dev xutils-dev zlib1g-dev |
Naturally all of the paths below are configurable, this is just what I used.
export PROJECT_PATH="/opt/qemu_android" export VIRGLRENDERER_PATH="${PROJECT_PATH}/virglrenderer" export QEMU_PATH="${PROJECT_PATH}/qemu" export LINUX_PATH="${PROJECT_PATH}/linux" export ANDROID_PATH="${PROJECT_PATH}/android" export ANDROID_TOOLS_PATH="${PROJECT_PATH}/android-tools" |
Virglrenderer creates a virtual 3D GPU, that allows the Qemu guest to use the graphics capabilities of the host machine.
git clone git://git.freedesktop.org/git/virglrenderer ${VIRGLRENDERER_PATH} cd ${VIRGLRENDERER_PATH} ./autogen.sh make sudo make install |
Qemu is a full system emulator, and supports a multitude of machine architectures. We're going to to use x86_64 but also build support for arm64/aarch64.
git clone git://git.qemu-project.org/qemu.git ${QEMU_PATH} mkdir ${QEMU_PATH}/build cd ${QEMU_PATH}/build ../configure --target-list=aarch64-softmmu,x86_64-softmmu --enable-gtk --with-gtkabi=3.0 --enable-kvm make -j |
Build trunk of mainline linux kernel.
Important: The below instructions use upstream/master but during testing of this guide, https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git and the fences branch was used due to SW_SYNC not yet being included in upstream. Inclusion is targeted for v4.9.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ${LINUX_PATH} cd ${LINUX_PATH} wget http://memcpy.io/files/2016-08-30/Kconfig -O ${LINUX_PATH}/.config make oldconfig make -j |
Important: If you decide not to use the .config linked in this step, a few Kconfig options need to be set:
CONFIG_ANDROID=y CONFIG_ANDROID_BINDER_IPC=y CONFIG_AUDIT=y CONFIG_HAVE_ARCH_AUDITSYSCALL=y CONFIG_AUDITSYSCALL=y CONFIG_AUDIT_WATCH=y CONFIG_AUDIT_TREE=y CONFIG_SECURITY_SELINUX=y CONFIG_SECURITY_SELINUX_BOOTPARAM=y CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1 CONFIG_SECURITY_SELINUX_DISABLE=y CONFIG_SECURITY_SELINUX_DEVELOP=y CONFIG_SECURITY_SELINUX_AVC_STATS=y CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=0 CONFIG_DEFAULT_SECURITY_SELINUX=y CONFIG_DEFAULT_SECURITY="selinux" CONFIG_VIRTIO_BLK=y CONFIG_SCSI_VIRTIO=y CONFIG_VIRTIO_NET=y CONFIG_VIRTIO_CONSOLE=y CONFIG_HW_RANDOM_VIRTIO=y CONFIG_DRM_VIRTIO_GPU=y CONFIG_VIRT_DRIVERS=y CONFIG_VIRTIO=y CONFIG_VIRTIO_PCI=y CONFIG_VIRTIO_PCI_LEGACY=y CONFIG_VIRTIO_BALLOON=y CONFIG_VIRTIO_INPUT=y CONFIG_VIRTIO_MMIO=y CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y CONFIG_NET_9P=y CONFIG_NET_9P_VIRTIO=y CONFIG_SYNC=y CONFIG_SW_SYNC=y CONFIG_SYNC_FILE=y |
Build the Android Open Source Project.
Important: When running source build/envsetup.sh make sure that you are using bash. I had issues running lunch using zsh.
mkdir ${ANDROID_PATH} cd ${ANDROID_PATH} repo init -u https://android.googlesource.com/platform/manifest -b master cd .repo git clone https://github.com/robherring/android_manifest.git -b android-6.0 local_manifests cd .. repo sync -j cd device/linaro/generic make defconfig make all cd ../../.. # The following snippet must be run in bash bash source build/envsetup.sh # Select linaro_x86_64-userdebug lunch make -j # We don't need to use bash any longer exit |
As of this writing SW_SYNC related patches by Gustavo Padovan has yet to be included into AOSP, and therefore has to be included included manually until it is upstreamed. After switching to this branch, the AOSP project has to be rebuilt again.
cd ${ANDROID_PATH}/external/drm_hwcomposer git remote add padovan git://git.collabora.com/git/user/padovan/android-system-core.git git fetch padovan git checkout padovan/master |
Fetch the make boot image script. This script later assembles the boot image, boot.img.
git clone https://android.googlesource.com/platform/system/core.git $ANDROID_TOOLS_PATH |
When running the below script, make sure that the all of the paths from step two have been exported.
wget http://memcpy.io/files/2016-08-30/boot_android_qemu.sh -O ${PROJECT_PATH}/boot_android_qemu.sh chmod +x ${PROJECT_PATH}/boot_android_qemu.sh ${PROJECT_PATH}/boot_android_qemu.sh x86_64 |
Hopefully this guide will have enabled you build the required software and run Android on Qemu with a virtual GPU. The post was has been a part of work undertaken by my employer Collabora.
08/10/2024
Having multiple developers work on pre-merge testing distributes the process and ensures that every contribution is rigorously tested before…
15/08/2024
After rigorous debugging, a new unit testing framework was added to the backend compiler for NVK. This is a walkthrough of the steps taken…
01/08/2024
We're reflecting on the steps taken as we continually seek to improve Linux kernel integration. This will include more detail about the…
27/06/2024
With each board running a mainline-first Linux software stack and tested in a CI loop with the LAVA test framework, the Farm showcased Collabora's…
26/06/2024
WirePlumber 0.5 arrived recently with many new and essential features including the Smart Filter Policy, enabling audio filters to automatically…
12/06/2024
Part 3 of the cmtp-responder series with a focus on USB gadgets explores several new elements including a unified build environment with…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (43)
Roman:
Feb 14, 2017 at 06:30 PM
Thanks for an article, Robert.
One question: did you try to run android on with qemu arm?
I did not succeed with it: built kernel for arm, android for linaro_arm-userdebug and qemu for arm-softmmu, but the first "[ 0.000000] Linux version 4.10.0-rc7+ ..." debug print does not appear in qemu output.
Reply to this comment
Reply to this comment
Robert Foss:
Feb 14, 2017 at 06:38 PM
I haven't actually been running qemu in ARM mode, just x86. The qemu ARM build was mostly a precaution, in case the need arose.
But I am seeing the first log line:
[ 0.000000] Linux version 4.9.0-rc3+ (hottuna@xps9550) (gcc version 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) ) #52 SMP Mon Feb 13 13:05:19 EST 2017
Reply to this comment
Reply to this comment
Roman:
Feb 14, 2017 at 07:20 PM
I guess you used padovan kernel with you Kconfig, right?
Curious how to enable bootloader logs in order to understand what goes wrong. Are you aware of how to do this?
Reply to this comment
Reply to this comment
Robert Foss:
Feb 14, 2017 at 08:45 PM
Yes, I used Padovans kernel with the linked kconfig.
No, I haven't really touched the bootloader.
Reply to this comment
Reply to this comment
Dmitriy S.:
Apr 15, 2017 at 03:27 PM
Hello, I carefully followed your instructions to compile Android and tried to run it in QEMU, but I got error after executing this command:
${PROJECT_PATH}/boot_android_qemu.sh x86_64
It says that file or directory 'x86_64' doesn't exist. It is because QEMU cannot find disk image which is the last argument in the script (the "$*" at the end). Maybe there need to be some other path?
Reply to this comment
Reply to this comment
Robert Foss:
Apr 20, 2017 at 07:08 PM
Hi Dmitriy,
Thanks for the question.
For the arch "x86_64" the build script seems to have an issue where it tries to use the x86 path instead of the x86_64 one.
I've updated the script now, please download it again.
I hope that solves your issues.
Rob.
Reply to this comment
Reply to this comment
Jungsoo Park:
Jun 12, 2017 at 12:44 AM
Thanks for an article, Robert.
I have a error message follow this article.
git clone https://github.com/robherring/android_manifest.git -b android-6.0 local_manifests -> fatal: Remote branch android-6.0 not found in upstream origin
If you have any other git site, please recommand.
thank you.
Reply to this comment
Reply to this comment
Robert Foss:
Jun 12, 2017 at 03:15 PM
Hey,
Thanks for noticing that. That branch must have been removed. Try using the master branch instead:
git clone https://github.com/robherring/android_manifest.git -b master local_manifests
This is however untested territory, so I can't promise that it will work.
Good luck!
Reply to this comment
Reply to this comment
Ines:
Jul 31, 2017 at 09:56 AM
Hello,
I carefully followed your instructions to compile Android and tried to run it in QEMU but i used version 7.1 by changing
git clone https://github.com/robherring/android_manifest.git -b android-6.0 local_manifests
with
git clone https://github.com/robherring/android_manifest.git -b android-7.1.0_r1
git clone https://github.com/robherring/android_manifest.git -b android-6.0 local_manifests
, but I got error after executing this command:
${PROJECT_PATH}/boot_android_qemu.sh x86_64
It says:
mkbootimg: error: argument --kernel: can't open '/home/ybenayed/qemu_android/linux/arch/x86_64/boot/bzImage': [Errno 2] No such file or directory: '/home/ybenayed/qemu_android/linux/arch/x86_64/boot/bzImage'
Reply to this comment
Reply to this comment
Robert Foss:
Jul 31, 2017 at 04:43 PM
Hi Ines,
The error you are seeing seems unrelated to the switch to Android 7.1, that being said, I would expect things to break when switching.
It is not an entirely straight forward process.
Rob.
Reply to this comment
Reply to this comment
Aidan Gauland:
Aug 03, 2017 at 08:48 PM
Thank you very much for this tutorial! It definitely saved me a lot of time in getting to this point. I now have Android booting, but the UI is unusable because I get a persistent "com.android.phone has stopped" error. Do I need to disable building some components of Android for hardware that the VM does not have?
(Notes: I built the latest stable release of qemu (2.9.0) and the Linux kernel (4.12.4) instead of master. I also had to skip the "make defconfig" line from the Android build, because it gave the error "mkdir: missing operand".)
Reply to this comment
Reply to this comment
Robert Foss:
Aug 16, 2017 at 07:42 PM
Hey Aidan,
The "com.android.phone has stopped" error is one that I haven't encountered unfortunately.
I think the best bet for trying to figure out what's causing it is to have a look at the logcat logs.
Rob.
Reply to this comment
Reply to this comment
Matan Dobrushin:
Aug 12, 2017 at 08:27 PM
Hy Robert,
Great guide!
I have an issue on the android section, i get this error:
~/Android/qemu_android/android/device/linaro/generic$ make defconfig
mkdir: missing operand
Try 'mkdir --help' for more information.
Makefile:134: *** failed to create output directory "". Stop.
Seems like the output directory isn't configured properly, any ideas?
Thanks!
Reply to this comment
Reply to this comment
Robert Foss:
Aug 16, 2017 at 07:44 PM
Hey Matan,
It would seem that the version of mkdir that you are running is missing support for some option.
Have a look in ~/Android/qemu_android/android/device/linaro/generic/Makefile line 134 to see what options are being given to mkdir, and try to replace the unsopported ones with something that your platform provides.
Rob.
Reply to this comment
Reply to this comment
Matan Dobrushin:
Aug 17, 2017 at 03:58 PM
Hey Rob,
Thank you for your reply.
I don't think the problem is mkdir, I'm using ubuntu 16.
The problem is that the argument passed to mkdir is an empty string, therefore it yells at me that I'm missing operand.
For some reason, $(KBUILD_OUTPUT) is not defined properly and therefore it is empty.
Matan
Reply to this comment
Reply to this comment
Julio:
Sep 14, 2017 at 08:28 AM
Hi!, to fix the error, add this:
cd device/linaro/generic
export KBUILD_OUTPUT="$(pwd)/../build"
make defconfig
Regards.
Reply to this comment
Reply to this comment
Masterboy:
Sep 16, 2017 at 11:44 AM
Thank you for the script. Could you update the script and the instructions because they don't work anymore? Could you tell me why you use export VIRGLRENDERER_PATH="${PROJECT_PATH}/virglrenderer" but don't use it in your qemu loading script?
Reply to this comment
Reply to this comment
Robert Foss:
Sep 20, 2017 at 08:30 PM
Hey,
You're quite welcome.
Unfortunately this script is not maintained, but you are very welcome to fork it.
Rob.
Reply to this comment
Reply to this comment
Ismail:
Sep 20, 2017 at 07:47 PM
When I run 'make oldconfig' or directly run 'make' with proper ARCH and CROSS_COMPILE parameters, it asks endless questions.
I want to use vexpress_defconfig, it doesnt work.
Have you ever tried with vexpress_defconfig to build it?
Reply to this comment
Reply to this comment
Robert Foss:
Sep 20, 2017 at 08:32 PM
Hey Ismail,
Unfortunately I don't have access to the versatile express devices, and have never tried building against that target.
Rob.
Reply to this comment
Reply to this comment
Ismail:
Sep 20, 2017 at 08:58 PM
Thanks for your prompt response. How can I deal with the endless y/n question on your .config after running make oldconfig" or "make" command?
Does it work with the most recent stabile kernel version?
Reply to this comment
Reply to this comment
Robert Foss:
Sep 20, 2017 at 11:11 PM
Hey again,
I would either start with a .config file that has been used successfully with this target before.
Often it can be found in "/proc/config.gz", if not try using the defaults. Just hold enter for a few seconds.
Rob.
Reply to this comment
Reply to this comment
Simon:
Nov 02, 2017 at 04:40 PM
Who can help me with my problem? I followed all instruction and I it doesn't start , it shows black screen with pinguin and "android" text, no graphic
log of console show:
binder: 1928:1928 transaction failed 29189, size 0-0
binder: 1928:1928 transaction failed 29189, size 0-0
binder: 1928:1928 transaction failed 29189, size 0-0
binder: 1928:1928 transaction failed 29189, size 0-0
binder: 1928:1928 transaction failed 29189, size 0-0
Reply to this comment
Reply to this comment
Robert Foss:
Nov 02, 2017 at 08:07 PM
Those logs are normal to see, and not indicative of the black screen.
As to what's causing the black screen, I can't really tell you.
I would suggest digging into the code and enabling more debugging.
Reply to this comment
Reply to this comment
Simon:
Nov 03, 2017 at 02:00 PM
Could you look to my log? https://justpaste.it/1d398
Reply to this comment
Reply to this comment
Robert Foss:
Nov 13, 2017 at 09:45 PM
Hi Simon,
Those logs don't reveal anything interesting.
Try running logcat on the device for more complete logs. However, I can't debug this issues for you.
Good luck!
Rob.
Reply to this comment
Reply to this comment
Robert Foss:
Nov 17, 2017 at 02:49 PM
I'm still not seeing anything out of place. Try running logcat.
Reply to this comment
Reply to this comment
nclman:
Nov 17, 2017 at 07:47 AM
Hi, I have been following Rob Herring and your work to run Android on QEMU, but no success so far. It seems like some graphic device issue.
I noticed that you did not require "CONFIG_DRM_BOCHS" in your kernel. But this was required by in Rob Herring's steps. Which should be "correct"?
>cd device/linaro/generic
>make defconfig
What is the expected output here? Is it a ".config" file? I have not been able to generate this but have no idea why.
Reply to this comment
Reply to this comment
Robert Foss:
Nov 17, 2017 at 02:52 PM
No, a .config isn't expected. Though the syntax us quite similar to the kernel one.
This will just propagate the configuration options from the device config to the right places.
I think running make all would cover the same ground.
Reply to this comment
Reply to this comment
nclman:
Nov 17, 2017 at 08:38 AM
Does anything look bad here? I had to copy some libs to /vendor/lib64/egl/ to get to this stage. Otherwise, system complains of missing files...
1579 1579 I SurfaceFlinger: SurfaceFlinger's main thread ready to run. Initializaing graphics H/W...
1579 1579 I SurfaceFlinger: Phase offset NS: 1000000
1579 1579 D libEGL : Emulator has vendor provided software renderer, qemu.gles is set to 2.
1579 1579 D vndksupportL Loading /vendor/lib64/egl/libGLES_mesa.so from current namespace instead of sphal namespace.
1579 1579 D libEGL : Loaded /vendor/lib64/egl/libGLES_mesa.so
1579 1579 D vndksupport: Loading /vendor/lib64/hw/gralloc.default.so from current namespace instead of sphal namespace.
1579 1579 W EGL-MAIN: fail to get drm fd
1579 1579 W libEGL : eglInitialize(...) failed (EGL_NOT_INITIALIZED)
Reply to this comment
Reply to this comment
Robert Foss:
Nov 17, 2017 at 02:55 PM
From the /vendor/lib64/egl/ I'm surmising that you are running Android O.
This guide is a bit outdated, and has never been put through its paces with Android O, and I would not expect it to work.
The error you're seeing relates to the graphics stack, and gralloc specifically. What's actually wrong, I can't really tell you.
Reply to this comment
Reply to this comment
nclman:
Nov 22, 2017 at 08:48 AM
What would be a recommended Android version to start off?
7.1.2?
7.1.1?
7.0.0?
Reply to this comment
Reply to this comment
Robert Foss:
Nov 22, 2017 at 02:58 PM
Hey nclman,
Unfortunately I no longer have my repo for this around, so I can't check, but Android 6 was the newest version available at the time this post was written.
Android 7.x.x isn't out of the question, but Android 8 is.
Rob.
Reply to this comment
Reply to this comment
nclman;:
Nov 24, 2017 at 08:37 AM
Seems like the toolchain for Android 6 isn't so readily available anymore.
Any chance you are able to try it out on a newer Android version?
Reply to this comment
Reply to this comment
Robert Foss:
Nov 28, 2017 at 07:27 PM
Hey nclman,
Which part of the toolchain is missing?
Unfortunately I'm not able to support a new Android version for the purpose of this blog post.
Rob.
Reply to this comment
Reply to this comment
Anthony Lee:
Nov 29, 2017 at 09:29 AM
I downloaded AOSP of android-8.0, and build images of target aosp_x86_64-eng (suppose this can be used by android-emulator/qemu-variant), and I git clone the up-to-date qemu for the emulator.
Since the AOSP already output raw images, unfortunately the AOS cannot boot or (even cannot mount fstab).
With android-emulator, I could:
emulator -kernel out/target/product/generic_x86_64/kernel-ranchu -show-kernel
-system system.img \
-ramdisk out/target/product/generic_x86_64/ramdisk.img \
-initdata out/target/product/generic_x86_64/userdata.img \
-data userdata-qemu.img \
-memory 2048 -debug init -selinux permissive \
-skindir android-sdk/platforms/android-27/skins/ -skin WXGA800-7in \
-qemu -enable-kvm -smp 4
However, this kernel_ranchu cannot apply to QEMU enhanced:
qemu-standalone/build/x86_64-softmmu/qemu-system-x86_64 \
-enable-kvm -smp 4 \
-append "console=tty0 console=ttyS0 debug drm.debug=0x0 androidboot.selinux=permissive" \
-m 2048 \
-serial mon:stdio \
-kernel kernel-ranchu \
-initrd ramdisk.img \
-drive index=0,if=none,id=system,file=system.img \
-device virtio-blk-pci,drive=system \
-drive index=1,if=none,id=cache,file=cache.img \
-device virtio-blk-pci,drive=cache \
-drive index=2,if=none,id=userdata,file=userdata.img \
-device virtio-blk-pci,drive=userdata \
-drive index=3,if=none,id=vendor,file=vendor.img \
-device virtio-blk-pci,drive=vendor \
-device virtio-gpu-pci,virgl -display gtk,gl=on
Shall we always use any possible *re-generated* ramdisk.img for correct init or later?
Reply to this comment
Reply to this comment
Robert Foss:
Nov 29, 2017 at 04:45 PM
Hey Anthony,
The last Android version this was tested on was 6.X.X, it _may_ work on 7.X.X, but not on 8.X.X.
As for specific kernel version, I can't really comment on them.
Sorry that I can't be of more help.
Rob.
Reply to this comment
Reply to this comment
M:
May 28, 2018 at 04:52 AM
git clone https://github.com/robherring/android_manifest.git -b android-6.0 local_manifests
Not found this branch
Reply to this comment
Reply to this comment
Robert Foss:
Jun 03, 2018 at 11:08 AM
Hey m,
It would seem that an external dependency has been changed. Unfortunately there isn't much I can do about it at this point.
However I'm hoping to be able to spin a new updated blogpost along these lines once some new Android features have landed in the virgl drivers.
Robert.
Reply to this comment
Reply to this comment
John:
Sep 28, 2018 at 11:49 PM
Hi! While running 'make defconfig' under device/linaro/generic I have the following error:
(...)
*** Default configuration is based on target 'defconfig'
*** Default configuration is based on target 'defconfig'
make[4847]: execvp: make: Argument list too long
/home/my_user/android/android/device/linaro/generic/Makefile:234: recipe for target 'scripts_basic' failed
make[4847]: *** [scripts_basic] Error 127
/home/my_user/android/android/device/linaro/generic/scripts/kconfig/Makefile:104: recipe for target 'defconfig' failed
make[4846]: *** [defconfig] Error 2
(...)
I'm running ubuntu 18.04. Any ideas? Thanks!
Reply to this comment
Reply to this comment
Robert Foss:
Oct 01, 2018 at 08:30 PM
Hey John,
Like noted above, this guide is no longer valid for >6.0 versions of android.
But! I think I have seen errors like that while using zsh instead of bash.
I hope that helps!
Rob.
Reply to this comment
Reply to this comment
Mario Zio:
May 22, 2021 at 05:39 PM
so,isn't the right time to update it ?
Reply to this comment
Reply to this comment
Mario Zio:
May 23, 2021 at 03:39 PM
Hello to everyone.
I’m trying several roads to be able to emulate a recent version of android on top of the jetson nano,but at the moment I failed. Someone of you can put my shoes on the right direction ? thanks.
Reply to this comment
Reply to this comment
Add a Comment