Skip to content

Commit bc29f84

Browse files
Radon10043Radon10043
committed
docs/linux: add instructions for fuzzing Linux x86-64 kernel on Android virtual device
Add new page for fuzzing Linux x86-64 kernel on Android virtual device. It explains how to build and run Android Generic System Image and Generic Kernel Image on Ubuntu host.
1 parent 1804e95 commit bc29f84

File tree

2 files changed

+253
-0
lines changed

2 files changed

+253
-0
lines changed

‎docs/linux/setup.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Instructions for a particular VM type or kernel architecture can be found on the
1010
- [Setup: Linux host, QEMU vm, riscv64 kernel](setup_linux-host_qemu-vm_riscv64-kernel.md)
1111
- [Setup: Linux host, QEMU vm, s390x kernel](setup_linux-host_qemu-vm_s390x-kernel.md)
1212
- [Setup: Linux host, Android device, arm32/64 kernel](setup_linux-host_android-device_arm-kernel.md)
13+
- [Setup: Linux host, Android virtual device, x86-64 kernel](setup_linux-host_android-virtual-device_x86-64-kernel.md)
1314
- [Setup: Linux isolated host](setup_linux-host_isolated.md)
1415
- [Setup: Ubuntu host, VMware vm, x86-64 kernel](setup_ubuntu-host_vmware-vm_x86-64-kernel.md)
1516

‎docs/linux/setup_linux-host_android-virtual-device_x86-64-kernel.md‎

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
# Setup: Linux host, Android virtual device, x86-64 kernel
2+
3+
This document details the steps involved in setting up a syzkaller instance fuzzing an `x86-64` linux kernel on an Android virtual device.
4+
5+
In the instructions below, the `$VAR` notation (e.g. `$GSI`, `$GKI`, etc.) is used to denote paths to directories that are either created when executing the instructions, or that you have to create yourself before running the instructions. Substitute the values for those variables manually.
6+
7+
Note:
8+
- All commands below assume root privileges.
9+
- It is recommended to have at least 64 GB of RAM and 500 GB of free disk space.
10+
11+
## Install prerequisites
12+
13+
Command:
14+
``` bash
15+
apt update
16+
apt install sudo git wget curl repo libncurses5 vim gcc make bison bc zip rsync language-pack-en-base
17+
```
18+
19+
## Cuttlefish
20+
21+
It is recommended to use [Cuttlefish](https://github.com/google/android-cuttlefish) to emulate Android devices. Build and install it from source (v1.16.0 as an example):
22+
23+
Command:
24+
``` bash
25+
apt install git devscripts equivs config-package-dev debhelper-compat golang curl
26+
git clone -b v1.16.0 https://github.com/google/android-cuttlefish
27+
cd android-cuttlefish
28+
tools/buildutils/build_packages.sh
29+
dpkg -i ./cuttlefish-base_*_*64.deb || sudo apt-get install -y -f
30+
dpkg -i ./cuttlefish-user_*_*64.deb || sudo apt-get install -y -f
31+
usermod -aG kvm,cvdnetwork,render root
32+
reboot
33+
```
34+
35+
## Generic System Images (GSI)
36+
37+
### Checkout GSI source
38+
39+
The GSI source checkout is close to 90 GB, and the build can take up about 300 GB of disk space.
40+
41+
Command:
42+
``` bash
43+
mkdir android13-gsi
44+
cd android13-gsi
45+
repo init -u https://android.googlesource.com/platform/manifest -b android13-gsi
46+
repo sync -c
47+
```
48+
49+
### Build GSI
50+
51+
Refresh the build environment and select the build target:
52+
53+
Command:
54+
``` bash
55+
source build/envsetup.sh
56+
lunch aosp_cf_x86_64_phone-userdebug
57+
```
58+
59+
The output should be as follows (may vary depending on the host):
60+
61+
``` text
62+
============================================
63+
PLATFORM_VERSION_CODENAME=REL
64+
PLATFORM_VERSION=13
65+
TARGET_PRODUCT=aosp_cf_x86_64_phone
66+
TARGET_BUILD_VARIANT=userdebug
67+
TARGET_BUILD_TYPE=release
68+
TARGET_ARCH=x86_64
69+
TARGET_ARCH_VARIANT=silvermont
70+
TARGET_2ND_ARCH=x86
71+
TARGET_2ND_ARCH_VARIANT=silvermont
72+
HOST_ARCH=x86_64
73+
HOST_2ND_ARCH=x86
74+
HOST_OS=linux
75+
HOST_OS_EXTRA=Linux-6.8.0-65-generic-x86_64-Ubuntu-22.04.4-LTS
76+
HOST_CROSS_OS=windows
77+
HOST_CROSS_ARCH=x86
78+
HOST_CROSS_2ND_ARCH=x86_64
79+
HOST_BUILD_TYPE=release
80+
BUILD_ID=TP1A.220624.019
81+
OUT_DIR=out
82+
PRODUCT_SOONG_NAMESPACES=device/generic/goldfish-opengl device/generic/goldfish device/generic/goldfish-opengl hardware/google/camera hardware/google/camera/devices/EmulatedCamera device/google/cuttlefish/apex/com.google.cf.wifi_hwsim external/mesa3d vendor/google_devices/common/proprietary/confirmatioui_hal
83+
============================================
84+
```
85+
86+
Start building:
87+
88+
Command:
89+
``` bash
90+
m
91+
```
92+
93+
You can now test your setup by launching the virtual device:
94+
95+
Command:
96+
```bash
97+
launch_cvd
98+
```
99+
100+
Open [http://localhost:8443](http://localhost:8443) in your browser, you should see a virtual device. Click `Connect` to interact with it as you would with a real phone. Press `Ctrl-C` in the terminal to stop the simulator.
101+
102+
## Kernel
103+
104+
### Checkout Android Generic Kernel Image (GKI) source
105+
106+
Command:
107+
``` bash
108+
mkdir common-android13-5.15
109+
cd common-android13-5.15
110+
repo init -u https://android.googlesource.com/kernel/manifest -b common-android13-5.15
111+
repo sync -c
112+
```
113+
114+
### Build GKI
115+
116+
We need to:
117+
- build the Android kernel with KASAN and KCOV;
118+
- enable `virtio-blk` so Cuttlefish can emulate the device as expected;
119+
- and minimize kernel size.
120+
121+
Add the following flags to the `update_kasan_config` function in `common-android13-5.15/common/build.config.gki_kasan.x86_64`:
122+
123+
``` text
124+
-e CONFIG_VIRTIO \
125+
-e CONFIG_VIRTIO_PCI \
126+
-e CONFIG_VIRTIO_BLK \
127+
-e CONFIG_VIRTIO_CONSOLE \
128+
-e CONFIG_VIRTIO_NET \
129+
-e CONFIG_CC_OPTIMIZE_FOR_SIZE \
130+
```
131+
132+
The modified `build.config.gki_kasan.x86_64` should look like this:
133+
134+
``` bash
135+
DEFCONFIG=gki_defconfig
136+
POST_DEFCONFIG_CMDS="check_defconfig && update_kasan_config"
137+
KERNEL_DIR=common
138+
LTO=none
139+
140+
function update_kasan_config() {
141+
${KERNEL_DIR}/scripts/config --file ${OUT_DIR}/.config \
142+
-e CONFIG_KASAN \
143+
-e CONFIG_KASAN_PANIC_ON_WARN \
144+
-e CONFIG_KCOV \
145+
-e CONFIG_PANIC_ON_WARN_DEFAULT_ENABLE \
146+
-e CONFIG_VIRTIO \
147+
-e CONFIG_VIRTIO_PCI \
148+
-e CONFIG_VIRTIO_BLK \
149+
-e CONFIG_VIRTIO_CONSOLE \
150+
-e CONFIG_VIRTIO_NET \
151+
-e CONFIG_CC_OPTIMIZE_FOR_SIZE \
152+
-d CONFIG_RANDOMIZE_BASE \
153+
-d CONFIG_KASAN_OUTLINE \
154+
--set-val CONFIG_FRAME_WARN 0 \
155+
-d CFI \
156+
-d CFI_PERMISSIVE \
157+
-d CFI_CLANG \
158+
-d SHADOW_CALL_STACK
159+
(cd ${OUT_DIR} && \
160+
make ${TOOL_ARGS} O=${OUT_DIR} olddefconfig)
161+
}
162+
```
163+
164+
Build the kernel. Using `LTO=none` can speed up the build.
165+
166+
Command:
167+
``` bash
168+
BUILD_CONFIG=common/build.config.gki_kasan.x86_64 LTO=none build/build.sh
169+
```
170+
171+
Build vendor modules with KASAN and KCOV:
172+
173+
Command:
174+
``` bash
175+
BUILD_CONFIG=common-modules/virtual-device/build.config.virtual_device_kasan.x86_64 LTO=none build/build.sh
176+
```
177+
178+
## syzkaller
179+
180+
### Build syzkaller
181+
182+
Build syzkaller as described [here](/docs/linux/setup.md#go-and-syzkaller).
183+
Then create a manager config like the following, replacing the environment
184+
variables `$GOPATH` and `$GKI` with their actual values.
185+
186+
``` json
187+
{
188+
"target": "linux/amd64",
189+
"http": "127.0.0.1:56741",
190+
"workdir": "$GOPATH/src/github.com/google/syzkaller/workdir/android/out",
191+
"kernel_obj": "$GKI/out/android13-5.15/dist",
192+
"syzkaller": "$GOPATH/src/github.com/google/syzkaller",
193+
"cover": true,
194+
"type": "adb",
195+
"vm": {
196+
"devices": ["0.0.0.0:6520"],
197+
"battery_check": true
198+
}
199+
}
200+
```
201+
202+
### Launch the virtual device
203+
204+
Launch the Android system with the KASAN and KCOV kernel.
205+
206+
Command:
207+
``` bash
208+
cd $GSI
209+
source build/envsetup.sh
210+
lunch aosp_cf_x86_64_phone-userdebug
211+
launch_cvd -daemon -kernel_path=$GKI/out/android13-5.15/dist/bzImage -initramfs_path=$GKI/out/android13-5.15/dist/initramfs.img
212+
```
213+
214+
Connect to the virtual device with adb:
215+
216+
Command:
217+
``` bash
218+
adb connect 0.0.0.0:6520
219+
```
220+
221+
List available virtual devices:
222+
223+
Command:
224+
``` bash
225+
adb devices
226+
```
227+
228+
### Run syzkaller
229+
230+
Run syzkaller manager:
231+
232+
Command:
233+
```bash
234+
cd $GOPATH/src/github.com/google/syzkaller
235+
./bin/syz-manager -config=android.cfg
236+
```
237+
238+
Now syzkaller should be running, you can check manager status with your web browser at `127.0.0.1:56741`.
239+
240+
If you get issues after `syz-manager` starts, consider running it with the `-debug` flag.
241+
242+
Here are some useful links:
243+
244+
- [github - google/android-cuttlefish](https://github.com/google/android-cuttlefish)
245+
- [AOSP - Cuttlefish virtual Android devices](https://source.android.com/docs/devices/cuttlefish)
246+
- [AOSP - Cuttlefish: Get started](https://source.android.com/docs/devices/cuttlefish/get-started)
247+
- [AOSP - Download the Android source](https://source.android.com/docs/setup/download)
248+
- [AOSP - Build Android](https://source.android.com/docs/setup/build/building)
249+
- [AOSP - Generic system images](https://source.android.com/docs/core/tests/vts/gsi#building-gsis)
250+
- [AOSP - Architecture overview](https://source.android.com/docs/core/architecture)
251+
- [AOSP - Build kernels](https://source.android.com/docs/setup/build/building-kernels)
252+
- [AOSP - Kernel branches and their build systems](https://source.android.com/docs/setup/reference/bazel-support)

0 commit comments

Comments
 (0)