Skip to content

Commit 8c8edd3

Browse files
xairyxairy
committed
docs: update USB documentation
1 parent d291dd2 commit 8c8edd3

File tree

1 file changed

+127
-48
lines changed

1 file changed

+127
-48
lines changed

‎docs/linux/external_fuzzing_usb.md‎

Lines changed: 127 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,25 @@
11
External USB fuzzing for Linux kernel
22
=====================================
33

4-
syzkaller supports fuzzing the Linux kernel USB subsystem from the external side.
5-
Instead of relying on external hardware (like [Facedancer](https://github.com/usb-tools/Facedancer)-based boards) or VM management software features (like QEMU [usbredir](https://www.spice-space.org/usbredir.html)), syzkaller fuzzes USB fully within a (potentially-virtualized) environment that runs the Linux kernel.
4+
syzkaller supports fuzzing the Linux kernel USB subsystem from the external USB device side.
5+
Instead of relying on physical hardware (like [Facedancer](https://github.com/usb-tools/Facedancer)-based boards) or VM management software features (like QEMU [usbredir](https://www.spice-space.org/usbredir.html)), syzkaller fuzzes USB fully within a (potentially-virtualized) environment that runs the Linux kernel.
66

77
The USB fuzzing support in syzkaller is based on:
88

9-
1. [Raw Gadget](https://github.com/xairy/raw-gadget) — kernel module that implements a low-level interface for the Linux USB Gadget subsystem (now in the mainline kernel);
10-
2. [Dummy HCD/UDC](https://github.com/xairy/raw-gadget/tree/master/dummy_hcd) — kernel module that sets up virtual USB Device and Host controllers that are connected to each other inside the kernel (existed in the mainline kernel for a long time);
9+
1. [Raw Gadget](https://github.com/xairy/raw-gadget) — a kernel module that implements a low-level interface for the Linux USB Gadget subsystem (now in the mainline kernel);
10+
2. [Dummy HCD/UDC](https://github.com/xairy/raw-gadget/tree/master/dummy_hcd) — a kernel module that sets up virtual USB Device and Host controllers that are connected to each other inside the kernel (existed in the mainline kernel for a long time);
1111
3. KCOV changes that allow collecting coverage [from background kernel threads and interrupts](https://docs.kernel.org/dev-tools/kcov.html#remote-coverage-collection) (now in the mainline kernel);
1212
4. syzkaller changes built on top of the other parts; see the [Internals](/docs/linux/external_fuzzing_usb.md#Internals) section.
1313

1414
Besides this documentation page, for details, see:
1515

16-
- [Coverage-Guided USB Fuzzing with Syzkaller](https://docs.google.com/presentation/d/1z-giB9kom17Lk21YEjmceiNUVYeI6yIaG5_gZ3vKC-M/edit?usp=sharing) talk ([video](https://www.youtube.com/watch?v=1MD5JV6LfxA)) from OffensiveCon 2019 (the talk was given while the USB fuzzing support was a work-in-progress, so some details are outdated);
16+
- [Coverage-Guided USB Fuzzing with Syzkaller](https://docs.google.com/presentation/d/1z-giB9kom17Lk21YEjmceiNUVYeI6yIaG5_gZ3vKC-M/edit?usp=sharing) talk ([video](https://www.youtube.com/watch?v=1MD5JV6LfxA)) from OffensiveCon 2019 (was given while the USB fuzzing support was a work-in-progress, so some details are outdated);
1717

18-
- [Fuzzing USB with Raw Gadget](https://docs.google.com/presentation/d/1sArf2cN5tAOaovlaL3KBPNDjYOk8P6tRrzfkclsbO_c/edit?usp=sharing) talk ([video](https://www.youtube.com/watch?v=AT3PQjKxa_c)) from BSides Munich 2022 (more up-to-date, but less in-depth).
18+
- [Fuzzing USB with Raw Gadget](https://docs.google.com/presentation/d/1sArf2cN5tAOaovlaL3KBPNDjYOk8P6tRrzfkclsbO_c/edit?usp=sharing) talk ([video](https://www.youtube.com/watch?v=AT3PQjKxa_c)) from BSides Munich 2022 (more up-to-date, but less in-depth);
1919

20-
See [this page](/docs/linux/found_bugs_usb.md) for the list of bugs found in the Linux kernel USB stack so far.
21-
22-
23-
## Internals
24-
25-
syzkaller defines 6 pseudo-syscalls for emulating USB devices for fuzzing (see the [syzlang descriptions](/sys/linux/vusb.txt) and the [C](/executor/common_usb.h) [implementations](/executor/common_usb_linux.h)):
26-
27-
1. `syz_usb_connect` — handles the enumeration process of a new USB device (in simple terms: connects a new USB device; in detail: handles all requests to the control endpoint until a `SET_CONFIGURATION` request is received);
28-
2. `syz_usb_connect_ath9k` — flavor of `syz_usb_connect` for connecting an `ath9k` USB device.
29-
Not implemented as a `$`variant of `syz_usb_connect`, as `ath9k` expects a compatible device to immediately handle the firmware download requests after the enumeration (after the `SET_CONFIGURATION` request);
30-
3. `syz_usb_disconnect` — disconnects a USB device;
31-
4. `syz_usb_control_io` — handles a post-enumeration control request (`IN` or `OUT`);
32-
5. `syz_usb_ep_write` — handles a non-control `IN` request on an endpoint;
33-
6. `syz_usb_ep_read` — handles a non-control `OUT` request on an endpoint.
34-
35-
The syzlang descriptions for these pseudo-syscalls are targeted at a few different layers:
36-
37-
1. The common USB enumeration code is targeted by the generic `syz_usb_connect` variant.
38-
In addition, this generic variant also briefly touches the enumeration code in various USB drivers: the USB device descriptor fields get [patched](/sys/linux/init_vusb.go) during the program generation;
20+
- [External fuzzing of Linux kernel USB drivers with syzkaller](https://docs.google.com/presentation/d/1ba7Au3Gt6dEQAsfZmjUdzjVWHKxE_EdaJGU9WOSF-Ts/edit?usp=sharing) talk (outlines steps for adding new basic USB descriptions).
3921

40-
2. The code of the class-specific drivers is targeted by `syz_usb_connect$hid`, `syz_usb_connect$cdc_ecm`, and other variants (accompanied by matching `syz_usb_control_io$*` and `syz_usb_ep_read/write$*` pseudo-syscalls).
41-
The descriptor fields for these `syz_usb_connect` variants are also intended to be patched during the program generation based on the driver class (to exercise various driver quirks), but so far, this has only been implemented for the HID class;
42-
43-
3. The code of the device-specific drivers is intended to be targeted by more `syz_usb_connect` variants whose descriptors do not get patched and are fully defined in syzlang instead.
44-
So far, the only such (partial) descriptions have been added for the `ath9k` driver (`syz_usb_connect_ath9k` and `syz_usb_ep_write$ath9k_ep*`).
22+
See [this page](/docs/linux/found_bugs_usb.md) for the list of bugs found in the Linux kernel USB stack so far.
4523

4624

4725
## Setting up
@@ -55,17 +33,17 @@ So far, the only such (partial) descriptions have been added for the `ath9k` dri
5533
Optionally, also enable the config options for the specific USB drivers that you wish to fuzz.
5634

5735
As an alternative, you can use the [config](/dashboard/config/linux/upstream-usb.config) from the syzbot USB fuzzing instance.
58-
This config has the options for many USB drivers commonly-used in distro kernels enabled;
36+
This config has many USB drivers commonly-used in distro kernels enabled;
5937

6038
3. Build the kernel;
6139

62-
4. Optionally, update syzkaller descriptions by [extracting the USB IDs](/docs/linux/external_fuzzing_usb.md#updating-syzkaller-usb-ids).
40+
4. Optionally, update syzkaller descriptions by [extracting the USB IDs](#updating-usb-ids).
6341

6442
This step is recommended if you wish to just rely on the existing syzlang descriptions to fuzz all USB drivers enabled in your kernel config.
6543

6644
If you plan to add new syzlang descriptions that target a specific USB driver, this step can be skipped;
6745

68-
5. Optionally, write syzlang descriptions for the USB driver you wish to fuzz;
46+
5. Optionally, [write](#internals) [syzlang](#dealing-with-complicated-descriptors) [descriptions](#handling-post-enumeration-control-requests) for the USB driver you wish to fuzz;
6947

7048
6. Enable `syz_usb_connect`, `syz_usb_disconnect`, `syz_usb_control_io`, `syz_usb_ep_write` and `syz_usb_ep_read` pseudo-syscalls (or any of their specific variants) in the manager config;
7149

@@ -80,30 +58,108 @@ So far, the only such (partial) descriptions have been added for the `ath9k` dri
8058
You should also see a decent amount of coverage in `drivers/usb/core/*` after the first few programs get added to the corpus.
8159

8260

83-
## Limitations
61+
## Internals
8462

85-
Most of the limitations of the USB fuzzing support in syzkaller stem from the features [missing](https://github.com/xairy/raw-gadget/tree/master?tab=readme-ov-file#limitations) from the Raw Gadget and Dummy HCD/UDC implementations (USB 3, isochronous transfers, etc).
63+
syzkaller defines 5 main pseudo-syscalls for fuzzing USB drivers (see the [syzlang descriptions](/sys/linux/vusb.txt) and the [C](/executor/common_usb.h) [implementations](/executor/common_usb_linux.h)):
8664

87-
In addition, `syz_usb_connect` only supports devices with a single configuration (but this can be fixed).
88-
This is not a critical limitation, as most kernel drivers are tied to specific interfaces and do not care about the configurations.
89-
However, there are USB devices whose drivers assume the device to have multiple specific configurations.
65+
1. `syz_usb_connect` — handles the enumeration process of a new USB device (in simple terms: connects a new USB device; in detail: handles all control requests up until a `SET_CONFIGURATION` request is received);
66+
2. `syz_usb_disconnect` — disconnects a USB device;
67+
3. `syz_usb_control_io` — handles a post-enumeration control request (`IN` or `OUT`);
68+
4. `syz_usb_ep_write` — handles a non-control `IN` request on an endpoint;
69+
5. `syz_usb_ep_read` — handles a non-control `OUT` request on an endpoint.
70+
71+
Additionally, there is the `syz_usb_connect_ath9k` pseudo-syscall targeted to handle a few [post-enumeration control requests](#handling-post-enumeration-control-requests) the `ath9k` driver expects.
72+
73+
The syzlang descriptions for these pseudo-syscalls are targeted at a few different parts of the USB subsystem:
74+
75+
1. The common USB enumeration code is targeted by the generic `syz_usb_connect` variant.
76+
77+
In addition, this generic variant also briefly touches the enumeration code in various USB drivers: the USB device descriptor fields get [patched](#usb-ids-patching) during the program generation;
78+
79+
2. The code of the class-specific drivers is targeted by `syz_usb_connect$hid`, `syz_usb_connect$cdc_ecm`, and other variants (accompanied by matching `syz_usb_control_io$*` and `syz_usb_ep_read/write$*` pseudo-syscalls).
80+
81+
The descriptor fields for these `syz_usb_connect` variants are also intended to be [patched](#usb-ids-patching) during the program generation based on the matching rules specific to the driver class (to exercise various driver quirks).
82+
So far, this has only been implemented for the HID ([see](/sys/linux/init_vusb.go) `generateUsbHidDeviceDescriptor`) and the printer ([see](/sys/linux/init_vusb.go) `generateUsbPrinterDeviceDescriptor`) classes;
83+
84+
3. The code of the device-specific drivers is intended to be targeted by more `syz_usb_connect` variants whose descriptors do not get patched and are fully defined in syzlang instead. (However, they can be patched as well for drivers that define quirks.)
85+
86+
So far, the only such (partial) descriptions have been added for the `ath9k`, `rtl8150`, `sierra_net`, and `lan78xx` drivers.
87+
88+
89+
## Dealing with complicated descriptors
90+
91+
Many USB drivers expect a fixed USB descriptors structure from the connected USB device.
92+
Writing syzlang descriptions for these drivers is straightforward: just describe the required structures in syzlang.
93+
94+
However, some drivers allow various permutations of the interface/endpoint descriptors (typically the case for class-specific drivers like UVC).
95+
96+
Describing a fixed descriptor structure for such drivers in syzlang would allow passing the driver descriptor checks and efficiently fuzz the post-enumeration communication.
97+
But it would also prevent syzkaller from exploring the various error-checking paths within the driver's enumeration code.
98+
On the other hand, defining a relaxed structure for the descriptors would make syzkaller often fail the driver's enumeration and thus would not allow efficiently fuzzing the post-enumeration communication.
99+
100+
There are two proposed ways to handle such drivers:
101+
102+
1. Writing relaxed descriptions and adding seed programs (aka runtests).
103+
104+
This works by only defining a single `syz_usb_connect` variant with the relaxed descriptions of the USB descriptors but also adding a [seed program](#seed-programs) (into `sys/linux/test/`) that contains the `syz_usb_connect` pseudo-syscall with the specific fixed values to pass the driver's enumeration checks;
90105

106+
2. Adding two `syz_usb_connect` variants.
91107

92-
## Runtests
108+
One with relaxed descriptions to explore the error-checking paths; and the other with fixed descriptions to allow fuzzing the post-enumeration communication.
93109

94-
There are a few [runtests](/sys/linux/test/) for the USB pseudo-syscalls.
95-
They are named starting with the `vusb` prefix and can be run as:
96110

97-
``` bash
98-
./bin/syz-manager -config usb-manager.cfg -mode run-tests -tests vusb
99-
```
111+
## Handling post-enumeration control requests
100112

113+
Many USB drivers finish their enumeration procedure with the `SET_CONFIGURATION` request.
114+
Following this, these drivers start functioning normally and are ready to accept both control and non-control requests.
101115

102-
## Updating syzkaller USB IDs
116+
However, some USB drivers expect certain control requests to be handled by the device directly following the `SET_CONFIGURATION` request.
117+
Without these requests being handled, these drivers abort their execution and disconnect the device.
103118

104-
syzkaller uses a list of hardcoded [USB IDs](/sys/linux/init_vusb_ids.go) that are [patched](/sys/linux/init_vusb.go) into `syz_usb_connect` (for the generic and the HID variants) during the program generation.
119+
There are two proposed ways to handle such drivers:
105120

106-
To update the syzkaller USB IDs to match the USB drivers enabled in your kernel config:
121+
1. Adding seed programs (aka runtests).
122+
123+
This works by adding a [seed program](#seed-programs) (into `sys/linux/test/`) that contains both the `syz_usb_connect` pseudo-syscall for the specific driver and a few `syz_usb_control_io` pseudo-sycalls that handle the required control requests.
124+
125+
This is the currently recommended way of handling such drivers, as it allows syzkaller to permute the expected post-`SET_CONFIGURATION` control requests and thus possibly trigger unexpected driver behavior during their handling.
126+
127+
See [#6283](https://github.com/google/syzkaller/pull/6283) for a reference implementation of this approach;
128+
129+
2. Modifying the behavior of the `syz_usb_connect` pseudo-syscall.
130+
131+
An alternative approach is to modify the C implementation of the `syz_usb_connect` pseudo-syscall (or add a similar new pseudo-syscall) to handle the required post-`SET_CONFIGURATION` requests directly from C.
132+
133+
Right now, this approach is only implemented for the `ath9k` driver via `syz_usb_connect_ath9k`.
134+
Unlike `syz_usb_connect`, `syz_usb_connect_ath9k` also handles the post-`SET_CONFIGURATION` firmware download requests expected by `ath9k`.
135+
136+
If this approach is to be used for other drivers, the proper implementation should rework and extend `syz_usb_connect_ath9k` instead of adding more similar pseudo-syscalls.
137+
For example, add a new argument to `syz_usb_connect` that allows specifiying (in syzlang) the responses to post-`SET_CONFIGURATION` control requests and which one them is the last one (i.e., the condition for exiting the pseudo-syscall).
138+
And then port the hardcoded responses from the C implementation of `syz_usb_connect_ath9k` into syzlang and extend the functionality based on the specific new encountered cases.
139+
140+
Note: There is also a way to describe unusual pre-`SET_CONFIGURATION` `GET_DESCRIPTOR` requests via the `conn_descs` argument of `syz_usb_connect`, but none of the class/driver-specific descriptions use this feature at the moment.
141+
However, this approach does not allow specifying the order of the responses if there are multiple requests of the same type and also does not allow handling pre-`SET_CONFIGURATION` non-`GET_DESCRIPTOR` requests, as no described drivers required this so far.
142+
143+
144+
## USB IDs patching
145+
146+
Many USB drivers implement various quirks depending of which device is connected.
147+
Such quirks are either defined in the mathing rule table for the driver (a table of `usb_device_id` structures; [see](https://elixir.bootlin.com/linux/v6.16/source/sound/usb/card.c#L1263) e.g. `usb_audio_ids`) or hardcoded in the drivers code (by manually checking e.g. the Vendor and Product IDs against certain values; [see](https://elixir.bootlin.com/linux/v6.16/source/drivers/usb/class/usblp.c#L213) e.g. `quirk_printers`).
148+
149+
To exercise these driver quirks, syzkaller has to provide certain values (aka IDs) within the USB descriptors for the emulated device.
150+
Listing these USB IDs manually in syzlang descriptions is inefficient (there are too many), so syzkaller employs the way of automatically [extracting the IDs](#updating-usb-ids) before building and then [patching them in](/sys/linux/init_vusb.go) during the program generation.
151+
152+
The IDs defined in the driver matching rules can be extracted (and patched in) automatically; see [Updating USB IDs](#updating-usb-ids).
153+
154+
However, the hardcoded IDs need to be specified manually (extacting them automatically is hard, as they are often embedded into the drivers code in non-standard ways).
155+
The proposed approach is to read the driver code to find out the hardcoded IDs and then modify the program generation code to embed them into the appropriate USB decriptors ([see](/sys/linux/init_vusb.go) e.g. `generateUsbPrinterDeviceDescriptor`).
156+
157+
158+
## Updating USB IDs
159+
160+
syzkaller relies on a list of automatically-extracted [USB IDs](/sys/linux/init_vusb_ids.go) that are [patched](/sys/linux/init_vusb.go) into `syz_usb_connect` (for the generic, the HID, and the printer variants) during the program generation.
161+
162+
To update the syzkaller USB IDs based on the USB drivers enabled in your kernel config:
107163

108164
1. Apply [this](/tools/syz-usbgen/usb_ids.patch) kernel patch;
109165

@@ -113,7 +169,7 @@ To update the syzkaller USB IDs to match the USB drivers enabled in your kernel
113169

114170
Assuming you have `CONFIG_USB_RAW_GADGET=y` enabled, you can just run the [keyboard emulation program](https://raw.githubusercontent.com/xairy/raw-gadget/master/examples/keyboard.c);
115171

116-
4. Use [syz-usbgen](/tools/syz-usbgen/usbgen.go) script to update [sys/linux/init_vusb_ids.go](/sys/linux/init_vusb_ids.go):
172+
4. Use [syz-usbgen](/tools/syz-usbgen/usbgen.go) to update [sys/linux/init_vusb_ids.go](/sys/linux/init_vusb_ids.go):
117173

118174
``` bash
119175
./bin/syz-usbgen $KERNEL_LOG ./sys/linux/init_vusb_ids.go
@@ -122,6 +178,29 @@ To update the syzkaller USB IDs to match the USB drivers enabled in your kernel
122178
5. Revert the applied kernel patch and rebuild the kernel before starting syzkaller.
123179

124180

181+
## Seed programs
182+
183+
There are a few [seed programs](/sys/linux/test/) (aka runtests) for the USB pseudo-syscalls (named starting with the `vusb` prefix).
184+
185+
These seed programs serve two purposes:
186+
187+
1. Allow syzkaller to [go](#dealing-with-complicated-descriptors) [deeper](#handling-post-enumeration-control-requests) into the driver code without having to write detailed syzlang descriptions;
188+
189+
2. Allow veryfing the USB fuzzing functionality and catching potential descriptions breaking changes by running the seed programs as runtests:
190+
191+
``` bash
192+
./bin/syz-manager -config usb-manager.cfg -mode run-tests -tests vusb
193+
```
194+
195+
## Limitations
196+
197+
Most of the limitations of the USB fuzzing support in syzkaller stem from the features [missing](https://github.com/xairy/raw-gadget/tree/master?tab=readme-ov-file#limitations) from the Raw Gadget and Dummy HCD/UDC implementations (USB 3, isochronous transfers, etc).
198+
199+
In addition, `syz_usb_connect` only supports devices with a single configuration (but this can be fixed).
200+
This is not a critical limitation, as most kernel drivers are tied to specific interfaces and do not care about the configurations.
201+
However, there are USB devices whose drivers assume the device to have multiple specific configurations.
202+
203+
125204
## Things to improve
126205

127206
The core support for USB fuzzing is in place, but there's still space for improvement:

0 commit comments

Comments
 (0)