|
| 1 | +# Bazel Rules User Guide |
| 2 | + |
| 3 | +## Contents |
| 4 | + |
| 5 | +* [Defining fuzz tests](#defining-fuzz-tests) |
| 6 | +* [Integrating in your project](#integrating-in-your-project) |
| 7 | +* [Advanced topics](#advanced-topics) |
| 8 | + * [Defining fuzzing engines](#defining-fuzzing-engines) |
| 9 | + * [Customizing fuzz tests](#customizing-fuzz-tests) |
| 10 | +* [Rule reference](#rule-reference) |
| 11 | + |
| 12 | +## Defining fuzz tests |
| 13 | + |
| 14 | +The rule library provides support for writing *in-process* fuzz tests, which consist of a driver function that receives a generated input string and feeds it to the API under test. To make a complete fuzz test executable, the driver is linked with a fuzzing engine, which implements the test generation logic. The rule library provides out-of-the-box support for the most popular fuzzing engines (e.g., [libFuzzer][libfuzzer-doc] and [Honggfuzz][honggfuzz-doc]), and an extension mechanism to define new fuzzing engines. |
| 15 | + |
| 16 | +A fuzzing rule wraps a raw fuzz test executable and provides additional tools, such as the specification of a corpus and dictionary and a launcher that knows how to invoke the fuzzing engine with the appropriate set of flags. |
| 17 | + |
| 18 | +### Defining fuzz tests |
| 19 | + |
| 20 | +A fuzz test is specified using a [`cc_fuzz_test` rule](/docs/cc-fuzzing-rules.md#cc_fuzz_test). In the most basic form, a fuzz test requires a source file that implements the fuzz driver entry point. Let's consider a simple example that fuzzes the [RE2](https://github.com/google/re2) regular expression library: |
| 21 | + |
| 22 | +```python |
| 23 | +# BUILD file. |
| 24 | + |
| 25 | +load("@rules_fuzzing//fuzzing:cc_defs.bzl", "cc_fuzz_test") |
| 26 | + |
| 27 | +cc_fuzz_test( |
| 28 | + name = "re2_fuzz_test", |
| 29 | + srcs = ["re2_fuzz_test.cc"], |
| 30 | + deps = [ |
| 31 | + "@re2", |
| 32 | + ], |
| 33 | +) |
| 34 | +``` |
| 35 | + |
| 36 | +The fuzz driver implements the special `LLVMFuzzerTestOneInput` function that receives the fuzzer-generated string and uses it to drive the API under test: |
| 37 | + |
| 38 | +```cpp |
| 39 | +// Implementation file. |
| 40 | + |
| 41 | +#include <cstdint> |
| 42 | +#include <cstddef> |
| 43 | +#include <string> |
| 44 | + |
| 45 | +#include "re2/re2.h" |
| 46 | + |
| 47 | +extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) { |
| 48 | + RE2 re(std::string(reinterpret_cast<const char*>(data), size), RE2::Quiet); |
| 49 | + return 0; |
| 50 | +} |
| 51 | +``` |
| 52 | +
|
| 53 | +### Building and running |
| 54 | +
|
| 55 | +To build a fuzz test, you need to specify which fuzzing engine and what instrumentation to use for tracking errors during the execution of the fuzzer. Let's build the RE2 fuzz test using [libFuzzer][libfuzzer-doc] and the [Address Sanitizer (ASAN)][asan-doc] instrumentation, which catches memory errors such as buffer overflows and use-after-frees: |
| 56 | +
|
| 57 | +```sh |
| 58 | +$ bazel build -c opt --config=asan-libfuzzer //examples:re2_fuzz_test |
| 59 | +``` |
| 60 | + |
| 61 | +You can directly invoke this fuzz test executable if you know libFuzzer's command line interface. But in practice, you don't have to. For each fuzz test `<name>`, the rules library generates a number of additional targets that provide higher-level functionality to simplify the interaction with the fuzz test. |
| 62 | + |
| 63 | +One such target is `<name>_run`, which provides a simple engine-agnostic interface for invoking fuzz tests. Let's run our libFuzzer example: |
| 64 | +
|
| 65 | +```sh |
| 66 | +$ bazel run -c opt --config=asan-libfuzzer //examples:re2_fuzz_test_run |
| 67 | +``` |
| 68 | +
|
| 69 | +The fuzz test will start running locally, and write the generated tests under a temporary path under `/tmp/fuzzing`. By default, the generated tests persist across runs, in order to make it easy to stop and resume runs (possibly under different engines and configurations). |
| 70 | +
|
| 71 | +Let's interrupt the fuzz test execution (Ctrl-C), and resume it using the Honggfuzz engine: |
| 72 | + |
| 73 | +```sh |
| 74 | +$ bazel run -c opt --config=asan-honggfuzz //examples:re2_fuzz_test_run |
| 75 | +``` |
| 76 | + |
| 77 | +The `<name>_run` target accepts a number of engine-agnostic flags. For example, the following command runs the fuzz test with an execution timeout and on a clean slate (removing any previously generated tests). Note the extra `--` separator between Bazel's own flags and the launcher flags: |
| 78 | + |
| 79 | +```sh |
| 80 | +$ bazel run -c opt --config=asan-libfuzzer //examples:re2_fuzz_test_run \ |
| 81 | + -- --clean --timeout_secs=30 |
| 82 | +``` |
| 83 | + |
| 84 | +### Specifying seed corpora |
| 85 | + |
| 86 | +You can use the `corpus` attribute to specify a set of files that the fuzz test can use as a seed corpus when running in continuous fuzzing mode. The following example shows how to include all the files in a directory in the seed corpus: |
| 87 | + |
| 88 | +```python |
| 89 | +cc_fuzz_test( |
| 90 | + name = "fuzz_test", |
| 91 | + srcs = ["fuzz_test.cc"], |
| 92 | + corpus = glob(["fuzz_test_corpus/**"]), |
| 93 | +) |
| 94 | +``` |
| 95 | + |
| 96 | +Specifying a seed corpus is a [best practice][seed-corpus] that helps the fuzzer make progress faster. |
| 97 | + |
| 98 | +### Specifying dictionaries |
| 99 | + |
| 100 | +Similarly, you can speed up fuzzing by specifying a dictionary using the `dicts` attribute. A dictionary is a set of string tokens that the fuzzer can use to construct and mutate test inputs. The attribute accepts a list of files with one dictionary token per line specified in the [AFL/libFuzzer format](http://llvm.org/docs/LibFuzzer.html#dictionaries): |
| 101 | + |
| 102 | +```python |
| 103 | +cc_fuzz_test( |
| 104 | + name = "fuzz_test", |
| 105 | + srcs = ["fuzz_test.cc"], |
| 106 | +) |
| 107 | +``` |
| 108 | + |
| 109 | +### The fuzz test launcher |
| 110 | + |
| 111 | +Each fuzz test `<fuzz_test>` gets a `<fuzz_test>_run` target that can be used to launch the fuzzing executable in "continuous fuzzing" mode. The launcher provides a uniform command line interface regardless of the fuzzing engine or sanitizer used. |
| 112 | + |
| 113 | +Currently, the launcher offers the following options: |
| 114 | + |
| 115 | +* `--[no]clean`: If set, cleans up the output directory of the target before fuzzing (default: 'false'). |
| 116 | +* `--fuzzing_output_root`: The root directory for storing all generated artifacts during fuzzing. (default: '/tmp/fuzzing') |
| 117 | +* `--[no]regression`: If set, the script will trigger the target as a regression test. (default: 'false') |
| 118 | +* `--timeout_secs`: The maximum duration, in seconds, of the fuzzer run launched. (default: '0', a non-negative integer) |
| 119 | + |
| 120 | +### Built-in fuzzing engines |
| 121 | + |
| 122 | +* `@rules_fuzzing//fuzzing/engines:libfuzzer` provides libFuzzer support. Must be used with the `libfuzzer` engine instrumentation. |
| 123 | + |
| 124 | +* `@rules_fuzzing//fuzzing/engines:honggfuzz` provides Honggfuzz support. Must be used with the `honggfuzz` engine instrumentation. Requires importing its dependencies using the `honggfuzz_dependencies()` WORKSPACE function. |
| 125 | + |
| 126 | +* `@rules_fuzzing//fuzzing/engines:replay` provides a simple engine that just executes a set of test files. It can be combined with a sanitizer and can be used for regression tests or replaying crashes. |
| 127 | + |
| 128 | +* `@rules_fuzzing_oss_fuzz//:oss_fuzz_engine` provides a fuzzing engine that reflects the environment configuration of an [OSS-Fuzz build][bazel-oss-fuzz]. This engine is useful in the `build.sh` script of an OSS-Fuzz project. Requires importing its dependencies using the `oss_fuzz_dependencies()` WORKSPACE function. |
| 129 | + |
| 130 | +### Configuration flags |
| 131 | + |
| 132 | +The fuzzing rules library defines the following Bazel configuration flags that affect how a fuzz test is built: |
| 133 | + |
| 134 | +* `--@rules_fuzzing//fuzzing:cc_engine` specifies the fuzzing engine used to build and run the fuzz test. This flag should point to a `cc_fuzzing_engine` target. |
| 135 | + |
| 136 | +* `--@rules_fuzzing//fuzzing:cc_engine_instrumentation` specifies the engine-specific instrumentation to use. Valid values are: |
| 137 | + * `none`: No instrumentation. |
| 138 | + * `libfuzzer`: The libFuzzer-specific instrumentation. It should be used in conjunction with the `@rules_fuzzing//fuzzing/engines:libfuzzer` engine. |
| 139 | + * `honggfuzz`: The Honggfuzz-specific instrumentation. It should be used in conjunction with the `@rules_fuzzing//fuzzing/engines:honggfuzz` engine. |
| 140 | + * `oss-fuzz`: The instrumentation captured from the environment during an [OSS-Fuzz build][bazel-oss-fuzz]. It uses the `$FUZZING_CFLAGS` and `$FUZZING_CXXFLAGS` variables, if set, or falls back to `$CFLAGS` and `$CXXFLAGS` otherwise. |
| 141 | + |
| 142 | +* `--@rules_fuzzing//fuzzing:cc_engine_sanitizer` specifies the sanitizer configuration used to detect bugs. Valid values are: |
| 143 | + * `none`: No sanitizer instrumentation. |
| 144 | + * `asan`: [Address Sanitizer (ASAN)][asan-doc]. |
| 145 | + * `msan`: [Memory Sanitizer (MSAN)][msan-doc]. |
| 146 | + * `msan-origin-tracking`: MSAN with [origin tracking][msan-origin-tracking] enabled (useful for debugging crash reproducers; available separately due to it being 1.5-2x slower). |
| 147 | + |
| 148 | +* `--@rules_fuzzing//fuzzing:cc_fuzzing_build_mode` is a bool flag that specifies whether the special [`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` macro][fuzzing-build-mode] is defined during the build. This is turned on by default and most users should not need to change this flag. |
| 149 | + |
| 150 | +## Integrating in your project |
| 151 | + |
| 152 | +### Configuring the .bazelrc file |
| 153 | + |
| 154 | +Each fuzz test is built with a fuzzing engine and instrumentation specified as [build setting flags](#configuration-flags). For example, running a fuzz test in the libFuzzer / ASAN configuration would look like: |
| 155 | + |
| 156 | +```sh |
| 157 | +$ bazel test \ |
| 158 | + --//fuzzing:cc_engine=//fuzzing/engines:libfuzzer \ |
| 159 | + --@rules_fuzzing//fuzzing:cc_engine_instrumentation=libfuzzer \ |
| 160 | + --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan \ |
| 161 | + //examples:re2_fuzz_test |
| 162 | +``` |
| 163 | + |
| 164 | +This command is clearly too verbose to be used manually, so we recommend combining these options as a `--config` setting in your project's [`.bazelrc` file][bazelrc-docs]. For the example above, we can define an `asan-libfuzzer` config setting as follows: |
| 165 | + |
| 166 | +``` |
| 167 | +build:asan-libfuzzer --//fuzzing:cc_engine=//fuzzing/engines:libfuzzer |
| 168 | +build:asan-libfuzzer --@rules_fuzzing//fuzzing:cc_engine_instrumentation=libfuzzer |
| 169 | +build:asan-libfuzzer --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan |
| 170 | +``` |
| 171 | + |
| 172 | +To run your fuzz test, now you can simply invoke: |
| 173 | + |
| 174 | +```sh |
| 175 | +$ bazel test --config=asan-libfuzzer //examples:re2_fuzz_test |
| 176 | +``` |
| 177 | + |
| 178 | +You can add to your `.bazelrc` file as many configurations as you need. Feel free to use the [`.bazelrc` file of this repository](/.bazelrc) as a starting point. |
| 179 | + |
| 180 | +## Advanced topics |
| 181 | + |
| 182 | +### Defining fuzzing engines |
| 183 | + |
| 184 | +> TODO: The documentation for this feature is coming soon. |
| 185 | +
|
| 186 | +A fuzzing engine launcher script receives configuration through the following environment variables: |
| 187 | + |
| 188 | +| Variable | Description | |
| 189 | +|----------------------------|-------------| |
| 190 | +| `FUZZER_BINARY` | The path to the fuzz target executable. | |
| 191 | +| `FUZZER_TIMEOUT_SECS` | If set, a positive integer representing the timeout in seconds for the entire fuzzer run. | |
| 192 | +| `FUZZER_IS_REGRESSION` | Set to `1` if the fuzzer should run in regression mode (just execute the input tests), or `0` if this is a continuous fuzzer run. | |
| 193 | +| `FUZZER_DICTIONARY_PATH` | If set, provides a path to a fuzzing dictionary file. | |
| 194 | +| `FUZZER_SEED_CORPUS_DIR` | If set, provides a directory path to a seed corpus. | |
| 195 | +| `FUZZER_OUTPUT_ROOT` | A writable path that can be used by the fuzzer during its execution (e.g., as a workspace or for generated artifacts). See the variables below for specific categories of output. | |
| 196 | +| `FUZZER_OUTPUT_CORPUS_DIR` | A path under `FUZZER_OUTPUT_ROOT` where the new generated tests should be stored. | |
| 197 | +| `FUZZER_ARTIFACTS_DIR` | A path under `FUZZER_OUTPUT_ROOT` where generated crashes and other relevant artifacts should be stored. | |
| 198 | + |
| 199 | +## Rule reference |
| 200 | + |
| 201 | +* [`cc_fuzz_test`](/docs/cc-fuzzing-rules.md#cc_fuzz_test) |
| 202 | +* [`cc_fuzzing_engine`](/docs/cc-fuzzing-rules.md#cc_fuzzing_engine) |
| 203 | + |
| 204 | +<!-- Links --> |
| 205 | + |
| 206 | +[asan-doc]: https://clang.llvm.org/docs/AddressSanitizer.html |
| 207 | +[bazel-oss-fuzz]: https://google.github.io/oss-fuzz/getting-started/new-project-guide/bazel/ |
| 208 | +[bazelrc-docs]: https://docs.bazel.build/versions/master/guide.html#bazelrc-the-bazel-configuration-file |
| 209 | +[fuzzing-build-mode]: https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode |
| 210 | +[honggfuzz-doc]: https://github.com/google/honggfuzz |
| 211 | +[libfuzzer-doc]: https://llvm.org/docs/LibFuzzer.html |
| 212 | +[msan-doc]: https://clang.llvm.org/docs/MemorySanitizer.html |
| 213 | +[msan-origin-tracking]: https://clang.llvm.org/docs/MemorySanitizer.html#origin-tracking |
| 214 | +[seed-corpus]: https://github.com/google/fuzzing/blob/master/docs/good-fuzz-target.md#seed-corpus |
0 commit comments