Skip to content

Commit 1609704

Browse files
committed
docs(rfc): libCRS builder sidecar and snapshot
Signed-off-by: Andrew Chin <achin34@gatech.edu>
1 parent 0641e09 commit 1609704

1 file changed

Lines changed: 154 additions & 1 deletion

File tree

docs/RFC.md

Lines changed: 154 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616
- [CRS-Compose Configuration](#crs-compose-configuration)
1717
- [LLM Configuration](#llm-configuration)
1818
- [CRS Integration Interface (libCRS)](#crs-integration-interface-libcrs)
19+
- [Custom Builders](#custom-builders)
20+
- [Snapshot and Builder Sidecar](#snapshot-and-builder-sidecar)
1921
- [Output format standards](#output-format-standards)
2022
- [Delta Scan Mode](#delta-scan-mode)
2123
- [CRS Ensemble Support](#crs-ensemble-support)
@@ -489,6 +491,8 @@ required_llms:
489491

490492
#### Dockerfile Conventions
491493

494+
To accommodate the OSS-CRS workflow, we suggest following certain conventions when integrating a CRS.
495+
492496
**Builder Dockerfile** (`target_build_phase`): Uses the target project image as base via build arg. This allows CRS-specific instrumentation on top of the OSS-Fuzz project build.
493497

494498
```dockerfile
@@ -529,7 +533,8 @@ CRSs interact with the oss-crs infrastructure through **libCRS**, a Python libra
529533
| `POV` | Proof of Vulnerability inputs |
530534
| `SEED` | Fuzzing corpus seeds |
531535
| `BUG_CANDIDATE` | Potential bug reports |
532-
| `PATCH` | Bug fix patches |
536+
| `PATCH` | Bug fix patches (outputted by patch CRS) |
537+
| `DIFF` | Code diffs for delta-mode analysis (input for CRS) |
533538

534539
#### Core API
535540

@@ -554,8 +559,43 @@ crs.fetch(DataType.SEED, directory) # Fetch shared data
554559
555560
# Service Discovery
556561
domain = crs.get_service_domain("litellm") # Get service endpoint
562+
563+
# Builder Sidecar Operations (for bug-fixing CRSs)
564+
exit_code = crs.apply_patch_build(patch_path, response_dir, builder="builder-asan")
565+
exit_code = crs.run_pov(pov_path, harness_name, build_id, response_dir, builder="builder-asan")
566+
exit_code = crs.run_test(build_id, response_dir, builder="builder-asan")
567+
```
568+
569+
#### CLI Commands
570+
571+
libCRS is also available as the `libCRS` CLI executable inside containers:
572+
573+
```bash
574+
# Build output management
575+
libCRS submit-build-output <src_path> <dst_path>
576+
libCRS download-build-output <src_path> <dst_path>
577+
libCRS skip-build-output <dst_path>
578+
579+
# Data auto-sync (runs as daemon, blocks until container exits)
580+
libCRS register-submit-dir <TYPE> <path> [--log <log_path>]
581+
libCRS register-fetch-dir <TYPE> <path> [--log <log_path>]
582+
libCRS register-shared-dir <local_path> <shared_path>
583+
584+
# Manual data operations
585+
libCRS submit <TYPE> <path>
586+
libCRS fetch <TYPE> <path>
587+
588+
# Service discovery
589+
libCRS get-service-domain <service_name>
590+
591+
# Builder sidecar operations
592+
libCRS apply-patch-build <patch_path> <response_dir> --builder <name>
593+
libCRS run-pov <pov_path> <response_dir> --harness <name> --build-id <id> --builder <name>
594+
libCRS run-test <response_dir> --build-id <id> --builder <name>
557595
```
558596

597+
Where `TYPE` is one of: `pov`, `seed`, `bug-candidate`, `patch`, `diff`.
598+
559599
#### Environment Variables
560600

561601
CRSs receive the following environment variables from OSS-CRS:
@@ -580,6 +620,119 @@ CRSs may use the environment variables however they like, but we suggest handlin
580620
- `OSS_CRS_LLM_API_URL` and `OSS_CRS_LLM_API_KEY` for making LLM requests
581621
- `OSS_CRS_CPUSET` and `OSS_CRS_MEMORY_LIMIT` in case compute-heavy tasks need to scale along with the resource constraints applied by the operator
582622

623+
#### Custom Builders
624+
625+
A CRS may want to perform several custom target project compilation passes. For example, fuzzers need custom instrumentation of the source code, and static analysis tools also need to process the project's source code to produce analysis reports. OSS-CRS supports multiple custom builders to accommodate these demands.
626+
627+
CRS authors can choose between framework-provided builders and custom Dockerfiles for the `target_build_phase`. The `dockerfile` field accepts two formats:
628+
629+
- **Framework-provided:** `oss-crs-infra:<module>` references a builder shipped with oss-crs (e.g., `oss-crs-infra:default-builder`). These handle the standard OSS-Fuzz compile flow and snapshot creation automatically.
630+
- **Custom Dockerfile:** A path to a CRS-provided Dockerfile (e.g., `oss-crs/dockerfiles/builder.Dockerfile`). Custom builders use `ARG target_base_image` / `FROM $target_base_image` to layer CRS-specific instrumentation on top of the OSS-Fuzz project build.
631+
632+
```yaml
633+
target_build_phase:
634+
# Framework-provided builder for snapshot creation
635+
- name: asan-snapshot
636+
dockerfile: oss-crs-infra:default-builder
637+
snapshot: true
638+
additional_env:
639+
SANITIZER: address
640+
641+
# Custom builder with explicit outputs
642+
- name: coverage-build
643+
dockerfile: oss-crs/dockerfiles/builder.Dockerfile
644+
additional_env:
645+
BUILD_TYPE: coverage
646+
outputs:
647+
- coverage/build
648+
```
649+
650+
The same `oss-crs-infra:` prefix can be used in `crs_run_phase` modules to run framework-provided services (e.g., a builder sidecar server).
651+
652+
#### Snapshot and Builder Sidecar
653+
654+
CRSs may want to reuse their builder environment for on-demand rebuilds at the run phase. OSS-CRS provides mechanisms for snapshotting the builder environment and launching the CRS builders in the run phase. This comes with the benefits of more performative rebuilds and re-use of the builder environment without additional run phase setup.
655+
656+
A **snapshot** is a Docker image that captures a fully compiled and instrumented target project. Snapshots enable **incremental rebuilds**: instead of compiling from scratch for every patch attempt, a CRS applies a unified diff to the snapshot and only recompiles the changed files.
657+
658+
**Enabling snapshots** requires two fields in `crs.yaml`:
659+
660+
1. **`snapshot: true`** on a `target_build_phase` entry — tells `build-target` to compile the project, install libCRS and build scripts, then commit the container as a snapshot image.
661+
662+
2. **`run_snapshot: true`** on a `crs_run_phase` module — uses the snapshot image as the module's base image. This module runs a **builder sidecar**, an HTTP service that accepts patch, POV, and test requests.
663+
664+
```yaml
665+
# crs.yaml — bug-fixing CRS with builder sidecar
666+
target_build_phase:
667+
- name: asan-snapshot
668+
dockerfile: oss-crs-infra:default-builder
669+
snapshot: true
670+
additional_env:
671+
SANITIZER: address
672+
673+
crs_run_phase:
674+
patcher:
675+
dockerfile: oss-crs/patcher.Dockerfile
676+
builder-asan:
677+
dockerfile: oss-crs-infra:default-builder
678+
run_snapshot: true
679+
```
680+
681+
In this configuration, oss-crs launches `builder-asan` from the snapshot image. The patcher module communicates with it via libCRS:
682+
683+
**Patch → Build → Validate workflow:**
684+
685+
```python
686+
from libCRS import CRSUtils
687+
from pathlib import Path
688+
689+
crs = CRSUtils()
690+
691+
# 1. Apply a patch and rebuild (POST to builder sidecar /build)
692+
build_exit = crs.apply_patch_build(
693+
patch_path=Path("/work/fix.patch"),
694+
response_dir=Path("/work/build-result"),
695+
builder="builder-asan",
696+
)
697+
# Response dir contains: build_exit_code, build.log, build_id
698+
699+
# 2. Run POV against the patched build (POST to /run-pov)
700+
build_id = Path("/work/build-result/build_id").read_text()
701+
pov_exit = crs.run_pov(
702+
pov_path=Path("/work/crash-input"),
703+
harness_name="target_fuzzer",
704+
build_id=build_id,
705+
response_dir=Path("/work/pov-result"),
706+
builder="builder-asan",
707+
)
708+
# exit code 0 = no crash = patch fixed the bug
709+
710+
# 3. Run project tests against the patched build (POST to /run-test)
711+
test_exit = crs.run_test(
712+
build_id=build_id,
713+
response_dir=Path("/work/test-result"),
714+
builder="builder-asan",
715+
)
716+
# exit code 0 = tests pass
717+
```
718+
719+
Or equivalently via the CLI:
720+
721+
```bash
722+
# Apply patch and rebuild
723+
libCRS apply-patch-build /work/fix.patch /work/build-result --builder builder-asan
724+
725+
# Verify the POV no longer crashes
726+
BUILD_ID=$(cat /work/build-result/build_id)
727+
libCRS run-pov /work/crash-input /work/pov-result \
728+
--harness target_fuzzer --build-id "$BUILD_ID" --builder builder-asan
729+
730+
# Run project tests
731+
libCRS run-test /work/test-result --build-id "$BUILD_ID" --builder builder-asan
732+
```
733+
734+
The builder sidecar handles the compilation, binary execution, and test running inside the snapshot environment. CRS authors only need to generate patches and interpret the results.
735+
583736
### Output format standards
584737

585738
For bug-finding CRSs, the output contains crashes, corpus, and crs-specific data:

0 commit comments

Comments
 (0)