-
-
Notifications
You must be signed in to change notification settings - Fork 26
ci: add criterion benchmarks and OSS-Fuzz integration #80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
eefa9c2
f39d160
effc8d4
23c8a29
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| [changelog] | ||
| header = """ | ||
| # Changelog | ||
|
|
||
| All notable changes to this project will be documented in this file. | ||
|
|
||
| The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), | ||
| and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). | ||
| """ | ||
| body = """ | ||
| {% if version %}\ | ||
| ## [{{ version | trim_start_matches(pat="v") }}] - {{ timestamp | date(format="%Y-%m-%d") }} | ||
| {% else %}\ | ||
| ## [Unreleased] | ||
| {% endif %}\ | ||
| {% for group, commits in commits | group_by(attribute="group") %} | ||
| ### {{ group | upper_first }} | ||
| {% for commit in commits %} | ||
| - {% if commit.scope %}**{{ commit.scope }}**: {% endif %}{{ commit.message | upper_first }}\ | ||
| {% if commit.breaking %} [**BREAKING**]{% endif %} \ | ||
| ([{{ commit.id | truncate(length=7, end="") }}](https://github.com/webrtc-rs/rtc/commit/{{ commit.id }}))\ | ||
| {% endfor %} | ||
| {% endfor %}\n | ||
| """ | ||
| footer = "" | ||
| trim = true | ||
|
|
||
| [git] | ||
| conventional_commits = true | ||
| filter_unconventional = true | ||
| split_commits = false | ||
| commit_preprocessors = [] | ||
| commit_parsers = [ | ||
| { message = "^feat", group = "Features" }, | ||
| { message = "^fix", group = "Bug Fixes" }, | ||
| { message = "^perf", group = "Performance" }, | ||
| { message = "^refactor", group = "Refactoring" }, | ||
| { message = "^doc", group = "Documentation" }, | ||
| { message = "^test", group = "Testing" }, | ||
| { message = "^chore|^ci|^build", group = "Miscellaneous" }, | ||
| { message = "^revert", group = "Reverted Commits" }, | ||
| ] | ||
| protect_breaking_commits = false | ||
| filter_commits = false | ||
| tag_pattern = "v[0-9].*" | ||
| skip_tags = "" | ||
| ignore_tags = "" | ||
| topo_order = false | ||
| sort_commits = "oldest" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| name: Benchmarks | ||
|
|
||
| on: | ||
| push: | ||
| branches: [master] | ||
| pull_request: | ||
| branches: [master] | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
|
|
||
| jobs: | ||
| benchmark: | ||
| name: Run criterion benchmarks | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install stable toolchain | ||
| uses: dtolnay/rust-toolchain@stable | ||
|
|
||
| - name: Cache cargo registry | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| ~/.cargo/registry | ||
| ~/.cargo/git | ||
| target | ||
| key: ${{ runner.os }}-bench-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: ${{ runner.os }}-bench- | ||
|
|
||
| - name: Run benchmarks (rtc-rtp) | ||
| run: cargo bench -p rtc-rtp -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-rtcp) | ||
| run: cargo bench -p rtc-rtcp -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-stun) | ||
| run: cargo bench -p rtc-stun -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-sdp) | ||
| run: cargo bench -p rtc-sdp -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-srtp) | ||
| run: cargo bench -p rtc-srtp -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-turn) | ||
| run: cargo bench -p rtc-turn -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Run benchmarks (rtc-media) | ||
| run: cargo bench -p rtc-media -- --output-format bencher | tee -a bench-all.txt | ||
|
|
||
| - name: Store benchmark results | ||
| uses: benchmark-action/github-action-benchmark@v1 | ||
| if: github.event_name == 'push' && github.ref == 'refs/heads/master' | ||
| with: | ||
| name: WebRTC-RS Criterion Benchmarks | ||
| tool: cargo | ||
| output-file-path: bench-all.txt | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| auto-push: true | ||
| # Alert if performance degrades by more than 10% | ||
| alert-threshold: '110%' | ||
| comment-on-alert: true | ||
| fail-on-alert: false | ||
| benchmark-data-dir-path: docs/benchmarks | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: Fuzz | ||
|
|
||
| on: | ||
| schedule: | ||
| # Run daily at 02:00 UTC | ||
| - cron: '0 2 * * *' | ||
| workflow_dispatch: | ||
| inputs: | ||
| duration: | ||
| description: 'Fuzzing duration per target in seconds' | ||
| required: false | ||
| default: '30' | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
| FUZZ_DURATION: ${{ github.event.inputs.duration || '30' }} | ||
|
|
||
| jobs: | ||
| fuzz: | ||
| name: Fuzz ${{ matrix.crate }} | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| crate: | ||
| - rtc-dtls | ||
| - rtc-sctp | ||
| - rtc-rtcp | ||
| - rtc-sdp | ||
| - rtc-stun | ||
| - rtc-rtp | ||
| steps: | ||
|
Comment on lines
+24
to
+32
|
||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install nightly toolchain (required by cargo-fuzz) | ||
| uses: dtolnay/rust-toolchain@nightly | ||
|
|
||
| - name: Install cargo-fuzz | ||
| run: cargo install cargo-fuzz --locked | ||
|
|
||
| - name: Cache cargo registry | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| ~/.cargo/registry | ||
| ~/.cargo/git | ||
| key: ${{ runner.os }}-fuzz-${{ matrix.crate }}-${{ hashFiles('**/Cargo.lock') }} | ||
| restore-keys: ${{ runner.os }}-fuzz-${{ matrix.crate }}- | ||
|
|
||
| - name: Run fuzz targets (${{ matrix.crate }}) | ||
| working-directory: ${{ matrix.crate }}/fuzz | ||
| run: | | ||
| TARGETS=$(cargo fuzz list) | ||
| for TARGET in $TARGETS; do | ||
| echo "==> Fuzzing $TARGET for ${FUZZ_DURATION}s" | ||
| cargo fuzz run "$TARGET" -- \ | ||
| -max_total_time="${FUZZ_DURATION}" \ | ||
| -error_exitcode=77 \ | ||
| 2>&1 | tail -5 | ||
| done | ||
|
|
||
| - name: Upload crash artifacts | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: fuzz-crashes-${{ matrix.crate }} | ||
| path: ${{ matrix.crate }}/fuzz/artifacts/** | ||
| if-no-files-found: ignore | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| name: Release | ||
|
||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - 'v[0-9]*.[0-9]*.[0-9]*' | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
|
|
||
| jobs: | ||
| changelog: | ||
| name: Generate Changelog | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Install git-cliff | ||
| uses: kenji-miyake/setup-git-cliff@v2 | ||
|
|
||
| - name: Generate changelog for this release | ||
| run: | | ||
| git cliff --current --output CHANGELOG.md | ||
|
|
||
| - name: Upload changelog artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: changelog | ||
| path: CHANGELOG.md | ||
|
|
||
| release: | ||
| name: Create GitHub Release | ||
| runs-on: ubuntu-latest | ||
| needs: changelog | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install stable toolchain | ||
| uses: dtolnay/rust-toolchain@stable | ||
|
|
||
| - name: Download changelog | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: changelog | ||
|
|
||
| - name: Create GitHub Release | ||
| uses: softprops/action-gh-release@v2 | ||
| with: | ||
| body_path: CHANGELOG.md | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| # Uncomment and configure when ready to publish to crates.io: | ||
| # - name: Publish crates | ||
| # run: | | ||
| # cargo publish -p rtc-shared --token ${{ secrets.CRATES_IO_TOKEN }} | ||
| # cargo publish -p rtc-rtp --token ${{ secrets.CRATES_IO_TOKEN }} | ||
| # cargo publish -p rtc-rtcp --token ${{ secrets.CRATES_IO_TOKEN }} | ||
| # # ... remaining crates in dependency order | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| FROM gcr.io/oss-fuzz-base/base-builder-rust | ||
|
|
||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| cmake \ | ||
| libssl-dev \ | ||
| pkg-config \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY . $SRC/webrtc-rs-rtc | ||
| COPY oss-fuzz/build.sh $SRC/build.sh | ||
| WORKDIR $SRC/webrtc-rs-rtc |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| #!/bin/bash -eu | ||
| # | ||
| # OSS-Fuzz build script for webrtc-rs-rtc | ||
| # https://google.github.io/oss-fuzz/getting-started/new-project-guide/rust-lang/ | ||
| # | ||
| # This script is run inside the OSS-Fuzz Docker container. | ||
| # $OUT is the output directory for fuzz target binaries. | ||
| # $LIB_FUZZING_ENGINE is the fuzzing engine flags. | ||
| # $CFLAGS / $CXXFLAGS / $RUSTFLAGS are set by the base image. | ||
|
|
||
| cd "$SRC/webrtc-rs-rtc" | ||
|
|
||
| # Build all fuzz targets for each crate. | ||
| # cargo-fuzz compiles with --release and links libFuzzer automatically | ||
| # when RUSTFLAGS contains the libfuzzer flags provided by oss-fuzz. | ||
|
|
||
| FUZZ_CRATES=( | ||
| rtc-dtls | ||
| rtc-sctp | ||
| rtc-rtcp | ||
| rtc-sdp | ||
| rtc-stun | ||
| rtc-rtp | ||
| ) | ||
|
|
||
| for CRATE in "${FUZZ_CRATES[@]}"; do | ||
| pushd "$CRATE/fuzz" | ||
|
|
||
| # List all fuzz targets for this crate (fail build if cargo-fuzz is broken) | ||
| TARGETS=$(cargo fuzz list) | ||
| if [ -z "$TARGETS" ]; then | ||
| echo "ERROR: No fuzz targets found for $CRATE" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| for TARGET in $TARGETS; do | ||
| cargo fuzz build \ | ||
| --fuzz-dir . \ | ||
| --release \ | ||
| "$TARGET" | ||
|
|
||
| # Copy the compiled binary to $OUT | ||
| OUTPUT_PATH="$OUT/${CRATE//-/_}_$TARGET" | ||
| if [ -f "target/x86_64-unknown-linux-gnu/release/$TARGET" ]; then | ||
| cp "target/x86_64-unknown-linux-gnu/release/$TARGET" "$OUTPUT_PATH" | ||
| elif [ -f "target/release/fuzzing/$TARGET" ]; then | ||
| cp "target/release/fuzzing/$TARGET" "$OUTPUT_PATH" | ||
| else | ||
| echo "ERROR: Failed to locate built fuzz target binary for $CRATE/$TARGET" >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
|
|
||
| popd | ||
| done | ||
|
|
||
| echo "OSS-Fuzz build complete. Targets in $OUT:" | ||
| ls "$OUT/" | grep -v '\.options$' || true |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| homepage: "https://github.com/webrtc-rs/rtc" | ||
|
||
| language: rust | ||
| primary_contact: "security@webrtc.rs" | ||
| auto_ccs: | ||
| - "security@webrtc.rs" | ||
| fuzzing_engines: | ||
| - libfuzzer | ||
| - afl | ||
| - honggfuzz | ||
| sanitizers: | ||
| - address | ||
| - undefined | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
benchmark-action/github-action-benchmarkis configured withauto-push: true, but this workflow/job doesn't requestcontents: writepermissions. On GitHub's default read-onlyGITHUB_TOKEN, the action won't be able to push benchmark data to the repo/gh-pages branch; add an explicitpermissions: contents: write(workflow- or job-level) for the step to work onpushruns.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in a subsequent commit. This comment should be marked outdated — GitHub's detection did not trigger because the fix was on adjacent lines.