ci: add criterion benchmarks and OSS-Fuzz integration by nightness · Pull Request #80 · webrtc-rs/rtc

nightness · 2026-04-01T16:42:13Z

Summary

Add .github/workflows/bench.yml to run criterion benchmarks on every push and track regressions
Add .github/workflows/fuzz.yml for scheduled and manual CI fuzzing runs
Add .github/workflows/release.yml for tag-triggered GitHub Releases with git-cliff changelogs
Add oss-fuzz/ config files (build.sh, Dockerfile, project.yaml) to enable Google OSS-Fuzz cloud fuzzing
Add .cliff.toml for conventional-commits-based changelog generation
6 crates already have fuzz targets; this wires them up for continuous cloud fuzzing

Test plan

Verify bench.yml consolidates all benchmark outputs into a single file for tracking
Verify bench.yml has contents: write permission for benchmark auto-push
Verify OSS-Fuzz build.sh uses correct cargo-fuzz flags and fails on missing binaries
Verify OSS-Fuzz build.sh fails on empty fuzz target lists instead of silent success
Verify release workflow uses valid glob tag pattern and has correct permissions
Verify release workflow job name accurately reflects its behavior
Verify fuzz artifact upload captures nested target directories
Verify changelog links resolve to valid GitHub commit URLs

🤖 Generated with Claude Code

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds CI automation for benchmarking and fuzzing, and introduces release/changelog tooling to support ongoing quality and release management.

Changes:

Add OSS-Fuzz integration files (project metadata, Dockerfile, build script) intended to build existing cargo-fuzz targets.
Add GitHub Actions workflows for Criterion benchmarks, scheduled fuzz runs, and tag-based releases (with git-cliff changelog generation).
Add git-cliff configuration for generating a conventional-commits-based CHANGELOG.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`oss-fuzz/project.yaml`	Declares OSS-Fuzz project metadata for the repository.
`oss-fuzz/build.sh`	Builds and copies Rust fuzz target binaries into `$OUT` for OSS-Fuzz.
`oss-fuzz/Dockerfile`	Defines the OSS-Fuzz builder image dependencies and sources.
`.github/workflows/release.yml`	Adds a tag-triggered workflow to generate changelog artifacts and create GitHub releases.
`.github/workflows/fuzz.yml`	Adds scheduled/manual CI fuzzing runs using `cargo-fuzz`.
`.github/workflows/bench.yml`	Adds CI benchmark runs and publishes benchmark history via `github-action-benchmark`.
`.cliff.toml`	Configures `git-cliff` changelog generation format and parsing rules.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-04T14:15:11Z

@@ -0,0 +1,12 @@
+homepage: "https://github.com/webrtc-rs/rtc"


PR description says the OSS-Fuzz configs are added under oss-fuzz/projects/webrtc-rs/, but the diff adds them under oss-fuzz/ (e.g., oss-fuzz/project.yaml). If the intent is to mirror the upstream OSS-Fuzz repository layout, these files should live under oss-fuzz/projects/<project-name>/ (and be named accordingly, e.g., project.yaml, Dockerfile, build.sh in that directory). Otherwise, the PR description should be updated to match the actual paths/layout.

Addressed in a subsequent commit. This comment should be marked outdated — GitHub's detection did not trigger because the fix was on adjacent lines.

Copilot · 2026-04-04T14:15:12Z

@@ -0,0 +1,60 @@
+name: Release


The workflow creates a GitHub Release via softprops/action-gh-release@v2, which requires contents: write permission for GITHUB_TOKEN in many org/repo default-permissions configurations. Consider adding an explicit permissions: block (e.g., contents: write) to avoid release creation failing due to restricted default token permissions.

Addressed in a subsequent commit. This comment should be marked outdated — GitHub's detection did not trigger because the fix was on adjacent lines.

codecov · 2026-04-04T14:17:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.22%. Comparing base (9feb4a3) to head (6eb5617).

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #80      +/-   ##
==========================================
+ Coverage   71.17%   71.22%   +0.05%     
==========================================
  Files         442      442              
  Lines       67330    67330              
==========================================
+ Hits        47922    47958      +36     
+ Misses      19408    19372      -36

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Replace placeholder emails with security@webrtc.rs in OSS-Fuzz config - Remove invalid -O flag and $LIB_FUZZING_ENGINE/$RUSTFLAGS args from cargo fuzz build - Add proper error handling when fuzz target binary is not found (fail instead of silent || true) - Fix release workflow tag pattern (glob-compatible) and add contents: write permission - Use glob pattern for fuzz crash artifact upload to capture nested target dirs - Consolidate all benchmark outputs into single file for benchmark-action tracking - Fix changelog commit links to use full GitHub URLs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-08T08:08:40Z

+      - name: Store benchmark results
+        uses: benchmark-action/github-action-benchmark@v1
+        if: github.event_name == 'push' && github.ref == 'refs/heads/master'
+        with:
+          name: WebRTC-RS Criterion Benchmarks
+          tool: cargo
+          output-file-path: bench-all.txt
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          auto-push: true
+          # Alert if performance degrades by more than 10%


benchmark-action/github-action-benchmark is configured with auto-push: true, but this workflow/job doesn't request contents: write permissions. On GitHub's default read-only GITHUB_TOKEN, the action won't be able to push benchmark data to the repo/gh-pages branch; add an explicit permissions: contents: write (workflow- or job-level) for the step to work on push runs.

Addressed in a subsequent commit. This comment should be marked outdated — GitHub's detection did not trigger because the fix was on adjacent lines.

Copilot · 2026-04-08T08:08:41Z

+      matrix:
+        crate:
+          - rtc-dtls
+          - rtc-sctp
+          - rtc-rtcp
+          - rtc-sdp
+          - rtc-stun
+          - rtc-rtp
+    steps:


PR description says "8 crates already have fuzz targets", but both this workflow matrix and oss-fuzz/build.sh only wire up 6 crates. In the current repo tree there are fuzz directories/targets under rtc-dtls, rtc-sctp, rtc-rtcp, rtc-sdp, rtc-stun, and rtc-rtp only; update the PR description or extend the matrix/script to include the additional fuzz crates if they exist.

Addressed in a subsequent commit. This comment should be marked outdated — GitHub's detection did not trigger because the fix was on adjacent lines.

- .github/workflows/bench.yml: runs cargo bench for all 7 crates with criterion benchmarks (rtp, rtcp, stun, sdp, srtp, turn, media) on every push/PR to master; uses github-action-benchmark to track regressions and alert on >10% slowdowns - .github/workflows/release.yml: on version tags, generates CHANGELOG.md via git-cliff and creates a GitHub Release with the generated notes - .cliff.toml: git-cliff config for conventional commits (feat/fix/perf/ refactor/doc/test/chore groups) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- .github/workflows/fuzz.yml: daily scheduled fuzzing of all 87 targets across rtc-dtls (29), rtc-sctp (30), rtc-rtcp (24), rtc-sdp (2), rtc-stun (1), rtc-rtp (1) using cargo-fuzz with libfuzzer; crashes uploaded as artifacts for triage; can also be triggered manually with configurable duration - oss-fuzz/: configuration to submit to Google's OSS-Fuzz continuous fuzzing platform (project.yaml, Dockerfile, build.sh); submit via PR to https://github.com/google/oss-fuzz to enable cloud-scale fuzzing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Replace placeholder emails with security@webrtc.rs in OSS-Fuzz config - Remove invalid -O flag and $LIB_FUZZING_ENGINE/$RUSTFLAGS args from cargo fuzz build - Add proper error handling when fuzz target binary is not found (fail instead of silent || true) - Fix release workflow tag pattern (glob-compatible) and add contents: write permission - Use glob pattern for fuzz crash artifact upload to capture nested target dirs - Consolidate all benchmark outputs into single file for benchmark-action tracking - Fix changelog commit links to use full GitHub URLs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Add `permissions: contents: write` to bench.yml for benchmark auto-push - Rename misleading `publish` job to `release` in release.yml - Remove silent error suppression from `cargo fuzz list` in oss-fuzz/build.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

nightness · 2026-04-10T05:26:07Z

Rebased onto upstream/master so this PR contains only its own changes. Previous branch structure caused merge conflicts when PRs were merged in sequence. Each PR is now independently mergeable.

rainliu requested a review from Copilot April 4, 2026 14:08

Copilot AI reviewed Apr 4, 2026

View reviewed changes

Copilot started reviewing on behalf of rainliu April 4, 2026 14:22 View session

nightness requested a review from Copilot April 8, 2026 08:00

Copilot started reviewing on behalf of nightness April 8, 2026 08:01 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

nightness and others added 4 commits April 10, 2026 00:16

nightness force-pushed the ci/benchmarks-fuzzing branch from af54113 to 23c8a29 Compare April 10, 2026 05:16

nightness mentioned this pull request Apr 10, 2026

Summary: 19 PRs ready for review #87

Open

		@@ -0,0 +1,12 @@
		homepage: "https://github.com/webrtc-rs/rtc"

Uh oh!

Conversation

nightness commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

nightness Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

Uh oh!

nightness Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Apr 4, 2026

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

nightness Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

nightness Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

nightness commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nightness commented Apr 1, 2026 •

edited

Loading