Skip to content

linux-kernel: T8496: Add kernel config needed for PWRU - #1170

Merged
c-po merged 1 commit into
vyos:currentfrom
robinchrist:T8496-kconfig-for-pwru
Apr 27, 2026
Merged

linux-kernel: T8496: Add kernel config needed for PWRU#1170
c-po merged 1 commit into
vyos:currentfrom
robinchrist:T8496-kconfig-for-pwru

Conversation

@robinchrist

Copy link
Copy Markdown
Contributor

PWRU (https://github.com/cilium/pwru) is a very useful tool to debug complex networking issues on Linux, as it allows you to trace how packets travel through the kernel functions

Change summary

This PR adds the kernel flags required for PWRU

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes)
  • Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
  • Other (please describe): Additional Kernel Flags enabling debug functionality

Related Task(s)

https://vyos.dev/T8496

Related PR(s)

How to test / Smoketest result

vyos@vyos-test-01:/config$ uname -r
6.6.135-vyos+test2
vyos@vyos-test-01:/config$ sudo ./pwru --output-limit-lines 5
2026/04/27 19:44:40 Attaching kprobes (via kprobe-multi)...
1388 / 1388 [--------------------------------------------------------------------------------------------------] 100.00% ? p/s
2026/04/27 19:44:40 Attached (ignored 0)
2026/04/27 19:44:40 Listening for events..
SKB                CPU PROCESS          NETNS      MARK/x        IFACE       PROTO  MTU   LEN   TUPLE FUNC
0xffff8efadaeeea00 1   ~/sbin/sshd:3330 0          0               0         0x0000 0     0     :0->:0() __build_skb_around
0xffff8efadaeeea00 1   ~/sbin/sshd:3330 0          0               0         0x0000 0     0     :0->:0() tcp_skb_entail
0xffff8efadaeeea00 1   ~/sbin/sshd:3330 0          0               0         0x0000 0     252   :0->:0() __tcp_transmit_skb
0xffff8efadaeeea00 1   ~/sbin/sshd:3330 0          0               0         0x0000 0     252   :0->:0() skb_clone
0xffff8efadaeeeae8 1   ~/sbin/sshd:3330 4026531840 0             eth0:2      0x0800 9170  192   10.151.0.31:43008->10.151.0.1:4789(udp) __skb_clone
2026/04/27 19:44:40 Printed 5 events, exiting program..
2026/04/27 19:44:40 Detaching kprobes...
5 / 5 [-------------------------------------------------------------------------------------------------------] 100.00% 23 p/s

Checklist:

  • I have read the CONTRIBUTING document
  • I have linked this PR to one or more Phabricator Task(s)
  • My commit headlines contain a valid Task id
  • My change requires a change to the documentation
  • I have updated the documentation accordingly

PWRU (https://github.com/cilium/pwru) is a very useful tool to debug complex networking issues on Linux, as it allows you to trace how packets travel through the kernel functions
@coderabbitai

coderabbitai Bot commented Apr 27, 2026

Copy link
Copy Markdown
📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Added kernel configuration enabling debugging and tracing capabilities, including debug information generation, BTF support, BPF-based tracing, kernel probes, performance event monitoring, and function tracing for improved system diagnostics and observability.

Walkthrough

A new Linux kernel configuration fragment for PWRU is added, enabling debug information, BTF debug info, and required tracing and BPF capabilities including kprobes, perf events, and function tracing.

Changes

Cohort / File(s) Summary
PWRU Kernel Configuration
scripts/package-build/linux-kernel/config/30-pwru.config
New kernel configuration fragment that enables debug info (CONFIG_DEBUG_INFO=y), BTF debug info, and required BPF/tracing capabilities (kprobes, perf events, BPF syscall interface, function tracer, fprobe) with guidance on removing conflicting default values.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: adding a kernel configuration for PWRU (a debugging tool for packet tracing), referencing task T8496.
Description check ✅ Passed The description provides relevant context about PWRU, explains the purpose of the changes, includes test results, and references the related task, all directly related to the kernel configuration addition.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown

👍
No issues in PR Title / Commit Title

@github-actions github-actions Bot added the current VyOS rolling release label Apr 27, 2026
@sever-sever

Copy link
Copy Markdown
Member

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Apr 27, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
scripts/package-build/linux-kernel/config/30-pwru.config (1)

9-14: Track artifact-size/build-time impact in CI for DEBUG_INFO+BTF.

This is a sensible change for PWRU, but these flags can materially increase kernel build time and debug artifact size. Consider adding/monitoring CI thresholds so regressions are caught early.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/package-build/linux-kernel/config/30-pwru.config` around lines 9 -
14, Add CI tracking for the DEBUG_INFO+BTF kernel config change: create a CI job
(e.g. "linux-kernel-debug-info-btf") that builds the kernel with
CONFIG_DEBUG_INFO=y, CONFIG_DEBUG_INFO_BTF=y and
CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y, record build time and output
artifact sizes (vmlinux, debug tarballs), and assert thresholds (or fail the
job) when regressions exceed predefined limits; update CI config to store and
graph these metrics so future changes to CONFIG_DEBUG_INFO or
CONFIG_DEBUG_INFO_BTF are automatically monitored and alerts are raised on
regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@scripts/package-build/linux-kernel/config/30-pwru.config`:
- Around line 9-14: Add CI tracking for the DEBUG_INFO+BTF kernel config change:
create a CI job (e.g. "linux-kernel-debug-info-btf") that builds the kernel with
CONFIG_DEBUG_INFO=y, CONFIG_DEBUG_INFO_BTF=y and
CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y, record build time and output
artifact sizes (vmlinux, debug tarballs), and assert thresholds (or fail the
job) when regressions exceed predefined limits; update CI config to store and
graph these metrics so future changes to CONFIG_DEBUG_INFO or
CONFIG_DEBUG_INFO_BTF are automatically monitored and alerts are raised on
regressions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fae2d8e0-ae77-4f6b-b8da-9d45ddb01f09

📥 Commits

Reviewing files that changed from the base of the PR and between 8d13ffa and ae4bfe8.

📒 Files selected for processing (1)
  • scripts/package-build/linux-kernel/config/30-pwru.config
📜 Review details
🧰 Additional context used
🔍 Remote MCP

Summary: Additional Context for PWRU Kernel Configuration PR Review

PWRU Tool Overview

PWRU ("Packet, where are you?") is an eBPF-based tool for tracing network packets in the Linux kernel with advanced filtering capabilities. Under the hood, pwru attaches eBPF debugging programs to all Linux kernel functions which are responsible for processing network packets, providing a finer-grained view into packet processing in the kernel than with traditional tools like tcpdump or Wireshark.

Kernel Requirements for PWRU

PWRU requires >= 5.3 kernel to run, >= 5.9 kernel for --output-skb, and >= 5.18 kernel for --backend=kprobe-multi. The test kernel referenced in the PR (6.6.135-vyos+test2) meets these requirements.

BTF (BPF Type Format) - Critical for PWRU

BTF (BPF Type Format) is the metadata format which encodes the debug info related to BPF program/map. BTF was extended to include function info for defined subroutines and line info for source/line information. The debug info is used for map pretty print, function signature, etc.

Tetragon (a related Cilium security observability tool) needs BTF support, and CO-RE allows portability of BPF programs but requires a kernel with BTF enabled. Most common Linux distributions now ship with BTF enabled via CONFIG_DEBUG_INFO_BTF=y.

It is now practical to have Linux kernel with always present embedded BTF type information at runtime by building the kernel with CONFIG_DEBUG_INFO_BTF=y option.

Kernel Configuration Requirements

The PR adds standard kernel configuration options needed for eBPF-based tracing:

  • CONFIG_DEBUG_INFO=y with DWARF version delegated to toolchain: Provides symbol information essential for function tracing
  • CONFIG_DEBUG_INFO_BTF=y: Enables embedded BTF metadata at runtime, critical for PWRU's portable BPF programs
  • Kprobes, perf events, BPF, and function tracer: These are standard eBPF requirements (CONFIG_BPF_EVENTS, CONFIG_BPF_SYSCALL, CONFIG_PERF_EVENTS) that enable attaching BPF programs to kernel function entry points

[::web_search::]

🔇 Additional comments (1)
scripts/package-build/linux-kernel/config/30-pwru.config (1)

3-20: Good config override pattern for enabling PWRU prerequisites.

Using # CONFIG_DEBUG_INFO_NONE is not set plus explicit CONFIG_DEBUG_INFO=y / CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y and the tracing/BPF flags is the right approach for this fragment-based merge flow.

@sever-sever sever-sever left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It allows eBPF-based tools for tracing network packets in the Linux kernel

@sever-sever
sever-sever requested a review from c-po April 27, 2026 12:31

@c-po c-po left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will increase Kernel size but according to research no noticeable runtime degradation when not in use.

@c-po
c-po merged commit d1bfa06 into vyos:current Apr 27, 2026
5 checks passed
@vyosbot vyosbot added mirror-initiated This PR initiated for mirror sync workflow mirror-completed and removed mirror-initiated This PR initiated for mirror sync workflow labels Apr 27, 2026
@robinchrist

Copy link
Copy Markdown
Contributor Author

One thing we might have to check is that the linux-image-6.6.135-vyos-dbg_6.6.135-1_amd64.deb is not included in the final image. I'm not exactly sure how the VyOS production pipeline works (whether it will install that package or not)

As far as I've understood, the -dbg package is not required and the BTF info is embedded in the kernel itself, the -dbg package contains the full debug info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

current VyOS rolling release mirror-completed

Development

Successfully merging this pull request may close these issues.

5 participants