26.7: kernel panic in bridge_input() via Suricata netmap on bridge0, not 26.1

[yoavain]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/src/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/src/issues?q=is%3Aissue

Describe the bug

Kernel panic (page fault in bridge_input+0xd8) when netmap attaches to an if_bridge interface on 26.7 / FreeBSD 15.1. Triggered by Suricata in inline netmap IPS mode bound to a bridge.

Last known working version: 26.1.11_10 (FreeBSD 15.0-RELEASE). The identical configuration has been stable there for months. On that version bridge0 currently shows options=100000<NETMAP> with Suricata attached and running.

bridge0 has no native netmap support, so netmap attaches a generic (emulated) adapter. The panic follows the first packet through it.

To Reproduce

1. Create a bridge over one or more physical interfaces and assign it as LAN
2. Services > Intrusion Detection > Administration > Settings
3. Set Capture mode to Netmap (IPS), select the bridge under Interfaces, check Enabled, Apply
4. System panics roughly 107 seconds after the Suricata service starts

The system boots fully first: all interfaces come up, PPPoE authenticates and obtains an address, Unbound and the web GUI start, and the console banner prints. The panic comes afterwards, once Suricata finishes loading rules and attaches netmap. It then reboots and repeats, which from the outside looks like a boot loop but is not one.

Reproduced twice, at identical uptime, on consecutive boots.

Expected behavior

Either netmap attaches to the bridge without panicking, as it did on FreeBSD 15.0, or it refuses to attach and returns an error. A kernel panic is not an acceptable failure mode for an unsupported interface type on a router.

Describe alternatives you considered

Divert mode instead of inline netmap, which avoids the netmap path entirely. That works, but sidesteps the bug rather than addressing it. Binding Suricata to the physical members instead of the bridge is not practical here (seven interfaces, and it would inspect intra-LAN traffic unnecessarily).

Relevant log files

Panic, transcribed from /var/log/system/latest.log with the syslog envelope removed:

[107] 147.957436 [ 321] generic_netmap_register Emulated adapter for bridge0 activated
[107]
[107]
[107] Fatal trap 12: page fault while in kernel mode
[107] cpuid = 4; apic id = 08
[107] fault virtual address= 0x18
[107] fault code = supervisor read data, page not present
[107] instruction pointer = 0x20:0xffffffff82eda008
[108] stack pointer = 0x28:0xfffffe011d027880
[108] frame pointer = 0x28:0xfffffe011d0278e0
[108] code segment = base 0x0, limit 0xfffff, type 0x1b
[108] = DPL 0, pres 1, long 1, def32 0, gran 1
[108] processor eflags = interrupt enabled, resume, IOPL = 0
[108] current process = 60970 (W#01-bridge0)
[108] rdi: 0000000000000028 rsi: fffff80225320376 rdx: 0000000000000000
[108] rcx: 0000000000000000 r8: 00000000bc571c01 r9: 0000000000000155
[108] rax: 0000000004000201 rbx: fffff80225320300 rbp: fffffe011d0278e0
[108] r10: 00000000b183d5d8 r11: 00000000525d7235 r12: 0000000000000000
[108] r13: fffff800241e1400 r14: fffff800247fb800 r15: fffff800015e5000
[108] trap number = 12
[108] panic: page fault
[108] cpuid = 4
[108] time = 1784838148
[108] KDB: stack backtrace:
[108] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe011d0275d0
[108] vpanic() at vpanic+0x136/frame 0xfffffe011d027700
[108] panic() at panic+0x43/frame 0xfffffe011d027760
[108] trap_pfault() at trap_pfault+0x37d/frame 0xfffffe011d0277b0
[108] calltrap() at calltrap+0x8/frame 0xfffffe011d0277b0
[108] --- trap 0xc, rip = 0xffffffff82eda008, rsp = 0xfffffe011d027880, rbp = 0xfffffe011d0278e0 ---
[108] bridge_input() at bridge_input+0xd8/frame 0xfffffe011d0278e0
[108] ether_nh_input() at ether_nh_input+0x2b2/frame 0xfffffe011d027940
[108] netisr_dispatch_src() at netisr_dispatch_src+0x9f/frame 0xfffffe011d027990
[108] ether_input() at ether_input+0x56/frame 0xfffffe011d0279e0
[108] nm_os_send_up() at nm_os_send_up+0x1e/frame 0xfffffe011d027a00
[108] netmap_send_up() at netmap_send_up+0x4e/frame 0xfffffe011d027a50
[108] netmap_txsync_to_host() at netmap_txsync_to_host+0x72/frame 0xfffffe011d027ad0
[108] netmap_ioctl() at netmap_ioctl+0x1a6/frame 0xfffffe011d027bb0
[108] freebsd_netmap_ioctl() at freebsd_netmap_ioctl+0x79/frame 0xfffffe011d027bf0
[108] devfs_ioctl() at devfs_ioctl+0xcb/frame 0xfffffe011d027c40
[108] vn_ioctl() at vn_ioctl+0xc8/frame 0xfffffe011d027cb0
[108] devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe011d027cd0
[108] kern_ioctl() at kern_ioctl+0x286/frame 0xfffffe011d027d40
[108] sys_ioctl() at sys_ioctl+0x101/frame 0xfffffe011d027e00
[108] amd64_syscall() at amd64_syscall+0x126/frame 0xfffffe011d027f30
[108] fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe011d027f30
[108] --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x82cff826a, rsp = 0x82eb04e48, rbp = 0x82eb04e70 ---
[108] KDB: enter: panic
---<<BOOT>>---

The netmap attach immediately preceding the second occurrence:

[107] 301.683015 [1087] generic_netmap_dtor Emulated netmap adapter for bridge0 destroyed
[107] bridge0: permanently promiscuous mode enabled
[107] 301.693515 [1182] generic_netmap_attach Emulated adapter for bridge0 created (prev was NULL)
[107] 301.754355 [ 321] generic_netmap_register Emulated adapter for bridge0 activated

The second occurrence is identical in structure, including bridge_input+0xd8, on a different CPU (cpuid = 0, process 80641 (W#01-bridge0)). The absolute instruction pointer differs (0xffffffff8277f008) because the module load address changed between boots.

savecore captured both to /var/crash/textdump.tar.15 and /var/crash/textdump.tar.16. I can attach ddb.txt on request. I am withholding msgbuf.txt from a public thread since it contains credentials and host key fingerprints, but can send it privately.

Additional context

All offloading is disabled in config as the netmap documentation requires:

<disablechecksumoffloading>1</disablechecksumoffloading>
<disablesegmentationoffloading>1</disablesegmentationoffloading>
<disablelargereceiveoffloading>1</disablelargereceiveoffloading>
<disablevlanhwfilter>1</disablevlanhwfilter>

and this is reflected at runtime:

igc0: options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>

os-intel-microcode is not installed, so the microcode issue noted in the 26.7 release announcement does not apply.

Forum thread:

Environment

OPNsense 26.7.1_1 (amd64)
FreeBSD 15.1-RELEASE-p1 stable/26.7-n283674-12334a596709
Intel® Core™ 3 N355, 8 cores, 16 GB RAM
Network: 8x Intel® I226-V (igc0-igc7), all reporting native netmap support
Storage: KIOXIA KBG40ZNS512G NVMe, ZFS root, UEFI boot

Topology: igc4 = WAN (PPPoE), remaining seven ports bridged as bridge0 (flat LAN)
Suricata bound to bridge0, Capture mode Netmap (IPS)

Last known working: OPNsense 26.1.11_10 (FreeBSD 15.0-RELEASE)

[yoavain]

Forum topic
https://forum.opnsense.org/index.php?topic=52539.0

[fichtner]

@yoavain thanks, we will look into it but it's full-blown holiday season so it could be multiple weeks until progress is made

[yoavain]

Thanks, I appreciate it.
Currently, I disabled Suricate so the upgrade can be completed.
I will probably wait a week and then enable it but only on the PPPoE connection rather than the bridge.
So, no pressure.

[Monviech]

While testing vlans on bridges I also saw this:

https://github.com/freebsd/freebsd-src/blob/deddfa3db956ebed584a859010a10b6d9a4e2e5d/sys/net/if_bridge.c#L874

But even if you create a bridge with no bridge members, it will set VLAN_HWTAGGING as option...

If that's the smoking gun though, unknown. :)