Interrupt counters on `struct intr_event` by ehem · Pull Request #1301 · freebsd/freebsd-src

ehem · 2024-06-21T00:07:56Z

Every architecture has counters just outside struct intr_event. As such it seems appropriate to try to move them onto struct intr_event to reduce differences between architectures.

There is also an issue of the interface between architecture and core. The core has been implementing hw.intrcnt and hw.intrnames, yet forcing architectures to handle the tables behind these. This results in rather troublesome mixing.

ehem · 2024-06-21T00:10:22Z

I do not guarantee I've sufficiently resolved issues brought up via Phabricator, even though I have tried. This seems the way to go.

sys/kern/kern_intr.c

ehem · 2024-06-21T01:05:01Z

sys/kern/kern_intr.c

+	if (!thread && !filter) {
+		++ie->ie_stray;
+		return (stray ? ie->ie_stray : 0);
+	}


Kind of a wishlist item. I'm looking at a device which is known to generate a stray interrupt during start. I would rather like to have it counted, yet returning FILTER_STRAY causes all interrupt controllers to mask the interrupt. Given the device, this would be Bad, as a result the present driver simply always returns FILTER_HANDLED.

ehem · 2024-06-21T01:07:10Z

sys/kern/kern_intr.c

+	if ((flags & INTR_MULTIPROC) && !(ie->ie_flags & IE_MULTIPROC)) {
+#if defined(INVARIANTS) || defined(DEBUG)
+		printf("%s: Requested multi-processor interupt, but got "
+		    "uniprocessor interrupt\n", name);
+#endif
+		return (ENODEV);
+	}
+#if defined(INVARIANTS) || defined(DEBUG)
+	if (!(flags & INTR_MULTIPROC) && (ie->ie_flags & IE_MULTIPROC))
+		printf("%s: Requested uniprocessor interupt, but got "
+		    "multi-processor interrupt\n", name);
+#endif
+


Seems something which really should have some way for a driver to indicate, "I'd damn well better get a PPI interrupt, or I'll panic()!"

ehem · 2024-07-23T20:19:59Z

I'm planning to update to FreeBSD's HEAD soon. There will be a rebase in a few hours.

ehem · 2024-08-05T23:06:51Z

sys/kern/kern_clock.c

+		if (!CK_SLIST_EMPTY(&clk_intr_event->ie_handlers))
+			intr_event_handle(clk_intr_event, NULL);


@jrtc27 was objecting to this on #1358, due to looking inside struct intr_event. Issue would be, what approach should instead be used? Perhaps clock interrupts aren't considered stray if there are no handlers? Perhaps a flag to indicate no handler shouldn't be considered stray?

ehem · 2024-08-08T01:05:14Z

I don't recall who forced per-processor for PPI counters due to indicating they would veto atomic increment. This does provide the valuable feature of showing which processors the PPI is occurring on. Otherwise you're left unsure whether your PPI interrupt being triggered on a single processor, or several. Also tells you how evenly it is spread. On balance I think this is distinctly superior to INTRNG's choice to use atomic increment.

ehem · 2024-09-17T02:14:13Z

Note, ARM64 has been tested and was helpful with confirming a fully operational PPI interrupt. x86 and PowerPC are untested. I guess the point here is, why is this information being copied back and forth between the events and the architecture tables, and simply being duplicated in both places? All of this really should be in one place.

github-actions · 2025-09-01T20:31:55Z

Thank you for taking the time to contribute to FreeBSD!

There is an issue that needs to be resolved:

Missing Signed-off-by lines (799877c, e9301af, cf90100, 4eb6ee6, 8c03828, d92899b, af47f74, eb54018, cd37810, 505cdb8, d925871, e500137, a33a3bb, d02228d, 5fa61ad, 4643e1a, a97fed9, 33d3033, 1c64227, 56430b1, 34de3ed, 2bc0eff, a03fd83, 44b7113, 8d61bbe, e5ff2b8, 038f843, caf4e18, c91eb49, 42a5417, b86c56a, d24d6d9, 5832aad)

Note

Please review CONTRIBUTING.md, then update and push your branch again.

ehem · 2025-09-01T20:35:43Z

Upon checking I remember why I originally tried this. Still potentially viable as an alternative solution to a pesky issue.

Note, this has simply been rebased and not properly checked. There is a good chance some of the commits have become broken in the interim, so exercise care with this.

ehem · 2025-09-02T02:46:00Z

Been some time since I had looked at this. Some things in the kernel look distinctly concerning. Seems the best ordering would be #1457, this (will need adaptation) and then adapting #1358.

Some interrupt types can be signaled on multiple processors. These need adjustment to their handling (notably adjustment to counter handling). Mark clk_intr_event as being multi-processor (it is). Differential Revision: https://reviews.freebsd.org/D38448

This should ensure devices needing a PPI interrupt receive a PPI interrupt. In debug kernels warn in case of mismatch between request and retrieved intr_event. Differential Revision: https://reviews.freebsd.org/D38448

Announce as these are allocated to assist debugging issues with these counters (as they are a limited resource).

This allows architectures to provide the interrupt names/counters from intr_event, instead of the architecture tables. Since the tables are inferior for normal interrupts, but functional for IPI interrupts this is a substantial step forward. Differential Revision: https://reviews.freebsd.org/D36610

The tables of interrupt names and counts are a layering violation. Moving the SYSCTL_PROC()s to architecture functions as a first step towards moving the handling to the architecture. Differential Revision: https://reviews.freebsd.org/D38454

The tables of interrupt names and counts are a layering violation. Let architecture code handle these as this form causes trouble for architecture interrupt implementations. Differential Revision: https://reviews.freebsd.org/D38454

For most operations using the count of interrupts instead of the raw table size is superior. Notably adding entries is faster. Computing the full table size is quite fast. In the longer term a single variable is smaller too. Differential Revision: https://reviews.freebsd.org/D38116

Clean out the now-unused counters/interrupt names for normal interrupts. The table is now strictly used for IPI interrupts, so all of this support can go. Differential Revision: https://reviews.freebsd.org/D38451

This allows architectures to provide the interrupt names/counters from intr_event, instead of the architecture tables. Since the tables are inferior for normal interrupts, but functional for IPI interrupts this is a substantial step forward. Differential Revision: https://reviews.freebsd.org/D36610

These are now strictly being used for counting IPI interrupts, thus can be removed from monoprocessor builds.

The tables of interrupt names and counts are a layering violation. Let architecture code handle these as this form causes trouble for architecture interrupt implementations. Differential Revision: https://reviews.freebsd.org/D38454

Since all interrupt table implementations use the same index variable to indicate current table size, use that instead of size variables. Differential Revision: https://reviews.freebsd.org/D38116

This allows architectures to provide the interrupt names/counters from intr_event, instead of the architecture tables. Since the tables are inferior for normal interrupts, but functional for IPI interrupts this is a substantial step forward. Differential Revision: https://reviews.freebsd.org/D36610

These counters are now used strictly for IPI/PPI interrupts. Purge support for normal interrupts as those are now *strictly* on intr_event. The comment at the nintrcnt computation was incorrect. Multiple hypervisor drivers use a per-core set of interrupts, but generally only one hypervisor driver will be loaded. Differential Revision: https://reviews.freebsd.org/D38453

The tables of interrupt names and counts are a layering violation. Let architecture code handle these as this form causes trouble for architecture interrupt implementations. Differential Revision: https://reviews.freebsd.org/D38454

Since all interrupt table implementations use the same index variable to indicate current table size, use that instead of size variables. Differential Revision: https://reviews.freebsd.org/D38116

This allows architectures to provide the interrupt names/counters from intr_event, instead of the architecture tables. Since the tables are inferior for normal interrupts, but functional for IPI interrupts this is a substantial step forward. Differential Revision: https://reviews.freebsd.org/D36610

The counters/names are now used strictly for IPI/PPI interrupts. Rather more of the source was shared between IPI and normal interrupts, so relatively little disappears. Differential Revision: https://reviews.freebsd.org/D38452

The sysctl()s are now handled by the architecture. The tables were a layering violation and have been removed. Let architecture implementations modify their handling depending on their needs. Differential Revision: https://reviews.freebsd.org/D38454

These are now strictly being used for counting IPI interrupts, thus can be removed from monoprocessor builds.

The "???" interrupt doesn't appear useful any more, so remove it. Differential Revision: https://reviews.freebsd.org/D37869

clk_intr_event is a clock interrupt, not a generalized interrupt. As such everything should be in clock.h/kern_clock.c instead of interrupt.h/kern_intr.c.

clk_intr_event was added strictly for ACPI. While additional uses could be added in the future, presently this is ACPI-only. Remove it from non-ACPI kernels. Correct the uses of intr_event_handle(). If no handlers are added by ACPI, invoking intr_event_handle() is incorrect as it is stray. Fixes: aba10e1 ("Allow swi_sched() to be called from NMI context.") Differential Revision: https://reviews.freebsd.org/D40776

If swi_add() is used to initialize an event without adding a handler, the name should be used as the device name. Otherwise there can be mysterious unnamed events lying around.

Comparable to the current situation. Minor savings, one caller simply passes the count directly through. Differential Revision: https://reviews.freebsd.org/D38449

This needs checking. I'm unsure of why the notyet is still here, but removing it /might/ be bad. Differential Revision: https://reviews.freebsd.org/D38449

Filter function can return more than just FILTER_STRAY. As such a filter might return FILTER_STRAY | FILTER_HANDLED to indicate stray counters should be incremented, but the interrupt should NOT be masked.

Reading event names doesn't require exclusive access. Differential Revision: https://reviews.freebsd.org/D38450

Some interrupt types can be signaled on multiple processors. These need adjustment to their handling (notably adjustment to counter handling). Mark clk_intr_event as being multi-processor. I'm doubting this is correct. Kind of makes sense to have SWIs have per-processor counters, on the flip side they do move. This should likely either be squashed back into the rest of the commit, or dropped. Differential Revision: https://reviews.freebsd.org/D38448

ehem commented Jun 21, 2024

View reviewed changes

sys/kern/kern_intr.c Show resolved Hide resolved

ehem commented Jun 21, 2024

View reviewed changes

sys/kern/kern_intr.c Show resolved Hide resolved

ehem commented Jun 21, 2024

View reviewed changes

ehem force-pushed the intrcnt branch from 251121b to f83c369 Compare July 23, 2024 22:30

ehem force-pushed the intrcnt branch from f83c369 to ed65fa5 Compare August 2, 2024 20:52

ehem mentioned this pull request Aug 3, 2024

Minimal collection for running on Xen/ARM64 #1358

Draft

3 tasks

ehem force-pushed the intrcnt branch 2 times, most recently from 8bd76cd to d862657 Compare August 5, 2024 23:01

ehem commented Aug 5, 2024

View reviewed changes

ehem force-pushed the intrcnt branch from d862657 to 68d3cbb Compare October 25, 2024 21:27

ehem force-pushed the intrcnt branch 2 times, most recently from 110d7c7 to e98b020 Compare November 6, 2024 20:44

ehem closed this Dec 11, 2024

ehem deleted the intrcnt branch December 11, 2024 22:45

ehem restored the intrcnt branch September 1, 2025 20:30

ehem reopened this Sep 1, 2025

ehem force-pushed the intrcnt branch from e98b020 to 5365ca6 Compare September 1, 2025 20:31

ehem marked this pull request as draft September 1, 2025 20:33

ehem force-pushed the intrcnt branch from 5365ca6 to bbf667f Compare September 2, 2025 02:32

ehem force-pushed the intrcnt branch from bbf667f to efed4f5 Compare October 6, 2025 19:47

ehem force-pushed the intrcnt branch 2 times, most recently from 1d67bf7 to a39d694 Compare October 23, 2025 21:55

ehem added 29 commits February 20, 2026 17:43

kern/intr: enforce PPI interrupt for PPI requests

d92899b

This should ensure devices needing a PPI interrupt receive a PPI interrupt. In debug kernels warn in case of mismatch between request and retrieved intr_event. Differential Revision: https://reviews.freebsd.org/D38448

kern/intr: add verbose boot message for multi-processor counters

af47f74

Announce as these are allocated to assist debugging issues with these counters (as they are a limited resource).

intrng: remove normal interrupts from intrcnt/intrnames

e500137

Clean out the now-unused counters/interrupt names for normal interrupts. The table is now strictly used for IPI interrupts, so all of this support can go. Differential Revision: https://reviews.freebsd.org/D38451

intrng: remove intrcnts from non-SMP builds

d02228d

These are now strictly being used for counting IPI interrupts, thus can be removed from monoprocessor builds.

intr/x86: switch to index vars from table size vars

4643e1a

Since all interrupt table implementations use the same index variable to indicate current table size, use that instead of size variables. Differential Revision: https://reviews.freebsd.org/D38116

intr/powerpc: switch to index var from table size vars

56430b1

Since all interrupt table implementations use the same index variable to indicate current table size, use that instead of size variables. Differential Revision: https://reviews.freebsd.org/D38116

intrng: remove intrcnts from non-SMP builds

44b7113

These are now strictly being used for counting IPI interrupts, thus can be removed from monoprocessor builds.

sys/intr: remove questionable interrupt report

8d61bbe

The "???" interrupt doesn't appear useful any more, so remove it. Differential Revision: https://reviews.freebsd.org/D37869

kern: move clk_intr_event to clock.[ch]

e5ff2b8

clk_intr_event is a clock interrupt, not a generalized interrupt. As such everything should be in clock.h/kern_clock.c instead of interrupt.h/kern_intr.c.

kern/intr: use swi_add() string as name if no handler

caf4e18

If swi_add() is used to initialize an event without adding a handler, the name should be used as the device name. Otherwise there can be mysterious unnamed events lying around.

kern/intr: switch intr_event_handle() to return stray count

c91eb49

Comparable to the current situation. Minor savings, one caller simply passes the count directly through. Differential Revision: https://reviews.freebsd.org/D38449

kern/intr: enable stray counting for non-acknowledged interrupts

42a5417

This needs checking. I'm unsure of why the notyet is still here, but removing it /might/ be bad. Differential Revision: https://reviews.freebsd.org/D38449

kern/intr: add handling for handled stray filter return

b86c56a

Filter function can return more than just FILTER_STRAY. As such a filter might return FILTER_STRAY | FILTER_HANDLED to indicate stray counters should be incremented, but the interrupt should NOT be masked.

kern/intr: replace mutex with read-write lock

d24d6d9

Reading event names doesn't require exclusive access. Differential Revision: https://reviews.freebsd.org/D38450

ehem force-pushed the intrcnt branch from 6558c3c to 5832aad Compare February 21, 2026 01:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Interrupt counters on `struct intr_event`#1301

Interrupt counters on `struct intr_event`#1301
ehem wants to merge 33 commits intofreebsd:mainfrom
ehem:intrcnt

ehem commented Jun 21, 2024

Uh oh!

ehem commented Jun 21, 2024

Uh oh!

Uh oh!

Uh oh!

ehem Jun 21, 2024

Uh oh!

ehem Jun 21, 2024

Uh oh!

ehem commented Jul 23, 2024

Uh oh!

ehem Aug 5, 2024

Uh oh!

ehem commented Aug 8, 2024

Uh oh!

ehem commented Sep 17, 2024

Uh oh!

github-actions bot commented Sep 1, 2025 •

edited

Loading

Uh oh!

ehem commented Sep 1, 2025

Uh oh!

ehem commented Sep 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		if (!CK_SLIST_EMPTY(&clk_intr_event->ie_handlers))
		intr_event_handle(clk_intr_event, NULL);

Comments

Conversation

ehem commented Jun 21, 2024

Uh oh!

ehem commented Jun 21, 2024

Uh oh!

Uh oh!

Uh oh!

ehem Jun 21, 2024

Choose a reason for hiding this comment

Uh oh!

ehem Jun 21, 2024

Choose a reason for hiding this comment

Uh oh!

ehem commented Jul 23, 2024

Uh oh!

ehem Aug 5, 2024

Choose a reason for hiding this comment

Uh oh!

ehem commented Aug 8, 2024

Uh oh!

ehem commented Sep 17, 2024

Uh oh!

github-actions bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ehem commented Sep 1, 2025

Uh oh!

ehem commented Sep 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot commented Sep 1, 2025 •

edited

Loading