Fix event handler `types` filter when using bulk export API #61787

kshi36 · 2025-11-25T18:42:37Z

Manual Tests

Test: types filter works correctly for clients using new bulk export API

When connected to a storage backend supporting bulk export API (eg. Teleport Cloud, Athena backend), and setting toml configuration file with the following options:
- Verify types="user.login" filter forwards only user.login events to fluentd audit events endpoint
- Verify skip-event-types="user.login" filter skips forwarding user.login events to fluentd audit events endpoint
- Verify skip-session-types="db.session.query" filter skips forwarding db.session.query events to fluentd session events endpoint
- Verify options above yield same results using both toml config file and CLI arguments

changelog: Fixed bug where event handler types filter is ignored for Teleport clients using Athena storage backend

tigrato · 2025-11-25T20:16:48Z

integrations/event-handler/events_job.go

 	// filter out unwanted event types (in the v1 event export logic this was an internal behavior
 	// of the event processing helper since it would perform conversion prior to storing the event
 	// in its internal buffer).
+	if len(j.app.Config.Types) > 0 && !slices.Contains(j.app.Config.Types, evt.Event.Type) {


No need for the length check since the slice contains does exactly that check before.
Should we create a map to avoid looping through config types?

Similar to skip events

My understanding was that if the types option was an empty slice, we would forward all event types (default behavior). Then slices.Contains would return false given the slice is empty, and the event type would be erroneously skipped.

I can change it into a map, although the length check will still be required

You are correct. I forgot about that

zmb3 · 2025-11-26T00:53:05Z

integrations/event-handler/events_job.go

 		return nil
 	}

+	j.app.log.DebugContext(ctx, "Not skipping event", "type", evt.Event.Type)


I'd remove this one. The event handler processes many thousands of events, we probably don't need to debug log the normal operation.

zmb3 · 2025-11-26T00:54:12Z

integrations/event-handler/events_job_test.go

+			require.NotRegexp(t, regexp.MustCompile("\"Event sent\".*type=role.created"), out.String())
+			require.Regexp(t, regexp.MustCompile("\"Event sent\".*type=join_token.create"), out.String())


Compile these regexes outside of the loop so you don't waste time recompiling them for each test case.

zmb3 · 2025-11-26T00:54:33Z

integrations/event-handler/README.md

@@ -38,7 +38,6 @@ You may specify configuration options via command line arguments, environment va
 | skip-session-types        | Comma-separated list of session event types to skip                                                   | FDFWD_SKIP_SESSION_TYPES        |
 | start-time                | Minimum event time (RFC3339 format)                                                                   | FDFWD_START_TIME                |
 | timeout                   | Polling timeout                                                                                       | FDFWD_TIMEOUT                   |
-| cursor                    | Start cursor value                                                                                    | FDFWD_CURSOR                    |


Why was this removed?

The cursor CLI option (and environment variable) do not seem to exist anymore

zmb3 · 2025-11-26T00:54:50Z

integrations/event-handler/session_events_job.go

@@ -357,6 +357,8 @@ Loop:

 			_, ok = j.app.Config.SkipSessionTypes[e.Type]
 			if !ok {
+				j.app.log.DebugContext(ctx, "Not skipping session event", "type", e.Type)


Similarly, I'd remove this one.

zmb3 · 2025-11-26T00:55:14Z

integrations/event-handler/teleport_event.go

 	}

-	switch e.GetType() {
+	switch evt.Type {


Was this change necessary?

Wasn't necessary, although it matches how we are operating on the newly created event (hence removing the unused method fields as well)

Although if there is some other drawback or inconvenience I can revert it

r0mant

Just a small comment in addition to what other people commented on already.

Good job figuring this out @kshi36!

r0mant · 2025-11-26T01:12:08Z

integrations/event-handler/events_job.go

 	// filter out unwanted event types (in the v1 event export logic this was an internal behavior
 	// of the event processing helper since it would perform conversion prior to storing the event
 	// in its internal buffer).
+	if len(j.app.Config.Types) > 0 && !slices.Contains(j.app.Config.Types, evt.Event.Type) {
+		j.app.log.DebugContext(ctx, "Skipping event from types filter", "type", evt.Event.Type)


To echo Zac's similar comments, I would consider removing all of these extra debug statements that are logged whenever an event is processed, whether it's skipped or not. They can still clog the logs if you e.g. choose to skip a frequent event.

I like the idea of logging a debug message when dropping an event. This can help troubleshoot issues like this one: #61729.

For the config I agree that this is not especially useful as we usually have access to the config.

kshi36 added 2 commits November 25, 2025 10:22

Fixed event handler types filter when using bulk export API

0ef504f

Fix license

f14e0fa

kshi36 changed the title ~~Fixed event handler types filter when using bulk export API~~ Fix event handler types filter when using bulk export API Nov 25, 2025

kshi36 added backport/branch/v17 backport/branch/v18 labels Nov 25, 2025

kshi36 marked this pull request as ready for review November 25, 2025 20:13

kshi36 requested review from bernardjkim, hugoShaka and r0mant November 25, 2025 20:13

github-actions bot requested review from marcoandredinis and tigrato November 25, 2025 20:14

github-actions bot added audit-log Issues related to Teleports Audit Log size/md labels Nov 25, 2025

tigrato reviewed Nov 25, 2025

View reviewed changes

zmb3 reviewed Nov 26, 2025

View reviewed changes

r0mant reviewed Nov 26, 2025

View reviewed changes

		require.NotRegexp(t, regexp.MustCompile("\"Event sent\".*type=role.created"), out.String())
		require.Regexp(t, regexp.MustCompile("\"Event sent\".*type=join_token.create"), out.String())

Fix event handler types filter when using bulk export API #61787

Are you sure you want to change the base?

Fix event handler types filter when using bulk export API #61787

Conversation

kshi36 commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

r0mant left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Fix event handler `types` filter when using bulk export API #61787

Fix event handler `types` filter when using bulk export API #61787

kshi36 commented Nov 25, 2025 •

edited

Loading