Skip to content

Conversation

pgellert
Copy link
Contributor

  1. Integrate the filter parser and predicate into the ListKafkaConnections admin API endpoint.
  2. Require an enterprise license for the ListKafkaConnections endpoint

Fixes https://redpandadata.atlassian.net/browse/CORE-13361
Fixes https://redpandadata.atlassian.net/browse/CORE-13573

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.2.x
  • v25.1.x
  • v24.3.x

Release Notes

  • none

@pgellert pgellert requested a review from a team October 10, 2025 14:22
@pgellert pgellert self-assigned this Oct 10, 2025
@pgellert pgellert requested review from Copilot and michael-redpanda and removed request for a team October 10, 2025 14:22
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR integrates filtering functionality and license requirements into the Kafka connections admin API endpoint. The changes enhance security by requiring an enterprise license and improve usability by allowing filtered queries.

  • Adds enterprise license validation to the list_kafka_connections endpoint
  • Integrates AIP filter parser and predicate for connection filtering
  • Updates tests to verify both filtering functionality and license enforcement

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/rptest/tests/list_kafka_connections_test.py Adds tests for filtering integration and license requirement enforcement
src/v/redpanda/admin/services/broker.h Updates constructor to accept feature table dependency for license checking
src/v/redpanda/admin/services/broker.cc Implements license validation and filter integration in list_kafka_connections
src/v/redpanda/admin/services/BUILD Adds dependencies for features, AIP filter, and protobuf RPC
src/v/redpanda/admin/server.cc Updates broker service instantiation to pass feature table

Integrate the filter parser and predicate into the ListKafkaConnections
admin API endpoint.
@pgellert
Copy link
Contributor Author

force-push: address copilot review

auto result = connection_gather_result{};

auto conn_ptrs = server.list_connections();
co_await ss::maybe_yield();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ss::coroutine::maybe_yield()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Comment on lines 134 to 135
throw serde::pb::rpc::permission_denied_exception(
fmt::format("Invalid license: {}", status()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

failed_precondition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to switch to that if you think that's more appropriate, I was debating between the two. I went with permission_denied for consistency with the existing SR ACLs endpoint:

throw ss::httpd::base_exception(
fmt::format("Invalid license: {}", status()),
ss::http::reply::status_type::forbidden);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading https://grpc.github.io/grpc/core/md_doc_statuscodes.html, failed_precondition seems more appropriate so I've switched to that.

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Oct 10, 2025

Retry command for Build#73952

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/random_node_operations_test.py::RedpandaNodeOperationsSmokeTest.test_node_ops_smoke_test@{"cloud_storage_type":1,"mixed_versions":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"compaction_mode":"chunked_sliding_window","enable_failures":true,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"compaction_mode":"adjacent_merge","enable_failures":false,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"chunked_sliding_window","enable_failures":false,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"chunked_sliding_window","enable_failures":true,"mixed_versions":false,"with_iceberg":true}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"adjacent_merge","enable_failures":true,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"sliding_window","enable_failures":false,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"sliding_window","enable_failures":true,"mixed_versions":false,"with_iceberg":true}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"chunked_sliding_window","enable_failures":true,"mixed_versions":false,"with_iceberg":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":1,"compaction_mode":"adjacent_merge","enable_failures":false,"mixed_versions":false,"with_iceberg":true}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"compaction_mode":"chunked_sliding_window","enable_failures":false,"mixed_versions":false,"with_iceberg":false}

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Oct 10, 2025

CI test results

test results on build#73952
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
CloudStorageScrubberTest test_scrubber {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23dd-4d15-9d74-91d4f90f8d17 FLAKY 20/21 upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=CloudStorageScrubberTest&test_method=test_scrubber
SegmentMsTest test_segment_rolling_with_retention_consumer null integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceac-2c8d-479e-b705-8395b76cae86 FLAKY 19/21 upstream reliability is '92.07119741100324'. current run reliability is '90.47619047619048'. drift is 1.59501 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=SegmentMsTest&test_method=test_segment_rolling_with_retention_consumer
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "adjacent_merge", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e4-492c-90eb-dc685e4512e0 FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "adjacent_merge", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e5-45ad-8f8c-1d71e6758853 FAIL 0/12 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "chunked_sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e7-4c5b-81ce-b7486540db25 FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "chunked_sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e8-4e43-940d-e35dbd6feb78 FAIL 0/1 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23db-434e-a83f-0955de93c114 FAIL 0/17 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23dd-4d15-9d74-91d4f90f8d17 FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "adjacent_merge", "enable_failures": false, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e0-4436-ae5f-8a4610fa6d34 FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "chunked_sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e3-4737-ab78-7842fd56d371 FAIL 0/17 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "sliding_window", "enable_failures": false, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e5-45ad-8f8c-1d71e6758853 FAIL 0/12 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23db-434e-a83f-0955de93c114 FAIL 0/17 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23dd-4d15-9d74-91d4f90f8d17 FAIL 0/1 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "chunked_sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23df-4814-91af-53234a17729c FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "chunked_sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e0-4436-ae5f-8a4610fa6d34 FAIL 0/1 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 2, "compaction_mode": "sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e2-4118-93b6-b56aab629242 FAIL 0/17 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e3-4737-ab78-7842fd56d371 FAIL 0/18 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e5-45ad-8f8c-1d71e6758853 FAIL 0/11 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "chunked_sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e8-4e43-940d-e35dbd6feb78 FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RandomNodeOperationsTest test_node_operations {"cloud_storage_type": 1, "compaction_mode": "sliding_window", "enable_failures": true, "mixed_versions": false, "with_iceberg": true} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23dd-4d15-9d74-91d4f90f8d17 FAIL 0/1 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RandomNodeOperationsTest&test_method=test_node_operations
RedpandaNodeOperationsSmokeTest test_node_ops_smoke_test {"cloud_storage_type": 1, "mixed_versions": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceac-2c89-4464-a1c3-e6547bcdf6bc FAIL 0/21 The test has failed across all retries https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RedpandaNodeOperationsSmokeTest&test_method=test_node_ops_smoke_test
RedpandaNodeOperationsSmokeTest test_node_ops_smoke_test {"cloud_storage_type": 1, "mixed_versions": false} integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e2-4118-93b6-b56aab629242 FAIL 0/17 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RedpandaNodeOperationsSmokeTest&test_method=test_node_ops_smoke_test
RpkDebugBundleTest test_debug_bundle null integration https://buildkite.com/redpanda/redpanda/builds/73952#0199ceae-23e7-4c5b-81ce-b7486540db25 FLAKY 20/21 upstream reliability is '99.73544973544973'. current run reliability is '95.23809523809523'. drift is 4.49735 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RpkDebugBundleTest&test_method=test_debug_bundle
test results on build#74061
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
TimeQueryTest test_timequery_with_local_gc null integration https://buildkite.com/redpanda/redpanda/builds/74061#0199dca3-448d-42eb-8069-8e333c02873a FLAKY 20/21 upstream reliability is '89.69957081545064'. current run reliability is '95.23809523809523'. drift is -5.53852 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=TimeQueryTest&test_method=test_timequery_with_local_gc

This endpoint requires an enterprise license
Carry-over improvement from an earlier PR
Carry-over cleanup from an earlier PR
@pgellert pgellert requested a review from a team as a code owner October 13, 2025 07:28
@pgellert
Copy link
Contributor Author

force-push: change error type + minor fixups from earlier PRs

@pgellert pgellert requested a review from Copilot October 13, 2025 07:32
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@pgellert pgellert requested a review from rockwotj October 13, 2025 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants