Skip to content

fleet dump CLI: Internal pagination when fetching resources#4588

Merged
p-se merged 1 commit intorancher:mainfrom
p-se:issue-4419
Feb 12, 2026
Merged

fleet dump CLI: Internal pagination when fetching resources#4588
p-se merged 1 commit intorancher:mainfrom
p-se:issue-4419

Conversation

@p-se
Copy link
Contributor

@p-se p-se commented Feb 2, 2026

Instead of holding all resources in memory, write them in batches. Also eases the load on the cluster by only fetching a certain amount of resources at a time.

Also fixes an issue with collecting duplicated namespaces from cluster resources.

Refers to #4419

Additional Information

Checklist

  • I have updated the documentation via a pull request in the
    fleet-docs repository.

@p-se p-se force-pushed the issue-4419 branch 2 times, most recently from ab01c6a to 7220488 Compare February 2, 2026 13:32
@p-se p-se changed the title Internal pagination when fetching resources fleet dump CLI: Internal pagination when fetching resources Feb 2, 2026
@p-se p-se marked this pull request as ready for review February 9, 2026 08:32
@p-se p-se requested a review from a team as a code owner February 9, 2026 08:32
Copilot AI review requested due to automatic review settings February 9, 2026 08:32
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds internal pagination support to the fleet dump CLI to avoid holding large resource lists in memory and reduce API server load, and fixes duplicated namespace collection from Cluster resources (ref #4419).

Changes:

  • Introduces FetchLimit plumbing through the dump command/options to control list pagination.
  • Implements pagination for dynamic client resource listing (Fleet resources/Contents) and for controller-runtime client listing in events/metrics paths.
  • Updates and extends unit tests to cover namespace de-duplication and pagination behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal/cmd/cli/dump/dump.go Adds FetchLimit option and pagination loops for multiple list operations; de-duplicates namespaces.
internal/cmd/cli/dump/dump_test.go Adds/updates tests for namespace de-duplication and pagination (including tar output).
internal/cmd/cli/dump.go Exposes --fetch-limit CLI flag and passes it into dump options.
internal/cmd/builder.go Extends command builder to support int64 CLI flags.
integrationtests/cli/dump/suite_test.go Import formatting change only.
e2e/single-cluster/cli_dump_test.go Import formatting change only.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@p-se p-se force-pushed the issue-4419 branch 3 times, most recently from 8201278 to 79458ac Compare February 9, 2026 13:44
opts := []client.ListOption{
client.InNamespace(svc.Namespace),
matchingLabels,
client.Limit(fetchLimit),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be surprised if we actually needed this limit here, considering that we're only using it to fetch monitoring services, but I guess it doesn't harm :)

@weyfonk weyfonk added this to Fleet Feb 11, 2026
@weyfonk weyfonk moved this to 👀 In review in Fleet Feb 11, 2026
@weyfonk weyfonk added this to the v2.14.0 milestone Feb 11, 2026
@p-se p-se requested a review from weyfonk February 11, 2026 16:28
@p-se p-se merged commit 8af9ad2 into rancher:main Feb 12, 2026
34 of 36 checks passed
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Fleet Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants