Skip to content

rbd: add debug logging to ProcessMetadata for CBT diagnostics#6203

Open
kaovilai wants to merge 2 commits intoceph:develfrom
kaovilai:rbd-add-debug-logging-ProcessMetadata
Open

rbd: add debug logging to ProcessMetadata for CBT diagnostics#6203
kaovilai wants to merge 2 commits intoceph:develfrom
kaovilai:rbd-add-debug-logging-ProcessMetadata

Conversation

@kaovilai
Copy link
Copy Markdown
Contributor

@kaovilai kaovilai commented Mar 24, 2026

Summary

Add debug log statements to ProcessMetadata in snap_diff.go to improve observability of the CBT (Changed Block Tracking) metadata streaming path.

Why this is useful beyond a specific bug

The ProcessMetadata function involves multiple steps (snapshot ID resolution, snap context setup, DiffIterateByID call) but currently has no logging between them. When GetMetadataAllocated returns unexpected results (empty streams, wrong block counts), there's no visibility into:

  • Which snapshot ID was resolved and whether it matches expectations
  • What Offset/Length/FromSnapID parameters were passed to DiffIterateByID
  • How many blocks were buffered but not yet sent

These are all DebugLog calls (not visible at default log levels), so there's no performance impact in production.

Changes

Three log.DebugLog calls added:

  1. After snapshot ID resolution — logs snapshot name, ID, and volume size
  2. Before DiffIterateByID — logs config parameters (Offset, Length, FromSnapID)
  3. After DiffIterateByID completes — logs remaining buffered block count

Fixes: #6206

Note

Responses generated with Claude

Copilot AI review requested due to automatic review settings March 24, 2026 23:43
@mergify mergify bot added component/rbd Issues related to RBD bug Something isn't working labels Mar 24, 2026
@kaovilai kaovilai force-pushed the rbd-add-debug-logging-ProcessMetadata branch from afeabf6 to b94a798 Compare March 24, 2026 23:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds targeted debug logging to the RBD snapshot metadata streaming path (ProcessMetadata) to help diagnose CBT cases where metadata streams are unexpectedly empty.

Changes:

  • Log resolved snapshot ID and volume size before setting the snapshot context.
  • Log DiffIterateByID configuration parameters (offset/length/from-snap).
  • Log completion status and the number of buffered blocks after DiffIterateByID returns.

@nixpanic nixpanic requested a review from Rakshith-R March 26, 2026 15:52
@nixpanic
Copy link
Copy Markdown
Member

Is this debugging useful outside the scope where a potential bug needs to be addressed?

Can you explain a little more about the issue that you are facing?

To get the PR accepted, you will need to squash the extra/correction commits into the original one.

Add debug log statements to ProcessMetadata to log key parameters
at each stage: snapshot ID and volume size after resolution,
DiffIterateByID config (Offset, Length, FromSnapID) before the
call, and remaining buffered block count after completion.

These logs are useful for diagnosing CBT (Changed Block Tracking)
issues in production where GetMetadataAllocated may return
unexpected results without visibility into the internal state.

Fixes: ceph#6206
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
@kaovilai kaovilai force-pushed the rbd-add-debug-logging-ProcessMetadata branch from 0c8b85e to f68485a Compare March 26, 2026 19:13
@kaovilai
Copy link
Copy Markdown
Contributor Author

squashed, pr body updated.

nixpanic
nixpanic previously approved these changes Mar 27, 2026
@nixpanic
Copy link
Copy Markdown
Member

@Rakshith-R , please have a look at this and #6200

Copy link
Copy Markdown
Contributor

@Rakshith-R Rakshith-R left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a small nit, LGTM

Co-authored-by: Rakshith R <rakshith.r.0001@gmail.com>
Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
@mergify mergify bot dismissed nixpanic’s stale review April 9, 2026 15:11

Pull request has been modified.

@kaovilai kaovilai requested a review from Rakshith-R April 9, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working component/rbd Issues related to RBD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rbd: getRBDSnapID and ProcessMetadata issues in snapshot metadata path

4 participants