Skip to content

RDMA service endpoint monitoring - #6856

Open
yegorskii wants to merge 2 commits into
mainfrom
users/yegorskii/cell-server-monitoring
Open

RDMA service endpoint monitoring#6856
yegorskii wants to merge 2 commits into
mainfrom
users/yegorskii/cell-server-monitoring

Conversation

@yegorskii

Copy link
Copy Markdown
Collaborator

Notes

RDMA connection monitoring for the blockstore server: a page showing which clients are connected and which volumes are mounted over each connection

The transport now reports connection lifecycle.

  • IServerSession — a connection descriptor (GetId/GetPeer/GetStartTs), valid only for the duration of the call it is passed to.

  • IServerRequest replaces the untyped void* context in HandleRequest/SendResponse/SendError. Hence the mechanical churn across every handler implementation: disk_agent, endpoints_rdma, rdma-test, rdma_test.

  • IServerHandler gained OnSessionCreated/OnSessionClosed — noexcept, with empty default bodies, so the other targets are untouched.

  • TServerSession caches Peer and StartTs in const fields; TRequest carries SessionId.

Where the notifications fire is load-bearing:

  • NotifySessionCreated() from Accept(), before Acquire() — otherwise the poller could release the session and report it closed before the handler ever learns it existed.

  • NotifySessionClosed() from Release(), which only runs once IsFlushed() holds, i.e. after every in-flight request has been answered.

That yields the invariant the rest depends on: connection announced → all of its mounts → connection closed, strictly in that order.

Registry (cloud/blockstore/libs/service_rdma/mount_registry.*, new)

  • TMountRegistry maps sessionId → TConnectionInfo{Peer, StartTs, Mounts}. It owns a single-threaded pool; every mutation is asynchronous and serialized on that thread.

The thread exists precisely for ordering: OnSessionCreated arrives on the CM thread, OnSessionClosed on a completion poller. On a multi-threaded pool the close could be applied before the open, leaving the connection in the registry forever.

Exactly one method creates entries — DoAddConnection. DoAddMount goes through FindPtr and ignores a mount for an unknown connection; otherwise it would be a second path for resurrecting a closed one. Removing a connection drops its mounts along with the entry.

Target (rdma_target.*)
TRdmaTarget forwards the four events to the registry and renders the table from it: SessionId / Peer / Connected / DiskId / ClientId / AccessMode / MountMode / MountSeqNumber. The RdmaTarget page is registered under /blockstore.

The feature is toggleable: a new ConnectionMonitoringEnabled field on TRdmaTarget in config/rdma.proto, defaulting to false. When off, the registry is not created, the thread is not started, the page is not registered, and the handler holds an empty pointer.

The factory signature barely moved — IMonitoringServicePtr monitoring was added, the return type stayed IStartablePtr.

Issue

Put links to the related issues here

@yegorskii yegorskii added the blockstore Add this label to run only cloud/blockstore build and tests on PR label Aug 20, 2026
@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1383s): some tests FAILED for commit b762604.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7070 7068 0 1 0 1 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 900s): all tests PASSED for commit b762604.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7073 7072 0 0 0 1 0

@yegorskii yegorskii added the rebase Add this label if you want to rebase your PR for test run label Aug 21, 2026
@yegorskii
yegorskii force-pushed the users/yegorskii/cell-server-monitoring branch from b762604 to fb48b13 Compare August 21, 2026 08:07
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1449s): some tests FAILED for commit fb48b13.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7086 7082 0 3 0 1 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 49s): all tests PASSED for commit fb48b13.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4 4 0 0 0 0 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blockstore Add this label to run only cloud/blockstore build and tests on PR rebase Add this label if you want to rebase your PR for test run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant