Metadata identity + unary command RPCs part 1 by asher-pem-arm · Pull Request #1918 · labgrid-project/labgrid

asher-pem-arm · 2026-07-02T13:22:01Z

Description

This is the first part of the coordinator unary-RPC rework. It introduces metadata-based identity for clients and exporters, reducing the coordinator’s dependency on startup messages sent through the long-lived streams.

This PR:

Adds client and server gRPC identity interceptors.
Sends client/exporter identity and user-agent information as gRPC metadata.
Initializes ClientStream and ExporterStream sessions from that metadata.
Deprecates stream startup identity messages while retaining them as a fallback for older clients and exporters.
Updates AcquirePlace, AllowPlace, and CreateReservation to infer caller identity from metadata or an existing stream session.
No new coordinator RPCs are introduced in this part.

Motivation

Decouple ClientStream’s live-update role from identity and independent RPCs, allowing services and short-lived tools to perform bounded authenticated operations without maintaining stream state. This simplifies client implementations, reconnection handling, horizontal scaling, and per-operation authorization.

Compatibility note

Old clients and exporters remain compatible with the new coordinator through the startup-message fallback.
However, new clients and exporters stop sending startup identity messages and are therefore incompatible with coordinators predating this change. Those coordinators cannot establish the required stream session.

Codecov Report

❌ Patch coverage is 47.50958% with 137 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.2%. Comparing base (64723a5) to head (04a49f6).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...abgrid/remote/generated/labgrid_coordinator_pb2.py	0.8%	123 Missing ⚠️
labgrid/remote/coordinator.py	76.1%	10 Missing ⚠️
labgrid/remote/identity.py	91.4%	3 Missing ⚠️
labgrid/remote/grpc/interceptor/server.py	95.6%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #1918     +/-   ##
========================================
+ Coverage    61.0%   61.2%   +0.1%     
========================================
  Files         182     185      +3     
  Lines       14881   15019    +138     
========================================
+ Hits         9083    9194    +111     
- Misses       5798    5825     +27

Flag	Coverage Δ
3.10	`61.2% <47.5%> (+0.1%)`	⬆️
3.11	`61.1% <47.5%> (+0.1%)`	⬆️
3.12	`61.1% <47.5%> (+0.1%)`	⬆️
3.13	`61.1% <47.5%> (+0.1%)`	⬆️
3.14	`61.1% <47.5%> (+0.1%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Emantor · 2026-07-02T13:40:10Z

I took the liberty to reformatted for the PR template, this should make the description more readable.

Emantor · 2026-07-02T13:44:17Z

+            client_call_details.metadata.add(USER_AGENT_KEY, self.user_agent)
+
+
+class IdentityClientUnaryUnaryInterceptor(UnaryUnaryClientInterceptor, BaseIdentityClientInterceptor):


Do we really need to child classes here? To me it looks like these only differ in parameters, and this won't matter for python as long as the number of parameters remain the same.

It would be nice if we can simplify this to one class that implements the logic.

grpc.aio.Channel sorts interceptors into per-RPC-type lists using isinstance(), so we would still need separate unary-unary and stream-stream interceptor classes.

But we could just have a IdentityClientInterceptor(UnaryUnaryClientInterceptor, StreamStreamClientInterceptor) or is there something baring that on the gRPC side?

Edit: Does not look like it, the gRPC example uses a similar pattern: https://github.com/grpc/grpc/blob/b86a3460af2160cd7a02bd51014bc9fa036b91ff/examples/python/interceptors/default_value/default_value_client_interceptor.py#L48

Emantor · 2026-07-09T13:03:20Z

During the review of this PR yesterday it came up that we are missing a good justification for separate unary RPC instead of extending the ClientStream with filtering. The unary RPCs introduced in this PR duplicate the information on the stream and applications should be able to use the ClientStream for the same information.

asher-pem-arm · 2026-07-09T21:23:37Z

Hey @Emantor ,

Thanks for sharing the concern. The primary motivation for this came out of our work on the Labgrid fleeting plugin, and the more we thought about it the more sense it made.

At the moment ClientStream is doing two jobs: it is both a watch/update mechanism and part of the session/identity for independent RPCs. That coupling is what we are trying to remove. A service, CI worker, web backend, or short-lived tool should be able to make a bounded RPC with identity/auth metadata on that RPC, get a response, and finish.
As we found on our end with our fleeting plugin implementation, requiring those clients to maintain a bidirectional stream means they also need reconnect logic, caching, sync handling, readiness state, and failure handling to safely perform simple operations. On the Labgrid fleeting plugin we managed to remove ~1000 lines of code/tests when we moved to using the unary RPC implementation.

This also matters for production/distributed deployments. If every client has to establish and maintain a ClientStream session, then coordinator endpoint changes, load balancer migration, reconnects, and horizontally scaled services all need to manage stream/session recovery. In many cases those clients gain no benefit from a long-lived session. The intent here is to decouple those concerns: clients that need ClientStream for live updates can still use it.

A stream also makes auth harder as the coordinator has to continuously filter updates for each client and ensure no unauthorised state is ever sent. For unary RPCs, the auth decision is scoped to one operation and one response.

If you feel this is useful context I can add this to the discussion as needed. Thanks

asher-pem-arm · 2026-07-13T13:49:41Z

@Emantor as discussed I've updated this PR to only include the client stream decoupling

Emantor · 2026-07-13T13:56:48Z

+            logging.debug("client identity provided in gRPC metadata")
+            logging.debug(identity)
+            self.clients[peer] = ClientSession(self, peer, identity.id, out_msg_queue, identity.user_agent)
+


The ClientSession creation can be moved into the startup handling, new clients will sent both for a release (old coordinators use the old StartupDone message, new ones the identity). The coordinator uses the identity for new clients and warns for connections of older clients.

With the introduction of auth we can than abort for older clients, which are not supported with auth.

updated in latest commit

Add client and server interceptors which attach labgrid identity metadata to gRPC calls and expose it to coordinator RPC handlers. Use the metadata identity to register client and exporter stream sessions while keeping startup-message handling as a deprecated fallback for older clients and exporters. Signed-off-by: Asher Pemberton <asher.pemberton@arm.com> Reviewed-by: Asher Pemberton <asher.pemberton@arm.com> # gatekeeper Co-authored-by: Luke Beardsmore <luke.beardsmore2@arm.com>

Allow AcquirePlace, ReleasePlace and CreateReservation to identify the caller from gRPC metadata instead of requiring identity to come only from an established ClientStream session. Keep the existing ClientStream session lookup as a fallback so older clients which still send startup messages on the stream continue to work. Signed-off-by: Asher Pemberton <asher.pemberton@arm.com> Reviewed-by: Asher Pemberton <asher.pemberton@arm.com> # gatekeeper Co-authored-by: Luke Beardsmore <luke.beardsmore2@arm.com>

Send StartupDone alongside identity metadata so new clients and exporters remain compatible with older coordinators. Create sessions immediately from identity metadata when available, while retaining StartupDone as a deprecated fallback. Signed-off-by: Asher Pemberton <asher.pemberton@arm.com> Reviewed-by: Asher Pemberton <asher.pemberton@arm.com> # gatekeeper

asher-pem-arm changed the title ~~Detach streams from unary rpcs master pr1~~ Metadata identity + unary command RPCs part 1 Jul 2, 2026

Emantor requested changes Jul 2, 2026

View reviewed changes

Emantor requested changes Jul 3, 2026

View reviewed changes

Comment thread labgrid/remote/coordinator.py

Bastian-Krause added the architectural change label Jul 7, 2026

asher-pem-arm force-pushed the detach-streams-from-unary-rpcs-master-pr1 branch from 8291b0e to 604b887 Compare July 13, 2026 13:35

Emantor requested changes Jul 13, 2026

View reviewed changes

asher-pem-arm mentioned this pull request Jul 13, 2026

Metadata identity + unary command RPCs part 3 #1933

Open

7 tasks

asher-pem-arm and others added 3 commits July 14, 2026 13:50

asher-pem-arm force-pushed the detach-streams-from-unary-rpcs-master-pr1 branch from 604b887 to 04a49f6 Compare July 14, 2026 13:01

asher-pem-arm requested a review from Emantor July 16, 2026 13:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Metadata identity + unary command RPCs part 1#1918

Metadata identity + unary command RPCs part 1#1918
asher-pem-arm wants to merge 3 commits into
labgrid-project:masterfrom
ARM-software:detach-streams-from-unary-rpcs-master-pr1

asher-pem-arm commented Jul 2, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

Emantor commented Jul 2, 2026

Uh oh!

Emantor Jul 2, 2026

Uh oh!

asher-pem-arm Jul 2, 2026

Uh oh!

Emantor Jul 3, 2026 •

edited

Loading

Uh oh!

Uh oh!

Emantor commented Jul 9, 2026

Uh oh!

asher-pem-arm commented Jul 9, 2026

Uh oh!

asher-pem-arm commented Jul 13, 2026

Uh oh!

Emantor Jul 13, 2026

Uh oh!

asher-pem-arm Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		client_call_details.metadata.add(USER_AGENT_KEY, self.user_agent)


		class IdentityClientUnaryUnaryInterceptor(UnaryUnaryClientInterceptor, BaseIdentityClientInterceptor):

Uh oh!

Uh oh!

Conversation

asher-pem-arm commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

This PR:

Motivation

Compatibility note

See also

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Emantor commented Jul 2, 2026

Uh oh!

Emantor Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

asher-pem-arm Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Emantor Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Emantor commented Jul 9, 2026

Uh oh!

asher-pem-arm commented Jul 9, 2026

Uh oh!

asher-pem-arm commented Jul 13, 2026

Uh oh!

Emantor Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

asher-pem-arm Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

asher-pem-arm commented Jul 2, 2026 •

edited

Loading

codecov Bot commented Jul 2, 2026 •

edited

Loading

Emantor Jul 3, 2026 •

edited

Loading