Skip to content

Commit 0ee3494

Browse files
levontilevonti
andauthored
fix(api-server): ignore duplicate task-api responses in RmfService (#1089)
RMF's request/response channel is a pub/sub pseudo-service, so more than one node can answer the same request_id (the RmfService docstring already notes clients must drop duplicated response ids). _handle_response dropped only *unknown* ids; for a known id it called fut.set_result() unconditionally. A second response for an already-resolved future raises asyncio.InvalidStateError inside the subscription callback, which runs in the rclpy.spin() thread (ros.py) and terminates it. After that the api-server node stops processing every subscription, so all task-api calls time out with HTTP 500 "rmf service timed out" even though the request was delivered and executed. Drop duplicate/late responses (fut.done()) so the spin thread survives, and add a regression test. Signed-off-by: levonti <levonti@ileo.cloud> Co-authored-by: levonti <levonti@ileo.cloud>
1 parent 62ab379 commit 0ee3494

2 files changed

Lines changed: 35 additions & 0 deletions

File tree

packages/api-server/api_server/rmf_io/rmf_service.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,18 @@ def _handle_response(self, msg: ApiResponse):
9494
f"Received response for unknown request id: {msg.request_id}"
9595
)
9696
return
97+
if fut.done():
98+
# RMF's request/response is a pub/sub pseudo-service, so more than
99+
# one node may answer the same request_id (see the class docstring:
100+
# clients must drop duplicated response ids). Calling set_result on
101+
# an already-resolved future raises asyncio.InvalidStateError inside
102+
# this subscription callback, which runs in the rclpy.spin() thread
103+
# (see ros.py) and would terminate it -- after which the node stops
104+
# processing every subscription and all later calls time out.
105+
logging.warning(
106+
f"Ignoring duplicate response for request id: {msg.request_id}"
107+
)
108+
return
97109
fut.set_result(msg.json_msg)
98110

99111

packages/api-server/api_server/rmf_io/test_rmf_service.py

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,3 +122,26 @@ async def run():
122122
self.assertListEqual(["hello", "world"], list(results))
123123

124124
self.loop.run_until_complete(run())
125+
126+
def test_duplicate_response_is_ignored(self):
127+
# Regression: RMF's request/response is a pub/sub pseudo-service, so
128+
# more than one node may answer the same request_id. A duplicate
129+
# ApiResponse for an already-resolved future used to raise
130+
# asyncio.InvalidStateError inside the subscription callback, which runs
131+
# in the rclpy spin thread and would terminate it -- silently breaking
132+
# every later task-api call. The handler must drop the duplicate.
133+
req_id = self._create_node_id()
134+
fut: asyncio.Future = self.loop.create_future()
135+
self.rmf_service._requests[req_id] = fut
136+
try:
137+
self.rmf_service._handle_response(
138+
ApiResponse(request_id=req_id, json_msg="first")
139+
)
140+
# second, duplicate delivery of the same request_id must not raise
141+
self.rmf_service._handle_response(
142+
ApiResponse(request_id=req_id, json_msg="second")
143+
)
144+
finally:
145+
self.rmf_service._requests.pop(req_id, None)
146+
self.assertTrue(fut.done())
147+
self.assertEqual("first", fut.result())

0 commit comments

Comments
 (0)