Fix memory leak when taking service responses addressed to other clients - #600
Fix memory leak when taking service responses addressed to other clients#600pavlo-yashchuk wants to merge 1 commit into
Conversation
rmw_take_response_request drains the reply reader in a loop, deserialising each sample into the same ros_data and skipping replies whose GUID belongs to another client of the service. The skipped sample's deserialised content was never finalised, so its sequence/string payload leaked (~1 payload per skipped reply). Fini and re-init the message between iterations before the next take. The refini closure is resolved once per reader in rmw_init_cs via get_typesupport (error-returning) rather than per sample.
|
Tick the box to add this pull request to the merge queue (same as
|
|
@jmachowinski |
fujitatomoya
left a comment
There was a problem hiding this comment.
@pavlo-yashchuk it does not look like comments on #589 are addressed in this PR? can you confirm?
|
Please update the description of the PR to use the default form to avoid future confusion. I also need a way to reproduce the memory leak. I still believe that this is not the correct fix, and that we should rather check the deserialization or the runtime functions. |
|
hey @jmachowinski yep, you are right, and it sits in desirialization. Ive asked claude to track changes and find the fix: The leak is in the old C-introspection deserializer, auto & data = *reinterpret_cast<typename GenericCSequence<T>::type *>(field);
deser >> dsize;
if (!GenericCSequence<T>::init(&data, dsize)) { ... } // rosidl_runtime_c__<T>__Sequence__init — no __fini first It calls This is already handled on the newer branches by #575, which routes deserialization through
So rolling/lyrical need nothing. Jazzy is the problem: it only got the #553/#562 perf backports, which kept the old Does that seem like a reasonable jazzy-only fix, or would you prefer backporting the deserializer change itself? |
Description
rmw_take_response_request drains the reply reader in a loop, deserializing each sample into the same ros_data and skipping replies that belong to other clients of the same service (GUID mismatch). The message content of the skipped iteration was never finalized, so the next deserialization overwrote sequence/string pointers, leaking the payload.
In our deployment, three services are polling the same ros image service; each service leaked ~13-30 mb/min. With these changes, ram usage is stable even after 5h
Is this user-facing behavior change?
Did you use Generative AI?
yes, was diagnosed and patch drafted with Claude code (Fable 5)
Additional Information
The leak needs two or more processes holding clients for the same service - single-client tests never hit it, since the caller always frees the last deserialized sample