Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR lowers the channel sizes for both the internal and external event streams.
Since these channels are implemented internally as a
VecDeque, we are allocating a largeRawVecequal to the channel capacity on initialization. For the internal event stream this is mostly inconsequential, since the vector holdsArcreferences (so even an array with 100,000 entries only takes up a few megabytes).On the other hand, for the external event stream, which holds structs like
QuorumProposal, this results in a ~200+ MB virtual allocation at startup. This memory is not immediately physically allocated to the program (because it will likely exceedMMAP_THRESHOLDfor the allocator), but instead becomes available as we append to the vector. Broadcasts to the channel will always append to the array in memory (becauseVecDequeis a circular buffer), increasing the physical memory usage. Even when we drop entries at the head of theVecDeque, we do not free any of the initial segments of memory (of course, doing this in a circular buffer would be counterproductive), so this manifests as a memory leak up until we reach the end of the underlying array at ~200 MB (or more).