Skip to content

Deserializing CQL collections causes frequent reallocations of Vec #1058

Open
@pkolaczk

Description

@pkolaczk

This is an issue I found when testing the performance of vector type, but apparently this applies to all collections.
The code uses itertools to collect fallible items into a Vec<Result<...>>:

                ListlikeIterator::<'frame, T>::deserialize(typ, v)
                    .and_then(|it| it.collect::<Result<_, DeserializationError>>())
                    .map_err(deser_error_replace_rust_name::<Self>)

Unfortunately itertools does not use the size hint in this case and does not reserve proper space in the returned vector.
Instead, it grows the vector as more items are added.

Also it the size hint for ListLikeIterator doesn't look correct anyways (it delegates to raw iterator, which provides the default None hint).

This can be trivially fixed by creating the vector with proper capacity first (we know the number of elements very early on and it is stored in the field of ListLikeIterator) and then adding the items with e.g. extend.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions