Skip to content

Multi threading phase 3: serialize multiple columns simultaneously #64

Open
@fstpackage

Description

@fstpackage

Some column-types can be serialized more efficiently than others. Character vector serialization is slow compared to integer, double, logical and factor serialization. We can process serialization of a character vector in parallel with the serialization of a column of another type. Combining serialization of a slow column with the serialization of a fast column enables maximum throughput of data and keeps the IO device busy.

In short: combine a column that can be (de-) serialized faster than IO speed with a column that can be (de-) serialized at lower than IO speed and process in parallel.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions