Skip to content

Flag the use of special columns when reading a fst file  #142

@MarcusKlik

Description

@MarcusKlik

A fst file might contain columns of non-standard types integer64, nanotime or ITime. To make sure these are processed and printed correctly, the corresponding packages need to be loaded (bit64, nanotime or data.table). The C++ core library (fstlib) returns column vectors with the appropriate class set, but the specific packages still need to be loaded.

For a large number of columns, scanning for these special column types using the class names can be very expensive.

To avoid running

sample_data <- data.frame(
  X = bit64::as.integer64(1:10),
  Y = 1:10)

# check for integer64 column
any(sapply(sample_data, inherits, "integer64"))
#> [1] TRUE

in read_fst() for all special types, the core library should return a vector of all column-types that were read. That vector can be used to selectively attach the correct packages, without the performance hit.

Metadata

Metadata

Assignees

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions