Merge branch 'release'

Author: MarcusKlik

.travis.yml

@@ -15,8 +15,6 @@ matrix:
   exclude:
   - r: release
     os: osx
-  - r: oldrel
-    os: linux
   - r: devel
     os: osx
 

DESCRIPTION

@@ -1,27 +1,34 @@
 Package: fst
 Type: Package
 Title: Lightning Fast Serialization of Data Frames for R
-Description: Multithreaded serialization of compressed data frames using the 'fst' format.
-    The 'fst' format allows for random access of stored data and compression with the
-    LZ4 and ZSTD compressors created by Yann Collet. The ZSTD compression library is owned by
-    Facebook Inc.
-Version: 0.8.2
-Date: 2017-12-18
+Description: Multithreaded serialization of compressed data frames using the
+    'fst' format. The 'fst' format allows for random access of stored data and
+    compression with the LZ4 and ZSTD compressors created by Yann Collet. The ZSTD
+    compression library is owned by Facebook Inc.
+Version: 0.8.4
+Date: 2018-01-25
 Authors@R: c(
     person("Mark", "Klik", email = "[email protected]", role = c("aut", "cre", "cph")),
     person("Yann", "Collet", role = c("ctb", "cph"),
       comment = "Yann Collet is author of the bundled LZ4 and ZSTD code and copyright holder of LZ4"),
     person("Facebook, Inc.", role = "cph", comment = "Bundled ZSTD code"))
 LazyData: true
-Depends: R (>= 3.0.0)
-Imports: Rcpp
+Depends:
+    R (>= 3.0.0)
+Imports:
+    Rcpp
 LinkingTo: Rcpp
 SystemRequirements: little-endian platform
 RoxygenNote: 6.0.1
-Suggests: testthat, bit64, data.table, lintr, nanotime
+Suggests:
+    testthat,
+    bit64,
+    data.table,
+    lintr,
+    nanotime
 License: AGPL-3 | file LICENSE
 Copyright: This package includes sources from the LZ4 library written
     by Yann Collet, sources of the ZSTD library owned by Facebook, Inc.
-    and sources of the fstlib library owned by Mark Klik 
+    and sources of the fstlib library owned by Mark Klik
 URL: https://fstpackage.github.io
 BugReports: https://github.com/fstpackage/fst/issues

NAMESPACE

@@ -1,8 +1,20 @@
 # Generated by roxygen2: do not edit by hand
 
+S3method("$",fst_table)
+S3method("[",fst_table)
+S3method("[[",fst_table)
+S3method(as.data.frame,fst_table)
+S3method(as.list,fst_table)
+S3method(dim,fst_table)
+S3method(dimnames,fst_table)
+S3method(names,fst_table)
+S3method(print,fst_table)
 S3method(print,fstmetadata)
+S3method(row.names,fst_table)
+S3method(str,fst_table)
 export(compress_fst)
 export(decompress_fst)
+export(fst)
 export(fst.metadata)
 export(hash_fst)
 export(metadata_fst)
@@ -14,4 +26,5 @@ export(write_fst)
 importFrom(Rcpp,sourceCpp)
 importFrom(parallel,detectCores)
 importFrom(utils,packageVersion)
+importFrom(utils,str)
 useDynLib(fst, .registration = TRUE)

NEWS.md

@@ -1,6 +1,29 @@
 
 **If you are viewing this file on CRAN, please check latest news on GitHub [here](https://github.com/fstpackage/fst/blob/develop/NEWS.md).**
 
+### Changes in v0.8.4
+
+The v0.8.4 release brings a `data.frame` interface to the `fst` package. Column and row selection can now be done directly from the `[` operator. In addition, it fixes some issues and prepares the package for the next build toolchain of CRAN.
+
+#### New features
+
+* A `data.frame` interface was added to the package. The user can create a reference obkect to a `fst` file with method `fst`. That reference can be used like a `data.frame` and will automatically make column- and row- selections in the referenced `fst` file.
+
+#### Bugs solved
+
+* Build issues with the dev build of R have been fixed. In particular, `fst` now builds correctly with the Clang 6.0 toolchain which will be released by CRAN shortly (thanks @kevinushey for reporting the problem and CRAN maintainers for the advance warning.
+
+* An error was fixed where compressing a short factor column with 128 to 32767 levels but only a single value, returned incorrect results (issue #128, thanks @martinblostein for reporting and help fixing the problem).
+
+* An error was fixed where columns f type 'ITime' were incorrectly serialized (issue #126, thanks @Giqles for reporting the problem).
+
+* An error was fixed where using `fst` as a dependency in another package and building that package in RStudio, crashed RStudio. The problem was that RStudio uses a fork to build or document a package. That fork made `fst` use OpenMP library methods, which leads to crashes on macOS. After the fix, no calls to any OpenMP library method are now made from `fst` when it's run from a forked process (issue #100 and issue #109, thanks to @eipi10, @PeteHaitch, @kevinushey, @thierrygosselin, @xiaodaigh and @jzzcutler for reporting the problem and help fix it).
+
+#### Documentation
+
+* Documentation for method `write_fst` was improved (issue #123, thanks @krlmlr for reporting and submitting a pull request).
+
+
 ### Changes in v0.8.2
 
 #### New features

R/fst.R

@@ -32,13 +32,14 @@
 #' @param x a data frame to write to disk
 #' @param path path to fst file
 #' @param compress value in the range 0 to 100, indicating the amount of compression to use.
+#'   Lower values mean larger file sizes.
 #' @param uniform_encoding If TRUE, all character vectors will be assumed to have elements with equal encoding.
 #' The encoding (latin1, UTF8 or native) of the first non-NA element will used as encoding for the whole column.
 #' This will be a correct assumption for most use cases.
 #' If \code{uniform.encoding} is set to FALSE, no such assumption will be made and all elements will be converted
 #' to the same encoding. The latter is a relatively expensive operation and will reduce write performance for
 #' character columns.
-#' @return \code{read_fst} returns a data frame with the selected columns and rows. \code{read_fst})
+#' @return \code{read_fst} returns a data frame with the selected columns and rows. \code{read_fst}
 #' invisibly returns \code{x} (so you can use this function in a pipeline).
 #' @examples
 #' # Sample dataset
@@ -97,13 +98,15 @@ metadata_fst <- function(path, old_format = FALSE) {
     stop("A logical value is expected for parameter 'old_format'.")
   }
 
-  metadata <- fstmetadata(normalizePath(path, mustWork = TRUE), old_format)
+  full_path <- normalizePath(path, mustWork = TRUE)
+
+  metadata <- fstmetadata(full_path, old_format)
 
   if (inherits(metadata, "fst_error")) {
     stop(metadata)
   }
 
-  colInfo <- list(path = path, nrOfRows = metadata$nrOfRows,
+  colInfo <- list(path = full_path, nrOfRows = metadata$nrOfRows,
     keys = metadata$keyNames, columnNames = metadata$colNames,
     columnBaseTypes = metadata$colBaseType, keyColIndex = metadata$keyColIndex,
     columnTypes = metadata$colType)
@@ -116,7 +119,7 @@ metadata_fst <- function(path, old_format = FALSE) {
 #' @export
 print.fstmetadata <- function(x, ...) {
   cat("<fst file>\n")
-  cat(x$nrOfRows, " rows, ", length(x$columnNames), " columns (", x$path,
+  cat(x$nrOfRows, " rows, ", length(x$columnNames), " columns (", basename(x$path),
     ")\n\n", sep = "")
 
   types <- c("unknown", "character", "factor", "ordered factor", "integer", "POSIXct", "difftime",