How to install arrow in R with all features enabled?

[rkrug]

M2 Mac mini. Using homebrew.

I have installed arrow using homebrew, and the R package using the standard install.packages(). I have the following features enabled:

> arrow_info()
Arrow package version: 20.0.0.2

Capabilities:
               
acero      TRUE
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3         TRUE
gcs       FALSE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli    FALSE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2       FALSE
jemalloc  FALSE
mimalloc   TRUE

Memory:
                  
Allocator mimalloc
Current    0 bytes
Max        0 bytes

Runtime:
                        
SIMD Level          none
Detected SIMD Level none

Build:
                                    
C++ Library Version           20.0.0
C++ Compiler              AppleClang
C++ Compiler Version 14.0.0.14000029
>

How can I install apache so that (at lest the compression features) are all installed?

Thanks

Rainer

[rkrug]

How can I install apache so that (at lest the compression features) are all installed?

Do I really have to install everything from source as described here https://arrow.apache.org/docs/8.0/r/articles/developers/setup.html or is there a simpler way?

Thanks.

[amoeba]

You don't have to install all the dependencies from source, no, but it's the simplest solution for users who just want to use all features. The build system can pick up pre-installed dependencies from the system though I'm not sure how well that works at the moment.

The reason we don't ship a feature-complete binary package has to do with difficulties around getting CRAN to build a feature-complete version of the package. The version hosted by R-universe is more feature-complete so that may be your simplest option. At the moment, the R-universe version has these features enabled:

Capabilities:

acero      TRUE
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3         TRUE
gcs        TRUE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli     TRUE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2        TRUE
jemalloc   TRUE
mimalloc   TRUE

[rkrug]

Let’s say, it worked for me without problems, and was, if I am not mistaken, done in less then a minute.

Good to know, that the RUniverse contains a nearly feature complete version - I will use it the next time. Thanks.

But I am wondering: why is lzo disabled? If I am not mistaken, is it a compression algorithm, and I have lzo installed using homebrew.

I have no idea what substrait is, and I do not think I will need it.

Thanks.

[amoeba]

I think Arrow C++ just doesn't support lzo (and that's what the R package is binding to).

[rkrug]

Solved by using suggestion here: #44859. In a nutshell:

Sys.setenv("NOT_CRAN" = "true")
install.packages("arrow", type = "source")