Documentation on "correct" way of handling `NA` in `list_of`

[bart1]

I was investigating the implementation of a list_of vector. For that I studies the poly example. I was interested in understanding how NA values should be handled. I tried two different ways of creating a vector with NA (see below) and they ended with different lists. Once with a NULL element and once with a NA_integer_, this results in quite different behavior both for printing and detecting NA's. From the behaviour it seems the NULL version is more like I would expect. However then this should be probably printing NA or missing. This might not really be an issue with the package code but more profit form the best practice being included in the documentation.

require(vctrs)
#> Loading required package: vctrs
poly <- function(...) {
  x <- vec_cast_common(..., .to = integer())
  new_poly(x)
}
new_poly <- function(x) {
  new_list_of(x, ptype = integer(), class = "vctrs_poly_list")
}

format.vctrs_poly_list <- function(x, ...) {
  format_one <- function(x) {
    if (length(x) == 0) {
      return("")
    }
    
    if (length(x) == 1) {
      format(x)
    } else {
      suffix <- c(paste0("\u22C5x^", seq(length(x) - 1, 1)), "")
      out <- paste0(x, suffix)
      out <- out[x != 0L]
      paste0(out, collapse = " + ")
    }
  }
  
  vapply(x, format_one, character(1))
}

obj_print_data.vctrs_poly_list <- function(x, ...) {
  if (length(x) != 0) {
    print(format(x), quote = FALSE)
  }
}

p <- poly(1, NA, c(1, 0, 1))
pp <- c(poly(2:1), NA, poly(1))
str(p)
#> vctrs_p_ [1:3] 
#> $ : int 1
#> $ : int NA
#> $ : int [1:3] 1 0 1
#> @ ptype: int(0)
str(pp)
#> vctrs_p_ [1:3] 
#> $ : int [1:2] 2 1
#> $ : NULL
#> $ : int 1
#> @ ptype: int(0)
p
#> <vctrs_poly_list[3]>
#> [1] 1         NA        1⋅x^2 + 1
pp
#> <vctrs_poly_list[3]>
#> [1] 2⋅x^1 + 1           1
is.na(p)
#> [1] FALSE FALSE FALSE
is.na(pp)
#> [1] FALSE  TRUE FALSE
vec_detect_missing(p)
#> [1] FALSE FALSE FALSE
vec_detect_missing(pp)
#> [1] FALSE  TRUE FALSE
p==p
#> [1] TRUE TRUE TRUE
pp==pp
#> [1] TRUE   NA TRUE
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.0 (2024-04-24)
#>  os       Ubuntu 22.04.4 LTS
#>  system   x86_64, linux-gnu
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Europe/Amsterdam
#>  date     2024-10-29
#>  pandoc   3.1.11 @ /usr/lib/rstudio/resources/app/bin/quarto/bin/tools/x86_64/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  cli           3.6.3   2024-06-21 [1] CRAN (R 4.4.0)
#>  digest        0.6.35  2024-03-11 [1] CRAN (R 4.4.0)
#>  evaluate      0.23    2023-11-01 [1] CRAN (R 4.4.0)
#>  fastmap       1.2.0   2024-05-15 [1] CRAN (R 4.4.0)
#>  fs            1.6.4   2024-04-25 [1] CRAN (R 4.4.0)
#>  glue          1.8.0   2024-09-30 [1] CRAN (R 4.4.0)
#>  htmltools     0.5.8.1 2024-04-04 [1] CRAN (R 4.4.0)
#>  knitr         1.46    2024-04-06 [1] CRAN (R 4.4.0)
#>  lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.4.0)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.4.0)
#>  purrr         1.0.2   2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo          1.26.0  2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils       2.12.3  2023-11-18 [1] CRAN (R 4.4.0)
#>  reprex        2.1.0   2024-01-11 [1] CRAN (R 4.4.0)
#>  rlang         1.1.4   2024-06-04 [1] CRAN (R 4.4.0)
#>  rmarkdown     2.26    2024-03-05 [1] CRAN (R 4.4.0)
#>  rstudioapi    0.16.0  2024-03-24 [1] CRAN (R 4.4.0)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.4.0)
#>  styler        1.10.3  2024-04-07 [1] CRAN (R 4.4.0)
#>  vctrs       * 0.6.5   2023-12-01 [1] CRAN (R 4.4.0)
#>  withr         3.0.1   2024-07-31 [1] CRAN (R 4.4.0)
#>  xfun          0.43    2024-03-25 [1] CRAN (R 4.4.0)
#>  yaml          2.3.9   2024-07-05 [1] CRAN (R 4.4.0)
#> 
#>  [1] /home/bart/R/x86_64-pc-linux-gnu-library/4.4
#>  [2] /usr/local/lib/R/site-library
#>  [3] /usr/lib/R/site-library
#>  [4] /usr/lib/R/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[bart1]

I realized the format function might want to return NA_character_ for consistent formating

[fabian-s]

Seconding this -- more documentation on best practices for creating & handling NAs would be very welcome.

[DavisVaughan]

The short answer is that if you have a list, then NULL is the way to signal a missing value in vctrs and is what you should use. Do not use NA_integer_ or anything else.

The NULL will work automatically with vec_detect_missing() and other vctrs utilities.

[DavisVaughan]

The behavior seen here is correct

# Here the `NA` is cast to `<integer>` and is stored as `NA_integer_` as an element of the list

p <- poly(1, NA, c(1, 0, 1))
str(p)
#> vctrs_p_ [1:3] 
#> $ : int 1
#> $ : int NA
#> $ : int [1:3] 1 0 1
#> @ ptype: int(0)

# Here the `NA` is cast to the missing element for `poly<integer>` which means it is internally a `list(NULL)`
# with poly related attributes added onto it. Then it is combined with the other poly objects.

pp <- c(poly(2:1), NA, poly(1))
str(pp)
#> vctrs_p_ [1:3] 
#> $ : int [1:2] 2 1
#> $ : NULL
#> $ : int 1
#> @ ptype: int(0)

[fabian-s]

Sorry if this derails the issue but, arguably, p[2] and pp[2] should end up being identical objects (i.e., "NA"-polynomials represented as a NULL-entry) in your example -- am I understanding correctly that it is the class authors' responsibility to make sure this kind of streamlining happens somehow if computations on vctrs-objects happen to result in invalid entries somewhere inside?
Is there any documentation/best practice code examples for this kind of NA handling?

[DavisVaughan]

If list(NULL) and list(NA_integer_) are the same from your class's point of view, then yes it is the class author's responsibility to handle this. From vctrs's perspective those are very different.

[bart1]

@DavisVaughan Thank you for the clarification! Having this in de documentation would be helpful