`vec_slice` not falling back to `[` for `vctrs_list_of` types?

[CoryMcCartan]

We (@christopherkenny and I) are implementing an adjacency list vctrs type. We have a custom [ that reindexes the integer "pointers." However, since our type inherits from vctrs_list_of, we can't seem to get vec_slice() to call [.

For example:

> a = print(adj(2, 1)) # two nodes connected with a single edge
<adj[2]>
[1] {2} {1}

> a[2:1] # expected: switching order yields same list
<adj[2]>
[1] {2} {1}

> vec_slice(a, 2:1) # wrong
<adj[2]>
[1] {1} {2}

We tried implementing an identity vec_proxy() per what seemed like a suggestion in the documentation. However, the only thing we got working was the following, which feels extremely non-ideal, though it fixes vec_slice() for us. (We have separate implementations for vec_proxy_equal(), vec_proxy_compare(), etc.)

vec_proxy.adj <- function(x, ...) {
    seq_along(x)
}
vec_restore.adj <- function(x, to, ...) {
    if (is.integer(x)) {
        to[x]
    } else {
        NextMethod()
    }
}

We see #1116 and #1411, which suggest that the vctrs team has some reasons for the current behavior, e.g., not making vec_slice() generic?

Are we missing an easy fix here, or is there a suggested way to make e.g. slicing a tibble with our class have the same behavior as slicing a data frame? Thanks.

[DavisVaughan]

Let me first say that I have no idea how adjacency lists are really supposed to work, so I'm guessing a bit here.

I have a gut feeling that the things you'd like to do with your adjacency list class are just not compatible with vctrs, but I'm not totally sure.

Remember that vctrs classes are meant to be used as columns in a data frame alongside other columns. So any kind of slicing must make sense alongside other columns.

You also must be able to support NA slicing and duplicate values, which it doesn't seem like you can.

library(adj)

x <- adj(c(2, 3), c(1, 3), c(1, 2))

df <- data.frame(
  idx = 1:3,
  x = x
)
df
#>   idx      x
#> 1   1 {2, 3}
#> 2   2 {1, 3}
#> 3   3 {1, 2}

# I'm not really sure what this means
df[c(3, 2, 1),]
#>   idx      x
#> 3   3 {3, 2}
#> 2   2 {3, 1}
#> 1   1 {2, 1}

# Would need to be able to do this
df[c(1, 1),]
#> Error in `[.adj`(xj, i): Duplicate indices in reindexing vector

# Would need to be able to do this
df[c(1, NA),]
#> Error in `xj[i]`:
#> ! Can't subset elements.
#> ✖ Subscript can't contain missing values.
#> ✖ It has a missing value at location 2.

# This doesn't work either
vctrs::vec_c(x, x)
#> Error in `to[x]`:
#> ! Can't subset elements.
#> ✖ Subscript can't contain missing values.
#> ✖ It has missing values at locations 1, 2, 3, 4, 5, etc.

---

One other code smell I noticed was your proxy/restore methods.

Something to be aware of is that vec_proxy() and vec_restore() should always be paired together. You should proxy something, manipulate the proxy, and then restore back. Avoid calling one without the other. Your current vec_restore() method makes me think you can call it without calling vec_proxy(), which is not correct from a vctrs perspective.

---

The point of vec_proxy() is to provide vctrs a C compatible version of your object that we can manipulate at the C level (via slicing or combining, etc). It must:

• Be one of the "native" vctrs vector types, like logical, integer, double, raw, character, or list. Or a data frame containing columns of that type.
• Have the same size as the original object
• Maintain enough information so that you can vec_restore() from it after our manipulations back to your original type. And vec_restore(x, to) itself must also retain the size of x throughout the restoration process.

So for something like vec_c(x, x), vctrs expects you to be able to handle something like this:

library(adj)

x <- adj(c(2, 3), c(1, 3), c(1, 2))

# "proxy"
proxy <- vctrs:::unstructure(x)
proxy
#> [[1]]
#> [1] 2 3
#> 
#> [[2]]
#> [1] 1 3
#> 
#> [[3]]
#> [1] 1 2

# "manipulate"
out <- c(
  proxy,
  proxy
)
out
#> [[1]]
#> [1] 2 3
#> 
#> [[2]]
#> [1] 1 3
#> 
#> [[3]]
#> [1] 1 2
#> 
#> [[4]]
#> [1] 2 3
#> 
#> [[5]]
#> [1] 1 3
#> 
#> [[6]]
#> [1] 1 2

# "restore"
# vec_restore(out, x)

From what I see with your implementation of c(), you want this

c(x, x)
#> <adj[6]>
#> [1] {2, 3} {1, 3} {1, 2} {5, 6} {4, 6} {4, 5}

Can you reconstruct that from the bare bones out that vctrs returns above? And would it always be guaranteed that vec_size(out) == vec_size(vec_restore(out, x)) for you? If not, unfortunately I don't think this class is compatible with the vctrs system.

A similar argument would apply with vec_slice().

---

vec_slice() says that it calls [, but only if a vec_proxy() method doesn't exist. But one does exist, it's vec_proxy.vctrs_list_of, which you inherit by being a list-of. So that prevents your [ method from being called.

I think I'd argue that your class should not inherit from list-of at all.

Something that is a list-of should act like a list from the user's POV, like to a user this is clearly still "just a list":

vctrs::list_of(1, 2, 3)
#> <list_of<double>[3]>
#> [[1]]
#> [1] 1
#> 
#> [[2]]
#> [1] 2
#> 
#> [[3]]
#> [1] 3

But that isn't the case for you. The list storage is an implementation detail, and you want your users to think of this as more of an atomic vector class.

So I think for you I'd actually just have a class structure of "adj". Then you'll have to teach vctrs that your class is actually a vector class. Read the Vectors and Scalars part of ?obj_is_vector closely, but the gist is that you will still need to provide some vec_proxy() method that satisfies what I laid out above. But that again means that your [ method won't be called, because now you've implemented vec_proxy().