Deriving concrete VectorOfArrays / ArrayOfSimilarArrays types from their component types

[oschulz]

Since v1, VectorOfArrays and ArrayOfSimilarArrays carry their element type as a type parameter (ET), derived from the data and structure types via Base.promote_op(view, ...), much like Base's Slices{P,SM,AX,S,N} derives S. Migrating downstream packages (UnROOT, EncodedArrays, ...) surfaced two consequences that we should think through for a future release.

1. Concrete types can no longer be spelled by hand

julia> isconcretetype(PartsView{Float64, Vector{Float64}, Vector{Int32}, Vector{Tuple{}}})
false

julia> eltype(PartsView{Float64, Vector{Float64}, Vector{Int32}, Vector{Tuple{}}})
Any

julia> isconcretetype(VectorOfSimilarVectors{Float64, Matrix{Float64}})
false

Before v1 these were concrete types. Downstream code that declares struct fields, buffers or Vector{...} element types this way now silently becomes type-unstable, and there is no public way to obtain ET for given component types.

2. The constructors are not inferable for nested data

using ArraysOfArrays, Test
v = PartsView(Float64[1, 2, 3], Int32[1, 3, 4])
f(x) = PartsView(x, Int32[1, 2])
@inferred f(v)
# ERROR: return type PartsView{SubArray{...}, PartsView{Float64, Vector{Float64}, Vector{Int32}, Vector{Tuple{}}, SubArray{...}}, Vector{Int32}, Vector{Tuple{}}, T} where T<:AbstractVector{SubArray{...}} does not match inferred return type ...

view(v, 1:1) itself infers fine, but the nested promote_op inside _voa_eltype gets widened during inference, so the return type keeps ET free. Any user function constructing a vector of vectors of vectors is type-unstable, and the idiomatic way for downstream code to ask for the result type, Base.promote_op(VectorOfVectors, VT, VI), is exact for plain and StructArray content but not for nested VectorOfArrays content.

Current downstream workaround

UnROOT needs the concrete type of vector fields statically (for type assertions and typed buffers) and currently uses

VectorOfVectors{eltype(VT), VT, VI, Vector{Tuple{}}, Base.promote_op(view, VT, UnitRange{Int})}

(JuliaHEP/UnROOT.jl#459). That replicates our private formula, including the choice of UnitRange{Int} as index type, which downstream code should not have to know about.

Directions to consider

• Inference-free ET rules for common data vector types (Array, VectorOfArrays, ...) with promote_op only as a fallback. This would fix the constructor inferability for nested data and make Base.promote_op(VectorOfArrays, ...) exact, the natural spelling for downstream code.
• Base.eltype for partially applied types, eltype(::Type{<:VectorOfArrays{T,N,M,VT,VI,VD}}) and the ArrayOfSimilarArrays{T,M,N,P} analogue, computed from the other parameters, so that a partial type can be completed as P{eltype(P)} without any inference in user code.
• A dedicated public function returning the concrete type for given component types (e.g. the result type of splitup(::VT, ::SplitParts{M,VI,VD})), if the above are not enough.
• Documentation: how to obtain concrete types, and a note in the v1 release notes that hand-spelled types are now abstract.

Open questions: whether the index type used for element views (UnitRange{Int}) becomes part of the public contract, how this interacts with StaticSlices and BaseSlicing, and whether custom subtypes of AbstractArrayOfSimilarArrays (which must name ET in their supertype, see EncodedArrays) need the same helpers.

Created by generative AI.