Better support for operations on `option[Vector[...]] `

[agoose77]

Vector Version

0.10.0

Python Version

3.10.7

OS / Environment

Linux (Ubuntu 22.04)
Conda-provisioned Python

Describe the bug

Outer options become inner-options after operations on vectors:

>>> x = vector.awk([{"x": 1, "y": 0, "z": 0}])
>>> y = vector.awk([{"x": 0, "y": 1, "z": 0}])
>>> x.cross(y).type
1 * Vector3D["x": int64, "y": int64, "z": int64]
>>> x.mask[[False]].cross(y).type
1 * Vector3D["x": ?int64, "y": ?int64, "z": ?int64]

Steps to Reproduce

Wrap a vector in an option type, and then operate on it e.g. with cross.

Expected Results

The option should survive outside the record.

Observed Results

The option moves inside the record.

Relevant log output

No response

[agoose77]

This is one of the motivations for the optiontype_outside_record in ak.zip. However, using this option alone might not be the safest way of fixing this bug; users might want options to exist inside the record. @jpivarski this is a question that goes beyond just vector. Do you have any thoughts / ideas here?

[agoose77]

(self note) maybe add an option to zip that allows it to broadcast below record.

[jpivarski]

We talked about this on Slack; having a general array-cast mechanism as described in scikit-hep/awkward#1807 (comment) would allow users and library designers to fix the type after the fact. Looking at use-cases like this helps, though, because it specifies how flexible the type casting would have to be.

For instance, given an array like

# * Vector3D[x: ?int64, y: ?int64, z: ?int64]

and desiring a type

# * option[Vector3D[x: int64, y: int64, z: int64]]

it would have to add the option[·] to Vector3D and remove the ? from all three fields in a single transformation, which checks that the fields have missing values in all the same entries. It can't do the more natural algorithm:

1. add option[·] to Vector3D as an UnmaskedArray because option-type is requested and not present in the array, then
2. recurse into each field separately, and
3. try to remove the ? from a field, only to discover that there's a non-zero number of missing values, and so raise an error.

I might need to start with a list of transformations that the algorithm needs to have.

[Superharz]

Is there a fix to this yet?

I just stumbled across this issue when I attempted to concatenate awkward arrays of Momentum4D Elements where some entries are None:

type: 120000 * option[var * Momentum4D[
    rho: float32,
    phi: float32,
    eta: float32,
    tau: float64
]]

The resulting array has lost the Nones but also the Momentum4D reference and only contains the 4 kinematic branches:

type: N * var * {
    rho: float32,
    phi: float32,
    eta: float32,
    tau: float64
}

Calling vector.awk on the array recreates the Momentum4D objects but only if the input does not already contain them. So it's not a solution to simply always call vector.awk on the output of the concatenate.

[Superharz]

Actually, calling vector.awk on the result creates a Vector4D and not a Momentum4D because of #482.