Skip to content

map(f) allocates infinite amount of memory #238

@shall-ow

Description

@shall-ow

This overload:

AxisArrays.jl/src/core.jl

Lines 431 to 435 in bbf1f27

function Base.map(f, As::AxisArray{T,N,D,Ax}...) where {T,N,D,Ax<:Tuple{Vararg{Axis}}}
matchingdims(As) || error("All axes must be identically-valued")
data = map(a -> a.data, As)
return AxisArray(map(f, data...), As[1].axes...)
end

can be called with an empty As, e.g. map(x->x). This calls matchingdims(()) which calls something like tuple(zip()...):
sizes(As::AxisArray...) = tuple(zip(map(a -> map(length, Base.axes(a)), As)...)...)
matchingdims(As::Tuple{Vararg{AxisArray}}) = all(equalvalued, sizes(As...))

Since zip() is an infinite iterator of empty tuples, Julia allocates an infinite amount of memory which is quite unexpected in this case (I stumbled upon this when I forgot an argument to map while using a package that has AxisArrays in its dependency graph and got Julia OOM-killed).
I believe this should be a MethodError instead, e.g. by requiring at least one array:

function Base.map(f, A::AxisArray{T,N,D,Ax}, As::AxisArray{T,N,D,Ax}...) where {T,N,D,Ax<:Tuple{Vararg{Axis}}}
    matchingdims((A, As...)) || error("All axes must be identically-valued")
    data = map(a -> a.data, (A, As...))
    return AxisArray(map(f, data...), A.axes...)
end

Maybe changing matchingdims(()) behavior makes sense as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions