Confused by `cat` behaviour

[alex-s-gardner]

I want to calculate the monthly mean of a 2D DimArray (DateTime by X) and produce a resulting 2D DimArray with dimensions of integer month and X

Create a DimArray with dimensions DataTime and x

using DimensionalData, Dates, Statistics
dTi = Ti(collect(Date(2000,1,1):Day(1):Date(2024,10,1)));
dX = X(collect(1:10));
da = rand(dTi,dX);

group by month and calculate mean

gda = groupby(da, dTi => Bins(month, 1:12));
gda = mean.(gda; dims=Ti);

now comes the part I don't get.

try dropping the DataTime dimension then concatenating

foo2 = dropdims.(gda; dims=Ti);
foo = cat(foo2...; dims=Ti)

somehow DateTime is still there. How did that happen? I explicitly dropped this dim.

┌ 10×12 DimArray{Float64, 2} ┐
├────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────── dims ┐
  ↓ X  Sampled{Int64} [1, 2, …, 9, 10] ForwardOrdered Irregular Points,
  → Ti Sampled{Date} [2012-01-16, …, 2012-12-01] ForwardOrdered Irregular Points
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
  ↓ →   2012-01-16   2012-02-15   2012-03-16  …   2012-10-01   2012-11-01   2012-12-01
  1    0.492005             0.504026             0.494869                0.495121             0.513776             0.507932
  2    0.496965             0.493398             0.51959                 0.487964             0.490996             0.499451
  3    0.488379             0.496261             0.507665                0.481631             0.534441             0.495114
  ⋮                                                                   ⋱  ⋮                                         
  8    0.487388             0.49639              0.514814                0.495875             0.504643             0.492865
  9    0.495462             0.491538             0.492786                0.501266             0.501263             0.504023
 10    0.506875             0.481782             0.500255             …  0.49605              0.51212              0.492466

How can I combine the grouped DimArray to give me a DimArray that inherits the Ti dimension from the groups of the the grouped DimArray such that:

↓ Ti Sampled{Int64} 1:12 ForwardOrdered Regular Points

I can get what I want doing the following:

foo = cat(foo2...; dims=dims(foo2, :Ti))

but it seems cumbersome that cat is not able to inherit the dimension from the groups... which makes me think that I'm not doing this correctly. I looked for a combine function by couldn't find one

[asinghvi17]

DD is being fancy here

julia> dropdims.(gda; dims=Ti)[1].refdims
(↓ Ti Sampled{Date} [Date("2012-01-16")] ForwardOrdered Irregular Points)

so when you cat, it probably checks refdims to recreate the original array.

Indeed if I rebuild to remove the refdims,

cat(rebuild.(dropdims.(gda; dims=Ti); refdims = ())...; dims = Ti)
┌ 10×12 DimArray{Float64, 2} ┐
├────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── dims ┐
  ↓ X  Sampled{Int64} [1, 2, …, 9, 10] ForwardOrdered Irregular Points,
  → Ti
└─────────────────────────────────────────────────────────.....

[rafaqz]

Yeah, because you don't specify the lookup we find the values in refdims.

I think you can just do dims=Ti() and it will be empty, otherwise dims=Ti(No lookup())

[rafaqz]

combine is in the breaking branch

[alex-s-gardner]

It looks like combine requires a reducing function. Is there a method to go from a grouped DimArray to a DimArray that inherits groups as a dimension?

i.e. this works but isn't as intuitive as it might seem

da = cat(gdf...; dims=dims(gdf, :Ti))

Would it make sense for:

da = cambine(gdf)

to do the same as above... maybe my case is too simple and it will break with increasing complexity.

[rafaqz]

Have to think about it but yes something like that should work too