Question about ReduceConverter

[Jingchang Shi (jingchangshi)]

In ReduceConverter, the reduction along the innermost axis is replaced by transpose followed by the reduction along the outer axis, as shown in the following codes

    if (axis == sourceType.getRank() - 1 && !isVectorReduce) {
      source = getTransposedValue(source, op.getLoc(), rewriter);
      axis = sourceType.getRank() - 2;
    }

Is it for optimization on SIMD based devices? Do you think the reduction along the inner axis is slower than the reduction along the outer axis with a transpose op? Or there are other reasons?

[Nhat Nguyen (nhat-nguyen)]

Jingchang Shi (@jingchangshi) Hey thank you for your question, and sorry for the late reply. I don't get notifications from the discussion so I missed this. Will get back to you with some thoughts as soon as possible.

[Nhat Nguyen (nhat-nguyen)]

Haishan Zhu (@haishanzzzz) would be the best person to ask this, here's a snippet from our old internal code -- hopefully it gives some more context:

// Transpose input if reducing the last dimension, this changes the
// accumulation from (1D vector -> scalar) to (2D matrix -> 1D vector),
// which improves ALU and memory efficiency on most SIMD and SIMT
// architectures.