Size the interpolate flops from the whole output tensor

[vineethsaivs]

_upsample_flops_compute counts only the output spatial shape when F.interpolate is called with size=, dropping the batch and channel dims. Its own scale_factor branch sizes the job from input.numel() and is correct, so the same op gets two different answers depending on which argument you pass.

Cause: return int(_prod(size)), 0. size is the output spatial shape only, and a scalar size applies to every spatial dim, not just one.

Fix: multiply by input.shape[:2], and raise a scalar size to input.ndim - 2.

Test: pytest tests/unit/profiling/flops_profiler/test_flops_profiler.py -m sequential -k "interpolate or elementwise or conv_transpose" -> 7 failed / 14 passed before, 21 passed after. New test_interpolate_flops asserts flops == result.numel() over 3d/4d/5d inputs, tuple and scalar size, with the scale_factor case as the unchanged control.

input	call	true	before	after
[8,3,32,32]	size=(64,64)	98304	4096	98304
[8,3,32,32]	size=64	98304	64	98304
[8,3,32,32]	scale_factor=2	98304	98304	98304
[2,3,8,8,8]	size=16	24576	16	24576

[ebarkhordar]

The size fix looks right. All eight of your parametrizations match result.numel() at a356a4f, in a clean python:3.11-slim container on torch 2.14.0+cpu.

The control is the part I would look at again. scale_factor only agrees when the scales are integers, because torch floors each output dim separately while the branch multiplies the scales together and truncates once:

(8,3,32,32) scale_factor=1.4        out (8,3,44,44)     flops 48168  numel 46464
(8,3,32,32) scale_factor=(1.4,2.6)  out (8,3,44,83)     flops 89456  numel 87648
(2,3,7,7,7) scale_factor=1.5        out (2,3,10,10,10)  flops  6945  numel  6000
(1,4,20,20) scale_factor=(0.7,1.9)  out (1,4,14,38)     flops  2127  numel  2128

That last one is off by one in the other direction, from 1600 * (0.7*1.9) landing just under 2128.

So it is the same bug one branch over, which is why scale_factor=2 passing as your control does not say much. Building it the way you built the size side, flops = _prod(input.shape[:2]) and then flops *= math.floor(dim * scale) across input.shape[2:], gives result.numel() on all ten cases I ran, your three included.

git log -G on that line reaches #7353, which corrected the exponent from len(input) to input.ndim - 2 and left the flooring alone, so it reads as unconsidered rather than deliberate. Your call whether it belongs in this PR.

[vineethsaivs]

You are right, and it is worse than the control being weak: the scale_factor branch is wrong on its own terms. Folded the fix in as d605f91.

Reproduced your four cases exactly on torch 2.11.0+cpu, so it is not version-specific:

(8,3,32,32) scale_factor=1.4       out (8,3,44,44)    flops 48168  numel 46464
(8,3,32,32) scale_factor=(1.4,2.6) out (8,3,44,83)    flops 89456  numel 87648
(2,3,7,7,7) scale_factor=1.5       out (2,3,10,10,10) flops  6945  numel  6000
(1,4,20,20) scale_factor=(0.7,1.9) out (1,4,14,38)    flops  2127  numel  2128

The branch now sizes the job the way the size branch does, from the batch and channel dims, and multiplies by int(dim * scale) per spatial dim. int rather than math.floor only because profiler.py imports no math and the scales are positive, so the two agree.

Seven scale_factor parametrizations added, four non-integer plus three integer ones as the controls that must not move:

$ pytest tests/unit/profiling/flops_profiler/test_flops_profiler.py -m sequential -k interpolate -q
before   4 failed, 11 passed     (exactly your four, with your numbers)
after   15 passed

Your read of #7353 matches what I see: it corrected the exponent and left the flooring alone, so this had never been looked at. yapf --style .style.yapf is clean.

Thanks for actually running it rather than eyeballing the diff. The scalar-size case in the other branch is the same shape of mistake, which is what made me look there first.

[ebarkhordar]

Checked d605f91 against real F.interpolate output on torch 2.14.0+cpu, over 21 shape and scale combinations including your four (mode="nearest", recompute_scale_factor=False). I parsed _prod and _upsample_flops_compute out of profiler.py at that SHA and called them directly, so this exercises the two functions rather than the profiler hooks.

             input          scale         real out     numel      old      new
    (8, 3, 32, 32)            1.4   (8, 3, 44, 44)     46464    48168    46464
    (1, 4, 20, 20)     (0.7, 1.9)   (1, 4, 14, 38)      2128     2127     2128
      (1, 2, 5, 5)     (1.1, 1.1)     (1, 2, 5, 5)        50       60       50
    (1, 2, 49, 49)           1.02   (1, 2, 49, 49)      4802     4996     4802
   (2, 3, 6, 6, 6) (1.3, 1.3, 1.3)  (2, 3, 7, 7, 7)      2058     2847     2058
(1, 1, 1000, 1000)          1.001 (1, 1, 1000, 1000)  1000000  1002000  1000000
    (8, 3, 32, 32)              2   (8, 3, 64, 64)      98304    98304    98304

21 cases, 13 wrong before, 0 wrong after

On int against math.floor: for positive scales those two never differ, so the place this could still have gone wrong is the double multiply, not the truncation. 1000 * 1.001 is 1000.9999999999999, and torch lands on 1000 there too, so both sides inherit the same rounding. Same for 49 * 1.02. They agreed on all 21. I did not run the repo's test suite, so your 15 passed is your number, not a second measurement of it.

[vineethsaivs]

Thanks for running it against real F.interpolate output rather than reading the diff. 21 shape and scale combinations, new == numel on every row, is a better check than my four cases.