Apply or expose rotation

[johan-sightic]

🚀 The feature

I have videos collected with an iPhone, some of which are rotated +/- 90 degrees. This shows up in the output of ffprobe:

$ ffprobe video.mp4 
...
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, smpte170m/bt709/bt709), 1440x1080, 80078 kb/s, 59.91 fps, 59.94 tbr, 600 tbn (default)
    Metadata:
      creation_time   : 2023-09-01T15:16:03.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees

PyAV exposes this on each frames as frame.rotation.

Could torchcodec expose this too in the metadata or even better apply the rotation when decoding?

Motivation, pitch

Currently I need to get the rotation in another way and transpose the return tensor. It would be nice to not have to deal with that

[NicolasHug]

Thanks for the feature request @johan-sightic . I think we'll want to support that, and natively return the frames in the orientation that respects the rotation metadata. That's potentially a BC-breaking change, but it probably qualifies as a bug-fix?

[johan-sightic]

That would be great! Yeah could be classified as a bug