KEEP - Kalman-Inspired Feature Propagation for Video Face Super-Resolution

[johndpope]

I cherry pick idea from here
https://github.com/jnjaby/KEEP

i integrate some logic to shovle attention gains / uncertainties from previous frames to help training converge
https://github.com/johndpope/IMF/blob/feat/keep/keep_model.py#L102

just POC - it's not using the official code above which has more optical flow depencies....
but with the few additions - seems to help a lot

https://wandb.ai/snoozie/IMF/runs/qp2dcd3s?nw=nwusersnoozie

currently there's a race between my 3090 - batchsize =2
running vanilla - https://wandb.ai/snoozie/IMF/runs/892ufzr3?nw=nwusersnoozie

and some cloud compute a100 - 40gb - batchsize = 5

[johndpope]

3090 - vanilla on the left - vs a100 - KEEP(my trimmed down version) on the right

in the vanilla - epoch 4 - im not happy with how it hasn't corrected the face orientation