Shouldn't Prototypical loss use torch.cdist instead of F.pairwise_distance?

[theolepage]

Hello,

I have a question regarding the following part used for the Prototypical loss computation.

voxceleb_trainer/loss/proto.py

Line 31 in 343af8b

output = -1 * (F.pairwise_distance(out_positive.unsqueeze(-1),out_anchor.unsqueeze(-1).transpose(0,2))**2)

From my understanding, output should be similar to the cosine similarity matrix used for the Angular Prototypical loss but based on Euclidean distances instead.

Thus, the output tensor should have a shape of $(N, N)$ (with $N$ the number of samples in the mini-batch) and values at $i, j$ should be the squared Euclidean distance between sample $i$ of out_positive and sample $j$ of out_anchor.

However, F.pairwise_distance computes the pairwise distance between out_positive and out_anchor and not the distance between each pair between two sets of row vectors like torch.cdist.

Visualization of the difference between F.pairwise_distance and F.cosine_similarity

As a result the output shape will be $(N, D)$ (with $D$ the output dimension of the model) and the following loss computation is not coherent.

Thanks.

[AlexGranger-scn]

Same confusion..... Have you solved this problem now? Thanks for your reply!

[deGennesMarc]

Same issue. In the spirit of @theolepage 's suggestion, I replaced the line with :
output = -torch.cdist(out_positive, out_anchor, p=2).pow(2)
but as of today it does not work for me.

Also it seems to me the definition of the prototypical loss from the "In defence of metric learning" paper is wrong as there should be a minus sign in front of the distances S_{j,k} in the softmax.