NT-Xent loss with multiple positive pairs

[annaVettoruzzo]

Does NTXent loss address the case with multiple positive pairs? If the label assignment has the same label for multiple samples, does this loss sum over all possible positive pairs (similarly to Ed 2 and 3 in [https://arxiv.org/abs/2201.11736])?

Thank you.

[KevinMusgrave]

Yes it will sum over all positive pairs, and the negative pairs for each positive pair will be based on the anchor of the positive pair.

For example, if the indices for a positive pair are (2, 5), then all of the negative pairs for this positive pair will be of the form (2, X).

So I think the implementation is the same as Equation 2 in the paper you linked.

See these related comments:

• [Question] Understanding terminology reference, embedding #600 (comment)
• NT-Xet loss? #6 (comment)