loss computation: mean and not sum

[CassNot]

Dear authors,

Thank you for your code!

We had a question concerning the loss implementation. We saw that for each minibatch, the mean is computed and not the sum as in the paper (https://arxiv.org/pdf/2004.11362.pdf - equation 2):

SupContrast/losses.py

Line 96 in 331aab5

loss = loss.view(anchor_count, batch_size).mean()

We were wondering if there was a reason for this choice.

Thank you

[HobbitLong]

Good catch! I think the eq 2 in the paper has ignored the 1/(2N).

[dave4422]

Hi,

I've been reviewing the implementation, and I noticed the line loss = loss.view(anchor_count, batch_size).mean(). Given the computations, it seems that the result would be equivalent to simply using loss.mean(). Could you kindly explain the rationale behind the reshaping here?

[dave4422]

I assume it's just for readability?

[HobbitLong]

yeah, it's just helping understand the shape (potentially may help understand what's going on).