Minimize relevancy score instead of maximize

[CorneliusHsiao]

Hi, thanks for the excellent work! I have a question regarding your implementation:

lerf/lerf/encoders/openclip_encoder.py

Line 96 in 3b2cb90

best_id = softmax[..., 0].argmin(dim=1) # rays x 2

And in your paper Sec.3.5 (Relevancy Score), you stated:

Intuitively, this score represents how much closer the rendered embedding is towards the query embedding compared to the canonical embeddings.

My understanding about your inline equation and code is:
you try to pick $\phi^i_{canon}$ that is closer to $\phi_{lang}$ compared to $\phi_{lang}$ from $\phi_{quer}$, because minimization over $i$ means maximization of similarity between $\phi^i_{canon}$ and $\phi_{lang}$.

My question is:
why is this minimization instead of maximization? I think we are looking for $\phi_{lang}$ that best matches $\phi_{quer}$ instead of $\phi^i_{canon}$, right? Is it because we want the embedding to fit to both $\phi_{quer}$ and $\phi^i_{canon}$ at the same time? From my experiment, I do see that results getting worse if I change min to max, but could you explain a little bit more please?

Much thanks!