Note

This page is a reference documentation. It only explains the class signature, and not how to use it. Please refer to the user guide for the big picture.

nidl.losses.DCLWLoss¶

class nidl.losses.DCLWLoss(sigma=0.5, temperature=0.1)[source]¶

Bases: DCLLoss

Decoupled Contrastive Loss (DCL) with von Mises-Fisher (vMF) weighting.

It implements the DCL with vMF weighting as described in [1]. See the documentation for DCLLoss for more details.

The vMF weighting function is defined as:

$w(z_i^{(1)}, z_i^{(2)}) = 2 - \frac{ \exp\!\big(\operatorname{sim}(z_i^{(1)}, z_i^{(2)})/\sigma\big) }{ \frac{1}{N}\sum\limits_{j=1}{N} \exp\!\big(\operatorname{sim}(z_i^{(1)}, z_i^{(2)})/\sigma\big) }$

where $N$ is the batch size, $\operatorname{sim}(z_i^(1), z_i^(2))$ denotes the cosine similarity between the normalized embeddings $z_i^(1)$ and $z_i^(2)$ , and $\sigma > 0$ is a temperature parameter controlling the concentration of the distribution.

Parameters:

sigma: float, default=0.5: Temperature parameter of the von Mises-Fisher weighting function.
temperature: float, default=0.1: Scale logits by the inverse of the temperature.

References

[1]

Yeh, Chun-Hsiao, et al. “Decoupled contrastive learning.” European conference on computer vision. Cham: Springer Nature Switzerland, 2022. https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136860653.pdf

__init__(sigma=0.5, temperature=0.1)[source]¶: Initialize internal Module state, shared by both nn.Module and ScriptModule.