nidl.losses: Available losses

Common losses.

Introduction

A loss is a torch.nn.Module (or a plain callable) implementing the objective function optimized during the training_step of an estimator. Losses are decoupled from the estimators that use them so that they can be reused, benchmarked or swapped independently.

Self-supervised learning losses

Losses used by the self-supervised learning embedding estimators (see API References).

InfoNCE([temperature])

Implementation of the InfoNCE loss [Re38fc64e0ed4-1], [Re38fc64e0ed4-2].

DCLLoss([temperature, pos_weight_fn])

Implementation of the Decoupled Contrastive Learning loss [R0cf4714be807-1]

DCLWLoss([sigma, temperature])

Decoupled Contrastive Loss (DCL) with von Mises-Fisher (vMF) weighting.

YAwareInfoNCE([kernel, bandwidth, temperature])

Implementation of the y-Aware InfoNCE loss [Ra2feb9ab43ec-1].

BarlowTwinsLoss([lambd])

Implementation of the Barlow Twins loss [Re83c9b545e4a-1].

DINOLoss([output_dim, warmup_teacher_temp, ...])

Implementation of the DINO loss [Re2e7efabd714-1].

Autoencoder losses

Losses used by the autoencoder estimators.

BetaVAELoss([beta, default_dist])

Compute the Beta-VAE loss [Rd208eb1f92d3-1].