nidl.estimators.ssl.utils: Available projection heads

Introduction

A projection head is a small torch.nn.Module mapping the representations produced by an encoder into the space where a self-supervised loss (see nidl.losses: Available losses) is applied. It is typically discarded after pre-training, once the encoder is used for downstream tasks.

Projection heads

ProjectionHead(blocks)

Base class for all projection and prediction heads in self-supervised estimators.

SimCLRProjectionHead([input_dim, ...])

Projection head used for SimCLR.

YAwareProjectionHead([input_dim, ...])

Projection head used for yAware contrastive learning.

BarlowTwinsProjectionHead([input_dim, ...])

Projection head used for Barlow Twins [R86bff2d09119-1].

DINOProjectionHead([input_dim, hidden_dim, ...])

Projection head used in DINO [Re0843dd52443-1].

 
        classDiagram
  Module <|-- ProjectionHead
  ProjectionHead <|-- BarlowTwinsProjectionHead
  ProjectionHead <|-- DINOProjectionHead
  ProjectionHead <|-- SimCLRProjectionHead
  ProjectionHead <|-- YAwareProjectionHead