Note

This page is a reference documentation. It only explains the class signature, and not how to use it. Please refer to the user guide for the big picture.

nidl.estimators.ssl.utils.SimCLRProjectionHead¶

class nidl.estimators.ssl.utils.SimCLRProjectionHead(input_dim=2048, hidden_dim=2048, output_dim=128)[source]¶

Bases: ProjectionHead

Projection head used for SimCLR.

This module implements the projection head as described in SimCLR [1]. The projection head is a multilayer perceptron (MLP) with one hidden layer and a ReLU non-linearity, defined as:

$\mathbf{z} = g(\mathbf{h}) = W_2 \cdot \sigma(W_1\cdot\mathbf{h})$

where $\sigma$ is the ReLU activation function.

References

[1]

Chen, T., et al. “A Simple Framework for Contrastive Learning of Visual Representations.” ICML, 2020. https://arxiv.org/abs/2002.05709

__init__(input_dim=2048, hidden_dim=2048, output_dim=128)[source]¶: Initialize internal Module state, shared by both nn.Module and ScriptModule.