Deep learning for NeuroImaging in Python.

Note

This page is a reference documentation. It only explains the class signature, and not how to use it. Please refer to the gallery for the big picture.

class nidl.estimators.ssl.utils.projection_heads.SimCLRProjectionHead(input_dim: int = 2048, hidden_dim: int = 2048, output_dim: int = 128)[source]¶

Bases: ProjectionHead

Projection head used for SimCLR.

This module implements the projection head as described in SimCLR [R19]. The projection head is a multilayer perceptron (MLP) with one hidden layer and a ReLU non-linearity, defined as:

$\mathbf{z} = g(\mathbf{h}) = W_2 \cdot \sigma(W_1\cdot\mathbf{h})$

where $\sigma$ is the ReLU activation function.

References

[R19] (1,2)

Chen, T., et al. “A Simple Framework for Contrastive Learning of Visual Representations.” ICML, 2020. https://arxiv.org/abs/2002.05709

Initialize internal Module state, shared by both nn.Module and ScriptModule.