Note

This page is a reference documentation. It only explains the class signature, and not how to use it. Please refer to the user guide for the big picture.

nidl.estimators.ssl.utils.BarlowTwinsProjectionHead

class nidl.estimators.ssl.utils.BarlowTwinsProjectionHead(input_dim=2048, hidden_dim=8192, output_dim=8192)

Bases: ProjectionHead

Projection head used for Barlow Twins [1].

It implements the upscaling of layer sizes (hidden and output layers of size 8192), with 3-layer MLPs as in [1] or [2]

References

[1] (1,2)

Zbontar, J., et al., “Barlow Twins: Self-Supervised Learning via Redundancy Reduction.” PMLR, 2021. https://proceedings.mlr.press/v139/zbontar21a

[2]

Siddiqui, S., et al., “Blockwise Self-Supervised Learning at Scale” TMLR, 2024. https://openreview.net/forum?id=M2m618iIPk

__init__(input_dim=2048, hidden_dim=8192, output_dim=8192)

Initialize internal Module state, shared by both nn.Module and ScriptModule.