nidl.callbacks: Available callbacks

This modules details the public API you should use and implement for a nidl compatible callback, as well as the callbacks available in nidl.

Introduction

A callback is a lightning.pytorch.callbacks.Callback class that allows you to add arbitrary self-contained programs to your training. At specific points during the flow of execution (hooks), the callback interface allows you to design programs that encapsulate a full set of functionality. It de-couples functionality that does not need to be in the lightning module and can be shared across projects.

Lightning provides a large set of callbacks described here. We propose in nidl critical callbacks for model probing and metrics computation. It allows a better decoupling of the model training and evaluation logic from the actual implementation of these functionalities.

        classDiagram
  Callback <|-- BatchTypingCallback
  Callback <|-- MetricsCallback
  Callback <|-- ModelProbing
  Callback <|-- ModelProbingCV
    

Probing callbacks

Classes for all callbacks performing model’s probing.

ModelProbing(train_dataloader, ...[, ...])	Callback to probe the representation of an embedding estimator on a dataset.
ModelProbingCV(dataloader, probe[, scoring, ...])	Callback to probe the representation of an embedding estimator on a dataset using cross-validation.

Metrics callback

Classes for all callbacks performing metrics computation.

MetricsCallback(metrics[, needs, ...])

Callback to compute and log metrics during training, validation and test of a PL model.

Typing callback

Classes for all callbacks checking the batch format given by a dataloader against the expected type.

BatchTypingCallback(*[, only_first_batch])

Check the batch format based on LightningModule step signatures.