Deep learning for NeuroImaging in Python.

Spherical VAE architecture.

Use either RePa - Rectangular Patch convolution method or DiNe - Direct Neighbor convolution method.

Notes

Debuging messages can be displayed by changing the log level using setup_logging(level='debug').

References

Representation Learning of Resting State fMRI with Variational Autoencoder, NeuroImage 2021.

Examples

>>> import torch
>>> from surfify.utils import icosahedron
>>> from surfify.models import SphericalVAE
>>> verts, tris = icosahedron(order=6)
>>> x = torch.zeros((1, 2, len(verts)))
>>> model = SphericalVAE(
>>>     input_channels=2, input_order=6, latent_dim=64,
>>>     conv_flts=[32, 32, 64, 64], conv_mode="DiNe", dine_size=1,
>>>     fusion_level=2, standard_ico=False")
>>> print(model)
>>> out = model(x, x)
>>> print(out[0].shape, out[1].shape)

Init class.

Parameters:

input_channels : int, default 1

the number of input channels.

input_order : int, default 5

the input icosahedron order.

latent_dim : int, default 64

the size of the stochastic latent state of the SVAE.

conv_flts : list of int

the size of convolutional filters.

conv_mode : str, default ‘DiNe’

use either ‘RePa’ - Rectangular Patch convolution method or ‘DiNe’ - 1 ring Direct Neighbor convolution method.

dine_size : int, default 1

the size of the spherical convolution filter, ie. the number of neighbor rings to be considered.

repa_size : int, default 5

the size of the rectangular grid in the tangent space.

repa_zoom : int, default 5

control the rectangular grid spacing in the tangent space by applying a multiplicative factor of 1 / repa_zoom.

dynamic_repa_zoom : bool, default False

dynamically adapt the RePa zoom by applying a multiplicative factor of log(order + 1) + 1.

fusion_level : int, default 1

at which max pooling level left and right hemisphere data are concatenated.

standard_ico : bool, default False

optionaly use surfify tesselation.

cachedir : str, default None

set this folder to use smart caching speedup.

decode(z)[source]¶

The decoder.

Parameters:

z : Tensor (samples, <latent_dim>)

the stochastic latent state z.

Returns:

left_recon_x : Tensor (samples, <input_channels>, n_vertices)

reconstructed left cortical texture.

right_recon_x : Tensor (samples, <input_channels>, n_vertices)

reconstructed right cortical texture.

encode(left_x, right_x)[source]¶

The encoder.

Parameters:

left_x : Tensor (samples, <input_channels>, azimuth, elevation)

input left cortical texture.

right_x : Tensor (samples, <input_channels>, azimuth, elevation)

input right cortical texture.

Returns:

q(z | x) : Normal (batch_size, <latent_dim>)

a Normal distribution.

forward(left_x, right_x, sample=True)[source]¶

The forward method.

Parameters:

left_x : Tensor (samples, <input_channels>, azimuth, elevation)

input left cortical texture.

right_x : Tensor (samples, <input_channels>, azimuth, elevation)

input right cortical texture.

Returns:

left_recon_x : Tensor (samples, <input_channels>, azimuth, elevation)

reconstructed left cortical texture.

right_recon_x : Tensor (samples, <input_channels>, azimuth, elevation)

reconstructed right cortical texture.

reparameterize(q, sample=True)[source]¶

Implement the reparametrization trick.