Self-Supervised Learning with Barlow Twins

This tutorial will show you how to fit and evaluate a Barlow Twins model [1] on the OpenBHB dataset using NIDL.

We will follow these steps using the NIDL library:

1. Load the OpenBHB dataset.

2. Define the data augmentations for self-supervised training.

3. Define the BarlowTwins model.

4. Train the model.

5. Visualize the model’s embedding using MDS and evaluate its performance on age prediction using linear regression and KNN.

As for the neuroimaging data, we will investigate two input representations:

• Voxel-based morphometry (VBM) maps, which are preprocessed gray matter density maps.

• Surface-based morphometry (SBM) maps, which are cortical thickness, mean curvature, gray matter volume and surface area maps projected onto a standard surface template.

  Both representations are available in the OpenBHB dataset. To make the training faster and reduce the memory footprint, we will consider regions of interest (ROIs) instead of the whole brain. For VBM, we will use the mean gray matter density averaged within each ROI of the Neuromorphometrics atlas (284 regions). For SBM, we will use the cortical thickness, mean curvature, gray matter volume and surface area averaged within each ROI of the Desikan-Killiany atlas (68 regions).

  The Barlow Twins model will be trained individually on both representations and we will compare their performance on age prediction.

  [1]

  Zbontar, J., et al., “Barlow Twins: Self-Supervised Learning via Redundancy Reduction.” PMLR, 2021. hhttps://proceedings.mlr.press/v139/zbontar21a

Setup

This notebook requires some packages besides nidl. Let’s first start with importing our standard libraries below:

import matplotlib.pyplot as plt
import numpy as np
import torchvision.transforms as transforms
from sklearn.linear_model import LinearRegression
from sklearn.manifold import MDS
from sklearn.metrics import mean_absolute_error, r2_score
from sklearn.neighbors import KNeighborsRegressor
from torch.utils.data import DataLoader
from torchvision.ops import MLP

from nidl.datasets import OpenBHB
from nidl.estimators.ssl import BarlowTwins
from nidl.transforms.transforms import MultiViewsTransform

We define some global parameters that will be used throughout the notebook:

data_dir = "/tmp/openBHB"
batch_size = 128
num_workers = 10
latent_size = 32

OpenBHB datasets and data augmentations for Barlow Twins training

We will use the OpenBHB dataset for pre-training the models. We will focus on the VBM ROI representation and the SBM ROI representation for this tutorial. Since they are tabular data, we will use random masking and adding Gaussian noise as data augmentation in contrastive learning.

# Hyperparameters for data augmentations
mask_prob = 0.8
noise_std = 0.5
contrast_transforms = transforms.Compose(
    [
        lambda x: x.flatten(),
        lambda x: (np.random.rand(*x.shape) > mask_prob).astype(np.float32)
        * x,  # random masking
        lambda x: x
        + (
            (np.random.rand() > 0.5) * np.random.randn(*x.shape) * noise_std
        ).astype(np.float32),  # random Gaussian noise
    ]
)

We first create the SSL dataloaders with VBM modality and age as weak label. We use the previous contrastive transforms for data augmentation.

dataloader_ssl_vbm = DataLoader(
    OpenBHB(
        data_dir,
        modality="vbm_roi",
        target=None,
        transforms=MultiViewsTransform(contrast_transforms, n_views=2),
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=True,
)
dataloader_ssl_vbm_test = DataLoader(
    OpenBHB(
        data_dir,
        modality="vbm_roi",
        target=None,
        split="val",
        transforms=MultiViewsTransform(contrast_transforms, n_views=2),
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()

Then, we create the SSL dataloaders with SBM modality on the Desikan-Killiany atlas and age as weak label. We only extract some surface features and we use the same contrastive transforms as for VBM.

# Extract only surface area, GM volume, cortical thickness, mean curvature for
# SBM maps
sbm_channels = [0, 1, 2, 5]


def sbm_transform(x):
    return x[sbm_channels].flatten()


def vbm_transform(x):
    return x.flatten()


dataloader_ssl_sbm = DataLoader(
    OpenBHB(
        data_dir,
        modality="fs_desikan_roi",
        target=None,
        transforms=MultiViewsTransform(
            transforms.Compose([sbm_transform, contrast_transforms]), n_views=2
        ),
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=True,
)
dataloader_ssl_sbm_test = DataLoader(
    OpenBHB(
        data_dir,
        modality="fs_desikan_roi",
        target=None,
        split="val",
        transforms=MultiViewsTransform(
            transforms.Compose([sbm_transform, contrast_transforms]), n_views=2
        ),
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()

Finally, we create the dataloaders for evaluating the learned representations on age prediction. We don’t apply any data augmentation here.

dataloader_vbm_train = DataLoader(
    OpenBHB(
        data_dir,
        modality="vbm_roi",
        target="age",
        split="train",
        transforms=vbm_transform,
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)

dataloader_vbm_test = DataLoader(
    OpenBHB(
        data_dir,
        modality="vbm_roi",
        target="age",
        split="val",
        transforms=vbm_transform,
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)

dataloader_sbm_train = DataLoader(
    OpenBHB(
        data_dir,
        modality="fs_desikan_roi",
        target="age",
        split="train",
        transforms=sbm_transform,
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)
dataloader_sbm_test = DataLoader(
    OpenBHB(
        data_dir,
        modality="fs_desikan_roi",
        target="age",
        split="val",
        transforms=sbm_transform,
        streaming=False,
    ),
    batch_size=batch_size,
    num_workers=num_workers,
    shuffle=False,
)

# Small hack to avoid returning the target in the dataloaders since we aim
# at transforming these datasets without their targets.
dataloader_vbm_train.dataset.target = None
dataloader_vbm_test.dataset.target = None
dataloader_sbm_train.dataset.target = None
dataloader_sbm_test.dataset.target = None

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:431: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()

Training of BarlowTwins models

We can now instantiate and train two Barlow Twins models (one for VBM and another for SBM).

Since we work with tabular data, we can use a simple MLP as encoder. For VBM data, the input dimension is 284 and we compress the data to a 32-d vector. SBM data is flattened to a 272-d vector (68 regions * 4 features) and we also compress it to a 32-d vector.

vbm_encoder = MLP(in_channels=284, hidden_channels=[64, latent_size])
sbm_encoder = MLP(in_channels=272, hidden_channels=[64, latent_size])

We limit the training to 10 epochs for the sake of time.

sigma = 4
vbm_model = BarlowTwins(
    encoder=vbm_encoder,
    proj_input_dim=latent_size,
    proj_hidden_dim=2 * latent_size,
    proj_output_dim=latent_size,
    lambd=0.005,
    max_epochs=10,
    learning_rate=1e-5,
    enable_checkpointing=False,
)

sbm_model = BarlowTwins(
    encoder=sbm_encoder,
    proj_input_dim=latent_size,
    proj_hidden_dim=2 * latent_size,
    proj_output_dim=latent_size,
    lambd=0.005,
    max_epochs=10,
    learning_rate=1e-5,
    enable_checkpointing=False,
)

We train both models on their respective dataloaders.

vbm_model.fit(
    dataloader_ssl_vbm,
    dataloader_ssl_vbm_test,
)

sbm_model.fit(
    dataloader_ssl_sbm,
    dataloader_ssl_sbm_test,
)

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/torch/utils/data/dataloader.py:437: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  self.check_worker_number_rationality()
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/pytorch_lightning/loops/fit_loop.py:321: The number of training batches (26) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.

Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking DataLoader 0:   0%|          | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0:  50%|█████     | 1/2 [00:00<00:00,  2.70it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00,  5.05it/s]


Training: |          | 0/? [00:00<?, ?it/s]
Training: |          | 0/? [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/26 [00:00<?, ?it/s]
Epoch 0:   4%|▍         | 1/26 [00:01<00:30,  0.83it/s]
Epoch 0:   4%|▍         | 1/26 [00:01<00:30,  0.82it/s, v_num=8, loss/train=28.40]
Epoch 0:   8%|▊         | 2/26 [00:01<00:15,  1.52it/s, v_num=8, loss/train=28.40]
Epoch 0:   8%|▊         | 2/26 [00:01<00:15,  1.51it/s, v_num=8, loss/train=29.80]
Epoch 0:  12%|█▏        | 3/26 [00:01<00:10,  2.15it/s, v_num=8, loss/train=29.80]
Epoch 0:  12%|█▏        | 3/26 [00:01<00:10,  2.15it/s, v_num=8, loss/train=32.70]
Epoch 0:  15%|█▌        | 4/26 [00:03<00:18,  1.19it/s, v_num=8, loss/train=32.70]
Epoch 0:  15%|█▌        | 4/26 [00:03<00:18,  1.19it/s, v_num=8, loss/train=31.50]
Epoch 0:  19%|█▉        | 5/26 [00:03<00:14,  1.47it/s, v_num=8, loss/train=31.50]
Epoch 0:  19%|█▉        | 5/26 [00:03<00:14,  1.47it/s, v_num=8, loss/train=32.40]
Epoch 0:  23%|██▎       | 6/26 [00:03<00:11,  1.73it/s, v_num=8, loss/train=32.40]
Epoch 0:  23%|██▎       | 6/26 [00:03<00:11,  1.73it/s, v_num=8, loss/train=30.90]
Epoch 0:  27%|██▋       | 7/26 [00:03<00:09,  2.00it/s, v_num=8, loss/train=30.90]
Epoch 0:  27%|██▋       | 7/26 [00:03<00:09,  2.00it/s, v_num=8, loss/train=33.70]
Epoch 0:  31%|███       | 8/26 [00:03<00:07,  2.26it/s, v_num=8, loss/train=33.70]
Epoch 0:  31%|███       | 8/26 [00:03<00:07,  2.26it/s, v_num=8, loss/train=32.40]
Epoch 0:  35%|███▍      | 9/26 [00:03<00:06,  2.44it/s, v_num=8, loss/train=32.40]
Epoch 0:  35%|███▍      | 9/26 [00:03<00:06,  2.44it/s, v_num=8, loss/train=32.50]
Epoch 0:  38%|███▊      | 10/26 [00:03<00:05,  2.68it/s, v_num=8, loss/train=32.50]
Epoch 0:  38%|███▊      | 10/26 [00:03<00:05,  2.68it/s, v_num=8, loss/train=29.90]
Epoch 0:  42%|████▏     | 11/26 [00:04<00:06,  2.29it/s, v_num=8, loss/train=29.90]
Epoch 0:  42%|████▏     | 11/26 [00:04<00:06,  2.29it/s, v_num=8, loss/train=32.50]
Epoch 0:  46%|████▌     | 12/26 [00:04<00:05,  2.41it/s, v_num=8, loss/train=32.50]
Epoch 0:  46%|████▌     | 12/26 [00:04<00:05,  2.41it/s, v_num=8, loss/train=32.90]
Epoch 0:  50%|█████     | 13/26 [00:05<00:05,  2.58it/s, v_num=8, loss/train=32.90]
Epoch 0:  50%|█████     | 13/26 [00:05<00:05,  2.58it/s, v_num=8, loss/train=30.90]
Epoch 0:  54%|█████▍    | 14/26 [00:05<00:05,  2.39it/s, v_num=8, loss/train=30.90]
Epoch 0:  54%|█████▍    | 14/26 [00:05<00:05,  2.39it/s, v_num=8, loss/train=32.50]
Epoch 0:  58%|█████▊    | 15/26 [00:05<00:04,  2.55it/s, v_num=8, loss/train=32.50]
Epoch 0:  58%|█████▊    | 15/26 [00:05<00:04,  2.55it/s, v_num=8, loss/train=32.50]
Epoch 0:  62%|██████▏   | 16/26 [00:05<00:03,  2.71it/s, v_num=8, loss/train=32.50]
Epoch 0:  62%|██████▏   | 16/26 [00:05<00:03,  2.71it/s, v_num=8, loss/train=32.00]
Epoch 0:  65%|██████▌   | 17/26 [00:06<00:03,  2.80it/s, v_num=8, loss/train=32.00]
Epoch 0:  65%|██████▌   | 17/26 [00:06<00:03,  2.80it/s, v_num=8, loss/train=30.60]
Epoch 0:  69%|██████▉   | 18/26 [00:06<00:02,  2.95it/s, v_num=8, loss/train=30.60]
Epoch 0:  69%|██████▉   | 18/26 [00:06<00:02,  2.94it/s, v_num=8, loss/train=33.70]
Epoch 0:  73%|███████▎  | 19/26 [00:06<00:02,  3.10it/s, v_num=8, loss/train=33.70]
Epoch 0:  73%|███████▎  | 19/26 [00:06<00:02,  3.10it/s, v_num=8, loss/train=34.20]
Epoch 0:  77%|███████▋  | 20/26 [00:06<00:01,  3.25it/s, v_num=8, loss/train=34.20]
Epoch 0:  77%|███████▋  | 20/26 [00:06<00:01,  3.25it/s, v_num=8, loss/train=32.10]
Epoch 0:  81%|████████  | 21/26 [00:06<00:01,  3.19it/s, v_num=8, loss/train=32.10]
Epoch 0:  81%|████████  | 21/26 [00:06<00:01,  3.19it/s, v_num=8, loss/train=32.00]
Epoch 0:  85%|████████▍ | 22/26 [00:06<00:01,  3.34it/s, v_num=8, loss/train=32.00]
Epoch 0:  85%|████████▍ | 22/26 [00:06<00:01,  3.34it/s, v_num=8, loss/train=33.20]
Epoch 0:  88%|████████▊ | 23/26 [00:06<00:00,  3.49it/s, v_num=8, loss/train=33.20]
Epoch 0:  88%|████████▊ | 23/26 [00:06<00:00,  3.49it/s, v_num=8, loss/train=29.60]
Epoch 0:  92%|█████████▏| 24/26 [00:06<00:00,  3.56it/s, v_num=8, loss/train=29.60]
Epoch 0:  92%|█████████▏| 24/26 [00:06<00:00,  3.56it/s, v_num=8, loss/train=29.60]
Epoch 0:  96%|█████████▌| 25/26 [00:06<00:00,  3.70it/s, v_num=8, loss/train=29.60]
Epoch 0:  96%|█████████▌| 25/26 [00:06<00:00,  3.70it/s, v_num=8, loss/train=31.80]
Epoch 0: 100%|██████████| 26/26 [00:06<00:00,  3.85it/s, v_num=8, loss/train=31.80]
Epoch 0: 100%|██████████| 26/26 [00:06<00:00,  3.85it/s, v_num=8, loss/train=35.90]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 173.64it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 232.13it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 265.18it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 291.60it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 308.35it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 142.71it/s]


Epoch 0: 100%|██████████| 26/26 [00:09<00:00,  2.84it/s, v_num=8, loss/train=35.90, loss/val=32.30]
Epoch 0: 100%|██████████| 26/26 [00:09<00:00,  2.84it/s, v_num=8, loss/train=35.90, loss/val=32.30]
Epoch 0:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=35.90, loss/val=32.30]
Epoch 1:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=35.90, loss/val=32.30]
Epoch 1:   4%|▍         | 1/26 [00:05<02:08,  0.19it/s, v_num=8, loss/train=35.90, loss/val=32.30]
Epoch 1:   4%|▍         | 1/26 [00:05<02:08,  0.19it/s, v_num=8, loss/train=33.40, loss/val=32.30]
Epoch 1:   8%|▊         | 2/26 [00:05<01:02,  0.38it/s, v_num=8, loss/train=33.40, loss/val=32.30]
Epoch 1:   8%|▊         | 2/26 [00:05<01:02,  0.38it/s, v_num=8, loss/train=31.50, loss/val=32.30]
Epoch 1:  12%|█▏        | 3/26 [00:05<00:43,  0.53it/s, v_num=8, loss/train=31.50, loss/val=32.30]
Epoch 1:  12%|█▏        | 3/26 [00:05<00:43,  0.53it/s, v_num=8, loss/train=32.70, loss/val=32.30]
Epoch 1:  15%|█▌        | 4/26 [00:05<00:31,  0.70it/s, v_num=8, loss/train=32.70, loss/val=32.30]
Epoch 1:  15%|█▌        | 4/26 [00:05<00:31,  0.70it/s, v_num=8, loss/train=32.20, loss/val=32.30]
Epoch 1:  19%|█▉        | 5/26 [00:05<00:24,  0.86it/s, v_num=8, loss/train=32.20, loss/val=32.30]
Epoch 1:  19%|█▉        | 5/26 [00:05<00:24,  0.86it/s, v_num=8, loss/train=36.90, loss/val=32.30]
Epoch 1:  23%|██▎       | 6/26 [00:05<00:19,  1.02it/s, v_num=8, loss/train=36.90, loss/val=32.30]
Epoch 1:  23%|██▎       | 6/26 [00:05<00:19,  1.02it/s, v_num=8, loss/train=32.20, loss/val=32.30]
Epoch 1:  27%|██▋       | 7/26 [00:05<00:16,  1.17it/s, v_num=8, loss/train=32.20, loss/val=32.30]
Epoch 1:  27%|██▋       | 7/26 [00:05<00:16,  1.17it/s, v_num=8, loss/train=30.00, loss/val=32.30]
Epoch 1:  31%|███       | 8/26 [00:06<00:13,  1.32it/s, v_num=8, loss/train=30.00, loss/val=32.30]
Epoch 1:  31%|███       | 8/26 [00:06<00:13,  1.32it/s, v_num=8, loss/train=35.30, loss/val=32.30]
Epoch 1:  35%|███▍      | 9/26 [00:06<00:11,  1.47it/s, v_num=8, loss/train=35.30, loss/val=32.30]
Epoch 1:  35%|███▍      | 9/26 [00:06<00:11,  1.47it/s, v_num=8, loss/train=33.70, loss/val=32.30]
Epoch 1:  38%|███▊      | 10/26 [00:06<00:09,  1.60it/s, v_num=8, loss/train=33.70, loss/val=32.30]
Epoch 1:  38%|███▊      | 10/26 [00:06<00:10,  1.60it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  42%|████▏     | 11/26 [00:10<00:14,  1.05it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  42%|████▏     | 11/26 [00:10<00:14,  1.05it/s, v_num=8, loss/train=30.30, loss/val=32.30]
Epoch 1:  46%|████▌     | 12/26 [00:10<00:12,  1.14it/s, v_num=8, loss/train=30.30, loss/val=32.30]
Epoch 1:  46%|████▌     | 12/26 [00:10<00:12,  1.14it/s, v_num=8, loss/train=33.90, loss/val=32.30]
Epoch 1:  50%|█████     | 13/26 [00:11<00:11,  1.14it/s, v_num=8, loss/train=33.90, loss/val=32.30]
Epoch 1:  50%|█████     | 13/26 [00:11<00:11,  1.14it/s, v_num=8, loss/train=31.30, loss/val=32.30]
Epoch 1:  54%|█████▍    | 14/26 [00:11<00:09,  1.23it/s, v_num=8, loss/train=31.30, loss/val=32.30]
Epoch 1:  54%|█████▍    | 14/26 [00:11<00:09,  1.23it/s, v_num=8, loss/train=34.10, loss/val=32.30]
Epoch 1:  58%|█████▊    | 15/26 [00:11<00:08,  1.31it/s, v_num=8, loss/train=34.10, loss/val=32.30]
Epoch 1:  58%|█████▊    | 15/26 [00:11<00:08,  1.31it/s, v_num=8, loss/train=30.20, loss/val=32.30]
Epoch 1:  62%|██████▏   | 16/26 [00:11<00:07,  1.39it/s, v_num=8, loss/train=30.20, loss/val=32.30]
Epoch 1:  62%|██████▏   | 16/26 [00:11<00:07,  1.39it/s, v_num=8, loss/train=34.00, loss/val=32.30]
Epoch 1:  65%|██████▌   | 17/26 [00:11<00:06,  1.48it/s, v_num=8, loss/train=34.00, loss/val=32.30]
Epoch 1:  65%|██████▌   | 17/26 [00:11<00:06,  1.48it/s, v_num=8, loss/train=31.90, loss/val=32.30]
Epoch 1:  69%|██████▉   | 18/26 [00:11<00:05,  1.56it/s, v_num=8, loss/train=31.90, loss/val=32.30]
Epoch 1:  69%|██████▉   | 18/26 [00:11<00:05,  1.56it/s, v_num=8, loss/train=33.40, loss/val=32.30]
Epoch 1:  73%|███████▎  | 19/26 [00:11<00:04,  1.64it/s, v_num=8, loss/train=33.40, loss/val=32.30]
Epoch 1:  73%|███████▎  | 19/26 [00:11<00:04,  1.64it/s, v_num=8, loss/train=28.70, loss/val=32.30]
Epoch 1:  77%|███████▋  | 20/26 [00:11<00:03,  1.73it/s, v_num=8, loss/train=28.70, loss/val=32.30]
Epoch 1:  77%|███████▋  | 20/26 [00:11<00:03,  1.73it/s, v_num=8, loss/train=28.30, loss/val=32.30]
Epoch 1:  81%|████████  | 21/26 [00:12<00:02,  1.72it/s, v_num=8, loss/train=28.30, loss/val=32.30]
Epoch 1:  81%|████████  | 21/26 [00:12<00:02,  1.72it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  85%|████████▍ | 22/26 [00:12<00:02,  1.80it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  85%|████████▍ | 22/26 [00:12<00:02,  1.80it/s, v_num=8, loss/train=32.10, loss/val=32.30]
Epoch 1:  88%|████████▊ | 23/26 [00:12<00:01,  1.88it/s, v_num=8, loss/train=32.10, loss/val=32.30]
Epoch 1:  88%|████████▊ | 23/26 [00:12<00:01,  1.88it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  92%|█████████▏| 24/26 [00:12<00:01,  1.96it/s, v_num=8, loss/train=32.00, loss/val=32.30]
Epoch 1:  92%|█████████▏| 24/26 [00:12<00:01,  1.96it/s, v_num=8, loss/train=32.90, loss/val=32.30]
Epoch 1:  96%|█████████▌| 25/26 [00:12<00:00,  2.04it/s, v_num=8, loss/train=32.90, loss/val=32.30]
Epoch 1:  96%|█████████▌| 25/26 [00:12<00:00,  2.04it/s, v_num=8, loss/train=31.90, loss/val=32.30]
Epoch 1: 100%|██████████| 26/26 [00:12<00:00,  2.12it/s, v_num=8, loss/train=31.90, loss/val=32.30]
Epoch 1: 100%|██████████| 26/26 [00:12<00:00,  2.12it/s, v_num=8, loss/train=37.00, loss/val=32.30]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 117.78it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 159.09it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 76.05it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 95.42it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 112.53it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 127.59it/s]


Epoch 1: 100%|██████████| 26/26 [00:14<00:00,  1.76it/s, v_num=8, loss/train=37.00, loss/val=31.30]
Epoch 1: 100%|██████████| 26/26 [00:14<00:00,  1.76it/s, v_num=8, loss/train=37.00, loss/val=31.30]
Epoch 1:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=37.00, loss/val=31.30]
Epoch 2:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=37.00, loss/val=31.30]
Epoch 2:   4%|▍         | 1/26 [00:03<01:33,  0.27it/s, v_num=8, loss/train=37.00, loss/val=31.30]
Epoch 2:   4%|▍         | 1/26 [00:03<01:33,  0.27it/s, v_num=8, loss/train=32.60, loss/val=31.30]
Epoch 2:   8%|▊         | 2/26 [00:05<01:00,  0.40it/s, v_num=8, loss/train=32.60, loss/val=31.30]
Epoch 2:   8%|▊         | 2/26 [00:05<01:00,  0.40it/s, v_num=8, loss/train=29.00, loss/val=31.30]
Epoch 2:  12%|█▏        | 3/26 [00:05<00:39,  0.59it/s, v_num=8, loss/train=29.00, loss/val=31.30]
Epoch 2:  12%|█▏        | 3/26 [00:05<00:39,  0.59it/s, v_num=8, loss/train=32.50, loss/val=31.30]
Epoch 2:  15%|█▌        | 4/26 [00:05<00:31,  0.69it/s, v_num=8, loss/train=32.50, loss/val=31.30]
Epoch 2:  15%|█▌        | 4/26 [00:05<00:31,  0.69it/s, v_num=8, loss/train=32.30, loss/val=31.30]
Epoch 2:  19%|█▉        | 5/26 [00:06<00:25,  0.81it/s, v_num=8, loss/train=32.30, loss/val=31.30]
Epoch 2:  19%|█▉        | 5/26 [00:06<00:25,  0.81it/s, v_num=8, loss/train=34.60, loss/val=31.30]
Epoch 2:  23%|██▎       | 6/26 [00:06<00:20,  0.96it/s, v_num=8, loss/train=34.60, loss/val=31.30]
Epoch 2:  23%|██▎       | 6/26 [00:06<00:20,  0.96it/s, v_num=8, loss/train=29.30, loss/val=31.30]
Epoch 2:  27%|██▋       | 7/26 [00:06<00:17,  1.11it/s, v_num=8, loss/train=29.30, loss/val=31.30]
Epoch 2:  27%|██▋       | 7/26 [00:06<00:17,  1.11it/s, v_num=8, loss/train=32.10, loss/val=31.30]
Epoch 2:  31%|███       | 8/26 [00:06<00:14,  1.25it/s, v_num=8, loss/train=32.10, loss/val=31.30]
Epoch 2:  31%|███       | 8/26 [00:06<00:14,  1.25it/s, v_num=8, loss/train=30.60, loss/val=31.30]
Epoch 2:  35%|███▍      | 9/26 [00:06<00:12,  1.37it/s, v_num=8, loss/train=30.60, loss/val=31.30]
Epoch 2:  35%|███▍      | 9/26 [00:06<00:12,  1.37it/s, v_num=8, loss/train=32.90, loss/val=31.30]
Epoch 2:  38%|███▊      | 10/26 [00:06<00:10,  1.51it/s, v_num=8, loss/train=32.90, loss/val=31.30]
Epoch 2:  38%|███▊      | 10/26 [00:06<00:10,  1.51it/s, v_num=8, loss/train=32.80, loss/val=31.30]
Epoch 2:  42%|████▏     | 11/26 [00:07<00:10,  1.48it/s, v_num=8, loss/train=32.80, loss/val=31.30]
Epoch 2:  42%|████▏     | 11/26 [00:07<00:10,  1.48it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  46%|████▌     | 12/26 [00:09<00:10,  1.30it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  46%|████▌     | 12/26 [00:09<00:10,  1.30it/s, v_num=8, loss/train=30.90, loss/val=31.30]
Epoch 2:  50%|█████     | 13/26 [00:09<00:09,  1.40it/s, v_num=8, loss/train=30.90, loss/val=31.30]
Epoch 2:  50%|█████     | 13/26 [00:09<00:09,  1.40it/s, v_num=8, loss/train=32.30, loss/val=31.30]
Epoch 2:  54%|█████▍    | 14/26 [00:11<00:09,  1.27it/s, v_num=8, loss/train=32.30, loss/val=31.30]
Epoch 2:  54%|█████▍    | 14/26 [00:11<00:09,  1.27it/s, v_num=8, loss/train=33.50, loss/val=31.30]
Epoch 2:  58%|█████▊    | 15/26 [00:11<00:08,  1.34it/s, v_num=8, loss/train=33.50, loss/val=31.30]
Epoch 2:  58%|█████▊    | 15/26 [00:11<00:08,  1.34it/s, v_num=8, loss/train=30.00, loss/val=31.30]
Epoch 2:  62%|██████▏   | 16/26 [00:11<00:07,  1.43it/s, v_num=8, loss/train=30.00, loss/val=31.30]
Epoch 2:  62%|██████▏   | 16/26 [00:11<00:07,  1.43it/s, v_num=8, loss/train=30.40, loss/val=31.30]
Epoch 2:  65%|██████▌   | 17/26 [00:11<00:05,  1.51it/s, v_num=8, loss/train=30.40, loss/val=31.30]
Epoch 2:  65%|██████▌   | 17/26 [00:11<00:05,  1.51it/s, v_num=8, loss/train=28.20, loss/val=31.30]
Epoch 2:  69%|██████▉   | 18/26 [00:11<00:04,  1.60it/s, v_num=8, loss/train=28.20, loss/val=31.30]
Epoch 2:  69%|██████▉   | 18/26 [00:11<00:04,  1.60it/s, v_num=8, loss/train=31.40, loss/val=31.30]
Epoch 2:  73%|███████▎  | 19/26 [00:11<00:04,  1.69it/s, v_num=8, loss/train=31.40, loss/val=31.30]
Epoch 2:  73%|███████▎  | 19/26 [00:11<00:04,  1.69it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=29.80, loss/val=31.30]
Epoch 2:  81%|████████  | 21/26 [00:11<00:02,  1.85it/s, v_num=8, loss/train=29.80, loss/val=31.30]
Epoch 2:  81%|████████  | 21/26 [00:11<00:02,  1.85it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  85%|████████▍ | 22/26 [00:11<00:02,  1.90it/s, v_num=8, loss/train=31.60, loss/val=31.30]
Epoch 2:  85%|████████▍ | 22/26 [00:11<00:02,  1.90it/s, v_num=8, loss/train=29.60, loss/val=31.30]
Epoch 2:  88%|████████▊ | 23/26 [00:11<00:01,  1.99it/s, v_num=8, loss/train=29.60, loss/val=31.30]
Epoch 2:  88%|████████▊ | 23/26 [00:11<00:01,  1.99it/s, v_num=8, loss/train=34.90, loss/val=31.30]
Epoch 2:  92%|█████████▏| 24/26 [00:11<00:00,  2.02it/s, v_num=8, loss/train=34.90, loss/val=31.30]
Epoch 2:  92%|█████████▏| 24/26 [00:11<00:00,  2.02it/s, v_num=8, loss/train=30.70, loss/val=31.30]
Epoch 2:  96%|█████████▌| 25/26 [00:12<00:00,  2.08it/s, v_num=8, loss/train=30.70, loss/val=31.30]
Epoch 2:  96%|█████████▌| 25/26 [00:12<00:00,  2.08it/s, v_num=8, loss/train=30.70, loss/val=31.30]
Epoch 2: 100%|██████████| 26/26 [00:12<00:00,  2.16it/s, v_num=8, loss/train=30.70, loss/val=31.30]
Epoch 2: 100%|██████████| 26/26 [00:12<00:00,  2.16it/s, v_num=8, loss/train=30.70, loss/val=31.30]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 73.41it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 107.98it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  4.53it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  6.02it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  7.50it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00,  8.97it/s]


Epoch 2: 100%|██████████| 26/26 [00:14<00:00,  1.80it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 2: 100%|██████████| 26/26 [00:14<00:00,  1.80it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 2:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 3:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 3:   4%|▍         | 1/26 [00:05<02:09,  0.19it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 3:   4%|▍         | 1/26 [00:05<02:09,  0.19it/s, v_num=8, loss/train=30.00, loss/val=33.40]
Epoch 3:   8%|▊         | 2/26 [00:05<01:09,  0.34it/s, v_num=8, loss/train=30.00, loss/val=33.40]
Epoch 3:   8%|▊         | 2/26 [00:05<01:09,  0.34it/s, v_num=8, loss/train=31.70, loss/val=33.40]
Epoch 3:  12%|█▏        | 3/26 [00:05<00:45,  0.50it/s, v_num=8, loss/train=31.70, loss/val=33.40]
Epoch 3:  12%|█▏        | 3/26 [00:05<00:45,  0.50it/s, v_num=8, loss/train=32.10, loss/val=33.40]
Epoch 3:  15%|█▌        | 4/26 [00:06<00:33,  0.66it/s, v_num=8, loss/train=32.10, loss/val=33.40]
Epoch 3:  15%|█▌        | 4/26 [00:06<00:33,  0.66it/s, v_num=8, loss/train=28.90, loss/val=33.40]
Epoch 3:  19%|█▉        | 5/26 [00:06<00:25,  0.82it/s, v_num=8, loss/train=28.90, loss/val=33.40]
Epoch 3:  19%|█▉        | 5/26 [00:06<00:25,  0.82it/s, v_num=8, loss/train=32.70, loss/val=33.40]
Epoch 3:  23%|██▎       | 6/26 [00:06<00:20,  0.97it/s, v_num=8, loss/train=32.70, loss/val=33.40]
Epoch 3:  23%|██▎       | 6/26 [00:06<00:20,  0.97it/s, v_num=8, loss/train=29.90, loss/val=33.40]
Epoch 3:  27%|██▋       | 7/26 [00:06<00:16,  1.13it/s, v_num=8, loss/train=29.90, loss/val=33.40]
Epoch 3:  27%|██▋       | 7/26 [00:06<00:16,  1.13it/s, v_num=8, loss/train=31.40, loss/val=33.40]
Epoch 3:  31%|███       | 8/26 [00:06<00:13,  1.29it/s, v_num=8, loss/train=31.40, loss/val=33.40]
Epoch 3:  31%|███       | 8/26 [00:06<00:14,  1.28it/s, v_num=8, loss/train=30.30, loss/val=33.40]
Epoch 3:  35%|███▍      | 9/26 [00:06<00:12,  1.40it/s, v_num=8, loss/train=30.30, loss/val=33.40]
Epoch 3:  35%|███▍      | 9/26 [00:06<00:12,  1.40it/s, v_num=8, loss/train=31.20, loss/val=33.40]
Epoch 3:  38%|███▊      | 10/26 [00:06<00:10,  1.53it/s, v_num=8, loss/train=31.20, loss/val=33.40]
Epoch 3:  38%|███▊      | 10/26 [00:06<00:10,  1.53it/s, v_num=8, loss/train=32.40, loss/val=33.40]
Epoch 3:  42%|████▏     | 11/26 [00:10<00:14,  1.03it/s, v_num=8, loss/train=32.40, loss/val=33.40]
Epoch 3:  42%|████▏     | 11/26 [00:10<00:14,  1.03it/s, v_num=8, loss/train=32.30, loss/val=33.40]
Epoch 3:  46%|████▌     | 12/26 [00:11<00:12,  1.09it/s, v_num=8, loss/train=32.30, loss/val=33.40]
Epoch 3:  46%|████▌     | 12/26 [00:11<00:12,  1.09it/s, v_num=8, loss/train=32.60, loss/val=33.40]
Epoch 3:  50%|█████     | 13/26 [00:11<00:11,  1.18it/s, v_num=8, loss/train=32.60, loss/val=33.40]
Epoch 3:  50%|█████     | 13/26 [00:11<00:11,  1.18it/s, v_num=8, loss/train=26.20, loss/val=33.40]
Epoch 3:  54%|█████▍    | 14/26 [00:11<00:09,  1.27it/s, v_num=8, loss/train=26.20, loss/val=33.40]
Epoch 3:  54%|█████▍    | 14/26 [00:11<00:09,  1.27it/s, v_num=8, loss/train=30.50, loss/val=33.40]
Epoch 3:  58%|█████▊    | 15/26 [00:11<00:08,  1.35it/s, v_num=8, loss/train=30.50, loss/val=33.40]
Epoch 3:  58%|█████▊    | 15/26 [00:11<00:08,  1.35it/s, v_num=8, loss/train=34.30, loss/val=33.40]
Epoch 3:  62%|██████▏   | 16/26 [00:11<00:06,  1.44it/s, v_num=8, loss/train=34.30, loss/val=33.40]
Epoch 3:  62%|██████▏   | 16/26 [00:11<00:06,  1.44it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 3:  65%|██████▌   | 17/26 [00:11<00:05,  1.53it/s, v_num=8, loss/train=30.70, loss/val=33.40]
Epoch 3:  65%|██████▌   | 17/26 [00:11<00:05,  1.53it/s, v_num=8, loss/train=32.50, loss/val=33.40]
Epoch 3:  69%|██████▉   | 18/26 [00:11<00:05,  1.59it/s, v_num=8, loss/train=32.50, loss/val=33.40]
Epoch 3:  69%|██████▉   | 18/26 [00:11<00:05,  1.59it/s, v_num=8, loss/train=33.30, loss/val=33.40]
Epoch 3:  73%|███████▎  | 19/26 [00:11<00:04,  1.68it/s, v_num=8, loss/train=33.30, loss/val=33.40]
Epoch 3:  73%|███████▎  | 19/26 [00:11<00:04,  1.68it/s, v_num=8, loss/train=30.90, loss/val=33.40]
Epoch 3:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=30.90, loss/val=33.40]
Epoch 3:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=30.20, loss/val=33.40]
Epoch 3:  81%|████████  | 21/26 [00:11<00:02,  1.76it/s, v_num=8, loss/train=30.20, loss/val=33.40]
Epoch 3:  81%|████████  | 21/26 [00:11<00:02,  1.76it/s, v_num=8, loss/train=33.90, loss/val=33.40]
Epoch 3:  85%|████████▍ | 22/26 [00:12<00:02,  1.82it/s, v_num=8, loss/train=33.90, loss/val=33.40]
Epoch 3:  85%|████████▍ | 22/26 [00:12<00:02,  1.82it/s, v_num=8, loss/train=34.40, loss/val=33.40]
Epoch 3:  88%|████████▊ | 23/26 [00:12<00:01,  1.90it/s, v_num=8, loss/train=34.40, loss/val=33.40]
Epoch 3:  88%|████████▊ | 23/26 [00:12<00:01,  1.90it/s, v_num=8, loss/train=31.50, loss/val=33.40]
Epoch 3:  92%|█████████▏| 24/26 [00:12<00:01,  1.98it/s, v_num=8, loss/train=31.50, loss/val=33.40]
Epoch 3:  92%|█████████▏| 24/26 [00:12<00:01,  1.98it/s, v_num=8, loss/train=33.60, loss/val=33.40]
Epoch 3:  96%|█████████▌| 25/26 [00:12<00:00,  2.06it/s, v_num=8, loss/train=33.60, loss/val=33.40]
Epoch 3:  96%|█████████▌| 25/26 [00:12<00:00,  2.06it/s, v_num=8, loss/train=31.80, loss/val=33.40]
Epoch 3: 100%|██████████| 26/26 [00:12<00:00,  2.14it/s, v_num=8, loss/train=31.80, loss/val=33.40]
Epoch 3: 100%|██████████| 26/26 [00:12<00:00,  2.14it/s, v_num=8, loss/train=28.20, loss/val=33.40]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 187.78it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 249.99it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 288.04it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 315.37it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 332.35it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 343.76it/s]


Epoch 3: 100%|██████████| 26/26 [00:14<00:00,  1.79it/s, v_num=8, loss/train=28.20, loss/val=30.20]
Epoch 3: 100%|██████████| 26/26 [00:14<00:00,  1.79it/s, v_num=8, loss/train=28.20, loss/val=30.20]
Epoch 3:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=28.20, loss/val=30.20]
Epoch 4:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=28.20, loss/val=30.20]
Epoch 4:   4%|▍         | 1/26 [00:04<01:58,  0.21it/s, v_num=8, loss/train=28.20, loss/val=30.20]
Epoch 4:   4%|▍         | 1/26 [00:04<01:59,  0.21it/s, v_num=8, loss/train=33.70, loss/val=30.20]
Epoch 4:   8%|▊         | 2/26 [00:05<01:00,  0.40it/s, v_num=8, loss/train=33.70, loss/val=30.20]
Epoch 4:   8%|▊         | 2/26 [00:05<01:00,  0.39it/s, v_num=8, loss/train=31.50, loss/val=30.20]
Epoch 4:  12%|█▏        | 3/26 [00:06<00:46,  0.50it/s, v_num=8, loss/train=31.50, loss/val=30.20]
Epoch 4:  12%|█▏        | 3/26 [00:06<00:46,  0.50it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  15%|█▌        | 4/26 [00:06<00:34,  0.64it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  15%|█▌        | 4/26 [00:06<00:34,  0.64it/s, v_num=8, loss/train=30.80, loss/val=30.20]
Epoch 4:  19%|█▉        | 5/26 [00:06<00:26,  0.79it/s, v_num=8, loss/train=30.80, loss/val=30.20]
Epoch 4:  19%|█▉        | 5/26 [00:06<00:26,  0.79it/s, v_num=8, loss/train=33.30, loss/val=30.20]
Epoch 4:  23%|██▎       | 6/26 [00:08<00:27,  0.73it/s, v_num=8, loss/train=33.30, loss/val=30.20]
Epoch 4:  23%|██▎       | 6/26 [00:08<00:27,  0.73it/s, v_num=8, loss/train=32.70, loss/val=30.20]
Epoch 4:  27%|██▋       | 7/26 [00:08<00:22,  0.84it/s, v_num=8, loss/train=32.70, loss/val=30.20]
Epoch 4:  27%|██▋       | 7/26 [00:08<00:22,  0.84it/s, v_num=8, loss/train=32.30, loss/val=30.20]
Epoch 4:  31%|███       | 8/26 [00:08<00:18,  0.96it/s, v_num=8, loss/train=32.30, loss/val=30.20]
Epoch 4:  31%|███       | 8/26 [00:08<00:18,  0.96it/s, v_num=8, loss/train=31.70, loss/val=30.20]
Epoch 4:  35%|███▍      | 9/26 [00:08<00:15,  1.07it/s, v_num=8, loss/train=31.70, loss/val=30.20]
Epoch 4:  35%|███▍      | 9/26 [00:08<00:15,  1.07it/s, v_num=8, loss/train=33.90, loss/val=30.20]
Epoch 4:  38%|███▊      | 10/26 [00:08<00:13,  1.18it/s, v_num=8, loss/train=33.90, loss/val=30.20]
Epoch 4:  38%|███▊      | 10/26 [00:08<00:13,  1.18it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  42%|████▏     | 11/26 [00:09<00:13,  1.11it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  42%|████▏     | 11/26 [00:09<00:13,  1.11it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  46%|████▌     | 12/26 [00:09<00:11,  1.20it/s, v_num=8, loss/train=31.30, loss/val=30.20]
Epoch 4:  46%|████▌     | 12/26 [00:09<00:11,  1.20it/s, v_num=8, loss/train=31.70, loss/val=30.20]
Epoch 4:  50%|█████     | 13/26 [00:10<00:10,  1.22it/s, v_num=8, loss/train=31.70, loss/val=30.20]
Epoch 4:  50%|█████     | 13/26 [00:10<00:10,  1.22it/s, v_num=8, loss/train=32.40, loss/val=30.20]
Epoch 4:  54%|█████▍    | 14/26 [00:10<00:09,  1.31it/s, v_num=8, loss/train=32.40, loss/val=30.20]
Epoch 4:  54%|█████▍    | 14/26 [00:10<00:09,  1.31it/s, v_num=8, loss/train=31.80, loss/val=30.20]
Epoch 4:  58%|█████▊    | 15/26 [00:10<00:07,  1.39it/s, v_num=8, loss/train=31.80, loss/val=30.20]
Epoch 4:  58%|█████▊    | 15/26 [00:10<00:07,  1.39it/s, v_num=8, loss/train=32.00, loss/val=30.20]
Epoch 4:  62%|██████▏   | 16/26 [00:11<00:07,  1.42it/s, v_num=8, loss/train=32.00, loss/val=30.20]
Epoch 4:  62%|██████▏   | 16/26 [00:11<00:07,  1.42it/s, v_num=8, loss/train=31.20, loss/val=30.20]
Epoch 4:  65%|██████▌   | 17/26 [00:11<00:05,  1.51it/s, v_num=8, loss/train=31.20, loss/val=30.20]
Epoch 4:  65%|██████▌   | 17/26 [00:11<00:05,  1.51it/s, v_num=8, loss/train=31.20, loss/val=30.20]
Epoch 4:  69%|██████▉   | 18/26 [00:11<00:05,  1.60it/s, v_num=8, loss/train=31.20, loss/val=30.20]
Epoch 4:  69%|██████▉   | 18/26 [00:11<00:05,  1.60it/s, v_num=8, loss/train=32.40, loss/val=30.20]
Epoch 4:  73%|███████▎  | 19/26 [00:11<00:04,  1.68it/s, v_num=8, loss/train=32.40, loss/val=30.20]
Epoch 4:  73%|███████▎  | 19/26 [00:11<00:04,  1.68it/s, v_num=8, loss/train=32.50, loss/val=30.20]
Epoch 4:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=32.50, loss/val=30.20]
Epoch 4:  77%|███████▋  | 20/26 [00:11<00:03,  1.77it/s, v_num=8, loss/train=33.60, loss/val=30.20]
Epoch 4:  81%|████████  | 21/26 [00:11<00:02,  1.78it/s, v_num=8, loss/train=33.60, loss/val=30.20]
Epoch 4:  81%|████████  | 21/26 [00:11<00:02,  1.78it/s, v_num=8, loss/train=34.80, loss/val=30.20]
Epoch 4:  85%|████████▍ | 22/26 [00:11<00:02,  1.86it/s, v_num=8, loss/train=34.80, loss/val=30.20]
Epoch 4:  85%|████████▍ | 22/26 [00:11<00:02,  1.86it/s, v_num=8, loss/train=33.40, loss/val=30.20]
Epoch 4:  88%|████████▊ | 23/26 [00:11<00:01,  1.92it/s, v_num=8, loss/train=33.40, loss/val=30.20]
Epoch 4:  88%|████████▊ | 23/26 [00:11<00:01,  1.92it/s, v_num=8, loss/train=32.00, loss/val=30.20]
Epoch 4:  92%|█████████▏| 24/26 [00:11<00:00,  2.00it/s, v_num=8, loss/train=32.00, loss/val=30.20]
Epoch 4:  92%|█████████▏| 24/26 [00:11<00:00,  2.00it/s, v_num=8, loss/train=30.90, loss/val=30.20]
Epoch 4:  96%|█████████▌| 25/26 [00:11<00:00,  2.09it/s, v_num=8, loss/train=30.90, loss/val=30.20]
Epoch 4:  96%|█████████▌| 25/26 [00:11<00:00,  2.09it/s, v_num=8, loss/train=33.60, loss/val=30.20]
Epoch 4: 100%|██████████| 26/26 [00:11<00:00,  2.17it/s, v_num=8, loss/train=33.60, loss/val=30.20]
Epoch 4: 100%|██████████| 26/26 [00:11<00:00,  2.17it/s, v_num=8, loss/train=32.30, loss/val=30.20]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 36.87it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00,  9.79it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  4.84it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  6.03it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  7.51it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00,  8.98it/s]


Epoch 4: 100%|██████████| 26/26 [00:14<00:00,  1.80it/s, v_num=8, loss/train=32.30, loss/val=31.60]
Epoch 4: 100%|██████████| 26/26 [00:14<00:00,  1.80it/s, v_num=8, loss/train=32.30, loss/val=31.60]
Epoch 4:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=32.30, loss/val=31.60]
Epoch 5:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=32.30, loss/val=31.60]
Epoch 5:   4%|▍         | 1/26 [00:04<01:59,  0.21it/s, v_num=8, loss/train=32.30, loss/val=31.60]
Epoch 5:   4%|▍         | 1/26 [00:04<01:59,  0.21it/s, v_num=8, loss/train=29.70, loss/val=31.60]
Epoch 5:   8%|▊         | 2/26 [00:06<01:13,  0.33it/s, v_num=8, loss/train=29.70, loss/val=31.60]
Epoch 5:   8%|▊         | 2/26 [00:06<01:13,  0.33it/s, v_num=8, loss/train=31.70, loss/val=31.60]
Epoch 5:  12%|█▏        | 3/26 [00:06<00:47,  0.48it/s, v_num=8, loss/train=31.70, loss/val=31.60]
Epoch 5:  12%|█▏        | 3/26 [00:06<00:47,  0.48it/s, v_num=8, loss/train=32.60, loss/val=31.60]
Epoch 5:  15%|█▌        | 4/26 [00:06<00:34,  0.63it/s, v_num=8, loss/train=32.60, loss/val=31.60]
Epoch 5:  15%|█▌        | 4/26 [00:06<00:34,  0.63it/s, v_num=8, loss/train=29.10, loss/val=31.60]
Epoch 5:  19%|█▉        | 5/26 [00:06<00:27,  0.76it/s, v_num=8, loss/train=29.10, loss/val=31.60]
Epoch 5:  19%|█▉        | 5/26 [00:06<00:27,  0.76it/s, v_num=8, loss/train=32.90, loss/val=31.60]
Epoch 5:  23%|██▎       | 6/26 [00:06<00:22,  0.89it/s, v_num=8, loss/train=32.90, loss/val=31.60]
Epoch 5:  23%|██▎       | 6/26 [00:06<00:22,  0.89it/s, v_num=8, loss/train=31.50, loss/val=31.60]
Epoch 5:  27%|██▋       | 7/26 [00:06<00:18,  1.03it/s, v_num=8, loss/train=31.50, loss/val=31.60]
Epoch 5:  27%|██▋       | 7/26 [00:06<00:18,  1.03it/s, v_num=8, loss/train=34.10, loss/val=31.60]
Epoch 5:  31%|███       | 8/26 [00:06<00:15,  1.17it/s, v_num=8, loss/train=34.10, loss/val=31.60]
Epoch 5:  31%|███       | 8/26 [00:06<00:15,  1.17it/s, v_num=8, loss/train=31.10, loss/val=31.60]
Epoch 5:  35%|███▍      | 9/26 [00:06<00:13,  1.30it/s, v_num=8, loss/train=31.10, loss/val=31.60]
Epoch 5:  35%|███▍      | 9/26 [00:06<00:13,  1.30it/s, v_num=8, loss/train=33.50, loss/val=31.60]
Epoch 5:  38%|███▊      | 10/26 [00:07<00:11,  1.42it/s, v_num=8, loss/train=33.50, loss/val=31.60]
Epoch 5:  38%|███▊      | 10/26 [00:07<00:11,  1.42it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 5:  42%|████▏     | 11/26 [00:08<00:11,  1.26it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 5:  42%|████▏     | 11/26 [00:08<00:11,  1.26it/s, v_num=8, loss/train=32.50, loss/val=31.60]
Epoch 5:  46%|████▌     | 12/26 [00:11<00:13,  1.07it/s, v_num=8, loss/train=32.50, loss/val=31.60]
Epoch 5:  46%|████▌     | 12/26 [00:11<00:13,  1.07it/s, v_num=8, loss/train=33.00, loss/val=31.60]
Epoch 5:  50%|█████     | 13/26 [00:11<00:11,  1.16it/s, v_num=8, loss/train=33.00, loss/val=31.60]
Epoch 5:  50%|█████     | 13/26 [00:11<00:11,  1.16it/s, v_num=8, loss/train=32.20, loss/val=31.60]
Epoch 5:  54%|█████▍    | 14/26 [00:11<00:09,  1.24it/s, v_num=8, loss/train=32.20, loss/val=31.60]
Epoch 5:  54%|█████▍    | 14/26 [00:11<00:09,  1.24it/s, v_num=8, loss/train=30.50, loss/val=31.60]
Epoch 5:  58%|█████▊    | 15/26 [00:11<00:08,  1.33it/s, v_num=8, loss/train=30.50, loss/val=31.60]
Epoch 5:  58%|█████▊    | 15/26 [00:11<00:08,  1.33it/s, v_num=8, loss/train=30.00, loss/val=31.60]
Epoch 5:  62%|██████▏   | 16/26 [00:11<00:07,  1.41it/s, v_num=8, loss/train=30.00, loss/val=31.60]
Epoch 5:  62%|██████▏   | 16/26 [00:11<00:07,  1.41it/s, v_num=8, loss/train=30.10, loss/val=31.60]
Epoch 5:  65%|██████▌   | 17/26 [00:11<00:06,  1.50it/s, v_num=8, loss/train=30.10, loss/val=31.60]
Epoch 5:  65%|██████▌   | 17/26 [00:11<00:06,  1.50it/s, v_num=8, loss/train=30.10, loss/val=31.60]
Epoch 5:  69%|██████▉   | 18/26 [00:11<00:05,  1.58it/s, v_num=8, loss/train=30.10, loss/val=31.60]
Epoch 5:  69%|██████▉   | 18/26 [00:11<00:05,  1.58it/s, v_num=8, loss/train=29.60, loss/val=31.60]
Epoch 5:  73%|███████▎  | 19/26 [00:11<00:04,  1.67it/s, v_num=8, loss/train=29.60, loss/val=31.60]
Epoch 5:  73%|███████▎  | 19/26 [00:11<00:04,  1.67it/s, v_num=8, loss/train=32.10, loss/val=31.60]
Epoch 5:  77%|███████▋  | 20/26 [00:11<00:03,  1.75it/s, v_num=8, loss/train=32.10, loss/val=31.60]
Epoch 5:  77%|███████▋  | 20/26 [00:11<00:03,  1.75it/s, v_num=8, loss/train=33.20, loss/val=31.60]
Epoch 5:  81%|████████  | 21/26 [00:11<00:02,  1.83it/s, v_num=8, loss/train=33.20, loss/val=31.60]
Epoch 5:  81%|████████  | 21/26 [00:11<00:02,  1.83it/s, v_num=8, loss/train=33.10, loss/val=31.60]
Epoch 5:  85%|████████▍ | 22/26 [00:12<00:02,  1.81it/s, v_num=8, loss/train=33.10, loss/val=31.60]
Epoch 5:  85%|████████▍ | 22/26 [00:12<00:02,  1.81it/s, v_num=8, loss/train=32.80, loss/val=31.60]
Epoch 5:  88%|████████▊ | 23/26 [00:12<00:01,  1.89it/s, v_num=8, loss/train=32.80, loss/val=31.60]
Epoch 5:  88%|████████▊ | 23/26 [00:12<00:01,  1.89it/s, v_num=8, loss/train=32.80, loss/val=31.60]
Epoch 5:  92%|█████████▏| 24/26 [00:12<00:01,  1.97it/s, v_num=8, loss/train=32.80, loss/val=31.60]
Epoch 5:  92%|█████████▏| 24/26 [00:12<00:01,  1.97it/s, v_num=8, loss/train=28.40, loss/val=31.60]
Epoch 5:  96%|█████████▌| 25/26 [00:12<00:00,  2.05it/s, v_num=8, loss/train=28.40, loss/val=31.60]
Epoch 5:  96%|█████████▌| 25/26 [00:12<00:00,  2.05it/s, v_num=8, loss/train=33.20, loss/val=31.60]
Epoch 5: 100%|██████████| 26/26 [00:12<00:00,  2.13it/s, v_num=8, loss/train=33.20, loss/val=31.60]
Epoch 5: 100%|██████████| 26/26 [00:12<00:00,  2.13it/s, v_num=8, loss/train=29.80, loss/val=31.60]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 174.36it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 239.46it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 278.46it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 169.81it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 194.82it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 214.15it/s]


Epoch 5: 100%|██████████| 26/26 [00:14<00:00,  1.78it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 5: 100%|██████████| 26/26 [00:14<00:00,  1.78it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 5:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 6:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 6:   4%|▍         | 1/26 [00:04<01:46,  0.23it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 6:   4%|▍         | 1/26 [00:04<01:46,  0.23it/s, v_num=8, loss/train=32.00, loss/val=32.60]
Epoch 6:   8%|▊         | 2/26 [00:05<01:01,  0.39it/s, v_num=8, loss/train=32.00, loss/val=32.60]
Epoch 6:   8%|▊         | 2/26 [00:05<01:01,  0.39it/s, v_num=8, loss/train=32.10, loss/val=32.60]
Epoch 6:  12%|█▏        | 3/26 [00:08<01:05,  0.35it/s, v_num=8, loss/train=32.10, loss/val=32.60]
Epoch 6:  12%|█▏        | 3/26 [00:08<01:05,  0.35it/s, v_num=8, loss/train=31.10, loss/val=32.60]
Epoch 6:  15%|█▌        | 4/26 [00:08<00:47,  0.46it/s, v_num=8, loss/train=31.10, loss/val=32.60]
Epoch 6:  15%|█▌        | 4/26 [00:08<00:47,  0.46it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 6:  19%|█▉        | 5/26 [00:08<00:36,  0.57it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 6:  19%|█▉        | 5/26 [00:08<00:36,  0.57it/s, v_num=8, loss/train=30.60, loss/val=32.60]
Epoch 6:  23%|██▎       | 6/26 [00:08<00:29,  0.68it/s, v_num=8, loss/train=30.60, loss/val=32.60]
Epoch 6:  23%|██▎       | 6/26 [00:08<00:29,  0.68it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 6:  27%|██▋       | 7/26 [00:08<00:24,  0.79it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 6:  27%|██▋       | 7/26 [00:08<00:24,  0.79it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 6:  31%|███       | 8/26 [00:08<00:20,  0.90it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 6:  31%|███       | 8/26 [00:08<00:20,  0.90it/s, v_num=8, loss/train=30.70, loss/val=32.60]
Epoch 6:  35%|███▍      | 9/26 [00:08<00:16,  1.01it/s, v_num=8, loss/train=30.70, loss/val=32.60]
Epoch 6:  35%|███▍      | 9/26 [00:08<00:16,  1.01it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 6:  38%|███▊      | 10/26 [00:08<00:14,  1.11it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 6:  38%|███▊      | 10/26 [00:08<00:14,  1.11it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 6:  42%|████▏     | 11/26 [00:09<00:12,  1.22it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 6:  42%|████▏     | 11/26 [00:09<00:12,  1.22it/s, v_num=8, loss/train=30.90, loss/val=32.60]
Epoch 6:  46%|████▌     | 12/26 [00:09<00:11,  1.24it/s, v_num=8, loss/train=30.90, loss/val=32.60]
Epoch 6:  46%|████▌     | 12/26 [00:09<00:11,  1.24it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 6:  50%|█████     | 13/26 [00:11<00:11,  1.14it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 6:  50%|█████     | 13/26 [00:11<00:11,  1.14it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 6:  54%|█████▍    | 14/26 [00:11<00:09,  1.23it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 6:  54%|█████▍    | 14/26 [00:11<00:09,  1.23it/s, v_num=8, loss/train=32.30, loss/val=32.60]
Epoch 6:  58%|█████▊    | 15/26 [00:11<00:08,  1.31it/s, v_num=8, loss/train=32.30, loss/val=32.60]
Epoch 6:  58%|█████▊    | 15/26 [00:11<00:08,  1.31it/s, v_num=8, loss/train=33.50, loss/val=32.60]
Epoch 6:  62%|██████▏   | 16/26 [00:11<00:07,  1.40it/s, v_num=8, loss/train=33.50, loss/val=32.60]
Epoch 6:  62%|██████▏   | 16/26 [00:11<00:07,  1.40it/s, v_num=8, loss/train=31.60, loss/val=32.60]
Epoch 6:  65%|██████▌   | 17/26 [00:11<00:06,  1.49it/s, v_num=8, loss/train=31.60, loss/val=32.60]
Epoch 6:  65%|██████▌   | 17/26 [00:11<00:06,  1.49it/s, v_num=8, loss/train=30.20, loss/val=32.60]
Epoch 6:  69%|██████▉   | 18/26 [00:11<00:05,  1.57it/s, v_num=8, loss/train=30.20, loss/val=32.60]
Epoch 6:  69%|██████▉   | 18/26 [00:11<00:05,  1.57it/s, v_num=8, loss/train=33.60, loss/val=32.60]
Epoch 6:  73%|███████▎  | 19/26 [00:11<00:04,  1.66it/s, v_num=8, loss/train=33.60, loss/val=32.60]
Epoch 6:  73%|███████▎  | 19/26 [00:11<00:04,  1.66it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 6:  77%|███████▋  | 20/26 [00:11<00:03,  1.74it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 6:  77%|███████▋  | 20/26 [00:11<00:03,  1.74it/s, v_num=8, loss/train=30.70, loss/val=32.60]
Epoch 6:  81%|████████  | 21/26 [00:11<00:02,  1.82it/s, v_num=8, loss/train=30.70, loss/val=32.60]
Epoch 6:  81%|████████  | 21/26 [00:11<00:02,  1.82it/s, v_num=8, loss/train=32.90, loss/val=32.60]
Epoch 6:  85%|████████▍ | 22/26 [00:11<00:02,  1.85it/s, v_num=8, loss/train=32.90, loss/val=32.60]
Epoch 6:  85%|████████▍ | 22/26 [00:11<00:02,  1.85it/s, v_num=8, loss/train=34.00, loss/val=32.60]
Epoch 6:  88%|████████▊ | 23/26 [00:12<00:01,  1.87it/s, v_num=8, loss/train=34.00, loss/val=32.60]
Epoch 6:  88%|████████▊ | 23/26 [00:12<00:01,  1.87it/s, v_num=8, loss/train=33.20, loss/val=32.60]
Epoch 6:  92%|█████████▏| 24/26 [00:12<00:01,  1.95it/s, v_num=8, loss/train=33.20, loss/val=32.60]
Epoch 6:  92%|█████████▏| 24/26 [00:12<00:01,  1.95it/s, v_num=8, loss/train=31.30, loss/val=32.60]
Epoch 6:  96%|█████████▌| 25/26 [00:12<00:00,  2.03it/s, v_num=8, loss/train=31.30, loss/val=32.60]
Epoch 6:  96%|█████████▌| 25/26 [00:12<00:00,  2.03it/s, v_num=8, loss/train=32.10, loss/val=32.60]
Epoch 6: 100%|██████████| 26/26 [00:12<00:00,  2.11it/s, v_num=8, loss/train=32.10, loss/val=32.60]
Epoch 6: 100%|██████████| 26/26 [00:12<00:00,  2.11it/s, v_num=8, loss/train=36.80, loss/val=32.60]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 124.29it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 166.77it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 188.88it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 112.42it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 47.46it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 55.52it/s]


Epoch 6: 100%|██████████| 26/26 [00:14<00:00,  1.76it/s, v_num=8, loss/train=36.80, loss/val=32.60]
Epoch 6: 100%|██████████| 26/26 [00:14<00:00,  1.76it/s, v_num=8, loss/train=36.80, loss/val=32.60]
Epoch 6:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=36.80, loss/val=32.60]
Epoch 7:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=36.80, loss/val=32.60]
Epoch 7:   4%|▍         | 1/26 [00:04<01:40,  0.25it/s, v_num=8, loss/train=36.80, loss/val=32.60]
Epoch 7:   4%|▍         | 1/26 [00:04<01:40,  0.25it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7:   8%|▊         | 2/26 [00:04<00:49,  0.49it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7:   8%|▊         | 2/26 [00:04<00:49,  0.49it/s, v_num=8, loss/train=31.10, loss/val=32.60]
Epoch 7:  12%|█▏        | 3/26 [00:04<00:36,  0.62it/s, v_num=8, loss/train=31.10, loss/val=32.60]
Epoch 7:  12%|█▏        | 3/26 [00:04<00:36,  0.62it/s, v_num=8, loss/train=31.00, loss/val=32.60]
Epoch 7:  15%|█▌        | 4/26 [00:05<00:30,  0.73it/s, v_num=8, loss/train=31.00, loss/val=32.60]
Epoch 7:  15%|█▌        | 4/26 [00:05<00:30,  0.73it/s, v_num=8, loss/train=28.80, loss/val=32.60]
Epoch 7:  19%|█▉        | 5/26 [00:05<00:23,  0.91it/s, v_num=8, loss/train=28.80, loss/val=32.60]
Epoch 7:  19%|█▉        | 5/26 [00:05<00:23,  0.91it/s, v_num=8, loss/train=31.90, loss/val=32.60]
Epoch 7:  23%|██▎       | 6/26 [00:05<00:18,  1.08it/s, v_num=8, loss/train=31.90, loss/val=32.60]
Epoch 7:  23%|██▎       | 6/26 [00:05<00:18,  1.08it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 7:  27%|██▋       | 7/26 [00:05<00:15,  1.26it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 7:  27%|██▋       | 7/26 [00:05<00:15,  1.26it/s, v_num=8, loss/train=32.20, loss/val=32.60]
Epoch 7:  31%|███       | 8/26 [00:05<00:12,  1.43it/s, v_num=8, loss/train=32.20, loss/val=32.60]
Epoch 7:  31%|███       | 8/26 [00:05<00:12,  1.43it/s, v_num=8, loss/train=30.10, loss/val=32.60]
Epoch 7:  35%|███▍      | 9/26 [00:05<00:10,  1.59it/s, v_num=8, loss/train=30.10, loss/val=32.60]
Epoch 7:  35%|███▍      | 9/26 [00:05<00:10,  1.59it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7:  38%|███▊      | 10/26 [00:05<00:09,  1.74it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7:  38%|███▊      | 10/26 [00:05<00:09,  1.74it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 7:  42%|████▏     | 11/26 [00:07<00:10,  1.42it/s, v_num=8, loss/train=29.80, loss/val=32.60]
Epoch 7:  42%|████▏     | 11/26 [00:07<00:10,  1.42it/s, v_num=8, loss/train=33.80, loss/val=32.60]
Epoch 7:  46%|████▌     | 12/26 [00:07<00:09,  1.55it/s, v_num=8, loss/train=33.80, loss/val=32.60]
Epoch 7:  46%|████▌     | 12/26 [00:07<00:09,  1.55it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 7:  50%|█████     | 13/26 [00:08<00:08,  1.53it/s, v_num=8, loss/train=30.40, loss/val=32.60]
Epoch 7:  50%|█████     | 13/26 [00:08<00:08,  1.53it/s, v_num=8, loss/train=31.80, loss/val=32.60]
Epoch 7:  54%|█████▍    | 14/26 [00:08<00:07,  1.57it/s, v_num=8, loss/train=31.80, loss/val=32.60]
Epoch 7:  54%|█████▍    | 14/26 [00:08<00:07,  1.57it/s, v_num=8, loss/train=31.50, loss/val=32.60]
Epoch 7:  58%|█████▊    | 15/26 [00:09<00:06,  1.62it/s, v_num=8, loss/train=31.50, loss/val=32.60]
Epoch 7:  58%|█████▊    | 15/26 [00:09<00:06,  1.62it/s, v_num=8, loss/train=30.50, loss/val=32.60]
Epoch 7:  62%|██████▏   | 16/26 [00:09<00:05,  1.72it/s, v_num=8, loss/train=30.50, loss/val=32.60]
Epoch 7:  62%|██████▏   | 16/26 [00:09<00:05,  1.72it/s, v_num=8, loss/train=29.00, loss/val=32.60]
Epoch 7:  65%|██████▌   | 17/26 [00:09<00:04,  1.83it/s, v_num=8, loss/train=29.00, loss/val=32.60]
Epoch 7:  65%|██████▌   | 17/26 [00:09<00:04,  1.83it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 7:  69%|██████▉   | 18/26 [00:09<00:04,  1.93it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 7:  69%|██████▉   | 18/26 [00:09<00:04,  1.93it/s, v_num=8, loss/train=30.60, loss/val=32.60]
Epoch 7:  73%|███████▎  | 19/26 [00:09<00:03,  1.99it/s, v_num=8, loss/train=30.60, loss/val=32.60]
Epoch 7:  73%|███████▎  | 19/26 [00:09<00:03,  1.99it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 7:  77%|███████▋  | 20/26 [00:09<00:02,  2.09it/s, v_num=8, loss/train=30.80, loss/val=32.60]
Epoch 7:  77%|███████▋  | 20/26 [00:09<00:02,  2.09it/s, v_num=8, loss/train=31.90, loss/val=32.60]
Epoch 7:  81%|████████  | 21/26 [00:09<00:02,  2.10it/s, v_num=8, loss/train=31.90, loss/val=32.60]
Epoch 7:  81%|████████  | 21/26 [00:09<00:02,  2.10it/s, v_num=8, loss/train=32.20, loss/val=32.60]
Epoch 7:  85%|████████▍ | 22/26 [00:10<00:01,  2.20it/s, v_num=8, loss/train=32.20, loss/val=32.60]
Epoch 7:  85%|████████▍ | 22/26 [00:10<00:01,  2.20it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 7:  88%|████████▊ | 23/26 [00:10<00:01,  2.26it/s, v_num=8, loss/train=33.70, loss/val=32.60]
Epoch 7:  88%|████████▊ | 23/26 [00:10<00:01,  2.26it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 7:  92%|█████████▏| 24/26 [00:10<00:00,  2.36it/s, v_num=8, loss/train=32.50, loss/val=32.60]
Epoch 7:  92%|█████████▏| 24/26 [00:10<00:00,  2.36it/s, v_num=8, loss/train=31.60, loss/val=32.60]
Epoch 7:  96%|█████████▌| 25/26 [00:10<00:00,  2.43it/s, v_num=8, loss/train=31.60, loss/val=32.60]
Epoch 7:  96%|█████████▌| 25/26 [00:10<00:00,  2.43it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7: 100%|██████████| 26/26 [00:10<00:00,  2.53it/s, v_num=8, loss/train=31.70, loss/val=32.60]
Epoch 7: 100%|██████████| 26/26 [00:10<00:00,  2.53it/s, v_num=8, loss/train=30.30, loss/val=32.60]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 148.32it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 201.70it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 39.54it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 45.60it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 55.38it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 64.55it/s]


Epoch 7: 100%|██████████| 26/26 [00:12<00:00,  2.04it/s, v_num=8, loss/train=30.30, loss/val=31.80]
Epoch 7: 100%|██████████| 26/26 [00:12<00:00,  2.04it/s, v_num=8, loss/train=30.30, loss/val=31.80]
Epoch 7:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.30, loss/val=31.80]
Epoch 8:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.30, loss/val=31.80]
Epoch 8:   4%|▍         | 1/26 [00:07<03:15,  0.13it/s, v_num=8, loss/train=30.30, loss/val=31.80]
Epoch 8:   4%|▍         | 1/26 [00:07<03:15,  0.13it/s, v_num=8, loss/train=29.60, loss/val=31.80]
Epoch 8:   8%|▊         | 2/26 [00:07<01:34,  0.25it/s, v_num=8, loss/train=29.60, loss/val=31.80]
Epoch 8:   8%|▊         | 2/26 [00:07<01:34,  0.25it/s, v_num=8, loss/train=31.40, loss/val=31.80]
Epoch 8:  12%|█▏        | 3/26 [00:07<01:00,  0.38it/s, v_num=8, loss/train=31.40, loss/val=31.80]
Epoch 8:  12%|█▏        | 3/26 [00:07<01:00,  0.38it/s, v_num=8, loss/train=31.80, loss/val=31.80]
Epoch 8:  15%|█▌        | 4/26 [00:07<00:43,  0.50it/s, v_num=8, loss/train=31.80, loss/val=31.80]
Epoch 8:  15%|█▌        | 4/26 [00:07<00:43,  0.50it/s, v_num=8, loss/train=33.00, loss/val=31.80]
Epoch 8:  19%|█▉        | 5/26 [00:08<00:33,  0.62it/s, v_num=8, loss/train=33.00, loss/val=31.80]
Epoch 8:  19%|█▉        | 5/26 [00:08<00:33,  0.62it/s, v_num=8, loss/train=33.10, loss/val=31.80]
Epoch 8:  23%|██▎       | 6/26 [00:08<00:27,  0.73it/s, v_num=8, loss/train=33.10, loss/val=31.80]
Epoch 8:  23%|██▎       | 6/26 [00:08<00:27,  0.73it/s, v_num=8, loss/train=34.10, loss/val=31.80]
Epoch 8:  27%|██▋       | 7/26 [00:08<00:22,  0.84it/s, v_num=8, loss/train=34.10, loss/val=31.80]
Epoch 8:  27%|██▋       | 7/26 [00:08<00:22,  0.84it/s, v_num=8, loss/train=31.90, loss/val=31.80]
Epoch 8:  31%|███       | 8/26 [00:08<00:18,  0.96it/s, v_num=8, loss/train=31.90, loss/val=31.80]
Epoch 8:  31%|███       | 8/26 [00:08<00:18,  0.96it/s, v_num=8, loss/train=30.70, loss/val=31.80]
Epoch 8:  35%|███▍      | 9/26 [00:08<00:15,  1.08it/s, v_num=8, loss/train=30.70, loss/val=31.80]
Epoch 8:  35%|███▍      | 9/26 [00:08<00:15,  1.08it/s, v_num=8, loss/train=33.90, loss/val=31.80]
Epoch 8:  38%|███▊      | 10/26 [00:08<00:13,  1.19it/s, v_num=8, loss/train=33.90, loss/val=31.80]
Epoch 8:  38%|███▊      | 10/26 [00:08<00:13,  1.19it/s, v_num=8, loss/train=32.70, loss/val=31.80]
Epoch 8:  42%|████▏     | 11/26 [00:10<00:14,  1.04it/s, v_num=8, loss/train=32.70, loss/val=31.80]
Epoch 8:  42%|████▏     | 11/26 [00:10<00:14,  1.04it/s, v_num=8, loss/train=29.20, loss/val=31.80]
Epoch 8:  46%|████▌     | 12/26 [00:10<00:12,  1.14it/s, v_num=8, loss/train=29.20, loss/val=31.80]
Epoch 8:  46%|████▌     | 12/26 [00:10<00:12,  1.14it/s, v_num=8, loss/train=33.40, loss/val=31.80]
Epoch 8:  50%|█████     | 13/26 [00:10<00:10,  1.23it/s, v_num=8, loss/train=33.40, loss/val=31.80]
Epoch 8:  50%|█████     | 13/26 [00:10<00:10,  1.23it/s, v_num=8, loss/train=30.20, loss/val=31.80]
Epoch 8:  54%|█████▍    | 14/26 [00:10<00:09,  1.29it/s, v_num=8, loss/train=30.20, loss/val=31.80]
Epoch 8:  54%|█████▍    | 14/26 [00:10<00:09,  1.29it/s, v_num=8, loss/train=31.40, loss/val=31.80]
Epoch 8:  58%|█████▊    | 15/26 [00:10<00:08,  1.37it/s, v_num=8, loss/train=31.40, loss/val=31.80]
Epoch 8:  58%|█████▊    | 15/26 [00:10<00:08,  1.37it/s, v_num=8, loss/train=33.20, loss/val=31.80]
Epoch 8:  62%|██████▏   | 16/26 [00:10<00:06,  1.46it/s, v_num=8, loss/train=33.20, loss/val=31.80]
Epoch 8:  62%|██████▏   | 16/26 [00:10<00:06,  1.46it/s, v_num=8, loss/train=31.50, loss/val=31.80]
Epoch 8:  65%|██████▌   | 17/26 [00:10<00:05,  1.55it/s, v_num=8, loss/train=31.50, loss/val=31.80]
Epoch 8:  65%|██████▌   | 17/26 [00:10<00:05,  1.55it/s, v_num=8, loss/train=32.10, loss/val=31.80]
Epoch 8:  69%|██████▉   | 18/26 [00:11<00:04,  1.63it/s, v_num=8, loss/train=32.10, loss/val=31.80]
Epoch 8:  69%|██████▉   | 18/26 [00:11<00:04,  1.63it/s, v_num=8, loss/train=32.60, loss/val=31.80]
Epoch 8:  73%|███████▎  | 19/26 [00:11<00:04,  1.72it/s, v_num=8, loss/train=32.60, loss/val=31.80]
Epoch 8:  73%|███████▎  | 19/26 [00:11<00:04,  1.72it/s, v_num=8, loss/train=34.20, loss/val=31.80]
Epoch 8:  77%|███████▋  | 20/26 [00:11<00:03,  1.81it/s, v_num=8, loss/train=34.20, loss/val=31.80]
Epoch 8:  77%|███████▋  | 20/26 [00:11<00:03,  1.81it/s, v_num=8, loss/train=33.70, loss/val=31.80]
Epoch 8:  81%|████████  | 21/26 [00:11<00:02,  1.75it/s, v_num=8, loss/train=33.70, loss/val=31.80]
Epoch 8:  81%|████████  | 21/26 [00:11<00:02,  1.75it/s, v_num=8, loss/train=31.20, loss/val=31.80]
Epoch 8:  85%|████████▍ | 22/26 [00:11<00:02,  1.84it/s, v_num=8, loss/train=31.20, loss/val=31.80]
Epoch 8:  85%|████████▍ | 22/26 [00:11<00:02,  1.84it/s, v_num=8, loss/train=31.70, loss/val=31.80]
Epoch 8:  88%|████████▊ | 23/26 [00:11<00:01,  1.92it/s, v_num=8, loss/train=31.70, loss/val=31.80]
Epoch 8:  88%|████████▊ | 23/26 [00:11<00:01,  1.92it/s, v_num=8, loss/train=31.80, loss/val=31.80]
Epoch 8:  92%|█████████▏| 24/26 [00:12<00:01,  1.96it/s, v_num=8, loss/train=31.80, loss/val=31.80]
Epoch 8:  92%|█████████▏| 24/26 [00:12<00:01,  1.96it/s, v_num=8, loss/train=32.90, loss/val=31.80]
Epoch 8:  96%|█████████▌| 25/26 [00:12<00:00,  2.05it/s, v_num=8, loss/train=32.90, loss/val=31.80]
Epoch 8:  96%|█████████▌| 25/26 [00:12<00:00,  2.05it/s, v_num=8, loss/train=31.70, loss/val=31.80]
Epoch 8: 100%|██████████| 26/26 [00:12<00:00,  2.13it/s, v_num=8, loss/train=31.70, loss/val=31.80]
Epoch 8: 100%|██████████| 26/26 [00:12<00:00,  2.13it/s, v_num=8, loss/train=30.20, loss/val=31.80]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 155.75it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 49.91it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 70.30it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 88.67it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 105.01it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 119.60it/s]


Epoch 8: 100%|██████████| 26/26 [00:14<00:00,  1.78it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 8: 100%|██████████| 26/26 [00:14<00:00,  1.78it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 8:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 9:   0%|          | 0/26 [00:00<?, ?it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 9:   4%|▍         | 1/26 [00:05<02:13,  0.19it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 9:   4%|▍         | 1/26 [00:05<02:14,  0.19it/s, v_num=8, loss/train=28.50, loss/val=31.60]
Epoch 9:   8%|▊         | 2/26 [00:05<01:06,  0.36it/s, v_num=8, loss/train=28.50, loss/val=31.60]
Epoch 9:   8%|▊         | 2/26 [00:05<01:06,  0.36it/s, v_num=8, loss/train=30.90, loss/val=31.60]
Epoch 9:  12%|█▏        | 3/26 [00:05<00:42,  0.54it/s, v_num=8, loss/train=30.90, loss/val=31.60]
Epoch 9:  12%|█▏        | 3/26 [00:05<00:42,  0.54it/s, v_num=8, loss/train=31.40, loss/val=31.60]
Epoch 9:  15%|█▌        | 4/26 [00:05<00:31,  0.71it/s, v_num=8, loss/train=31.40, loss/val=31.60]
Epoch 9:  15%|█▌        | 4/26 [00:05<00:31,  0.70it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 9:  19%|█▉        | 5/26 [00:05<00:24,  0.87it/s, v_num=8, loss/train=30.20, loss/val=31.60]
Epoch 9:  19%|█▉        | 5/26 [00:05<00:24,  0.87it/s, v_num=8, loss/train=30.50, loss/val=31.60]
Epoch 9:  23%|██▎       | 6/26 [00:05<00:19,  1.03it/s, v_num=8, loss/train=30.50, loss/val=31.60]
Epoch 9:  23%|██▎       | 6/26 [00:05<00:19,  1.03it/s, v_num=8, loss/train=31.50, loss/val=31.60]
Epoch 9:  27%|██▋       | 7/26 [00:05<00:15,  1.19it/s, v_num=8, loss/train=31.50, loss/val=31.60]
Epoch 9:  27%|██▋       | 7/26 [00:05<00:15,  1.19it/s, v_num=8, loss/train=31.80, loss/val=31.60]
Epoch 9:  31%|███       | 8/26 [00:05<00:13,  1.34it/s, v_num=8, loss/train=31.80, loss/val=31.60]
Epoch 9:  31%|███       | 8/26 [00:05<00:13,  1.34it/s, v_num=8, loss/train=30.80, loss/val=31.60]
Epoch 9:  35%|███▍      | 9/26 [00:06<00:11,  1.49it/s, v_num=8, loss/train=30.80, loss/val=31.60]
Epoch 9:  35%|███▍      | 9/26 [00:06<00:11,  1.49it/s, v_num=8, loss/train=35.30, loss/val=31.60]
Epoch 9:  38%|███▊      | 10/26 [00:06<00:09,  1.64it/s, v_num=8, loss/train=35.30, loss/val=31.60]
Epoch 9:  38%|███▊      | 10/26 [00:06<00:09,  1.64it/s, v_num=8, loss/train=32.70, loss/val=31.60]
Epoch 9:  42%|████▏     | 11/26 [00:09<00:12,  1.19it/s, v_num=8, loss/train=32.70, loss/val=31.60]
Epoch 9:  42%|████▏     | 11/26 [00:09<00:12,  1.18it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  46%|████▌     | 12/26 [00:09<00:10,  1.29it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  46%|████▌     | 12/26 [00:09<00:10,  1.29it/s, v_num=8, loss/train=29.60, loss/val=31.60]
Epoch 9:  50%|█████     | 13/26 [00:09<00:09,  1.39it/s, v_num=8, loss/train=29.60, loss/val=31.60]
Epoch 9:  50%|█████     | 13/26 [00:09<00:09,  1.39it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  54%|█████▍    | 14/26 [00:09<00:08,  1.49it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  54%|█████▍    | 14/26 [00:09<00:08,  1.49it/s, v_num=8, loss/train=33.60, loss/val=31.60]
Epoch 9:  58%|█████▊    | 15/26 [00:09<00:06,  1.60it/s, v_num=8, loss/train=33.60, loss/val=31.60]
Epoch 9:  58%|█████▊    | 15/26 [00:09<00:06,  1.60it/s, v_num=8, loss/train=32.20, loss/val=31.60]
Epoch 9:  62%|██████▏   | 16/26 [00:09<00:05,  1.70it/s, v_num=8, loss/train=32.20, loss/val=31.60]
Epoch 9:  62%|██████▏   | 16/26 [00:09<00:05,  1.70it/s, v_num=8, loss/train=34.00, loss/val=31.60]
Epoch 9:  65%|██████▌   | 17/26 [00:09<00:05,  1.71it/s, v_num=8, loss/train=34.00, loss/val=31.60]
Epoch 9:  65%|██████▌   | 17/26 [00:09<00:05,  1.71it/s, v_num=8, loss/train=31.20, loss/val=31.60]
Epoch 9:  69%|██████▉   | 18/26 [00:10<00:04,  1.76it/s, v_num=8, loss/train=31.20, loss/val=31.60]
Epoch 9:  69%|██████▉   | 18/26 [00:10<00:04,  1.76it/s, v_num=8, loss/train=32.00, loss/val=31.60]
Epoch 9:  73%|███████▎  | 19/26 [00:10<00:03,  1.85it/s, v_num=8, loss/train=32.00, loss/val=31.60]
Epoch 9:  73%|███████▎  | 19/26 [00:10<00:03,  1.85it/s, v_num=8, loss/train=31.60, loss/val=31.60]
Epoch 9:  77%|███████▋  | 20/26 [00:10<00:03,  1.95it/s, v_num=8, loss/train=31.60, loss/val=31.60]
Epoch 9:  77%|███████▋  | 20/26 [00:10<00:03,  1.95it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  81%|████████  | 21/26 [00:10<00:02,  1.99it/s, v_num=8, loss/train=31.00, loss/val=31.60]
Epoch 9:  81%|████████  | 21/26 [00:10<00:02,  1.99it/s, v_num=8, loss/train=28.90, loss/val=31.60]
Epoch 9:  85%|████████▍ | 22/26 [00:10<00:01,  2.08it/s, v_num=8, loss/train=28.90, loss/val=31.60]
Epoch 9:  85%|████████▍ | 22/26 [00:10<00:01,  2.08it/s, v_num=8, loss/train=33.70, loss/val=31.60]
Epoch 9:  88%|████████▊ | 23/26 [00:10<00:01,  2.17it/s, v_num=8, loss/train=33.70, loss/val=31.60]
Epoch 9:  88%|████████▊ | 23/26 [00:10<00:01,  2.17it/s, v_num=8, loss/train=27.60, loss/val=31.60]
Epoch 9:  92%|█████████▏| 24/26 [00:10<00:00,  2.27it/s, v_num=8, loss/train=27.60, loss/val=31.60]
Epoch 9:  92%|█████████▏| 24/26 [00:10<00:00,  2.27it/s, v_num=8, loss/train=30.30, loss/val=31.60]
Epoch 9:  96%|█████████▌| 25/26 [00:10<00:00,  2.36it/s, v_num=8, loss/train=30.30, loss/val=31.60]
Epoch 9:  96%|█████████▌| 25/26 [00:10<00:00,  2.36it/s, v_num=8, loss/train=31.30, loss/val=31.60]
Epoch 9: 100%|██████████| 26/26 [00:10<00:00,  2.45it/s, v_num=8, loss/train=31.30, loss/val=31.60]
Epoch 9: 100%|██████████| 26/26 [00:10<00:00,  2.45it/s, v_num=8, loss/train=34.10, loss/val=31.60]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 150.34it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 204.60it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 228.03it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 238.31it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 168.26it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 186.95it/s]


Epoch 9: 100%|██████████| 26/26 [00:13<00:00,  1.99it/s, v_num=8, loss/train=34.10, loss/val=31.50]
Epoch 9: 100%|██████████| 26/26 [00:13<00:00,  1.99it/s, v_num=8, loss/train=34.10, loss/val=31.50]
Epoch 9: 100%|██████████| 26/26 [00:13<00:00,  1.99it/s, v_num=8, loss/train=34.10, loss/val=31.50]

Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking: |          | 0/? [00:00<?, ?it/s]
Sanity Checking DataLoader 0:   0%|          | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0:  50%|█████     | 1/2 [00:00<00:00,  6.77it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00, 12.23it/s]


Training: |          | 0/? [00:00<?, ?it/s]
Training: |          | 0/? [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/26 [00:00<?, ?it/s]
Epoch 0:   4%|▍         | 1/26 [00:00<00:03,  7.33it/s]
Epoch 0:   4%|▍         | 1/26 [00:00<00:03,  7.31it/s, v_num=9, loss/train=31.10]
Epoch 0:   8%|▊         | 2/26 [00:00<00:10,  2.30it/s, v_num=9, loss/train=31.10]
Epoch 0:   8%|▊         | 2/26 [00:00<00:10,  2.29it/s, v_num=9, loss/train=29.50]
Epoch 0:  12%|█▏        | 3/26 [00:01<00:15,  1.52it/s, v_num=9, loss/train=29.50]
Epoch 0:  12%|█▏        | 3/26 [00:01<00:15,  1.52it/s, v_num=9, loss/train=32.80]
Epoch 0:  15%|█▌        | 4/26 [00:02<00:11,  1.92it/s, v_num=9, loss/train=32.80]
Epoch 0:  15%|█▌        | 4/26 [00:02<00:11,  1.92it/s, v_num=9, loss/train=31.50]
Epoch 0:  19%|█▉        | 5/26 [00:02<00:09,  2.24it/s, v_num=9, loss/train=31.50]
Epoch 0:  19%|█▉        | 5/26 [00:02<00:09,  2.24it/s, v_num=9, loss/train=31.70]
Epoch 0:  23%|██▎       | 6/26 [00:02<00:07,  2.60it/s, v_num=9, loss/train=31.70]
Epoch 0:  23%|██▎       | 6/26 [00:02<00:07,  2.60it/s, v_num=9, loss/train=30.90]
Epoch 0:  27%|██▋       | 7/26 [00:02<00:06,  2.96it/s, v_num=9, loss/train=30.90]
Epoch 0:  27%|██▋       | 7/26 [00:02<00:06,  2.96it/s, v_num=9, loss/train=29.00]
Epoch 0:  31%|███       | 8/26 [00:02<00:05,  3.33it/s, v_num=9, loss/train=29.00]
Epoch 0:  31%|███       | 8/26 [00:02<00:05,  3.33it/s, v_num=9, loss/train=30.30]
Epoch 0:  35%|███▍      | 9/26 [00:02<00:04,  3.69it/s, v_num=9, loss/train=30.30]
Epoch 0:  35%|███▍      | 9/26 [00:02<00:04,  3.69it/s, v_num=9, loss/train=30.80]
Epoch 0:  38%|███▊      | 10/26 [00:02<00:03,  4.04it/s, v_num=9, loss/train=30.80]
Epoch 0:  38%|███▊      | 10/26 [00:02<00:03,  4.04it/s, v_num=9, loss/train=30.00]
Epoch 0:  42%|████▏     | 11/26 [00:03<00:05,  2.88it/s, v_num=9, loss/train=30.00]
Epoch 0:  42%|████▏     | 11/26 [00:03<00:05,  2.87it/s, v_num=9, loss/train=30.70]
Epoch 0:  46%|████▌     | 12/26 [00:05<00:06,  2.30it/s, v_num=9, loss/train=30.70]
Epoch 0:  46%|████▌     | 12/26 [00:05<00:06,  2.29it/s, v_num=9, loss/train=29.40]
Epoch 0:  50%|█████     | 13/26 [00:05<00:05,  2.47it/s, v_num=9, loss/train=29.40]
Epoch 0:  50%|█████     | 13/26 [00:05<00:05,  2.47it/s, v_num=9, loss/train=32.00]
Epoch 0:  54%|█████▍    | 14/26 [00:05<00:04,  2.52it/s, v_num=9, loss/train=32.00]
Epoch 0:  54%|█████▍    | 14/26 [00:05<00:04,  2.52it/s, v_num=9, loss/train=30.00]
Epoch 0:  58%|█████▊    | 15/26 [00:05<00:04,  2.69it/s, v_num=9, loss/train=30.00]
Epoch 0:  58%|█████▊    | 15/26 [00:05<00:04,  2.69it/s, v_num=9, loss/train=32.50]
Epoch 0:  62%|██████▏   | 16/26 [00:05<00:03,  2.83it/s, v_num=9, loss/train=32.50]
Epoch 0:  62%|██████▏   | 16/26 [00:05<00:03,  2.83it/s, v_num=9, loss/train=32.10]
Epoch 0:  65%|██████▌   | 17/26 [00:05<00:03,  2.99it/s, v_num=9, loss/train=32.10]
Epoch 0:  65%|██████▌   | 17/26 [00:05<00:03,  2.99it/s, v_num=9, loss/train=33.80]
Epoch 0:  69%|██████▉   | 18/26 [00:05<00:02,  3.14it/s, v_num=9, loss/train=33.80]
Epoch 0:  69%|██████▉   | 18/26 [00:05<00:02,  3.14it/s, v_num=9, loss/train=31.20]
Epoch 0:  73%|███████▎  | 19/26 [00:05<00:02,  3.31it/s, v_num=9, loss/train=31.20]
Epoch 0:  73%|███████▎  | 19/26 [00:05<00:02,  3.31it/s, v_num=9, loss/train=33.70]
Epoch 0:  77%|███████▋  | 20/26 [00:05<00:01,  3.47it/s, v_num=9, loss/train=33.70]
Epoch 0:  77%|███████▋  | 20/26 [00:05<00:01,  3.47it/s, v_num=9, loss/train=29.60]
Epoch 0:  81%|████████  | 21/26 [00:07<00:01,  2.71it/s, v_num=9, loss/train=29.60]
Epoch 0:  81%|████████  | 21/26 [00:07<00:01,  2.71it/s, v_num=9, loss/train=28.40]
Epoch 0:  85%|████████▍ | 22/26 [00:08<00:01,  2.70it/s, v_num=9, loss/train=28.40]
Epoch 0:  85%|████████▍ | 22/26 [00:08<00:01,  2.70it/s, v_num=9, loss/train=32.20]
Epoch 0:  88%|████████▊ | 23/26 [00:08<00:01,  2.68it/s, v_num=9, loss/train=32.20]
Epoch 0:  88%|████████▊ | 23/26 [00:08<00:01,  2.68it/s, v_num=9, loss/train=30.60]
Epoch 0:  92%|█████████▏| 24/26 [00:08<00:00,  2.78it/s, v_num=9, loss/train=30.60]
Epoch 0:  92%|█████████▏| 24/26 [00:08<00:00,  2.78it/s, v_num=9, loss/train=31.00]
Epoch 0:  96%|█████████▌| 25/26 [00:08<00:00,  2.86it/s, v_num=9, loss/train=31.00]
Epoch 0:  96%|█████████▌| 25/26 [00:08<00:00,  2.86it/s, v_num=9, loss/train=32.90]
Epoch 0: 100%|██████████| 26/26 [00:08<00:00,  2.98it/s, v_num=9, loss/train=32.90]
Epoch 0: 100%|██████████| 26/26 [00:08<00:00,  2.98it/s, v_num=9, loss/train=29.60]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 125.75it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 172.66it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  6.90it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  9.12it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  7.75it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00,  9.27it/s]


Epoch 0: 100%|██████████| 26/26 [00:13<00:00,  1.88it/s, v_num=9, loss/train=29.60, loss/val=32.50]
Epoch 0: 100%|██████████| 26/26 [00:13<00:00,  1.88it/s, v_num=9, loss/train=29.60, loss/val=32.50]
Epoch 0:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.60, loss/val=32.50]
Epoch 1:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.60, loss/val=32.50]
Epoch 1:   4%|▍         | 1/26 [00:06<02:42,  0.15it/s, v_num=9, loss/train=29.60, loss/val=32.50]
Epoch 1:   4%|▍         | 1/26 [00:06<02:42,  0.15it/s, v_num=9, loss/train=31.80, loss/val=32.50]
Epoch 1:   8%|▊         | 2/26 [00:06<01:22,  0.29it/s, v_num=9, loss/train=31.80, loss/val=32.50]
Epoch 1:   8%|▊         | 2/26 [00:06<01:22,  0.29it/s, v_num=9, loss/train=31.20, loss/val=32.50]
Epoch 1:  12%|█▏        | 3/26 [00:09<01:16,  0.30it/s, v_num=9, loss/train=31.20, loss/val=32.50]
Epoch 1:  12%|█▏        | 3/26 [00:09<01:16,  0.30it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  15%|█▌        | 4/26 [00:10<00:55,  0.40it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  15%|█▌        | 4/26 [00:10<00:55,  0.40it/s, v_num=9, loss/train=34.30, loss/val=32.50]
Epoch 1:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=34.30, loss/val=32.50]
Epoch 1:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=32.80, loss/val=32.50]
Epoch 1:  23%|██▎       | 6/26 [00:10<00:34,  0.58it/s, v_num=9, loss/train=32.80, loss/val=32.50]
Epoch 1:  23%|██▎       | 6/26 [00:10<00:34,  0.58it/s, v_num=9, loss/train=27.40, loss/val=32.50]
Epoch 1:  27%|██▋       | 7/26 [00:10<00:28,  0.67it/s, v_num=9, loss/train=27.40, loss/val=32.50]
Epoch 1:  27%|██▋       | 7/26 [00:10<00:28,  0.67it/s, v_num=9, loss/train=28.10, loss/val=32.50]
Epoch 1:  31%|███       | 8/26 [00:10<00:23,  0.76it/s, v_num=9, loss/train=28.10, loss/val=32.50]
Epoch 1:  31%|███       | 8/26 [00:10<00:23,  0.76it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=31.10, loss/val=32.50]
Epoch 1:  38%|███▊      | 10/26 [00:10<00:17,  0.92it/s, v_num=9, loss/train=31.10, loss/val=32.50]
Epoch 1:  38%|███▊      | 10/26 [00:10<00:17,  0.92it/s, v_num=9, loss/train=31.30, loss/val=32.50]
Epoch 1:  42%|████▏     | 11/26 [00:12<00:16,  0.89it/s, v_num=9, loss/train=31.30, loss/val=32.50]
Epoch 1:  42%|████▏     | 11/26 [00:12<00:16,  0.89it/s, v_num=9, loss/train=30.50, loss/val=32.50]
Epoch 1:  46%|████▌     | 12/26 [00:14<00:16,  0.84it/s, v_num=9, loss/train=30.50, loss/val=32.50]
Epoch 1:  46%|████▌     | 12/26 [00:14<00:16,  0.84it/s, v_num=9, loss/train=30.40, loss/val=32.50]
Epoch 1:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=30.40, loss/val=32.50]
Epoch 1:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=28.60, loss/val=32.50]
Epoch 1:  54%|█████▍    | 14/26 [00:16<00:14,  0.85it/s, v_num=9, loss/train=28.60, loss/val=32.50]
Epoch 1:  54%|█████▍    | 14/26 [00:16<00:14,  0.84it/s, v_num=9, loss/train=30.80, loss/val=32.50]
Epoch 1:  58%|█████▊    | 15/26 [00:17<00:12,  0.88it/s, v_num=9, loss/train=30.80, loss/val=32.50]
Epoch 1:  58%|█████▊    | 15/26 [00:17<00:12,  0.88it/s, v_num=9, loss/train=33.10, loss/val=32.50]
Epoch 1:  62%|██████▏   | 16/26 [00:17<00:10,  0.93it/s, v_num=9, loss/train=33.10, loss/val=32.50]
Epoch 1:  62%|██████▏   | 16/26 [00:17<00:10,  0.93it/s, v_num=9, loss/train=29.40, loss/val=32.50]
Epoch 1:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=29.40, loss/val=32.50]
Epoch 1:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=31.90, loss/val=32.50]
Epoch 1:  69%|██████▉   | 18/26 [00:17<00:07,  1.04it/s, v_num=9, loss/train=31.90, loss/val=32.50]
Epoch 1:  69%|██████▉   | 18/26 [00:17<00:07,  1.04it/s, v_num=9, loss/train=31.00, loss/val=32.50]
Epoch 1:  73%|███████▎  | 19/26 [00:17<00:06,  1.10it/s, v_num=9, loss/train=31.00, loss/val=32.50]
Epoch 1:  73%|███████▎  | 19/26 [00:17<00:06,  1.10it/s, v_num=9, loss/train=34.10, loss/val=32.50]
Epoch 1:  77%|███████▋  | 20/26 [00:17<00:05,  1.16it/s, v_num=9, loss/train=34.10, loss/val=32.50]
Epoch 1:  77%|███████▋  | 20/26 [00:17<00:05,  1.16it/s, v_num=9, loss/train=31.80, loss/val=32.50]
Epoch 1:  81%|████████  | 21/26 [00:17<00:04,  1.21it/s, v_num=9, loss/train=31.80, loss/val=32.50]
Epoch 1:  81%|████████  | 21/26 [00:17<00:04,  1.21it/s, v_num=9, loss/train=27.30, loss/val=32.50]
Epoch 1:  85%|████████▍ | 22/26 [00:18<00:03,  1.22it/s, v_num=9, loss/train=27.30, loss/val=32.50]
Epoch 1:  85%|████████▍ | 22/26 [00:18<00:03,  1.22it/s, v_num=9, loss/train=30.90, loss/val=32.50]
Epoch 1:  88%|████████▊ | 23/26 [00:18<00:02,  1.24it/s, v_num=9, loss/train=30.90, loss/val=32.50]
Epoch 1:  88%|████████▊ | 23/26 [00:18<00:02,  1.24it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=32.20, loss/val=32.50]
Epoch 1:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=30.60, loss/val=32.50]
Epoch 1:  96%|█████████▌| 25/26 [00:18<00:00,  1.32it/s, v_num=9, loss/train=30.60, loss/val=32.50]
Epoch 1:  96%|█████████▌| 25/26 [00:18<00:00,  1.32it/s, v_num=9, loss/train=30.30, loss/val=32.50]
Epoch 1: 100%|██████████| 26/26 [00:18<00:00,  1.37it/s, v_num=9, loss/train=30.30, loss/val=32.50]
Epoch 1: 100%|██████████| 26/26 [00:18<00:00,  1.37it/s, v_num=9, loss/train=37.60, loss/val=32.50]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 79.08it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:01,  3.88it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  5.80it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  7.69it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  9.58it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 11.43it/s]


Epoch 1: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=37.60, loss/val=31.10]
Epoch 1: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=37.60, loss/val=31.10]
Epoch 1:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=37.60, loss/val=31.10]
Epoch 2:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=37.60, loss/val=31.10]
Epoch 2:   4%|▍         | 1/26 [00:10<04:11,  0.10it/s, v_num=9, loss/train=37.60, loss/val=31.10]
Epoch 2:   4%|▍         | 1/26 [00:10<04:11,  0.10it/s, v_num=9, loss/train=33.00, loss/val=31.10]
Epoch 2:   8%|▊         | 2/26 [00:10<02:02,  0.20it/s, v_num=9, loss/train=33.00, loss/val=31.10]
Epoch 2:   8%|▊         | 2/26 [00:10<02:02,  0.20it/s, v_num=9, loss/train=29.20, loss/val=31.10]
Epoch 2:  12%|█▏        | 3/26 [00:11<01:24,  0.27it/s, v_num=9, loss/train=29.20, loss/val=31.10]
Epoch 2:  12%|█▏        | 3/26 [00:11<01:24,  0.27it/s, v_num=9, loss/train=31.30, loss/val=31.10]
Epoch 2:  15%|█▌        | 4/26 [00:11<01:01,  0.36it/s, v_num=9, loss/train=31.30, loss/val=31.10]
Epoch 2:  15%|█▌        | 4/26 [00:11<01:01,  0.36it/s, v_num=9, loss/train=30.30, loss/val=31.10]
Epoch 2:  19%|█▉        | 5/26 [00:11<00:47,  0.45it/s, v_num=9, loss/train=30.30, loss/val=31.10]
Epoch 2:  19%|█▉        | 5/26 [00:11<00:47,  0.45it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  23%|██▎       | 6/26 [00:11<00:37,  0.53it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  23%|██▎       | 6/26 [00:11<00:37,  0.53it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  27%|██▋       | 7/26 [00:11<00:30,  0.62it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  27%|██▋       | 7/26 [00:11<00:30,  0.62it/s, v_num=9, loss/train=33.60, loss/val=31.10]
Epoch 2:  31%|███       | 8/26 [00:11<00:25,  0.70it/s, v_num=9, loss/train=33.60, loss/val=31.10]
Epoch 2:  31%|███       | 8/26 [00:11<00:25,  0.70it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  35%|███▍      | 9/26 [00:11<00:21,  0.78it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  35%|███▍      | 9/26 [00:11<00:21,  0.78it/s, v_num=9, loss/train=28.60, loss/val=31.10]
Epoch 2:  38%|███▊      | 10/26 [00:11<00:18,  0.87it/s, v_num=9, loss/train=28.60, loss/val=31.10]
Epoch 2:  38%|███▊      | 10/26 [00:11<00:18,  0.87it/s, v_num=9, loss/train=33.50, loss/val=31.10]
Epoch 2:  42%|████▏     | 11/26 [00:16<00:22,  0.68it/s, v_num=9, loss/train=33.50, loss/val=31.10]
Epoch 2:  42%|████▏     | 11/26 [00:16<00:22,  0.68it/s, v_num=9, loss/train=29.50, loss/val=31.10]
Epoch 2:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=29.50, loss/val=31.10]
Epoch 2:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=30.20, loss/val=31.10]
Epoch 2:  50%|█████     | 13/26 [00:16<00:16,  0.79it/s, v_num=9, loss/train=30.20, loss/val=31.10]
Epoch 2:  50%|█████     | 13/26 [00:16<00:16,  0.79it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  54%|█████▍    | 14/26 [00:16<00:14,  0.85it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  54%|█████▍    | 14/26 [00:16<00:14,  0.85it/s, v_num=9, loss/train=30.60, loss/val=31.10]
Epoch 2:  58%|█████▊    | 15/26 [00:16<00:12,  0.90it/s, v_num=9, loss/train=30.60, loss/val=31.10]
Epoch 2:  58%|█████▊    | 15/26 [00:16<00:12,  0.90it/s, v_num=9, loss/train=30.10, loss/val=31.10]
Epoch 2:  62%|██████▏   | 16/26 [00:16<00:10,  0.95it/s, v_num=9, loss/train=30.10, loss/val=31.10]
Epoch 2:  62%|██████▏   | 16/26 [00:16<00:10,  0.95it/s, v_num=9, loss/train=33.50, loss/val=31.10]
Epoch 2:  65%|██████▌   | 17/26 [00:16<00:08,  1.01it/s, v_num=9, loss/train=33.50, loss/val=31.10]
Epoch 2:  65%|██████▌   | 17/26 [00:16<00:08,  1.01it/s, v_num=9, loss/train=26.70, loss/val=31.10]
Epoch 2:  69%|██████▉   | 18/26 [00:16<00:07,  1.07it/s, v_num=9, loss/train=26.70, loss/val=31.10]
Epoch 2:  69%|██████▉   | 18/26 [00:16<00:07,  1.07it/s, v_num=9, loss/train=32.90, loss/val=31.10]
Epoch 2:  73%|███████▎  | 19/26 [00:16<00:06,  1.13it/s, v_num=9, loss/train=32.90, loss/val=31.10]
Epoch 2:  73%|███████▎  | 19/26 [00:16<00:06,  1.13it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  77%|███████▋  | 20/26 [00:16<00:05,  1.19it/s, v_num=9, loss/train=31.00, loss/val=31.10]
Epoch 2:  77%|███████▋  | 20/26 [00:16<00:05,  1.19it/s, v_num=9, loss/train=30.10, loss/val=31.10]
Epoch 2:  81%|████████  | 21/26 [00:18<00:04,  1.12it/s, v_num=9, loss/train=30.10, loss/val=31.10]
Epoch 2:  81%|████████  | 21/26 [00:18<00:04,  1.12it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  85%|████████▍ | 22/26 [00:18<00:03,  1.17it/s, v_num=9, loss/train=31.80, loss/val=31.10]
Epoch 2:  85%|████████▍ | 22/26 [00:18<00:03,  1.17it/s, v_num=9, loss/train=31.20, loss/val=31.10]
Epoch 2:  88%|████████▊ | 23/26 [00:19<00:02,  1.19it/s, v_num=9, loss/train=31.20, loss/val=31.10]
Epoch 2:  88%|████████▊ | 23/26 [00:19<00:02,  1.19it/s, v_num=9, loss/train=29.50, loss/val=31.10]
Epoch 2:  92%|█████████▏| 24/26 [00:19<00:01,  1.24it/s, v_num=9, loss/train=29.50, loss/val=31.10]
Epoch 2:  92%|█████████▏| 24/26 [00:19<00:01,  1.24it/s, v_num=9, loss/train=33.30, loss/val=31.10]
Epoch 2:  96%|█████████▌| 25/26 [00:19<00:00,  1.29it/s, v_num=9, loss/train=33.30, loss/val=31.10]
Epoch 2:  96%|█████████▌| 25/26 [00:19<00:00,  1.29it/s, v_num=9, loss/train=29.40, loss/val=31.10]
Epoch 2: 100%|██████████| 26/26 [00:19<00:00,  1.34it/s, v_num=9, loss/train=29.40, loss/val=31.10]
Epoch 2: 100%|██████████| 26/26 [00:19<00:00,  1.34it/s, v_num=9, loss/train=29.50, loss/val=31.10]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 192.14it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 258.29it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 296.91it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 321.67it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 338.00it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 348.74it/s]


Epoch 2: 100%|██████████| 26/26 [00:24<00:00,  1.07it/s, v_num=9, loss/train=29.50, loss/val=29.90]
Epoch 2: 100%|██████████| 26/26 [00:24<00:00,  1.07it/s, v_num=9, loss/train=29.50, loss/val=29.90]
Epoch 2:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.50, loss/val=29.90]
Epoch 3:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.50, loss/val=29.90]
Epoch 3:   4%|▍         | 1/26 [00:06<02:44,  0.15it/s, v_num=9, loss/train=29.50, loss/val=29.90]
Epoch 3:   4%|▍         | 1/26 [00:06<02:44,  0.15it/s, v_num=9, loss/train=31.10, loss/val=29.90]
Epoch 3:   8%|▊         | 2/26 [00:07<01:24,  0.28it/s, v_num=9, loss/train=31.10, loss/val=29.90]
Epoch 3:   8%|▊         | 2/26 [00:07<01:24,  0.28it/s, v_num=9, loss/train=31.30, loss/val=29.90]
Epoch 3:  12%|█▏        | 3/26 [00:09<01:13,  0.31it/s, v_num=9, loss/train=31.30, loss/val=29.90]
Epoch 3:  12%|█▏        | 3/26 [00:09<01:13,  0.31it/s, v_num=9, loss/train=30.30, loss/val=29.90]
Epoch 3:  15%|█▌        | 4/26 [00:09<00:53,  0.41it/s, v_num=9, loss/train=30.30, loss/val=29.90]
Epoch 3:  15%|█▌        | 4/26 [00:09<00:53,  0.41it/s, v_num=9, loss/train=30.60, loss/val=29.90]
Epoch 3:  19%|█▉        | 5/26 [00:10<00:44,  0.47it/s, v_num=9, loss/train=30.60, loss/val=29.90]
Epoch 3:  19%|█▉        | 5/26 [00:10<00:44,  0.47it/s, v_num=9, loss/train=31.60, loss/val=29.90]
Epoch 3:  23%|██▎       | 6/26 [00:10<00:35,  0.57it/s, v_num=9, loss/train=31.60, loss/val=29.90]
Epoch 3:  23%|██▎       | 6/26 [00:10<00:35,  0.57it/s, v_num=9, loss/train=31.10, loss/val=29.90]
Epoch 3:  27%|██▋       | 7/26 [00:10<00:28,  0.66it/s, v_num=9, loss/train=31.10, loss/val=29.90]
Epoch 3:  27%|██▋       | 7/26 [00:10<00:28,  0.66it/s, v_num=9, loss/train=32.50, loss/val=29.90]
Epoch 3:  31%|███       | 8/26 [00:10<00:24,  0.75it/s, v_num=9, loss/train=32.50, loss/val=29.90]
Epoch 3:  31%|███       | 8/26 [00:10<00:24,  0.75it/s, v_num=9, loss/train=33.40, loss/val=29.90]
Epoch 3:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=33.40, loss/val=29.90]
Epoch 3:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=30.30, loss/val=29.90]
Epoch 3:  38%|███▊      | 10/26 [00:10<00:17,  0.93it/s, v_num=9, loss/train=30.30, loss/val=29.90]
Epoch 3:  38%|███▊      | 10/26 [00:10<00:17,  0.92it/s, v_num=9, loss/train=32.20, loss/val=29.90]
Epoch 3:  42%|████▏     | 11/26 [00:12<00:17,  0.86it/s, v_num=9, loss/train=32.20, loss/val=29.90]
Epoch 3:  42%|████▏     | 11/26 [00:12<00:17,  0.86it/s, v_num=9, loss/train=30.10, loss/val=29.90]
Epoch 3:  46%|████▌     | 12/26 [00:14<00:16,  0.83it/s, v_num=9, loss/train=30.10, loss/val=29.90]
Epoch 3:  46%|████▌     | 12/26 [00:14<00:16,  0.83it/s, v_num=9, loss/train=35.20, loss/val=29.90]
Epoch 3:  50%|█████     | 13/26 [00:15<00:15,  0.83it/s, v_num=9, loss/train=35.20, loss/val=29.90]
Epoch 3:  50%|█████     | 13/26 [00:15<00:15,  0.83it/s, v_num=9, loss/train=34.90, loss/val=29.90]
Epoch 3:  54%|█████▍    | 14/26 [00:16<00:13,  0.87it/s, v_num=9, loss/train=34.90, loss/val=29.90]
Epoch 3:  54%|█████▍    | 14/26 [00:16<00:13,  0.87it/s, v_num=9, loss/train=33.00, loss/val=29.90]
Epoch 3:  58%|█████▊    | 15/26 [00:16<00:12,  0.90it/s, v_num=9, loss/train=33.00, loss/val=29.90]
Epoch 3:  58%|█████▊    | 15/26 [00:16<00:12,  0.90it/s, v_num=9, loss/train=32.70, loss/val=29.90]
Epoch 3:  62%|██████▏   | 16/26 [00:16<00:10,  0.95it/s, v_num=9, loss/train=32.70, loss/val=29.90]
Epoch 3:  62%|██████▏   | 16/26 [00:16<00:10,  0.95it/s, v_num=9, loss/train=28.10, loss/val=29.90]
Epoch 3:  65%|██████▌   | 17/26 [00:16<00:08,  1.01it/s, v_num=9, loss/train=28.10, loss/val=29.90]
Epoch 3:  65%|██████▌   | 17/26 [00:16<00:08,  1.01it/s, v_num=9, loss/train=33.80, loss/val=29.90]
Epoch 3:  69%|██████▉   | 18/26 [00:16<00:07,  1.07it/s, v_num=9, loss/train=33.80, loss/val=29.90]
Epoch 3:  69%|██████▉   | 18/26 [00:16<00:07,  1.07it/s, v_num=9, loss/train=34.50, loss/val=29.90]
Epoch 3:  73%|███████▎  | 19/26 [00:16<00:06,  1.13it/s, v_num=9, loss/train=34.50, loss/val=29.90]
Epoch 3:  73%|███████▎  | 19/26 [00:16<00:06,  1.13it/s, v_num=9, loss/train=32.50, loss/val=29.90]
Epoch 3:  77%|███████▋  | 20/26 [00:16<00:05,  1.19it/s, v_num=9, loss/train=32.50, loss/val=29.90]
Epoch 3:  77%|███████▋  | 20/26 [00:16<00:05,  1.19it/s, v_num=9, loss/train=28.40, loss/val=29.90]
Epoch 3:  81%|████████  | 21/26 [00:16<00:04,  1.25it/s, v_num=9, loss/train=28.40, loss/val=29.90]
Epoch 3:  81%|████████  | 21/26 [00:16<00:04,  1.25it/s, v_num=9, loss/train=31.70, loss/val=29.90]
Epoch 3:  85%|████████▍ | 22/26 [00:18<00:03,  1.20it/s, v_num=9, loss/train=31.70, loss/val=29.90]
Epoch 3:  85%|████████▍ | 22/26 [00:18<00:03,  1.20it/s, v_num=9, loss/train=25.50, loss/val=29.90]
Epoch 3:  88%|████████▊ | 23/26 [00:18<00:02,  1.25it/s, v_num=9, loss/train=25.50, loss/val=29.90]
Epoch 3:  88%|████████▊ | 23/26 [00:18<00:02,  1.25it/s, v_num=9, loss/train=30.50, loss/val=29.90]
Epoch 3:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=30.50, loss/val=29.90]
Epoch 3:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=30.40, loss/val=29.90]
Epoch 3:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=30.40, loss/val=29.90]
Epoch 3:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=28.70, loss/val=29.90]
Epoch 3: 100%|██████████| 26/26 [00:19<00:00,  1.37it/s, v_num=9, loss/train=28.70, loss/val=29.90]
Epoch 3: 100%|██████████| 26/26 [00:19<00:00,  1.37it/s, v_num=9, loss/train=30.20, loss/val=29.90]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 59.79it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 15.41it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  8.43it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 11.16it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 13.38it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 15.84it/s]


Epoch 3: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=30.20, loss/val=30.40]
Epoch 3: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=30.20, loss/val=30.40]
Epoch 3:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.20, loss/val=30.40]
Epoch 4:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.20, loss/val=30.40]
Epoch 4:   4%|▍         | 1/26 [00:10<04:31,  0.09it/s, v_num=9, loss/train=30.20, loss/val=30.40]
Epoch 4:   4%|▍         | 1/26 [00:10<04:31,  0.09it/s, v_num=9, loss/train=31.60, loss/val=30.40]
Epoch 4:   8%|▊         | 2/26 [00:10<02:11,  0.18it/s, v_num=9, loss/train=31.60, loss/val=30.40]
Epoch 4:   8%|▊         | 2/26 [00:10<02:11,  0.18it/s, v_num=9, loss/train=30.00, loss/val=30.40]
Epoch 4:  12%|█▏        | 3/26 [00:11<01:24,  0.27it/s, v_num=9, loss/train=30.00, loss/val=30.40]
Epoch 4:  12%|█▏        | 3/26 [00:11<01:24,  0.27it/s, v_num=9, loss/train=29.70, loss/val=30.40]
Epoch 4:  15%|█▌        | 4/26 [00:11<01:01,  0.36it/s, v_num=9, loss/train=29.70, loss/val=30.40]
Epoch 4:  15%|█▌        | 4/26 [00:11<01:01,  0.36it/s, v_num=9, loss/train=30.30, loss/val=30.40]
Epoch 4:  19%|█▉        | 5/26 [00:11<00:46,  0.45it/s, v_num=9, loss/train=30.30, loss/val=30.40]
Epoch 4:  19%|█▉        | 5/26 [00:11<00:46,  0.45it/s, v_num=9, loss/train=32.50, loss/val=30.40]
Epoch 4:  23%|██▎       | 6/26 [00:11<00:37,  0.53it/s, v_num=9, loss/train=32.50, loss/val=30.40]
Epoch 4:  23%|██▎       | 6/26 [00:11<00:37,  0.53it/s, v_num=9, loss/train=31.50, loss/val=30.40]
Epoch 4:  27%|██▋       | 7/26 [00:11<00:30,  0.62it/s, v_num=9, loss/train=31.50, loss/val=30.40]
Epoch 4:  27%|██▋       | 7/26 [00:11<00:30,  0.62it/s, v_num=9, loss/train=29.10, loss/val=30.40]
Epoch 4:  31%|███       | 8/26 [00:11<00:25,  0.71it/s, v_num=9, loss/train=29.10, loss/val=30.40]
Epoch 4:  31%|███       | 8/26 [00:11<00:25,  0.71it/s, v_num=9, loss/train=31.90, loss/val=30.40]
Epoch 4:  35%|███▍      | 9/26 [00:11<00:21,  0.79it/s, v_num=9, loss/train=31.90, loss/val=30.40]
Epoch 4:  35%|███▍      | 9/26 [00:11<00:21,  0.79it/s, v_num=9, loss/train=27.10, loss/val=30.40]
Epoch 4:  38%|███▊      | 10/26 [00:11<00:18,  0.87it/s, v_num=9, loss/train=27.10, loss/val=30.40]
Epoch 4:  38%|███▊      | 10/26 [00:11<00:18,  0.87it/s, v_num=9, loss/train=31.90, loss/val=30.40]
Epoch 4:  42%|████▏     | 11/26 [00:16<00:21,  0.68it/s, v_num=9, loss/train=31.90, loss/val=30.40]
Epoch 4:  42%|████▏     | 11/26 [00:16<00:21,  0.68it/s, v_num=9, loss/train=33.80, loss/val=30.40]
Epoch 4:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=33.80, loss/val=30.40]
Epoch 4:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=28.00, loss/val=30.40]
Epoch 4:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=28.00, loss/val=30.40]
Epoch 4:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=32.30, loss/val=30.40]
Epoch 4:  54%|█████▍    | 14/26 [00:16<00:13,  0.86it/s, v_num=9, loss/train=32.30, loss/val=30.40]
Epoch 4:  54%|█████▍    | 14/26 [00:16<00:13,  0.86it/s, v_num=9, loss/train=31.60, loss/val=30.40]
Epoch 4:  58%|█████▊    | 15/26 [00:16<00:11,  0.92it/s, v_num=9, loss/train=31.60, loss/val=30.40]
Epoch 4:  58%|█████▊    | 15/26 [00:16<00:11,  0.92it/s, v_num=9, loss/train=31.10, loss/val=30.40]
Epoch 4:  62%|██████▏   | 16/26 [00:16<00:10,  0.98it/s, v_num=9, loss/train=31.10, loss/val=30.40]
Epoch 4:  62%|██████▏   | 16/26 [00:16<00:10,  0.98it/s, v_num=9, loss/train=30.80, loss/val=30.40]
Epoch 4:  65%|██████▌   | 17/26 [00:17<00:09,  0.97it/s, v_num=9, loss/train=30.80, loss/val=30.40]
Epoch 4:  65%|██████▌   | 17/26 [00:17<00:09,  0.97it/s, v_num=9, loss/train=31.70, loss/val=30.40]
Epoch 4:  69%|██████▉   | 18/26 [00:17<00:07,  1.02it/s, v_num=9, loss/train=31.70, loss/val=30.40]
Epoch 4:  69%|██████▉   | 18/26 [00:17<00:07,  1.02it/s, v_num=9, loss/train=32.30, loss/val=30.40]
Epoch 4:  73%|███████▎  | 19/26 [00:17<00:06,  1.08it/s, v_num=9, loss/train=32.30, loss/val=30.40]
Epoch 4:  73%|███████▎  | 19/26 [00:17<00:06,  1.08it/s, v_num=9, loss/train=26.90, loss/val=30.40]
Epoch 4:  77%|███████▋  | 20/26 [00:17<00:05,  1.13it/s, v_num=9, loss/train=26.90, loss/val=30.40]
Epoch 4:  77%|███████▋  | 20/26 [00:17<00:05,  1.13it/s, v_num=9, loss/train=35.50, loss/val=30.40]
Epoch 4:  81%|████████  | 21/26 [00:19<00:04,  1.09it/s, v_num=9, loss/train=35.50, loss/val=30.40]
Epoch 4:  81%|████████  | 21/26 [00:19<00:04,  1.09it/s, v_num=9, loss/train=30.40, loss/val=30.40]
Epoch 4:  85%|████████▍ | 22/26 [00:19<00:03,  1.14it/s, v_num=9, loss/train=30.40, loss/val=30.40]
Epoch 4:  85%|████████▍ | 22/26 [00:19<00:03,  1.14it/s, v_num=9, loss/train=31.50, loss/val=30.40]
Epoch 4:  88%|████████▊ | 23/26 [00:19<00:02,  1.19it/s, v_num=9, loss/train=31.50, loss/val=30.40]
Epoch 4:  88%|████████▊ | 23/26 [00:19<00:02,  1.19it/s, v_num=9, loss/train=30.90, loss/val=30.40]
Epoch 4:  92%|█████████▏| 24/26 [00:19<00:01,  1.24it/s, v_num=9, loss/train=30.90, loss/val=30.40]
Epoch 4:  92%|█████████▏| 24/26 [00:19<00:01,  1.24it/s, v_num=9, loss/train=30.30, loss/val=30.40]
Epoch 4:  96%|█████████▌| 25/26 [00:19<00:00,  1.29it/s, v_num=9, loss/train=30.30, loss/val=30.40]
Epoch 4:  96%|█████████▌| 25/26 [00:19<00:00,  1.29it/s, v_num=9, loss/train=32.20, loss/val=30.40]
Epoch 4: 100%|██████████| 26/26 [00:19<00:00,  1.34it/s, v_num=9, loss/train=32.20, loss/val=30.40]
Epoch 4: 100%|██████████| 26/26 [00:19<00:00,  1.34it/s, v_num=9, loss/train=34.30, loss/val=30.40]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 117.74it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:01,  3.58it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  5.35it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  7.11it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  8.85it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 10.58it/s]


Epoch 4: 100%|██████████| 26/26 [00:24<00:00,  1.08it/s, v_num=9, loss/train=34.30, loss/val=31.40]
Epoch 4: 100%|██████████| 26/26 [00:24<00:00,  1.08it/s, v_num=9, loss/train=34.30, loss/val=31.40]
Epoch 4:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=34.30, loss/val=31.40]
Epoch 5:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=34.30, loss/val=31.40]
Epoch 5:   4%|▍         | 1/26 [00:07<03:13,  0.13it/s, v_num=9, loss/train=34.30, loss/val=31.40]
Epoch 5:   4%|▍         | 1/26 [00:07<03:13,  0.13it/s, v_num=9, loss/train=31.10, loss/val=31.40]
Epoch 5:   8%|▊         | 2/26 [00:10<02:03,  0.19it/s, v_num=9, loss/train=31.10, loss/val=31.40]
Epoch 5:   8%|▊         | 2/26 [00:10<02:04,  0.19it/s, v_num=9, loss/train=32.10, loss/val=31.40]
Epoch 5:  12%|█▏        | 3/26 [00:10<01:19,  0.29it/s, v_num=9, loss/train=32.10, loss/val=31.40]
Epoch 5:  12%|█▏        | 3/26 [00:10<01:19,  0.29it/s, v_num=9, loss/train=29.80, loss/val=31.40]
Epoch 5:  15%|█▌        | 4/26 [00:10<00:57,  0.38it/s, v_num=9, loss/train=29.80, loss/val=31.40]
Epoch 5:  15%|█▌        | 4/26 [00:10<00:57,  0.38it/s, v_num=9, loss/train=31.70, loss/val=31.40]
Epoch 5:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=31.70, loss/val=31.40]
Epoch 5:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=29.60, loss/val=31.40]
Epoch 5:  23%|██▎       | 6/26 [00:10<00:34,  0.57it/s, v_num=9, loss/train=29.60, loss/val=31.40]
Epoch 5:  23%|██▎       | 6/26 [00:10<00:34,  0.57it/s, v_num=9, loss/train=31.90, loss/val=31.40]
Epoch 5:  27%|██▋       | 7/26 [00:10<00:28,  0.67it/s, v_num=9, loss/train=31.90, loss/val=31.40]
Epoch 5:  27%|██▋       | 7/26 [00:10<00:28,  0.66it/s, v_num=9, loss/train=33.50, loss/val=31.40]
Epoch 5:  31%|███       | 8/26 [00:10<00:23,  0.76it/s, v_num=9, loss/train=33.50, loss/val=31.40]
Epoch 5:  31%|███       | 8/26 [00:10<00:23,  0.76it/s, v_num=9, loss/train=30.70, loss/val=31.40]
Epoch 5:  35%|███▍      | 9/26 [00:10<00:19,  0.85it/s, v_num=9, loss/train=30.70, loss/val=31.40]
Epoch 5:  35%|███▍      | 9/26 [00:10<00:20,  0.85it/s, v_num=9, loss/train=30.60, loss/val=31.40]
Epoch 5:  38%|███▊      | 10/26 [00:10<00:17,  0.94it/s, v_num=9, loss/train=30.60, loss/val=31.40]
Epoch 5:  38%|███▊      | 10/26 [00:10<00:17,  0.94it/s, v_num=9, loss/train=34.50, loss/val=31.40]
Epoch 5:  42%|████▏     | 11/26 [00:13<00:17,  0.84it/s, v_num=9, loss/train=34.50, loss/val=31.40]
Epoch 5:  42%|████▏     | 11/26 [00:13<00:17,  0.84it/s, v_num=9, loss/train=31.90, loss/val=31.40]
Epoch 5:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=31.90, loss/val=31.40]
Epoch 5:  46%|████▌     | 12/26 [00:16<00:18,  0.74it/s, v_num=9, loss/train=32.60, loss/val=31.40]
Epoch 5:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=32.60, loss/val=31.40]
Epoch 5:  50%|█████     | 13/26 [00:16<00:16,  0.80it/s, v_num=9, loss/train=32.20, loss/val=31.40]
Epoch 5:  54%|█████▍    | 14/26 [00:16<00:14,  0.86it/s, v_num=9, loss/train=32.20, loss/val=31.40]
Epoch 5:  54%|█████▍    | 14/26 [00:16<00:14,  0.86it/s, v_num=9, loss/train=31.00, loss/val=31.40]
Epoch 5:  58%|█████▊    | 15/26 [00:16<00:12,  0.92it/s, v_num=9, loss/train=31.00, loss/val=31.40]
Epoch 5:  58%|█████▊    | 15/26 [00:16<00:12,  0.92it/s, v_num=9, loss/train=28.10, loss/val=31.40]
Epoch 5:  62%|██████▏   | 16/26 [00:16<00:10,  0.97it/s, v_num=9, loss/train=28.10, loss/val=31.40]
Epoch 5:  62%|██████▏   | 16/26 [00:16<00:10,  0.97it/s, v_num=9, loss/train=33.70, loss/val=31.40]
Epoch 5:  65%|██████▌   | 17/26 [00:16<00:08,  1.03it/s, v_num=9, loss/train=33.70, loss/val=31.40]
Epoch 5:  65%|██████▌   | 17/26 [00:16<00:08,  1.03it/s, v_num=9, loss/train=29.10, loss/val=31.40]
Epoch 5:  69%|██████▉   | 18/26 [00:16<00:07,  1.09it/s, v_num=9, loss/train=29.10, loss/val=31.40]
Epoch 5:  69%|██████▉   | 18/26 [00:16<00:07,  1.09it/s, v_num=9, loss/train=29.30, loss/val=31.40]
Epoch 5:  73%|███████▎  | 19/26 [00:16<00:06,  1.15it/s, v_num=9, loss/train=29.30, loss/val=31.40]
Epoch 5:  73%|███████▎  | 19/26 [00:16<00:06,  1.15it/s, v_num=9, loss/train=33.90, loss/val=31.40]
Epoch 5:  77%|███████▋  | 20/26 [00:16<00:04,  1.21it/s, v_num=9, loss/train=33.90, loss/val=31.40]
Epoch 5:  77%|███████▋  | 20/26 [00:16<00:04,  1.21it/s, v_num=9, loss/train=28.70, loss/val=31.40]
Epoch 5:  81%|████████  | 21/26 [00:17<00:04,  1.19it/s, v_num=9, loss/train=28.70, loss/val=31.40]
Epoch 5:  81%|████████  | 21/26 [00:17<00:04,  1.19it/s, v_num=9, loss/train=30.20, loss/val=31.40]
Epoch 5:  85%|████████▍ | 22/26 [00:18<00:03,  1.17it/s, v_num=9, loss/train=30.20, loss/val=31.40]
Epoch 5:  85%|████████▍ | 22/26 [00:18<00:03,  1.17it/s, v_num=9, loss/train=32.30, loss/val=31.40]
Epoch 5:  88%|████████▊ | 23/26 [00:18<00:02,  1.22it/s, v_num=9, loss/train=32.30, loss/val=31.40]
Epoch 5:  88%|████████▊ | 23/26 [00:18<00:02,  1.22it/s, v_num=9, loss/train=31.30, loss/val=31.40]
Epoch 5:  92%|█████████▏| 24/26 [00:19<00:01,  1.26it/s, v_num=9, loss/train=31.30, loss/val=31.40]
Epoch 5:  92%|█████████▏| 24/26 [00:19<00:01,  1.26it/s, v_num=9, loss/train=29.90, loss/val=31.40]
Epoch 5:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=29.90, loss/val=31.40]
Epoch 5:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=29.50, loss/val=31.40]
Epoch 5: 100%|██████████| 26/26 [00:19<00:00,  1.36it/s, v_num=9, loss/train=29.50, loss/val=31.40]
Epoch 5: 100%|██████████| 26/26 [00:19<00:00,  1.36it/s, v_num=9, loss/train=39.50, loss/val=31.40]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 104.38it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:01<00:02,  1.50it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:01<00:01,  2.20it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:01<00:00,  2.93it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:01<00:00,  3.53it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:01<00:00,  4.23it/s]


Epoch 5: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=39.50, loss/val=30.30]
Epoch 5: 100%|██████████| 26/26 [00:23<00:00,  1.09it/s, v_num=9, loss/train=39.50, loss/val=30.30]
Epoch 5:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=39.50, loss/val=30.30]
Epoch 6:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=39.50, loss/val=30.30]
Epoch 6:   4%|▍         | 1/26 [00:07<03:09,  0.13it/s, v_num=9, loss/train=39.50, loss/val=30.30]
Epoch 6:   4%|▍         | 1/26 [00:07<03:09,  0.13it/s, v_num=9, loss/train=33.50, loss/val=30.30]
Epoch 6:   8%|▊         | 2/26 [00:08<01:46,  0.22it/s, v_num=9, loss/train=33.50, loss/val=30.30]
Epoch 6:   8%|▊         | 2/26 [00:08<01:46,  0.22it/s, v_num=9, loss/train=29.20, loss/val=30.30]
Epoch 6:  12%|█▏        | 3/26 [00:09<01:10,  0.33it/s, v_num=9, loss/train=29.20, loss/val=30.30]
Epoch 6:  12%|█▏        | 3/26 [00:09<01:10,  0.33it/s, v_num=9, loss/train=32.20, loss/val=30.30]
Epoch 6:  15%|█▌        | 4/26 [00:09<00:50,  0.43it/s, v_num=9, loss/train=32.20, loss/val=30.30]
Epoch 6:  15%|█▌        | 4/26 [00:09<00:50,  0.43it/s, v_num=9, loss/train=32.20, loss/val=30.30]
Epoch 6:  19%|█▉        | 5/26 [00:09<00:39,  0.53it/s, v_num=9, loss/train=32.20, loss/val=30.30]
Epoch 6:  19%|█▉        | 5/26 [00:09<00:39,  0.53it/s, v_num=9, loss/train=33.60, loss/val=30.30]
Epoch 6:  23%|██▎       | 6/26 [00:09<00:31,  0.64it/s, v_num=9, loss/train=33.60, loss/val=30.30]
Epoch 6:  23%|██▎       | 6/26 [00:09<00:31,  0.64it/s, v_num=9, loss/train=32.50, loss/val=30.30]
Epoch 6:  27%|██▋       | 7/26 [00:10<00:29,  0.65it/s, v_num=9, loss/train=32.50, loss/val=30.30]
Epoch 6:  27%|██▋       | 7/26 [00:10<00:29,  0.65it/s, v_num=9, loss/train=29.80, loss/val=30.30]
Epoch 6:  31%|███       | 8/26 [00:10<00:24,  0.74it/s, v_num=9, loss/train=29.80, loss/val=30.30]
Epoch 6:  31%|███       | 8/26 [00:10<00:24,  0.74it/s, v_num=9, loss/train=30.30, loss/val=30.30]
Epoch 6:  35%|███▍      | 9/26 [00:10<00:20,  0.82it/s, v_num=9, loss/train=30.30, loss/val=30.30]
Epoch 6:  35%|███▍      | 9/26 [00:10<00:20,  0.82it/s, v_num=9, loss/train=32.30, loss/val=30.30]
Epoch 6:  38%|███▊      | 10/26 [00:10<00:17,  0.91it/s, v_num=9, loss/train=32.30, loss/val=30.30]
Epoch 6:  38%|███▊      | 10/26 [00:10<00:17,  0.91it/s, v_num=9, loss/train=31.60, loss/val=30.30]
Epoch 6:  42%|████▏     | 11/26 [00:14<00:20,  0.74it/s, v_num=9, loss/train=31.60, loss/val=30.30]
Epoch 6:  42%|████▏     | 11/26 [00:14<00:20,  0.74it/s, v_num=9, loss/train=28.90, loss/val=30.30]
Epoch 6:  46%|████▌     | 12/26 [00:15<00:17,  0.80it/s, v_num=9, loss/train=28.90, loss/val=30.30]
Epoch 6:  46%|████▌     | 12/26 [00:15<00:17,  0.80it/s, v_num=9, loss/train=29.00, loss/val=30.30]
Epoch 6:  50%|█████     | 13/26 [00:15<00:15,  0.83it/s, v_num=9, loss/train=29.00, loss/val=30.30]
Epoch 6:  50%|█████     | 13/26 [00:15<00:15,  0.83it/s, v_num=9, loss/train=31.40, loss/val=30.30]
Epoch 6:  54%|█████▍    | 14/26 [00:15<00:13,  0.89it/s, v_num=9, loss/train=31.40, loss/val=30.30]
Epoch 6:  54%|█████▍    | 14/26 [00:15<00:13,  0.89it/s, v_num=9, loss/train=31.90, loss/val=30.30]
Epoch 6:  58%|█████▊    | 15/26 [00:15<00:11,  0.94it/s, v_num=9, loss/train=31.90, loss/val=30.30]
Epoch 6:  58%|█████▊    | 15/26 [00:15<00:11,  0.94it/s, v_num=9, loss/train=29.30, loss/val=30.30]
Epoch 6:  62%|██████▏   | 16/26 [00:15<00:09,  1.00it/s, v_num=9, loss/train=29.30, loss/val=30.30]
Epoch 6:  62%|██████▏   | 16/26 [00:15<00:09,  1.00it/s, v_num=9, loss/train=30.80, loss/val=30.30]
Epoch 6:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=30.80, loss/val=30.30]
Epoch 6:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=31.20, loss/val=30.30]
Epoch 6:  69%|██████▉   | 18/26 [00:17<00:07,  1.05it/s, v_num=9, loss/train=31.20, loss/val=30.30]
Epoch 6:  69%|██████▉   | 18/26 [00:17<00:07,  1.05it/s, v_num=9, loss/train=33.10, loss/val=30.30]
Epoch 6:  73%|███████▎  | 19/26 [00:17<00:06,  1.11it/s, v_num=9, loss/train=33.10, loss/val=30.30]
Epoch 6:  73%|███████▎  | 19/26 [00:17<00:06,  1.11it/s, v_num=9, loss/train=31.00, loss/val=30.30]
Epoch 6:  77%|███████▋  | 20/26 [00:17<00:05,  1.16it/s, v_num=9, loss/train=31.00, loss/val=30.30]
Epoch 6:  77%|███████▋  | 20/26 [00:17<00:05,  1.16it/s, v_num=9, loss/train=28.10, loss/val=30.30]
Epoch 6:  81%|████████  | 21/26 [00:18<00:04,  1.12it/s, v_num=9, loss/train=28.10, loss/val=30.30]
Epoch 6:  81%|████████  | 21/26 [00:18<00:04,  1.12it/s, v_num=9, loss/train=28.10, loss/val=30.30]
Epoch 6:  85%|████████▍ | 22/26 [00:18<00:03,  1.18it/s, v_num=9, loss/train=28.10, loss/val=30.30]
Epoch 6:  85%|████████▍ | 22/26 [00:18<00:03,  1.18it/s, v_num=9, loss/train=32.90, loss/val=30.30]
Epoch 6:  88%|████████▊ | 23/26 [00:19<00:02,  1.20it/s, v_num=9, loss/train=32.90, loss/val=30.30]
Epoch 6:  88%|████████▊ | 23/26 [00:19<00:02,  1.20it/s, v_num=9, loss/train=30.00, loss/val=30.30]
Epoch 6:  92%|█████████▏| 24/26 [00:19<00:01,  1.26it/s, v_num=9, loss/train=30.00, loss/val=30.30]
Epoch 6:  92%|█████████▏| 24/26 [00:19<00:01,  1.26it/s, v_num=9, loss/train=31.90, loss/val=30.30]
Epoch 6:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=31.90, loss/val=30.30]
Epoch 6:  96%|█████████▌| 25/26 [00:19<00:00,  1.31it/s, v_num=9, loss/train=31.50, loss/val=30.30]
Epoch 6: 100%|██████████| 26/26 [00:19<00:00,  1.36it/s, v_num=9, loss/train=31.50, loss/val=30.30]
Epoch 6: 100%|██████████| 26/26 [00:19<00:00,  1.36it/s, v_num=9, loss/train=30.70, loss/val=30.30]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 192.66it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 249.06it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 288.91it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 316.29it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 337.43it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 347.97it/s]


Epoch 6: 100%|██████████| 26/26 [00:23<00:00,  1.08it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 6: 100%|██████████| 26/26 [00:23<00:00,  1.08it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 6:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 7:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 7:   4%|▍         | 1/26 [00:06<02:53,  0.14it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 7:   4%|▍         | 1/26 [00:06<02:53,  0.14it/s, v_num=9, loss/train=31.90, loss/val=31.70]
Epoch 7:   8%|▊         | 2/26 [00:08<01:38,  0.24it/s, v_num=9, loss/train=31.90, loss/val=31.70]
Epoch 7:   8%|▊         | 2/26 [00:08<01:38,  0.24it/s, v_num=9, loss/train=31.50, loss/val=31.70]
Epoch 7:  12%|█▏        | 3/26 [00:09<01:11,  0.32it/s, v_num=9, loss/train=31.50, loss/val=31.70]
Epoch 7:  12%|█▏        | 3/26 [00:09<01:11,  0.32it/s, v_num=9, loss/train=30.90, loss/val=31.70]
Epoch 7:  15%|█▌        | 4/26 [00:09<00:51,  0.42it/s, v_num=9, loss/train=30.90, loss/val=31.70]
Epoch 7:  15%|█▌        | 4/26 [00:09<00:51,  0.42it/s, v_num=9, loss/train=29.90, loss/val=31.70]
Epoch 7:  19%|█▉        | 5/26 [00:09<00:39,  0.53it/s, v_num=9, loss/train=29.90, loss/val=31.70]
Epoch 7:  19%|█▉        | 5/26 [00:09<00:39,  0.53it/s, v_num=9, loss/train=30.30, loss/val=31.70]
Epoch 7:  23%|██▎       | 6/26 [00:09<00:32,  0.62it/s, v_num=9, loss/train=30.30, loss/val=31.70]
Epoch 7:  23%|██▎       | 6/26 [00:09<00:32,  0.62it/s, v_num=9, loss/train=32.00, loss/val=31.70]
Epoch 7:  27%|██▋       | 7/26 [00:09<00:26,  0.72it/s, v_num=9, loss/train=32.00, loss/val=31.70]
Epoch 7:  27%|██▋       | 7/26 [00:09<00:26,  0.72it/s, v_num=9, loss/train=32.10, loss/val=31.70]
Epoch 7:  31%|███       | 8/26 [00:09<00:22,  0.80it/s, v_num=9, loss/train=32.10, loss/val=31.70]
Epoch 7:  31%|███       | 8/26 [00:09<00:22,  0.80it/s, v_num=9, loss/train=28.30, loss/val=31.70]
Epoch 7:  35%|███▍      | 9/26 [00:10<00:18,  0.90it/s, v_num=9, loss/train=28.30, loss/val=31.70]
Epoch 7:  35%|███▍      | 9/26 [00:10<00:19,  0.89it/s, v_num=9, loss/train=34.00, loss/val=31.70]
Epoch 7:  38%|███▊      | 10/26 [00:10<00:16,  0.99it/s, v_num=9, loss/train=34.00, loss/val=31.70]
Epoch 7:  38%|███▊      | 10/26 [00:10<00:16,  0.99it/s, v_num=9, loss/train=33.50, loss/val=31.70]
Epoch 7:  42%|████▏     | 11/26 [00:13<00:19,  0.79it/s, v_num=9, loss/train=33.50, loss/val=31.70]
Epoch 7:  42%|████▏     | 11/26 [00:13<00:19,  0.79it/s, v_num=9, loss/train=30.00, loss/val=31.70]
Epoch 7:  46%|████▌     | 12/26 [00:13<00:16,  0.86it/s, v_num=9, loss/train=30.00, loss/val=31.70]
Epoch 7:  46%|████▌     | 12/26 [00:13<00:16,  0.86it/s, v_num=9, loss/train=31.40, loss/val=31.70]
Epoch 7:  50%|█████     | 13/26 [00:14<00:14,  0.87it/s, v_num=9, loss/train=31.40, loss/val=31.70]
Epoch 7:  50%|█████     | 13/26 [00:14<00:14,  0.87it/s, v_num=9, loss/train=33.70, loss/val=31.70]
Epoch 7:  54%|█████▍    | 14/26 [00:16<00:13,  0.87it/s, v_num=9, loss/train=33.70, loss/val=31.70]
Epoch 7:  54%|█████▍    | 14/26 [00:16<00:13,  0.87it/s, v_num=9, loss/train=29.50, loss/val=31.70]
Epoch 7:  58%|█████▊    | 15/26 [00:16<00:11,  0.93it/s, v_num=9, loss/train=29.50, loss/val=31.70]
Epoch 7:  58%|█████▊    | 15/26 [00:16<00:11,  0.93it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 7:  62%|██████▏   | 16/26 [00:16<00:10,  0.99it/s, v_num=9, loss/train=30.70, loss/val=31.70]
Epoch 7:  62%|██████▏   | 16/26 [00:16<00:10,  0.99it/s, v_num=9, loss/train=30.10, loss/val=31.70]
Epoch 7:  65%|██████▌   | 17/26 [00:16<00:08,  1.05it/s, v_num=9, loss/train=30.10, loss/val=31.70]
Epoch 7:  65%|██████▌   | 17/26 [00:16<00:08,  1.05it/s, v_num=9, loss/train=28.10, loss/val=31.70]
Epoch 7:  69%|██████▉   | 18/26 [00:16<00:07,  1.11it/s, v_num=9, loss/train=28.10, loss/val=31.70]
Epoch 7:  69%|██████▉   | 18/26 [00:16<00:07,  1.11it/s, v_num=9, loss/train=31.30, loss/val=31.70]
Epoch 7:  73%|███████▎  | 19/26 [00:16<00:06,  1.17it/s, v_num=9, loss/train=31.30, loss/val=31.70]
Epoch 7:  73%|███████▎  | 19/26 [00:16<00:06,  1.17it/s, v_num=9, loss/train=29.60, loss/val=31.70]
Epoch 7:  77%|███████▋  | 20/26 [00:16<00:04,  1.22it/s, v_num=9, loss/train=29.60, loss/val=31.70]
Epoch 7:  77%|███████▋  | 20/26 [00:16<00:04,  1.22it/s, v_num=9, loss/train=33.60, loss/val=31.70]
Epoch 7:  81%|████████  | 21/26 [00:18<00:04,  1.14it/s, v_num=9, loss/train=33.60, loss/val=31.70]
Epoch 7:  81%|████████  | 21/26 [00:18<00:04,  1.14it/s, v_num=9, loss/train=31.90, loss/val=31.70]
Epoch 7:  85%|████████▍ | 22/26 [00:18<00:03,  1.20it/s, v_num=9, loss/train=31.90, loss/val=31.70]
Epoch 7:  85%|████████▍ | 22/26 [00:18<00:03,  1.20it/s, v_num=9, loss/train=27.00, loss/val=31.70]
Epoch 7:  88%|████████▊ | 23/26 [00:18<00:02,  1.22it/s, v_num=9, loss/train=27.00, loss/val=31.70]
Epoch 7:  88%|████████▊ | 23/26 [00:18<00:02,  1.22it/s, v_num=9, loss/train=31.40, loss/val=31.70]
Epoch 7:  92%|█████████▏| 24/26 [00:18<00:01,  1.27it/s, v_num=9, loss/train=31.40, loss/val=31.70]
Epoch 7:  92%|█████████▏| 24/26 [00:18<00:01,  1.27it/s, v_num=9, loss/train=34.20, loss/val=31.70]
Epoch 7:  96%|█████████▌| 25/26 [00:18<00:00,  1.32it/s, v_num=9, loss/train=34.20, loss/val=31.70]
Epoch 7:  96%|█████████▌| 25/26 [00:18<00:00,  1.32it/s, v_num=9, loss/train=29.40, loss/val=31.70]
Epoch 7: 100%|██████████| 26/26 [00:18<00:00,  1.37it/s, v_num=9, loss/train=29.40, loss/val=31.70]
Epoch 7: 100%|██████████| 26/26 [00:18<00:00,  1.37it/s, v_num=9, loss/train=29.50, loss/val=31.70]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 111.53it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 158.03it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 18.40it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 17.26it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  8.56it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 10.22it/s]


Epoch 7: 100%|██████████| 26/26 [00:23<00:00,  1.08it/s, v_num=9, loss/train=29.50, loss/val=31.80]
Epoch 7: 100%|██████████| 26/26 [00:23<00:00,  1.08it/s, v_num=9, loss/train=29.50, loss/val=31.80]
Epoch 7:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.50, loss/val=31.80]
Epoch 8:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=29.50, loss/val=31.80]
Epoch 8:   4%|▍         | 1/26 [00:07<03:02,  0.14it/s, v_num=9, loss/train=29.50, loss/val=31.80]
Epoch 8:   4%|▍         | 1/26 [00:07<03:03,  0.14it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:   8%|▊         | 2/26 [00:09<01:49,  0.22it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:   8%|▊         | 2/26 [00:09<01:49,  0.22it/s, v_num=9, loss/train=31.00, loss/val=31.80]
Epoch 8:  12%|█▏        | 3/26 [00:09<01:10,  0.33it/s, v_num=9, loss/train=31.00, loss/val=31.80]
Epoch 8:  12%|█▏        | 3/26 [00:09<01:10,  0.33it/s, v_num=9, loss/train=33.30, loss/val=31.80]
Epoch 8:  15%|█▌        | 4/26 [00:09<00:51,  0.42it/s, v_num=9, loss/train=33.30, loss/val=31.80]
Epoch 8:  15%|█▌        | 4/26 [00:09<00:51,  0.42it/s, v_num=9, loss/train=30.50, loss/val=31.80]
Epoch 8:  19%|█▉        | 5/26 [00:10<00:44,  0.47it/s, v_num=9, loss/train=30.50, loss/val=31.80]
Epoch 8:  19%|█▉        | 5/26 [00:10<00:44,  0.47it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  23%|██▎       | 6/26 [00:10<00:35,  0.56it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  23%|██▎       | 6/26 [00:10<00:35,  0.56it/s, v_num=9, loss/train=29.20, loss/val=31.80]
Epoch 8:  27%|██▋       | 7/26 [00:10<00:29,  0.65it/s, v_num=9, loss/train=29.20, loss/val=31.80]
Epoch 8:  27%|██▋       | 7/26 [00:10<00:29,  0.65it/s, v_num=9, loss/train=33.80, loss/val=31.80]
Epoch 8:  31%|███       | 8/26 [00:10<00:24,  0.73it/s, v_num=9, loss/train=33.80, loss/val=31.80]
Epoch 8:  31%|███       | 8/26 [00:10<00:24,  0.73it/s, v_num=9, loss/train=28.00, loss/val=31.80]
Epoch 8:  35%|███▍      | 9/26 [00:11<00:20,  0.81it/s, v_num=9, loss/train=28.00, loss/val=31.80]
Epoch 8:  35%|███▍      | 9/26 [00:11<00:20,  0.81it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  38%|███▊      | 10/26 [00:11<00:17,  0.90it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  38%|███▊      | 10/26 [00:11<00:17,  0.90it/s, v_num=9, loss/train=31.70, loss/val=31.80]
Epoch 8:  42%|████▏     | 11/26 [00:14<00:19,  0.76it/s, v_num=9, loss/train=31.70, loss/val=31.80]
Epoch 8:  42%|████▏     | 11/26 [00:14<00:19,  0.76it/s, v_num=9, loss/train=31.90, loss/val=31.80]
Epoch 8:  46%|████▌     | 12/26 [00:14<00:16,  0.83it/s, v_num=9, loss/train=31.90, loss/val=31.80]
Epoch 8:  46%|████▌     | 12/26 [00:14<00:16,  0.83it/s, v_num=9, loss/train=29.90, loss/val=31.80]
Epoch 8:  50%|█████     | 13/26 [00:14<00:14,  0.89it/s, v_num=9, loss/train=29.90, loss/val=31.80]
Epoch 8:  50%|█████     | 13/26 [00:14<00:14,  0.89it/s, v_num=9, loss/train=28.30, loss/val=31.80]
Epoch 8:  54%|█████▍    | 14/26 [00:15<00:12,  0.93it/s, v_num=9, loss/train=28.30, loss/val=31.80]
Epoch 8:  54%|█████▍    | 14/26 [00:15<00:12,  0.93it/s, v_num=9, loss/train=32.20, loss/val=31.80]
Epoch 8:  58%|█████▊    | 15/26 [00:17<00:12,  0.88it/s, v_num=9, loss/train=32.20, loss/val=31.80]
Epoch 8:  58%|█████▊    | 15/26 [00:17<00:12,  0.88it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  62%|██████▏   | 16/26 [00:17<00:10,  0.93it/s, v_num=9, loss/train=30.70, loss/val=31.80]
Epoch 8:  62%|██████▏   | 16/26 [00:17<00:10,  0.93it/s, v_num=9, loss/train=30.00, loss/val=31.80]
Epoch 8:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=30.00, loss/val=31.80]
Epoch 8:  65%|██████▌   | 17/26 [00:17<00:09,  0.99it/s, v_num=9, loss/train=29.00, loss/val=31.80]
Epoch 8:  69%|██████▉   | 18/26 [00:17<00:07,  1.04it/s, v_num=9, loss/train=29.00, loss/val=31.80]
Epoch 8:  69%|██████▉   | 18/26 [00:17<00:07,  1.04it/s, v_num=9, loss/train=31.50, loss/val=31.80]
Epoch 8:  73%|███████▎  | 19/26 [00:17<00:06,  1.10it/s, v_num=9, loss/train=31.50, loss/val=31.80]
Epoch 8:  73%|███████▎  | 19/26 [00:17<00:06,  1.10it/s, v_num=9, loss/train=31.50, loss/val=31.80]
Epoch 8:  77%|███████▋  | 20/26 [00:17<00:05,  1.15it/s, v_num=9, loss/train=31.50, loss/val=31.80]
Epoch 8:  77%|███████▋  | 20/26 [00:17<00:05,  1.15it/s, v_num=9, loss/train=30.60, loss/val=31.80]
Epoch 8:  81%|████████  | 21/26 [00:18<00:04,  1.14it/s, v_num=9, loss/train=30.60, loss/val=31.80]
Epoch 8:  81%|████████  | 21/26 [00:18<00:04,  1.14it/s, v_num=9, loss/train=29.90, loss/val=31.80]
Epoch 8:  85%|████████▍ | 22/26 [00:18<00:03,  1.19it/s, v_num=9, loss/train=29.90, loss/val=31.80]
Epoch 8:  85%|████████▍ | 22/26 [00:18<00:03,  1.19it/s, v_num=9, loss/train=31.40, loss/val=31.80]
Epoch 8:  88%|████████▊ | 23/26 [00:18<00:02,  1.24it/s, v_num=9, loss/train=31.40, loss/val=31.80]
Epoch 8:  88%|████████▊ | 23/26 [00:18<00:02,  1.24it/s, v_num=9, loss/train=31.10, loss/val=31.80]
Epoch 8:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=31.10, loss/val=31.80]
Epoch 8:  92%|█████████▏| 24/26 [00:18<00:01,  1.29it/s, v_num=9, loss/train=34.40, loss/val=31.80]
Epoch 8:  96%|█████████▌| 25/26 [00:19<00:00,  1.30it/s, v_num=9, loss/train=34.40, loss/val=31.80]
Epoch 8:  96%|█████████▌| 25/26 [00:19<00:00,  1.30it/s, v_num=9, loss/train=31.30, loss/val=31.80]
Epoch 8: 100%|██████████| 26/26 [00:19<00:00,  1.35it/s, v_num=9, loss/train=31.30, loss/val=31.80]
Epoch 8: 100%|██████████| 26/26 [00:19<00:00,  1.35it/s, v_num=9, loss/train=30.10, loss/val=31.80]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 125.80it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 11.65it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 11.95it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 15.78it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 19.54it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 23.22it/s]


Epoch 8: 100%|██████████| 26/26 [00:24<00:00,  1.08it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 8: 100%|██████████| 26/26 [00:24<00:00,  1.08it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 8:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 9:   0%|          | 0/26 [00:00<?, ?it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 9:   4%|▍         | 1/26 [00:09<04:08,  0.10it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 9:   4%|▍         | 1/26 [00:09<04:08,  0.10it/s, v_num=9, loss/train=29.60, loss/val=30.00]
Epoch 9:   8%|▊         | 2/26 [00:09<01:59,  0.20it/s, v_num=9, loss/train=29.60, loss/val=30.00]
Epoch 9:   8%|▊         | 2/26 [00:09<01:59,  0.20it/s, v_num=9, loss/train=32.50, loss/val=30.00]
Epoch 9:  12%|█▏        | 3/26 [00:10<01:16,  0.30it/s, v_num=9, loss/train=32.50, loss/val=30.00]
Epoch 9:  12%|█▏        | 3/26 [00:10<01:16,  0.30it/s, v_num=9, loss/train=29.50, loss/val=30.00]
Epoch 9:  15%|█▌        | 4/26 [00:10<00:55,  0.40it/s, v_num=9, loss/train=29.50, loss/val=30.00]
Epoch 9:  15%|█▌        | 4/26 [00:10<00:55,  0.40it/s, v_num=9, loss/train=31.70, loss/val=30.00]
Epoch 9:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=31.70, loss/val=30.00]
Epoch 9:  19%|█▉        | 5/26 [00:10<00:43,  0.48it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  23%|██▎       | 6/26 [00:10<00:35,  0.57it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  23%|██▎       | 6/26 [00:10<00:35,  0.57it/s, v_num=9, loss/train=30.50, loss/val=30.00]
Epoch 9:  27%|██▋       | 7/26 [00:10<00:28,  0.66it/s, v_num=9, loss/train=30.50, loss/val=30.00]
Epoch 9:  27%|██▋       | 7/26 [00:10<00:28,  0.66it/s, v_num=9, loss/train=29.70, loss/val=30.00]
Epoch 9:  31%|███       | 8/26 [00:10<00:24,  0.75it/s, v_num=9, loss/train=29.70, loss/val=30.00]
Epoch 9:  31%|███       | 8/26 [00:10<00:24,  0.75it/s, v_num=9, loss/train=31.10, loss/val=30.00]
Epoch 9:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=31.10, loss/val=30.00]
Epoch 9:  35%|███▍      | 9/26 [00:10<00:20,  0.84it/s, v_num=9, loss/train=28.50, loss/val=30.00]
Epoch 9:  38%|███▊      | 10/26 [00:10<00:17,  0.92it/s, v_num=9, loss/train=28.50, loss/val=30.00]
Epoch 9:  38%|███▊      | 10/26 [00:10<00:17,  0.92it/s, v_num=9, loss/train=31.10, loss/val=30.00]
Epoch 9:  42%|████▏     | 11/26 [00:15<00:20,  0.72it/s, v_num=9, loss/train=31.10, loss/val=30.00]
Epoch 9:  42%|████▏     | 11/26 [00:15<00:20,  0.72it/s, v_num=9, loss/train=33.70, loss/val=30.00]
Epoch 9:  46%|████▌     | 12/26 [00:15<00:17,  0.78it/s, v_num=9, loss/train=33.70, loss/val=30.00]
Epoch 9:  46%|████▌     | 12/26 [00:15<00:17,  0.78it/s, v_num=9, loss/train=32.10, loss/val=30.00]
Epoch 9:  50%|█████     | 13/26 [00:15<00:15,  0.84it/s, v_num=9, loss/train=32.10, loss/val=30.00]
Epoch 9:  50%|█████     | 13/26 [00:15<00:15,  0.84it/s, v_num=9, loss/train=31.60, loss/val=30.00]
Epoch 9:  54%|█████▍    | 14/26 [00:15<00:13,  0.91it/s, v_num=9, loss/train=31.60, loss/val=30.00]
Epoch 9:  54%|█████▍    | 14/26 [00:15<00:13,  0.91it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 9:  58%|█████▊    | 15/26 [00:16<00:11,  0.93it/s, v_num=9, loss/train=30.10, loss/val=30.00]
Epoch 9:  58%|█████▊    | 15/26 [00:16<00:11,  0.93it/s, v_num=9, loss/train=31.70, loss/val=30.00]
Epoch 9:  62%|██████▏   | 16/26 [00:16<00:10,  0.98it/s, v_num=9, loss/train=31.70, loss/val=30.00]
Epoch 9:  62%|██████▏   | 16/26 [00:16<00:10,  0.98it/s, v_num=9, loss/train=29.00, loss/val=30.00]
Epoch 9:  65%|██████▌   | 17/26 [00:16<00:08,  1.04it/s, v_num=9, loss/train=29.00, loss/val=30.00]
Epoch 9:  65%|██████▌   | 17/26 [00:16<00:08,  1.04it/s, v_num=9, loss/train=30.40, loss/val=30.00]
Epoch 9:  69%|██████▉   | 18/26 [00:16<00:07,  1.10it/s, v_num=9, loss/train=30.40, loss/val=30.00]
Epoch 9:  69%|██████▉   | 18/26 [00:16<00:07,  1.10it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  73%|███████▎  | 19/26 [00:16<00:06,  1.16it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  73%|███████▎  | 19/26 [00:16<00:06,  1.16it/s, v_num=9, loss/train=31.90, loss/val=30.00]
Epoch 9:  77%|███████▋  | 20/26 [00:16<00:04,  1.22it/s, v_num=9, loss/train=31.90, loss/val=30.00]
Epoch 9:  77%|███████▋  | 20/26 [00:16<00:04,  1.22it/s, v_num=9, loss/train=31.30, loss/val=30.00]
Epoch 9:  81%|████████  | 21/26 [00:18<00:04,  1.16it/s, v_num=9, loss/train=31.30, loss/val=30.00]
Epoch 9:  81%|████████  | 21/26 [00:18<00:04,  1.16it/s, v_num=9, loss/train=29.50, loss/val=30.00]
Epoch 9:  85%|████████▍ | 22/26 [00:18<00:03,  1.21it/s, v_num=9, loss/train=29.50, loss/val=30.00]
Epoch 9:  85%|████████▍ | 22/26 [00:18<00:03,  1.21it/s, v_num=9, loss/train=31.40, loss/val=30.00]
Epoch 9:  88%|████████▊ | 23/26 [00:18<00:02,  1.26it/s, v_num=9, loss/train=31.40, loss/val=30.00]
Epoch 9:  88%|████████▊ | 23/26 [00:18<00:02,  1.26it/s, v_num=9, loss/train=31.50, loss/val=30.00]
Epoch 9:  92%|█████████▏| 24/26 [00:18<00:01,  1.31it/s, v_num=9, loss/train=31.50, loss/val=30.00]
Epoch 9:  92%|█████████▏| 24/26 [00:18<00:01,  1.31it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  96%|█████████▌| 25/26 [00:18<00:00,  1.34it/s, v_num=9, loss/train=31.80, loss/val=30.00]
Epoch 9:  96%|█████████▌| 25/26 [00:18<00:00,  1.34it/s, v_num=9, loss/train=30.90, loss/val=30.00]
Epoch 9: 100%|██████████| 26/26 [00:18<00:00,  1.39it/s, v_num=9, loss/train=30.90, loss/val=30.00]
Epoch 9: 100%|██████████| 26/26 [00:18<00:00,  1.39it/s, v_num=9, loss/train=33.50, loss/val=30.00]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]

Validation DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 99.24it/s]

Validation DataLoader 0:  33%|███▎      | 2/6 [00:01<00:03,  1.27it/s]

Validation DataLoader 0:  50%|█████     | 3/6 [00:01<00:01,  1.86it/s]

Validation DataLoader 0:  67%|██████▋   | 4/6 [00:01<00:00,  2.47it/s]

Validation DataLoader 0:  83%|████████▎ | 5/6 [00:01<00:00,  3.08it/s]

Validation DataLoader 0: 100%|██████████| 6/6 [00:01<00:00,  3.69it/s]


Epoch 9: 100%|██████████| 26/26 [00:23<00:00,  1.11it/s, v_num=9, loss/train=33.50, loss/val=30.40]
Epoch 9: 100%|██████████| 26/26 [00:23<00:00,  1.11it/s, v_num=9, loss/train=33.50, loss/val=30.40]
Epoch 9: 100%|██████████| 26/26 [00:23<00:00,  1.11it/s, v_num=9, loss/train=33.50, loss/val=30.40]

BarlowTwins(
  (encoder): MLP(
    (0): Linear(in_features=272, out_features=64, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.0, inplace=False)
    (3): Linear(in_features=64, out_features=32, bias=True)
    (4): Dropout(p=0.0, inplace=False)
  )
  (projection_head): BarlowTwinsProjectionHead(
    (layers): Sequential(
      (0): Linear(in_features=32, out_features=64, bias=False)
      (1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, bias=True, track_running_stats=True)
      (2): ReLU()
      (3): Linear(in_features=64, out_features=64, bias=False)
      (4): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, bias=True, track_running_stats=True)
      (5): ReLU()
      (6): Linear(in_features=64, out_features=32, bias=True)
    )
  )
  (loss): BarlowTwinsLoss()
)

Visualization and evaluation of the learned representations

In order to visualize the learned representations of both models, we apply a widely used dimensionality reduction technique: Multi-Dimensional Scaling (MDS). This technique project the points in a lower-dimensional space such that the pairwise distances between points are preserved as much as possible. Then, we evaluate the learned representations on age prediction using linear regression and KNN regression.

We first extract the embeddings of the training and test sets for both VBM and SBM data.

Z_train_vbm = vbm_model.transform(dataloader_vbm_train)
Z_test_vbm = vbm_model.transform(dataloader_vbm_test)
Z_train_sbm = sbm_model.transform(dataloader_sbm_train)
Z_test_sbm = sbm_model.transform(dataloader_sbm_test)

Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting DataLoader 0:   0%|          | 0/26 [00:00<?, ?it/s]
Predicting DataLoader 0:   4%|▍         | 1/26 [00:00<00:00, 173.05it/s]
Predicting DataLoader 0:   8%|▊         | 2/26 [00:00<00:00, 283.58it/s]
Predicting DataLoader 0:  12%|█▏        | 3/26 [00:00<00:00, 172.71it/s]
Predicting DataLoader 0:  15%|█▌        | 4/26 [00:01<00:07,  2.97it/s]
Predicting DataLoader 0:  19%|█▉        | 5/26 [00:01<00:05,  3.69it/s]
Predicting DataLoader 0:  23%|██▎       | 6/26 [00:01<00:04,  4.42it/s]
Predicting DataLoader 0:  27%|██▋       | 7/26 [00:01<00:03,  5.06it/s]
Predicting DataLoader 0:  31%|███       | 8/26 [00:01<00:03,  5.75it/s]
Predicting DataLoader 0:  35%|███▍      | 9/26 [00:01<00:02,  6.11it/s]
Predicting DataLoader 0:  38%|███▊      | 10/26 [00:01<00:02,  6.68it/s]
Predicting DataLoader 0:  42%|████▏     | 11/26 [00:03<00:04,  3.51it/s]
Predicting DataLoader 0:  46%|████▌     | 12/26 [00:03<00:03,  3.82it/s]
Predicting DataLoader 0:  50%|█████     | 13/26 [00:03<00:03,  4.14it/s]
Predicting DataLoader 0:  54%|█████▍    | 14/26 [00:04<00:03,  3.43it/s]
Predicting DataLoader 0:  58%|█████▊    | 15/26 [00:04<00:03,  3.18it/s]
Predicting DataLoader 0:  62%|██████▏   | 16/26 [00:04<00:02,  3.38it/s]
Predicting DataLoader 0:  65%|██████▌   | 17/26 [00:04<00:02,  3.44it/s]
Predicting DataLoader 0:  69%|██████▉   | 18/26 [00:04<00:02,  3.63it/s]
Predicting DataLoader 0:  73%|███████▎  | 19/26 [00:04<00:01,  3.82it/s]
Predicting DataLoader 0:  77%|███████▋  | 20/26 [00:04<00:01,  4.01it/s]
Predicting DataLoader 0:  81%|████████  | 21/26 [00:05<00:01,  3.90it/s]
Predicting DataLoader 0:  85%|████████▍ | 22/26 [00:05<00:00,  4.08it/s]
Predicting DataLoader 0:  88%|████████▊ | 23/26 [00:05<00:00,  4.26it/s]
Predicting DataLoader 0:  92%|█████████▏| 24/26 [00:05<00:00,  4.39it/s]
Predicting DataLoader 0:  96%|█████████▌| 25/26 [00:05<00:00,  4.48it/s]
Predicting DataLoader 0: 100%|██████████| 26/26 [00:05<00:00,  4.66it/s]
Predicting DataLoader 0: 100%|██████████| 26/26 [00:05<00:00,  4.66it/s]

Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]
Predicting DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 398.32it/s]
Predicting DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00, 63.67it/s]
Predicting DataLoader 0:  50%|█████     | 3/6 [00:00<00:00, 92.79it/s]
Predicting DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00, 120.30it/s]
Predicting DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00, 146.58it/s]
Predicting DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 171.08it/s]
Predicting DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 169.11it/s]

Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting DataLoader 0:   0%|          | 0/26 [00:00<?, ?it/s]
Predicting DataLoader 0:   4%|▍         | 1/26 [00:00<00:00, 51.34it/s]
Predicting DataLoader 0:   8%|▊         | 2/26 [00:00<00:09,  2.53it/s]
Predicting DataLoader 0:  12%|█▏        | 3/26 [00:01<00:12,  1.86it/s]
Predicting DataLoader 0:  15%|█▌        | 4/26 [00:01<00:08,  2.48it/s]
Predicting DataLoader 0:  19%|█▉        | 5/26 [00:01<00:06,  3.08it/s]
Predicting DataLoader 0:  23%|██▎       | 6/26 [00:01<00:05,  3.67it/s]
Predicting DataLoader 0:  27%|██▋       | 7/26 [00:01<00:04,  4.26it/s]
Predicting DataLoader 0:  31%|███       | 8/26 [00:01<00:03,  4.84it/s]
Predicting DataLoader 0:  35%|███▍      | 9/26 [00:01<00:03,  5.42it/s]
Predicting DataLoader 0:  38%|███▊      | 10/26 [00:01<00:02,  5.99it/s]
Predicting DataLoader 0:  42%|████▏     | 11/26 [00:06<00:09,  1.60it/s]
Predicting DataLoader 0:  46%|████▌     | 12/26 [00:06<00:08,  1.74it/s]
Predicting DataLoader 0:  50%|█████     | 13/26 [00:06<00:06,  1.88it/s]
Predicting DataLoader 0:  54%|█████▍    | 14/26 [00:06<00:05,  2.02it/s]
Predicting DataLoader 0:  58%|█████▊    | 15/26 [00:06<00:05,  2.15it/s]
Predicting DataLoader 0:  62%|██████▏   | 16/26 [00:06<00:04,  2.29it/s]
Predicting DataLoader 0:  65%|██████▌   | 17/26 [00:07<00:03,  2.37it/s]
Predicting DataLoader 0:  69%|██████▉   | 18/26 [00:07<00:03,  2.50it/s]
Predicting DataLoader 0:  73%|███████▎  | 19/26 [00:07<00:02,  2.42it/s]
Predicting DataLoader 0:  77%|███████▋  | 20/26 [00:07<00:02,  2.54it/s]
Predicting DataLoader 0:  81%|████████  | 21/26 [00:10<00:02,  2.05it/s]
Predicting DataLoader 0:  85%|████████▍ | 22/26 [00:10<00:01,  2.14it/s]
Predicting DataLoader 0:  88%|████████▊ | 23/26 [00:10<00:01,  2.24it/s]
Predicting DataLoader 0:  92%|█████████▏| 24/26 [00:10<00:00,  2.34it/s]
Predicting DataLoader 0:  96%|█████████▌| 25/26 [00:10<00:00,  2.44it/s]
Predicting DataLoader 0: 100%|██████████| 26/26 [00:10<00:00,  2.53it/s]
Predicting DataLoader 0: 100%|██████████| 26/26 [00:10<00:00,  2.53it/s]

Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting: |          | 0/? [00:00<?, ?it/s]
Predicting DataLoader 0:   0%|          | 0/6 [00:00<?, ?it/s]
Predicting DataLoader 0:  17%|█▋        | 1/6 [00:00<00:00, 55.11it/s]
Predicting DataLoader 0:  33%|███▎      | 2/6 [00:00<00:00,  4.97it/s]
Predicting DataLoader 0:  50%|█████     | 3/6 [00:00<00:00,  7.44it/s]
Predicting DataLoader 0:  67%|██████▋   | 4/6 [00:00<00:00,  7.06it/s]
Predicting DataLoader 0:  83%|████████▎ | 5/6 [00:00<00:00,  8.81it/s]
Predicting DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 10.56it/s]
Predicting DataLoader 0: 100%|██████████| 6/6 [00:00<00:00, 10.55it/s]

We also extract the ages of the subjects for coloring the points in the visualizations and for evaluating the representations on age prediction.

y_train_vbm = [y for (_, y) in dataloader_vbm_train.dataset.samples]
y_test_vbm = [y for (_, y) in dataloader_vbm_test.dataset.samples]
y_train_sbm = [y for (_, y) in dataloader_sbm_train.dataset.samples]
y_test_sbm = [y for (_, y) in dataloader_sbm_test.dataset.samples]

We then apply MDS on the test set and visualize the results. The points are colored according to the age of the subjects.

def plot_mds_side_by_side(Z_vbm, Z_sbm, y_vbm, y_sbm):
    """Run MDS on VBM and SBM embeddings and plot side-by-side scatter
    plots."""
    mds = MDS(n_components=2, n_init=4, max_iter=300)

    # Fit-transform embeddings
    Z_vbm_mds = mds.fit_transform(Z_vbm.cpu())
    Z_sbm_mds = mds.fit_transform(Z_sbm.cpu())

    # Side-by-side plots
    fig, axes = plt.subplots(1, 2, figsize=(12, 5))

    sc1 = axes[0].scatter(
        Z_vbm_mds[:, 0], Z_vbm_mds[:, 1], c=y_vbm, cmap="viridis", alpha=0.8
    )
    axes[0].set_title("VBM - MDS projection")
    axes[0].set_xlabel("Dim 1")
    axes[0].set_ylabel("Dim 2")
    plt.colorbar(sc1, ax=axes[0], label="Age")

    sc2 = axes[1].scatter(
        Z_sbm_mds[:, 0], Z_sbm_mds[:, 1], c=y_sbm, cmap="viridis", alpha=0.8
    )
    axes[1].set_title("SBM - MDS projection")
    axes[1].set_xlabel("Dim 1")
    axes[1].set_ylabel("Dim 2")
    plt.colorbar(sc2, ax=axes[1], label="Age")

    plt.suptitle("MDS projections of test embeddings", fontsize=14)
    plt.tight_layout()
    plt.show()


plot_mds_side_by_side(Z_test_vbm, Z_test_sbm, y_test_vbm, y_test_sbm)

Finally, we evaluate the learned representations on age prediction using linear regression and KNN regression. We report the mean absolute error and the R^2 coefficient between the true and predicted ages on the test set for each model.

def evaluate_and_predict(model, Z_train, Z_test, y_train, y_test):
    """Train model and return predictions + metrics."""
    model.fit(Z_train.cpu(), y_train)
    y_pred = model.predict(Z_test.cpu())
    mae = mean_absolute_error(y_test, y_pred)
    r2 = r2_score(y_test, y_pred)
    return y_pred, mae, r2


def plot_comparison(models, embeddings):
    """
    Plot side-by-side scatter plots for each model and modality.
    models: dict of {name: model}
    embeddings: dict of {modality: (Z_train, Z_test, y_train, y_test)}
    """
    n_models = len(models)
    n_modalities = len(embeddings)

    fig, axes = plt.subplots(
        n_models,
        n_modalities,
        figsize=(6 * n_modalities, 5 * n_models),
        sharex=True,
        sharey=True,
    )
    for row, (model_name, model) in enumerate(models.items()):
        for col, (modality, (Z_train, Z_test, y_train, y_test)) in enumerate(
            embeddings.items()
        ):
            y_pred, mae, r2 = evaluate_and_predict(
                model, Z_train, Z_test, y_train, y_test
            )

            ax = axes[row, col]
            ax.scatter(
                y_test,
                y_pred,
                alpha=0.7,
                color="orange" if modality == "SBM" else "steelblue",
            )
            ax.plot(
                [np.min(y_test), np.max(y_test)],
                [np.min(y_test), np.max(y_test)],
                "r--",
                lw=2,
                label="Ideal",
            )
            ax.set_title(
                f"{modality} - {model_name}\nMAE={mae:.2f}, R²={r2:.2f}"
            )
            ax.set_xlabel("True Age")
            if col == 0:
                ax.set_ylabel("Predicted Age")
            ax.legend()
            ax.grid(True)

    plt.suptitle("Model Comparison: VBM vs SBM", fontsize=16, y=1.02)
    plt.tight_layout()
    plt.show()


# Define models and embeddings
models = {
    "Linear Regression": LinearRegression(),
    "KNN (k=5)": KNeighborsRegressor(n_neighbors=5),
}

embeddings = {
    "VBM": (Z_train_vbm, Z_test_vbm, y_train_vbm, y_test_vbm),
    "SBM": (Z_train_sbm, Z_test_sbm, y_train_sbm, y_test_sbm),
}

# Run comparison
plot_comparison(models, embeddings)

Observations: From the MDS visualizations, we can observe that both VBM and SBM embeddings show a gradient of ages, indicating that the models have learned to organize the data in a way that reflects age similarity. However, the VBM embeddings appear to have a more continuous distribution of ages compared to SBM. This suggests that VBM may capture age-related features more effectively than SBM in this context. This is confirmed when looking at the age prediction results, where VBM outperforms SBM for both linear regression and KNN regression. However, the results can be improved by working with the original 3d brain scans instead of the ROI-averaged data.

Estimated memory usage: 121 MB