timesead.models.common

Submodules

Classes

`AE`	Simple AE Implementation
`AnomalyDetector`	Base class for all neural network modules.
`MSEReconstructionAnomalyDetector`	Base class for all neural network modules.
`MAEReconstructionAnomalyDetector`	Base class for all neural network modules.
`PredictionAnomalyDetector`	Base class for all neural network modules.
`GAN`	Base class for all neural network modules.
`GANDiscriminatorLoss`	Base class for all neural network modules.
`GANGeneratorLoss`	Base class for all neural network modules.
`GANGeneratorLossMod`	Base class for all neural network modules.
`WassersteinGeneratorLoss`	Base class for all neural network modules.
`WassersteinDiscriminatorLoss`	Base class for all neural network modules.
`MLP`	Base class for all neural network modules.
`RNN`	Base class for all neural network modules.
`TCN`	Creates a TCN layer.
`TCNResidualBlock`	Base class for all neural network modules.
`DenseVAEEncoder`	Base class for all neural network modules.
`VAE`	VAE Implementation that supports normal distribution with diagonal cov matrix in the latent space
`VAELoss`	Base class for all neural network modules.

Package Contents

class timesead.models.common.AE(encoder: torch.nn.Module, decoder: torch.nn.Module, return_latent: bool = False)

Bases: torch.nn.Module

Simple AE Implementation

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

encoder (torch.nn.Module)
decoder (torch.nn.Module)
return_latent (bool)

encoder

decoder

return_latent = False

forward(x: torch.Tensor) → torch.Tensor | Tuple[torch.Tensor, torch.Tensor]

Parameters:: x (torch.Tensor)
Return type:: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

class timesead.models.common.AnomalyDetector

Bases: torch.nn.Module, abc.ABC

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

abstract compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

abstract format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: torch.Tensor

get_labels_and_scores(dataset: torch.utils.data.DataLoader) → Tuple[torch.Tensor, torch.Tensor]

Parameters:: dataset (torch.utils.data.DataLoader)
Return type:: Tuple[torch.Tensor, torch.Tensor]

class timesead.models.common.MSEReconstructionAnomalyDetector(model: timesead.models.BaseModel, batch_first: bool = True)

Bases: AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

model (timesead.models.BaseModel)
batch_first (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

batch_first = True

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.common.MAEReconstructionAnomalyDetector(model: timesead.models.BaseModel, batch_first: bool = True)

Bases: MSEReconstructionAnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

model (timesead.models.BaseModel)
batch_first (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

class timesead.models.common.PredictionAnomalyDetector

Bases: AnomalyDetector, abc.ABC

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

get_labels_and_scores(dataset: torch.utils.data.DataLoader) → Tuple[torch.Tensor, torch.Tensor]

Parameters:: dataset (torch.utils.data.DataLoader)
Return type:: Tuple[torch.Tensor, torch.Tensor]

class timesead.models.common.GAN(generator: torch.nn.Module, discriminator: torch.nn.Module)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

generator (torch.nn.Module)
discriminator (torch.nn.Module)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

generator

discriminator

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: Tuple[torch.Tensor, Ellipsis]

class timesead.models.common.GANDiscriminatorLoss

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

This is the original GAN loss, i.e., - E[log(D(x))] - E[log(1 - D(G(z)))]

cross_entropy

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.common.GANGeneratorLoss

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

This is the original GAN loss, i.e., E[log(1 - D(G(z)))]

cross_entropy

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.common.GANGeneratorLossMod

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

This is a modified version of original GAN loss, i.e., -E[log(D(G(z)))]

cross_entropy

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.common.WassersteinGeneratorLoss

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.common.WassersteinDiscriminatorLoss(gan: GAN = None, gradient_penalty: float = 10)

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

gan (GAN)
gradient_penalty (float)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

gradient_penalty_coeff = 10

gan = None

static gradient_penalty(discriminator, real_input: torch.Tensor, fake_input: torch.Tensor) → torch.Tensor

Parameters:

real_input (torch.Tensor)
fake_input (torch.Tensor)

Return type:

torch.Tensor

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.common.MLP(input_features: int, hidden_layers: int | Sequence[int], output_features: int, activation: Callable = torch.nn.Identity(), activation_after_last_layer: bool = False)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

input_features (int)
hidden_layers (Union[int, Sequence[int]])
output_features (int)
activation (Callable)
activation_after_last_layer (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

activation

activation_after_last_layer = False

layers

forward(x: torch.Tensor) → torch.Tensor

Parameters:: x (torch.Tensor)
Return type:: torch.Tensor

class timesead.models.common.RNN(layer_type: str, model: str, input_dimension: int, hidden_dimensions: Sequence[int] | int, n_recurrent_layers: int | None = None, recurrent_activation: str = 'tanh', recurrent_bias: bool = True, bidirectional: bool = False, projection_size: int = 0, dilation: Sequence[int] | None = None, dropout: float = 0.0)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

layer_type (str)
model (str)
input_dimension (int)
hidden_dimensions (Union[Sequence[int], int])
n_recurrent_layers (Optional[int])
recurrent_activation (str)
recurrent_bias (bool)
bidirectional (bool)
projection_size (int)
dilation (Optional[Sequence[int]])
dropout (float)

General framework of a recurrent neural network.

Parameters:

layer_type (str) – The type of recurrent layer (RNN, LSTM, GRU).
model (str) – Type of model (s2s, s2as, s2fh, s2mh)
input_dimension (int) – Number of dimensions of the feature space of the input data.
hidden_dimensions (Union[List[int], int]) – Number of dimensions of each hidden state for all recurrent layers.
n_recurrent_layers (Optional[int]) – Number of recurrent layers, if hidden_dimensions is the same for all layers (int).
recurrent_activation (str) – Activation function to use in each recurrent layer (has to be defined in torch.nn).
recurrent_bias (bool) – Whether to use bias in the computations of the recurrent layers.
bidirectional (bool) – Use bidirectional recurrent layers.
projection_size (int) – Parameter relevant for LSTM layers only. See pytorch docs for details.
dilation (Optional[Sequence[int]])
dropout (float)

dilation = None

dropout

recurrent_layers

model

apply_dilated_layer(layer: torch.nn.Module, inputs: torch.Tensor | torch.nn.utils.rnn.PackedSequence, dilation: int, hidden: torch.Tensor | Tuple[torch.Tensor, torch.Tensor] = None) → Tuple[torch.Tensor | torch.nn.utils.rnn.PackedSequence, torch.Tensor | Tuple[torch.Tensor, torch.Tensor]]

Parameters:

layer (torch.nn.Module)
inputs (Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence])
dilation (int)
hidden (Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]])

Return type:

Tuple[Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence], Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]]

forward(inputs: torch.Tensor | torch.nn.utils.rnn.PackedSequence, hidden_states: torch.Tensor | Tuple[torch.Tensor, torch.Tensor] = None, return_hidden: bool = False) → Tuple[torch.Tensor, Ellipsis] | torch.Tensor | torch.nn.utils.rnn.PackedSequence

Apply the base RNN to the sequence and return the hidden states from every step in the sequence.

Parameters:

inputs (Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence]) – Tensor or sequence of shape (T, B, …)
hidden_states (Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]) – Hidden states of shape (num_layers, B, …). For LSTM this must be a tuple of two tensors.
return_hidden (bool) – Whether the method should return hidden states of the RNN too

Returns:

If return_hidden=False this is a tensor or PackedSequence of shape (T, B, …), otherwise a Tuple (output, hidden)

Return type:

Union[Tuple[torch.Tensor, Ellipsis], torch.Tensor, torch.nn.utils.rnn.PackedSequence]

class timesead.models.common.TCN(input_dim: int, nb_filters: int | Sequence[int] = 64, kernel_size: int = 3, nb_stacks: int = 1, dilations: List[int] = (1, 2, 4, 8, 16, 32), padding: str = 'same', use_skip_connections: bool = True, dropout_rate: float = 0.0, return_sequences: bool = False, activation: str | Callable = 'relu', use_batch_norm: bool = False, use_layer_norm: bool = False)

Bases: torch.nn.Module

Creates a TCN layer.

Parameters:

nb_filters (Union[int, Sequence[int]]) – The number of filters to use in the convolutional layers. Can be a list.
kernel_size (int) – The size of the kernel to use in each convolutional layer.
dilations (List[int]) – The list of the dilations. Example is: [1, 2, 4, 8, 16, 32, 64].
nb_stacks (int) – The number of stacks of residual blocks to use.
padding (str) – The padding to use in the convolutional layers, ‘causal’ or ‘same’.
use_skip_connections (bool) – Boolean. If we want to add skip connections from input to each residual blocK.
return_sequences (bool) – Boolean. Whether to return the last output in the output sequence, or the full sequence.
activation (Union[str, Callable]) – The activation used in the residual blocks o = Activation(x + F(x)).
dropout_rate (float) – Float between 0 and 1. Fraction of the input units to drop.
use_batch_norm (bool) – Whether to use batch normalization in the residual layers or not.
use_layer_norm (bool) – Whether to use layer normalization in the residual layers or not.
input_dim (int)

Returns:

A TCN layer.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

return_sequences = False

use_skip_connections = True

dilations = (1, 2, 4, 8, 16, 32)

nb_stacks = 1

kernel_size = 3

residual_blocks

property receptive_field

forward(x: torch.Tensor) → torch.Tensor

Parameters:: x (torch.Tensor)
Return type:: torch.Tensor

class timesead.models.common.TCNResidualBlock(input_dim: int, dilation_rate: int, nb_filters: int, kernel_size: int, padding: str, activation: str | Callable = 'relu', dropout_rate: float = 0, use_batch_norm: bool = False, use_layer_norm: bool = False)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

input_dim (int)
dilation_rate (int)
nb_filters (int)
kernel_size (int)
padding (str)
activation (Union[str, Callable])
dropout_rate (float)
use_batch_norm (bool)
use_layer_norm (bool)

Defines the residual block for the WaveNet TCN. Input needs to be of shape (B, D, T).

Parameters:

dilation_rate (int) – The dilation power of 2 we are using for this residual block
nb_filters (int) – The number of convolutional filters to use in this block
kernel_size (int) – The size of the convolutional kernel
padding (str) – The padding used in the convolutional layers, ‘same’ or ‘causal’.
activation (Union[str, Callable]) – The final activation used in o = Activation(x + F(x))
dropout_rate (float) – Float between 0 and 1. Fraction of the input units to drop.
use_batch_norm (bool) – Whether to use batch normalization in the residual layers or not.
use_layer_norm (bool) – Whether to use layer normalization in the residual layers or not.
input_dim (int)

activation = 'relu'

dropout

forward(x: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor]

Returns: A tuple where the first element is the residual model tensor, and the second: is the skip connection tensor.

Parameters:: x (torch.Tensor)
Return type:: Tuple[torch.Tensor, torch.Tensor]

class timesead.models.common.DenseVAEEncoder(input_dim: int, hidden_dims: Sequence[int] = (100, 100), latent_dim: int = 10, activation=torch.nn.ReLU())

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

input_dim (int)
hidden_dims (Sequence[int])
latent_dim (int)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

latent_dim = 10

mlp

softplus

forward(x: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor]

Parameters:: x (torch.Tensor)
Return type:: Tuple[torch.Tensor, torch.Tensor]

class timesead.models.common.VAE(encoder: torch.nn.Module, decoder: torch.nn.Module, logvar_out: bool = True)

Bases: torch.nn.Module

VAE Implementation that supports normal distribution with diagonal cov matrix in the latent space and the output

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

encoder (torch.nn.Module)
decoder (torch.nn.Module)
logvar_out (bool)

encoder

decoder

log_var = True

forward(x: torch.Tensor, return_latent_sample: bool = False, num_samples: int = 1, force_sample: bool = False) → Tuple[torch.Tensor, Ellipsis]

Parameters:

x (torch.Tensor)
return_latent_sample (bool)
num_samples (int)
force_sample (bool)

Return type:

Tuple[torch.Tensor, Ellipsis]

class timesead.models.common.VAELoss(size_average=None, reduce=None, reduction: str = 'mean', logvar_out: bool = True)

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

reduction (str)
logvar_out (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

logvar_out = True

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor