timesead.models.common ====================== .. py:module:: timesead.models.common Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/timesead/models/common/ae/index /autoapi/timesead/models/common/anomaly_detector/index /autoapi/timesead/models/common/gan/index /autoapi/timesead/models/common/mlp/index /autoapi/timesead/models/common/rnn/index /autoapi/timesead/models/common/tcn/index /autoapi/timesead/models/common/vae/index Classes ------- .. autoapisummary:: timesead.models.common.AE timesead.models.common.AnomalyDetector timesead.models.common.MSEReconstructionAnomalyDetector timesead.models.common.MAEReconstructionAnomalyDetector timesead.models.common.PredictionAnomalyDetector timesead.models.common.GAN timesead.models.common.GANDiscriminatorLoss timesead.models.common.GANGeneratorLoss timesead.models.common.GANGeneratorLossMod timesead.models.common.WassersteinGeneratorLoss timesead.models.common.WassersteinDiscriminatorLoss timesead.models.common.MLP timesead.models.common.RNN timesead.models.common.TCN timesead.models.common.TCNResidualBlock timesead.models.common.DenseVAEEncoder timesead.models.common.VAE timesead.models.common.VAELoss Package Contents ---------------- .. py:class:: AE(encoder: torch.nn.Module, decoder: torch.nn.Module, return_latent: bool = False) Bases: :py:obj:`torch.nn.Module` Simple AE Implementation Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: encoder .. py:attribute:: decoder .. py:attribute:: return_latent :value: False .. py:method:: forward(x: torch.Tensor) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]] .. py:class:: AnomalyDetector Bases: :py:obj:`torch.nn.Module`, :py:obj:`abc.ABC` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:method:: compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor :abstractmethod: Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (B,) that contains the anomaly scores for this batch .. py:method:: compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor :abstractmethod: Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (N,) that contains the anomaly scores for this batch .. py:method:: fit(dataset: torch.utils.data.DataLoader) -> None :abstractmethod: Fit this anomaly detector on a dataset. Note that we assume only normal data here. :param dataset: A dataset .. py:method:: format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor :abstractmethod: Format the labels for a batch of targets. The output tensor must have the same shape as the output of `compute_online_anomaly_score` when called with the corresponding inputs for this batch. :param targets: tuple of target tensors :return: Tensor of shape (B,) that contains the ground truth labels for this batch .. py:method:: forward(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor .. py:method:: get_labels_and_scores(dataset: torch.utils.data.DataLoader) -> Tuple[torch.Tensor, torch.Tensor] .. py:class:: MSEReconstructionAnomalyDetector(model: timesead.models.BaseModel, batch_first: bool = True) Bases: :py:obj:`AnomalyDetector` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: model .. py:attribute:: batch_first :value: True .. py:method:: fit(dataset: torch.utils.data.DataLoader) -> None Fit this anomaly detector on a dataset. Note that we assume only normal data here. :param dataset: A dataset .. py:method:: compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (B,) that contains the anomaly scores for this batch .. py:method:: compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor :abstractmethod: Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (N,) that contains the anomaly scores for this batch .. py:method:: format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor Format the labels for a batch of targets. The output tensor must have the same shape as the output of `compute_online_anomaly_score` when called with the corresponding inputs for this batch. :param targets: tuple of target tensors :return: Tensor of shape (B,) that contains the ground truth labels for this batch .. py:class:: MAEReconstructionAnomalyDetector(model: timesead.models.BaseModel, batch_first: bool = True) Bases: :py:obj:`MSEReconstructionAnomalyDetector` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:method:: compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (B,) that contains the anomaly scores for this batch .. py:method:: compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) -> torch.Tensor :abstractmethod: Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of `format_targets` when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window. :param inputs: tuple of input tensors :return: Tensor of shape (N,) that contains the anomaly scores for this batch .. py:class:: PredictionAnomalyDetector Bases: :py:obj:`AnomalyDetector`, :py:obj:`abc.ABC` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:method:: get_labels_and_scores(dataset: torch.utils.data.DataLoader) -> Tuple[torch.Tensor, torch.Tensor] .. py:class:: GAN(generator: torch.nn.Module, discriminator: torch.nn.Module) Bases: :py:obj:`torch.nn.Module` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: generator .. py:attribute:: discriminator .. py:method:: forward(inputs: Tuple[torch.Tensor, Ellipsis]) -> Tuple[torch.Tensor, Ellipsis] .. py:class:: GANDiscriminatorLoss Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool This is the original GAN loss, i.e., - E[log(D(x))] - E[log(1 - D(G(z)))] .. py:attribute:: cross_entropy .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor .. py:class:: GANGeneratorLoss Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool This is the original GAN loss, i.e., E[log(1 - D(G(z)))] .. py:attribute:: cross_entropy .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor .. py:class:: GANGeneratorLossMod Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool This is a modified version of original GAN loss, i.e., -E[log(D(G(z)))] .. py:attribute:: cross_entropy .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor .. py:class:: WassersteinGeneratorLoss Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor .. py:class:: WassersteinDiscriminatorLoss(gan: GAN = None, gradient_penalty: float = 10) Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: gradient_penalty_coeff :value: 10 .. py:attribute:: gan :value: None .. py:method:: gradient_penalty(discriminator, real_input: torch.Tensor, fake_input: torch.Tensor) -> torch.Tensor :staticmethod: .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor .. py:class:: MLP(input_features: int, hidden_layers: Union[int, Sequence[int]], output_features: int, activation: Callable = torch.nn.Identity(), activation_after_last_layer: bool = False) Bases: :py:obj:`torch.nn.Module` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: activation .. py:attribute:: activation_after_last_layer :value: False .. py:attribute:: layers .. py:method:: forward(x: torch.Tensor) -> torch.Tensor .. py:class:: RNN(layer_type: str, model: str, input_dimension: int, hidden_dimensions: Union[Sequence[int], int], n_recurrent_layers: Optional[int] = None, recurrent_activation: str = 'tanh', recurrent_bias: bool = True, bidirectional: bool = False, projection_size: int = 0, dilation: Optional[Sequence[int]] = None, dropout: float = 0.0) Bases: :py:obj:`torch.nn.Module` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool General framework of a recurrent neural network. :param layer_type: The type of recurrent layer (RNN, LSTM, GRU). :type layer_type: str :param model: Type of model (s2s, s2as, s2fh, s2mh) :type model: str :param input_dimension: Number of dimensions of the feature space of the input data. :type input_dimension: int :param hidden_dimensions: Number of dimensions of each hidden state for all recurrent layers. :type hidden_dimensions: Union[List[int], int] :param n_recurrent_layers: Number of recurrent layers, if hidden_dimensions is the same for all layers (int). :type n_recurrent_layers: Optional[int] :param recurrent_activation: Activation function to use in each recurrent layer (has to be defined in torch.nn). :type recurrent_activation: str :param recurrent_bias: Whether to use bias in the computations of the recurrent layers. :type recurrent_bias: bool :param bidirectional: Use bidirectional recurrent layers. :type bidirectional: bool :param projection_size: Parameter relevant for LSTM layers only. See pytorch docs for details. :type projection_size: int .. py:attribute:: dilation :value: None .. py:attribute:: dropout .. py:attribute:: recurrent_layers .. py:attribute:: model .. py:method:: apply_dilated_layer(layer: torch.nn.Module, inputs: Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence], dilation: int, hidden: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]] = None) -> Tuple[Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence], Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]] .. py:method:: forward(inputs: Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence], hidden_states: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]] = None, return_hidden: bool = False) -> Union[Tuple[torch.Tensor, Ellipsis], torch.Tensor, torch.nn.utils.rnn.PackedSequence] Apply the base RNN to the sequence and return the hidden states from every step in the sequence. :param inputs: Tensor or sequence of shape (T, B, ...) :param hidden_states: Hidden states of shape (num_layers, B, ...). For LSTM this must be a tuple of two tensors. :param return_hidden: Whether the method should return hidden states of the RNN too :return: If return_hidden=False this is a tensor or PackedSequence of shape (T, B, ...), otherwise a Tuple (output, hidden) .. py:class:: TCN(input_dim: int, nb_filters: Union[int, Sequence[int]] = 64, kernel_size: int = 3, nb_stacks: int = 1, dilations: List[int] = (1, 2, 4, 8, 16, 32), padding: str = 'same', use_skip_connections: bool = True, dropout_rate: float = 0.0, return_sequences: bool = False, activation: Union[str, Callable] = 'relu', use_batch_norm: bool = False, use_layer_norm: bool = False) Bases: :py:obj:`torch.nn.Module` Creates a TCN layer. :param nb_filters: The number of filters to use in the convolutional layers. Can be a list. :param kernel_size: The size of the kernel to use in each convolutional layer. :param dilations: The list of the dilations. Example is: [1, 2, 4, 8, 16, 32, 64]. :param nb_stacks: The number of stacks of residual blocks to use. :param padding: The padding to use in the convolutional layers, 'causal' or 'same'. :param use_skip_connections: Boolean. If we want to add skip connections from input to each residual blocK. :param return_sequences: Boolean. Whether to return the last output in the output sequence, or the full sequence. :param activation: The activation used in the residual blocks o = Activation(x + F(x)). :param dropout_rate: Float between 0 and 1. Fraction of the input units to drop. :param use_batch_norm: Whether to use batch normalization in the residual layers or not. :param use_layer_norm: Whether to use layer normalization in the residual layers or not. :returns: A TCN layer. Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: return_sequences :value: False .. py:attribute:: use_skip_connections :value: True .. py:attribute:: dilations :value: (1, 2, 4, 8, 16, 32) .. py:attribute:: nb_stacks :value: 1 .. py:attribute:: kernel_size :value: 3 .. py:attribute:: residual_blocks .. py:property:: receptive_field .. py:method:: forward(x: torch.Tensor) -> torch.Tensor .. py:class:: TCNResidualBlock(input_dim: int, dilation_rate: int, nb_filters: int, kernel_size: int, padding: str, activation: Union[str, Callable] = 'relu', dropout_rate: float = 0, use_batch_norm: bool = False, use_layer_norm: bool = False) Bases: :py:obj:`torch.nn.Module` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Defines the residual block for the WaveNet TCN. Input needs to be of shape (B, D, T). :param dilation_rate: The dilation power of 2 we are using for this residual block :param nb_filters: The number of convolutional filters to use in this block :param kernel_size: The size of the convolutional kernel :param padding: The padding used in the convolutional layers, 'same' or 'causal'. :param activation: The final activation used in o = Activation(x + F(x)) :param dropout_rate: Float between 0 and 1. Fraction of the input units to drop. :param use_batch_norm: Whether to use batch normalization in the residual layers or not. :param use_layer_norm: Whether to use layer normalization in the residual layers or not. .. py:attribute:: activation :value: 'relu' .. py:attribute:: dropout .. py:method:: forward(x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor] Returns: A tuple where the first element is the residual model tensor, and the second is the skip connection tensor. .. py:class:: DenseVAEEncoder(input_dim: int, hidden_dims: Sequence[int] = (100, 100), latent_dim: int = 10, activation=torch.nn.ReLU()) Bases: :py:obj:`torch.nn.Module` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: latent_dim :value: 10 .. py:attribute:: mlp .. py:attribute:: softplus .. py:method:: forward(x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor] .. py:class:: VAE(encoder: torch.nn.Module, decoder: torch.nn.Module, logvar_out: bool = True) Bases: :py:obj:`torch.nn.Module` VAE Implementation that supports normal distribution with diagonal cov matrix in the latent space and the output Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: encoder .. py:attribute:: decoder .. py:attribute:: log_var :value: True .. py:method:: forward(x: torch.Tensor, return_latent_sample: bool = False, num_samples: int = 1, force_sample: bool = False) -> Tuple[torch.Tensor, Ellipsis] .. py:class:: VAELoss(size_average=None, reduce=None, reduction: str = 'mean', logvar_out: bool = True) Bases: :py:obj:`timesead.optim.loss.Loss` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: logvar_out :value: True .. py:method:: forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) -> torch.Tensor