timesead.models.reconstruction

Submodules

Classes

`LSTMAE`	Generic LSTMAE implementation
`LSTMAEMalhotra2016`	Implementation of Malhotra 2016 (https://arxiv.org/pdf/1607.00148.pdf, default parameters)
`LSTMAEMirza2018`	Mirza 2018 (http://repository.bilkent.edu.tr/bitstream/handle/11693/50234/Computer_network_intrusion_detection_using_sequential_LSTM_neural_networks_autoencoders.pdf?sequence=1)
`LSTMAEAnomalyDetector`	Base class for all neural network modules.
`MSCRED`	input is signature matrices of shape (Seq_len, Batch, Channel, Height, Width)
`MSCREDLoss`	Base class for all neural network modules.
`MSCREDAnomalyDetector`	Base class for all neural network modules.
`MSCREDAnomalyDetectorOrig`	Base class for all neural network modules.
`SignatureMatrixTransform`
`TCNAE`	A class used to represent the Temporal Convolutional Autoencoder (TCN-AE).
`TCNAEAnomalyDetector`	Base class for all neural network modules.
`USADModel`	Base class for all neural network modules.
`BasicAE`	What I believe to be the basic "AE" model from the USAD paper
`USADDecoder1Loss`	Base class for all neural network modules.
`USADDecoder2Loss`	Base class for all neural network modules.
`USADAnomalyDetector`	Base class for all neural network modules.
`AnomalyTransformer`	Base class for all neural network modules.
`AnomTransf_Loss`	Base class for all neural network modules.
`AnomTransf_Trainer`
`AnomTransf_AnomalyDetector`	Base class for all neural network modules.
`TimesNet`	Paper link: https://openreview.net/pdf?id=ju_Uqw384Oq
`Autoformer`	Autoformer is the first method to achieve the series-wise connection,
`FEDformer`	FEDformer performs the attention mechanism on frequency domain and achieved O(N) complexity
`ETSformer`	Paper link: https://arxiv.org/abs/2202.01381

Package Contents

class timesead.models.reconstruction.LSTMAE(input_dimension: int, hidden_dimensions=None, latent_pooling: str | Callable = 'last', decoder_class: Type[LSTMAEDecoder] = LSTMAEDecoderReverse, return_latent: bool = False)

Bases: timesead.models.common.AE, timesead.models.BaseModel

Generic LSTMAE implementation

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

input_dimension (int)
latent_pooling (Union[str, Callable])
decoder_class (Type[LSTMAEDecoder])
return_latent (bool)

encode(x: torch.Tensor) → List[torch.Tensor]

Parameters:: x (torch.Tensor)
Return type:: List[torch.Tensor]

forward(inputs: Tuple[torch.Tensor]) → torch.Tensor | Tuple[torch.Tensor, torch.Tensor]

Parameters:: inputs (Tuple[torch.Tensor]) – Tuple with a single tensor of shape (T, B, D)
Returns:: tensor of shape (T, B, D)
Return type:: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

class timesead.models.reconstruction.LSTMAEMalhotra2016(input_dimension: int, hidden_dimensions=None)

Bases: LSTMAE

Implementation of Malhotra 2016 (https://arxiv.org/pdf/1607.00148.pdf, default parameters)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: input_dimension (int)

class timesead.models.reconstruction.LSTMAEMirza2018(input_dimension: int, hidden_dimensions: List[int] = [64], latent_pooling: str = 'mean')

Bases: LSTMAE

Mirza 2018 (http://repository.bilkent.edu.tr/bitstream/handle/11693/50234/Computer_network_intrusion_detection_using_sequential_LSTM_neural_networks_autoencoders.pdf?sequence=1)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

input_dimension (int)
hidden_dimensions (List[int])
latent_pooling (str)

sigmoid

forward(inputs: Tuple[torch.Tensor]) → torch.Tensor | Tuple[torch.Tensor, torch.Tensor]

Parameters:: inputs (Tuple[torch.Tensor]) – Tuple with a single tensor of shape (T, B, D)
Returns:: tensor of shape (T, B, D)
Return type:: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]

class timesead.models.reconstruction.LSTMAEAnomalyDetector(model: LSTMAE)

Bases: timesead.models.common.AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: model (LSTMAE)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.MSCRED(n_features: int, in_channels: int, c_out: int = 256, small_model: bool = False, chi: float = 5.0)

Bases: timesead.models.BaseModel

input is signature matrices of shape (Seq_len, Batch, Channel, Height, Width)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

n_features (int)
in_channels (int)
c_out (int)
small_model (bool)
chi (float)

small_model = False

chi = 5.0

enc1

lstm1

forward(inputs: Tuple[torch.Tensor]) → torch.Tensor

Parameters:: inputs (Tuple[torch.Tensor])
Return type:: torch.Tensor

class timesead.models.reconstruction.MSCREDLoss

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

mse_loss

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.reconstruction.MSCREDAnomalyDetector(model: MSCRED)

Bases: timesead.models.common.MSEReconstructionAnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: model (MSCRED)

This is what Florian uses, but in the paper they describe sth. completely different. They compute the number of badly reconstructed entries in the signature matrix (i.e., higher than some threshold) and use that count as the anomaly score

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.MSCREDAnomalyDetectorOrig(model: MSCRED, error_threshold: float = 0.5)

Bases: timesead.models.common.AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

model (MSCRED)
error_threshold (float)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

error_threshold = 0.5

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.SignatureMatrixTransform(parent: timesead.data.transforms.Transform, wins: Tuple[int] = (10, 30, 60), seg_interval: int = 10, h: int = 5)

Bases: timesead.data.transforms.WindowTransform

Parameters:

parent (timesead.data.transforms.Transform)
wins (Tuple[int])
seg_interval (int)
h (int)

wins = (10, 30, 60)

seg_interval = 10

h = 5

property num_features: int | Tuple[int, Ellipsis]

Return type:: Union[int, Tuple[int, Ellipsis]]

class timesead.models.reconstruction.TCNAE(input_dimension: int, dilations: List[int] = (1, 2, 4, 8, 16), nb_filters: int | List[int] = 20, kernel_size: int = 20, nb_stacks: int = 1, padding: str = 'same', dropout_rate: float = 0.0, filters_conv1d: int = 8, activation_conv1d: str | Callable = 'linear', latent_sample_rate: int = 42, pooler: Type[torch.nn.Module] = torch.nn.AvgPool1d)

Bases: timesead.models.BaseModel

A class used to represent the Temporal Convolutional Autoencoder (TCN-AE). Loss for this is logcosh

Parameters:

ts_dimension (int) – The dimension of the time series (default is 1)
dilations (tuple) – The dilation rates used in the TCN-AE model (default is (1, 2, 4, 8, 16))
nb_filters (int) – The number of filters used in the dilated convolutional layers. All dilated conv. layers use the same number of filters (default is 20)
input_dimension (int)
kernel_size (int)
nb_stacks (int)
padding (str)
dropout_rate (float)
filters_conv1d (int)
activation_conv1d (Union[str, Callable])
latent_sample_rate (int)
pooler (Type[torch.nn.Module])

tcn_enc

conv1d

activation = 'linear'

pooler

tcn_dec

linear

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – Tuple with single Tensor of shape (B, T, D)
Returns:
Return type:: torch.Tensor

class timesead.models.reconstruction.TCNAEAnomalyDetector(model: TCNAE, offline_window_size: int = 128)

Bases: timesead.models.common.AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

model (TCNAE)
offline_window_size (int)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

offline_window_size = 128

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.USADModel(w_size: int, z_size: int)

Bases: timesead.models.BaseModel

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

w_size (int)
z_size (int)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

encoder

decoder1

decoder2

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – Tuple with one tensor of shape (B, T, D)
Returns:
Return type:: Tuple[torch.Tensor, Ellipsis]

grouped_parameters()

class timesead.models.reconstruction.BasicAE(w_size: int, z_size: int = 40)

Bases: timesead.models.BaseModel

What I believe to be the basic “AE” model from the USAD paper

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

w_size (int)
z_size (int)

encoder

decoder

forward(inputs: Tuple[torch.Tensor]) → torch.Tensor

Parameters:: inputs (Tuple[torch.Tensor])
Return type:: torch.Tensor

class timesead.models.reconstruction.USADDecoder1Loss(size_average=None, reduce=None, reduction: str = 'mean')

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: reduction (str)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.reconstruction.USADDecoder2Loss(size_average=None, reduce=None, reduction: str = 'mean')

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: reduction (str)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.reconstruction.USADAnomalyDetector(model: USADModel, alpha: float = 0.5)

Bases: timesead.models.common.AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

model (USADModel)
alpha (float)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

alpha = 0.5

beta = 0.5

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.AnomalyTransformer(win_size: int, input_dim: int, d_model: int = 512, n_heads: int = 8, e_layers: int = 3, d_ff: int = 512, dropout: float = 0.0, activation: str = 'gelu', output_attention: bool = True)

Bases: timesead.models.BaseModel

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

win_size (int)
input_dim (int)
d_model (int)
n_heads (int)
e_layers (int)
d_ff (int)
dropout (float)
activation (str)
output_attention (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

output_attention = True

embedding

encoder

projection

forward(x: torch.Tensor) → Tuple[torch.Tensor, Ellipsis]

Parameters:: x (torch.Tensor)
Return type:: Tuple[torch.Tensor, Ellipsis]

class timesead.models.reconstruction.AnomTransf_Loss(lamb: float = 3.0)

Bases: timesead.optim.loss.Loss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: lamb (float)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

lamb = 3.0

mse_loss

calc_association_disc(series: List[torch.Tensor], priors: List[torch.Tensor]) → Tuple[torch.Tensor, torch.Tensor]

Parameters:

series (List[torch.Tensor])
priors (List[torch.Tensor])

Return type:

Tuple[torch.Tensor, torch.Tensor]

forward(predictions: Tuple[torch.Tensor, Ellipsis], targets: Tuple[torch.Tensor, Ellipsis] = None, *args, **kwargs) → torch.Tensor

Parameters:

predictions (Tuple[torch.Tensor, Ellipsis])
targets (Tuple[torch.Tensor, Ellipsis])

Return type:

torch.Tensor

class timesead.models.reconstruction.AnomTransf_Trainer(*args, **kwargs)

Bases: timesead.optim.trainer.Trainer

validate_batch(network: torch.nn.Module, val_metrics: Dict[str, Callable], b_inputs: Tuple[torch.Tensor, Ellipsis], b_targets: Tuple[torch.Tensor, Ellipsis], *args, **kwargs) → Dict[str, float]

Parameters:

network (torch.nn.Module)
val_metrics (Dict[str, Callable])
b_inputs (Tuple[torch.Tensor, Ellipsis])
b_targets (Tuple[torch.Tensor, Ellipsis])

Return type:

Dict[str, float]

train_batch(network: AnomalyTransformer, losses: List[timesead.optim.loss.Loss], optimizers: List[torch.optim.Optimizer], epoch: int, num_epochs: int, b_inputs: Tuple[torch.Tensor, Ellipsis], b_targets: Tuple[torch.Tensor, Ellipsis]) → List[float]

Parameters:

network (AnomalyTransformer)
losses (List[timesead.optim.loss.Loss])
optimizers (List[torch.optim.Optimizer])
epoch (int)
num_epochs (int)
b_inputs (Tuple[torch.Tensor, Ellipsis])
b_targets (Tuple[torch.Tensor, Ellipsis])

Return type:

List[float]

class timesead.models.reconstruction.AnomTransf_AnomalyDetector(model: AnomalyTransformer)

Bases: timesead.models.common.AnomalyDetector

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.
Parameters:: model (AnomalyTransformer)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

model

compute_online_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the online anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (B,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

abstract compute_offline_anomaly_score(inputs: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Compute the offline anomaly score for a batch of inputs. The output tensor must have the same shape as the output of format_targets when called with the corresponding targets for this batch. This method expects a window (or a batch of windows) as its input and should return a score for the last point in the window.

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis]) – tuple of input tensors
Returns:: Tensor of shape (N,) that contains the anomaly scores for this batch
Return type:: torch.Tensor

fit(dataset: torch.utils.data.DataLoader) → None

Fit this anomaly detector on a dataset. Note that we assume only normal data here.

Parameters:: dataset (torch.utils.data.DataLoader) – A dataset
Return type:: None

format_online_targets(targets: Tuple[torch.Tensor, Ellipsis]) → torch.Tensor

Format the labels for a batch of targets. The output tensor must have the same shape as the output of compute_online_anomaly_score when called with the corresponding inputs for this batch.

Parameters:: targets (Tuple[torch.Tensor, Ellipsis]) – tuple of target tensors
Returns:: Tensor of shape (B,) that contains the ground truth labels for this batch
Return type:: torch.Tensor

class timesead.models.reconstruction.TimesNet(window_size: int, input_dim: int, top_k: int = 5, d_model: int = 64, d_ff: int = 64, num_kernels: int = 8, e_layers: int = 2, dropout: float = 0.1)

Bases: timesead.models.BaseModel

Paper link: https://openreview.net/pdf?id=ju_Uqw384Oq

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

window_size (int)
input_dim (int)
top_k (int)
d_model (int)
d_ff (int)
num_kernels (int)
e_layers (int)
dropout (float)

seq_len

model

enc_embedding

layer = 2

layer_norm

projection

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: Tuple[torch.Tensor, Ellipsis]

class timesead.models.reconstruction.Autoformer(window_size: int, input_dim: int, moving_avg: int = 25, model_dim: int = 128, dropout: float = 0.1, attention_factor: int = 1, num_heads: int = 8, fcn_dim: int = 128, activation: str = 'gelu', encoder_layers: int = 3)

Bases: timesead.models.BaseModel

Autoformer is the first method to achieve the series-wise connection, with inherent O(LlogL) complexity Paper link: https://openreview.net/pdf?id=I55UqU-M11y

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

window_size (int)
input_dim (int)
moving_avg (int)
model_dim (int)
dropout (float)
attention_factor (int)
num_heads (int)
fcn_dim (int)
activation (str)
encoder_layers (int)

seq_len

decomp

enc_embedding

encoder

projection

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: Tuple[torch.Tensor, Ellipsis]

class timesead.models.reconstruction.FEDformer(window_size: int, input_dim: int, moving_avg: int = 25, model_dim: int = 128, dropout: float = 0.1, num_heads: int = 8, fcn_dim: int = 128, activation: str = 'gelu', encoder_layers: int = 3, version: str = 'fourier', mode_select: str = 'random', modes: int = 32)

Bases: timesead.models.BaseModel

FEDformer performs the attention mechanism on frequency domain and achieved O(N) complexity Paper link: https://proceedings.mlr.press/v162/zhou22g.html

version: str, for FEDformer, there are two versions to choose, options: [Fourier, Wavelets]. mode_select: str, for FEDformer, there are two mode selection method, options: [random, low]. modes: int, modes to be selected.

Parameters:

window_size (int)
input_dim (int)
moving_avg (int)
model_dim (int)
dropout (float)
num_heads (int)
fcn_dim (int)
activation (str)
encoder_layers (int)
version (str)
mode_select (str)
modes (int)

seq_len

version = 'fourier'

mode_select = 'random'

modes = 32

decomp

enc_embedding

encoder

projection

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: Tuple[torch.Tensor, Ellipsis]

class timesead.models.reconstruction.ETSformer(window_size: int, input_dim: int, model_dim: int = 128, dropout: float = 0.1, num_heads: int = 8, fcn_dim: int = 128, encoder_layers: int = 3, activation: str = 'gelu', top_k: int = 5)

Bases: timesead.models.BaseModel

Paper link: https://arxiv.org/abs/2202.01381

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:

window_size (int)
input_dim (int)
model_dim (int)
dropout (float)
num_heads (int)
fcn_dim (int)
encoder_layers (int)
activation (str)
top_k (int)

seq_len

enc_embedding

encoder

decoder

forward(inputs: Tuple[torch.Tensor, Ellipsis]) → Tuple[torch.Tensor, Ellipsis]

Parameters:: inputs (Tuple[torch.Tensor, Ellipsis])
Return type:: Tuple[torch.Tensor, Ellipsis]