timesead.models.layers

Submodules

• timesead.models.layers.anom_attention
• timesead.models.layers.autocorrelation
• timesead.models.layers.autoformer_encdec
• timesead.models.layers.causal_conv
• timesead.models.layers.conv_block
• timesead.models.layers.conv_lstm
• timesead.models.layers.embed
• timesead.models.layers.fourier_correlation
• timesead.models.layers.inception
• timesead.models.layers.kervolution
• timesead.models.layers.multi_wavelet_correlation
• timesead.models.layers.planar_nf
• timesead.models.layers.same_pad

Classes

CausalConv1d	Base class for all neural network modules.
ConvLSTM	Base class for all neural network modules.
ConvLSTMCell	Base class for all neural network modules.
Kernel	Base class for all neural network modules.
LinearKernel	Base class for all neural network modules.
PolynomialKernel	Base class for all neural network modules.
RBFKernel	Base class for all neural network modules.
Kerv1d	Applies a 1D kervolution over an input signal composed of several inputplanes.
PlanarFlow	Base class for all neural network modules.
PlanarTransform	Implementation of the invertible transformation used in planar flow
SameZeroPad1d	Base class for all neural network modules.
SameCausalZeroPad1d	Base class for all neural network modules.
SameZeroPad2d	Pads the input tensor boundaries with zero.
AnomalyAttention	Base class for all neural network modules.
AttentionLayer	Base class for all neural network modules.
DataEmbedding	Base class for all neural network modules.
ConvBlock	Base class for all neural network modules.
AutoCorrelationLayer	Base class for all neural network modules.
AutoCorrelation	AutoCorrelation Mechanism with the following two phases:
FourierBlock	Base class for all neural network modules.
FourierCrossAttention	Base class for all neural network modules.
MultiWaveletCross	1D Multiwavelet Cross Attention layer.
MultiWaveletTransform	1D multiwavelet block.

Functions

calc_causal_same_pad(→ int)
calc_same_pad(→ Tuple[int, int])

Package Contents

class timesead.models.layers.CausalConv1d(in_channels: int, out_channels: int, kernel_size: torch.nn.common_types._size_1_t, stride: torch.nn.common_types._size_1_t = 1, dilation: torch.nn.common_types._size_1_t = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros', device=None, dtype=None)

Bases: torch.nn.Conv1d

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• in_channels (int)

• out_channels (int)

• kernel_size (torch.nn.common_types._size_1_t)

• stride (torch.nn.common_types._size_1_t)

• dilation (torch.nn.common_types._size_1_t)

• groups (int)

• bias (bool)

• padding_mode (str)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

causal_padding

forward(input: torch.Tensor) → torch.Tensor

Parameters:

input (torch.Tensor)

Return type:

torch.Tensor

class timesead.models.layers.ConvLSTM(*args, **kwargs)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

lstm

forward(x, hidden=None, memory=None)

input shape: (T, B, C, H, W)

class timesead.models.layers.ConvLSTMCell(in_channels: int, hid_channels: int, kernel_size: int | Tuple[int, int], spatial_size: Tuple[int, int])

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• in_channels (int)

• hid_channels (int)

• kernel_size (Union[int, Tuple[int, int]])

• spatial_size (Tuple[int, int])

Initialize internal Module state, shared by both nn.Module and ScriptModule.

hid_channels

x2h

h2h

c2c

reset_parameters()

forward(x, h, c)

class timesead.models.layers.Kernel(learnable_parameters: bool = False)

Bases: torch.nn.Module, abc.ABC

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

learnable_parameters (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

learnable_parameters = False

forward(x1: torch.Tensor, x2: torch.Tensor, diag_only: bool = False) → torch.Tensor

Compute the kernel function for inputs x1 and x2

Parameters:

• x1 (torch.Tensor) – A tensor of shape ([B1], D)

• x2 (torch.Tensor) – A tensor of shape ([B2], D)

• diag_only (bool) – Whether the entire kernel matrix should be computed or only the diagonal

Returns:

A tensor of shape ([B1] + [B2]) if diag_only = False, else a tensor of shape ([B]), where [B] is the result of broadcasting [B1] and [B2].

Return type:

torch.Tensor

class timesead.models.layers.LinearKernel(learnable_parameters: bool = False)

Bases: Kernel

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

learnable_parameters (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

class timesead.models.layers.PolynomialKernel(degree: int = 2, c0: float = 0.0, learnable_parameters: bool = False)

Bases: Kernel

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• degree (int)

• c0 (float)

• learnable_parameters (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

degree = 2

c0 = 0.0

class timesead.models.layers.RBFKernel(gamma: float = 1.0, learnable_parameters: bool = False)

Bases: Kernel

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• gamma (float)

• learnable_parameters (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

gamma = 1.0

class timesead.models.layers.Kerv1d(in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros', kernel: str | Kernel = 'linear', learnable_kernel: bool = False)

Bases: torch.nn.Conv1d

Applies a 1D kervolution over an input signal composed of several inputplanes.

Parameters:

• in_channels (int) – Number of channels in the input image

• out_channels (int) – Number of channels produced by the convolution

• kernel_size (int or tuple) – Size of the convolving kernel

• stride (int or tuple, optional) – Stride of the convolution. Default: 1

• padding (int or tuple, optional) – Zero-padding added to both sides of the input. Default: 0

• padding_mode (string, optional) – zeros

• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1

• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

• kernel (str or Kernel) – ‘linear’

• learnable_kernel (bool) – Learnable kernel parameters. Default: False

Shape:

• Input: \((N, C_{in}, L_{in})\)

• Output: \((N, C_{out}, L_{out})\) where

  \[L_{out} = \left\lfloor\frac{L_{in} + 2 \times \text{padding} - \text{dilation} \times (\text{kernel_size} - 1) - 1}{\text{stride}} + 1\right\rfloor\]

Examples

>>> m = Kerv1d(16, 33, 3, kernel='rbf')
>>> input = torch.randn(20, 16, 70)
>>> output = m(input)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

unfold

kernel = 'linear'

forward(x: torch.Tensor) → torch.Tensor

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

class timesead.models.layers.PlanarFlow(dim: int, num_layers: int = 6)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• dim (int)

• num_layers (int)

Make a planar flow by stacking planar transformations in sequence.

Parameters:

• dim (int) – Dimensionality of the distribution to be estimated.

• num_layers (int) – Number of transformations in the flow.

layers

forward(z: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor]

Parameters:

z (torch.Tensor)

Return type:

Tuple[torch.Tensor, torch.Tensor]

class timesead.models.layers.PlanarTransform(dim: int, epsilon: float = 0.0001)

Bases: torch.nn.Module

Implementation of the invertible transformation used in planar flow

f(z) = z + u * h(dot(w.T, z) + b)

See Section 4.1 in https://arxiv.org/pdf/1505.05770.pdf.

Initialise weights and bias.

Parameters:

• dim (int) – Dimensionality of the distribution to be estimated.

• epsilon (float)

epsilon = 0.0001

w

b

u

forward(z: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor]

Parameters:

z (torch.Tensor)

Return type:

Tuple[torch.Tensor, torch.Tensor]

get_u_hat() → None

Enforce w^T u >= -1. When using h(.) = tanh(.), this is a sufficient condition for invertibility of the transformation f(z). See Appendix A.1.

Return type:

None

timesead.models.layers.calc_causal_same_pad(kernel_size: int, stride: int = 1, in_shape: int = 1, dilation: int = 1) → int

Parameters:

• kernel_size (int)

• stride (int)

• in_shape (int)

• dilation (int)

Return type:

int

timesead.models.layers.calc_same_pad(kernel_size: int, stride: int = 1, in_shape: int = 1, dilation: int = 1) → Tuple[int, int]

Parameters:

• kernel_size (int)

• stride (int)

• in_shape (int)

• dilation (int)

Return type:

Tuple[int, int]

class timesead.models.layers.SameZeroPad1d(kernel_size: int, stride: int = 1, in_shape: int = 1, dilation: int = 1)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• kernel_size (int)

• stride (int)

• in_shape (int)

• dilation (int)

Replicates the “SAME” pad algorithm from Tensorflow. Note that Tensorflow will always assume stride = 1, whereas this implementation also takes different strides into account.

Parameters:

• kernel_size (int) – Kernel size that will be used

• stride (int) – Stride that will be used

• in_shape (int) – Size of the input. This is only needed if stride != 1

• dilation (int) – Dilation that will be used

padding

forward(x: torch.Tensor) → torch.Tensor

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

class timesead.models.layers.SameCausalZeroPad1d(kernel_size: int, stride: int = 1, in_shape: int = 1, dilation: int = 1)

Bases: SameZeroPad1d

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• kernel_size (int)

• stride (int)

• in_shape (int)

• dilation (int)

Replicates the “causal” pad algorithm from Tensorflow. Note that Tensorflow will always assume stride = 1, whereas this implementation also takes different strides into account.

Parameters:

• kernel_size (int) – Kernel size that will be used

• stride (int) – Stride that will be used

• in_shape (int) – Size of the input. This is only needed if stride != 1

• dilation (int) – Dilation that will be used

padding

class timesead.models.layers.SameZeroPad2d(kernel_size: torch.nn.common_types._size_2_t, stride: torch.nn.common_types._size_2_t = 1, in_shape: torch.nn.common_types._size_2_t = 1, dilation: torch.nn.common_types._size_2_t = 1)

Bases: torch.nn.ZeroPad2d

Pads the input tensor boundaries with zero.

For N-dimensional padding, use torch.nn.functional.pad().

Parameters:

• padding (int, tuple) – the size of the padding. If is int, uses the same padding in all boundaries. If a 4-tuple, uses (\(\text{padding\_left}\), \(\text{padding\_right}\), \(\text{padding\_top}\), \(\text{padding\_bottom}\))

• kernel_size (torch.nn.common_types._size_2_t)

• stride (torch.nn.common_types._size_2_t)

• in_shape (torch.nn.common_types._size_2_t)

• dilation (torch.nn.common_types._size_2_t)

Shape:

• Input: \((N, C, H_{in}, W_{in})\) or \((C, H_{in}, W_{in})\).

• Output: \((N, C, H_{out}, W_{out})\) or \((C, H_{out}, W_{out})\), where

  \(H_{out} = H_{in} + \text{padding\_top} + \text{padding\_bottom}\)

  \(W_{out} = W_{in} + \text{padding\_left} + \text{padding\_right}\)

Examples:

>>> # xdoctest: +IGNORE_WANT("non-deterministic")
>>> m = nn.ZeroPad2d(2)
>>> input = torch.randn(1, 1, 3, 3)
>>> input
tensor([[[[-0.1678, -0.4418,  1.9466],
          [ 0.9604, -0.4219, -0.5241],
          [-0.9162, -0.5436, -0.6446]]]])
>>> m(input)
tensor([[[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000, -0.1678, -0.4418,  1.9466,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.9604, -0.4219, -0.5241,  0.0000,  0.0000],
          [ 0.0000,  0.0000, -0.9162, -0.5436, -0.6446,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]])
>>> # using different paddings for different sides
>>> m = nn.ZeroPad2d((1, 1, 2, 0))
>>> m(input)
tensor([[[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000, -0.1678, -0.4418,  1.9466,  0.0000],
          [ 0.0000,  0.9604, -0.4219, -0.5241,  0.0000],
          [ 0.0000, -0.9162, -0.5436, -0.6446,  0.0000]]]])

Replicates the “SAME” pad algorithm from Tensorflow. Note that Tensorflow will always assume stride = 1, whereas this implementation also takes different strides into account.

Parameters:

• kernel_size (torch.nn.common_types._size_2_t) – Kernel size that will be used

• stride (torch.nn.common_types._size_2_t) – Stride that will be used

• in_shape (torch.nn.common_types._size_2_t) – Size of the input. This is only needed if stride != 1

• dilation (torch.nn.common_types._size_2_t) – Dilation that will be used

class timesead.models.layers.AnomalyAttention(win_size: int, mask_flag: bool = True, scale: float | None = None, attention_dropout: float = 0.0, output_attention: bool = False)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• win_size (int)

• mask_flag (bool)

• scale (Optional[float])

• attention_dropout (float)

• output_attention (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

scale = None

mask_flag = True

output_attention = False

dropout

forward(queries, keys, values, sigma, attn_mask)

class timesead.models.layers.AttentionLayer(attention: torch.nn.Module, d_model: int, n_heads: int, d_keys: int | None = None, d_values: int | None = None)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• attention (torch.nn.Module)

• d_model (int)

• n_heads (int)

• d_keys (Optional[int])

• d_values (Optional[int])

Initialize internal Module state, shared by both nn.Module and ScriptModule.

norm

inner_attention

query_projection

key_projection

value_projection

sigma_projection

out_projection

n_heads

forward(queries, keys, values, attn_mask)

class timesead.models.layers.DataEmbedding(c_in: int, d_model: int, dropout: float = 0.0, use_pos: bool = True)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• c_in (int)

• d_model (int)

• dropout (float)

• use_pos (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

value_embedding

position_embedding

dropout

forward(x)

class timesead.models.layers.ConvBlock(conv_layer, out_channels: int, activation, batch_norm: bool = False)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• out_channels (int)

• batch_norm (bool)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

conv

activation

norm

forward(x: torch.Tensor, *args, **kwargs) → torch.Tensor

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

class timesead.models.layers.AutoCorrelationLayer(correlation, d_model, n_heads, d_keys=None, d_values=None)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

inner_correlation

query_projection

key_projection

value_projection

out_projection

n_heads

forward(queries, keys, values, attn_mask)

class timesead.models.layers.AutoCorrelation(factor=1, scale=None, attention_dropout=0.1, output_attention=False)

Bases: torch.nn.Module

AutoCorrelation Mechanism with the following two phases: (1) period-based dependencies discovery (2) time delay aggregation This block can replace the self-attention family mechanism seamlessly.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

factor = 1

scale = None

output_attention = False

dropout

time_delay_agg_training(values, corr)

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the training phase.

time_delay_agg_inference(values, corr)

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the inference phase.

time_delay_agg_full(values, corr)

Standard version of Autocorrelation

forward(queries, keys, values, attn_mask)

class timesead.models.layers.FourierBlock(in_channels, out_channels, seq_len, num_heads=8, modes=0, mode_select_method='random')

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

index

scale

weights1

weights2

compl_mul1d(order, x, weights)

forward(q, k, v, mask)

class timesead.models.layers.FourierCrossAttention(in_channels, out_channels, seq_len_q, seq_len_kv, modes=64, mode_select_method='random', activation='tanh', policy=0, num_heads=8)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

activation = 'tanh'

in_channels

out_channels

index_q

index_kv

scale

weights1

weights2

compl_mul1d(order, x, weights)

forward(q, k, v, mask)

class timesead.models.layers.MultiWaveletCross(in_channels, out_channels, seq_len_q, seq_len_kv, modes, c=64, k=8, ich=512, L=0, base='legendre', mode_select_method='random', initializer=None, activation='tanh', **kwargs)

Bases: torch.nn.Module

1D Multiwavelet Cross Attention layer.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

c = 64

k = 8

L = 0

max_item = 3

attn1

attn2

attn3

attn4

T0

Lk

Lq

Lv

out

modes1

forward(q, k, v, mask=None)

wavelet_transform(x)

evenOdd(x)

class timesead.models.layers.MultiWaveletTransform(ich=1, k=8, alpha=16, c=128, nCZ=1, L=0, base='legendre', attention_dropout=0.1)

Bases: torch.nn.Module

1D multiwavelet block.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

k = 8

c = 128

L = 0

nCZ = 1

Lk0

Lk1

ich = 1

MWT_CZ

forward(queries, keys, values, attn_mask)