timesead.models.layers.autocorrelation

Classes

AutoCorrelation	AutoCorrelation Mechanism with the following two phases:
AutoCorrelationLayer	Base class for all neural network modules.

Module Contents

class timesead.models.layers.autocorrelation.AutoCorrelation(factor=1, scale=None, attention_dropout=0.1, output_attention=False)

Bases: torch.nn.Module

AutoCorrelation Mechanism with the following two phases: (1) period-based dependencies discovery (2) time delay aggregation This block can replace the self-attention family mechanism seamlessly.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

factor = 1

scale = None

output_attention = False

dropout

time_delay_agg_training(values, corr)

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the training phase.

time_delay_agg_inference(values, corr)

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the inference phase.

time_delay_agg_full(values, corr)

Standard version of Autocorrelation

forward(queries, keys, values, attn_mask)

class timesead.models.layers.autocorrelation.AutoCorrelationLayer(correlation, d_model, n_heads, d_keys=None, d_values=None)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

inner_correlation

query_projection

key_projection

value_projection

out_projection

n_heads

forward(queries, keys, values, attn_mask)