timesead.models.common.rnn

Implementations of basic recurrent neural networks.

Classes

RNN

Base class for all neural network modules.

Module Contents

class timesead.models.common.rnn.RNN(layer_type: str, model: str, input_dimension: int, hidden_dimensions: Sequence[int] | int, n_recurrent_layers: int | None = None, recurrent_activation: str = 'tanh', recurrent_bias: bool = True, bidirectional: bool = False, projection_size: int = 0, dilation: Sequence[int] | None = None, dropout: float = 0.0)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

  • layer_type (str)

  • model (str)

  • input_dimension (int)

  • hidden_dimensions (Union[Sequence[int], int])

  • n_recurrent_layers (Optional[int])

  • recurrent_activation (str)

  • recurrent_bias (bool)

  • bidirectional (bool)

  • projection_size (int)

  • dilation (Optional[Sequence[int]])

  • dropout (float)

General framework of a recurrent neural network.

Parameters:

  • layer_type (str) – The type of recurrent layer (RNN, LSTM, GRU).

  • model (str) – Type of model (s2s, s2as, s2fh, s2mh)

  • input_dimension (int) – Number of dimensions of the feature space of the input data.

  • hidden_dimensions (Union[List[int], int]) – Number of dimensions of each hidden state for all recurrent layers.

  • n_recurrent_layers (Optional[int]) – Number of recurrent layers, if hidden_dimensions is the same for all layers (int).

  • recurrent_activation (str) – Activation function to use in each recurrent layer (has to be defined in torch.nn).

  • recurrent_bias (bool) – Whether to use bias in the computations of the recurrent layers.

  • bidirectional (bool) – Use bidirectional recurrent layers.

  • projection_size (int) – Parameter relevant for LSTM layers only. See pytorch docs for details.

  • dilation (Optional[Sequence[int]])

  • dropout (float)

dilation = None
dropout
recurrent_layers
model
apply_dilated_layer(layer: torch.nn.Module, inputs: torch.Tensor | torch.nn.utils.rnn.PackedSequence, dilation: int, hidden: torch.Tensor | Tuple[torch.Tensor, torch.Tensor] = None) → Tuple[torch.Tensor | torch.nn.utils.rnn.PackedSequence, torch.Tensor | Tuple[torch.Tensor, torch.Tensor]]
Parameters:
Return type:

Tuple[Union[torch.Tensor, torch.nn.utils.rnn.PackedSequence], Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]]

forward(inputs: torch.Tensor | torch.nn.utils.rnn.PackedSequence, hidden_states: torch.Tensor | Tuple[torch.Tensor, torch.Tensor] = None, return_hidden: bool = False) → Tuple[torch.Tensor, Ellipsis] | torch.Tensor | torch.nn.utils.rnn.PackedSequence

Apply the base RNN to the sequence and return the hidden states from every step in the sequence.

Parameters:
Returns:

If return_hidden=False this is a tensor or PackedSequence of shape (T, B, …), otherwise a Tuple (output, hidden)

Return type:

Union[Tuple[torch.Tensor, Ellipsis], torch.Tensor, torch.nn.utils.rnn.PackedSequence]