Time Series With LSTM

Time series prediction involves predicting future values of a time-dependent variable based on previous observations using statistical or machine learning models.

A time series dataset, such as stock market prices, meteorological data, or sales numbers, is examined and used to predict future values by examining historical patterns and tendencies.

It is possible to predict time series using a variety of models, ranging from simple statistical methods like moving averages and exponential smoothing to more advanced machine learning methods such as ARIMA, LSTM, or neural networks. For accurate forecasting, these models take into account seasonality, trends, and other inherent patterns in time series data.

Among other applications, time series prediction is used in financial projections, demand forecasting, and predictive maintenance. We will explore and implement time series prediction using LSTM in this article. Let’s begin.

Time series prediction is commonly performed using LSTM (Long Short-Term Memory) networks, which are powerful variants of Recurrent Neural Networks (RNNs). Using PyTorch, we will implement an LSTM for time series prediction.

With PyTorch, the first step is to load and preprocess the time series data before creating an LSTM. Data loading, normalization, and partitioning into training and testing subsets are some of the substeps included in this process.

The first step is to load the time series data

To begin with, the time series data must be loaded, which can be done in a number of ways depending on the format of the data. CSV files can be loaded into data frames using the pandas library, for example. Here is an example code snippet that shows how to load time series data from a CSV file:

import pandas as pd

# Load the time series data from a CSV file
df = pd.read_csv('time_series_data.csv')

A Pandas DataFrame object called df is created by loading time series data from a CSV file called time_series_data.csv.

The first line imports the Pandas library and assigns it the alias ‘pd’. To make it easier to refer to the library later in the code, this is a common convention.

A DataFrame object named ‘df’ is created from the contents of the CSV file ‘time_series_data.csv’ using the Pandas function ‘read_csv()’. The function returns a DataFrame object containing the data from the CSV file as an argument.

This code loads time series data from a CSV file into a Python DataFrame object using the Pandas library. With Pandas and other Python libraries, the data can be manipulated, visualized, and analyzed.

Step 2: Standardize the Time Series Data

The next step is to standardize the time series data. The LSTM’s performance can be enhanced by standardizing the data, which ensures that all values are on a uniform scale. A prevalent method of standardization is to scale the data to values between 0 and 1, using the minimum and maximum values of the data. An example of standardizing time series data can be found in the code snippet below:

import numpy as np

# Normalize the time series data
data = df['value'].values.astype(float)
data = (data - np.min(data)) / (np.max(data) - np.min(data))

Using the NumPy library, this code normalizes time series data loaded into a Pandas DataFrame object ‘df’.

In the first line, the NumPy library is imported and assigned the alias ‘np’.

The second line extracts the values from the ‘value’ column of the DataFrame object ‘df’ and converts them to a NumPy array of floating-point numbers using the ‘astype()’ method.

NumPy array ‘data’ is normalized in the third line. It subtracts the minimum value of the array from each element, then divides by the range (i.e., the difference between the maximum and minimum values). The result is a normalized array with values ranging from 0 to 1.

The code performs a common preprocessing step for time series data by scaling the values to a common range. It will be easier to compare different time series or to feed the data into machine learning algorithms if this is done to avoid issues caused by varying scales of the data.

The code above converts the value column into a numpy array of floats. Using the minimum and maximum values of the data, we normalize the data.

Step 3: Split the Time Series Data into Training and Testing Sets

In the next step, the time series data will be split into training and testing sets. The training set is used to train the LSTM model, while the testing set is used to evaluate its performance. It is common to use 80% of the data for training and 20% for testing when splitting the data. This code snippet shows how to split time series data into training and testing sets:

# Split the time series data into training and testing sets
train_data = data[:int(len(data)*0.8)]
test_data = data[int(len(data)*0.8):]

Using Python’s slicing syntax, this code splits the normalized time series data into training and testing sets.

The first line creates a variable ‘train_data’ that contains the first 80% of the array ‘data’. The slicing syntax ‘data[:int(len(data)*0.8)]’ means to take all the elements from the beginning of the array up to but not including the element at index ‘int(len(data)*0.8)’.

Using the second line, a new variable ‘test_data’ is created that contains the remaining 20% of the ‘data’ array. The slicing syntax ‘data[int(len(data)*0.8):]’ refers to all elements from the element at index ‘int(len(data)*0.8)’ to the end of the array.

We can train a machine learning model on the training set and then evaluate its performance on the testing set by splitting the data into training and testing sets. By doing this, we can ensure that the model does not overfit to the training data and is able to generalize well.

Using numpy array slicing, we divide the data into training and testing subsets based on the length of the data.

Step 4: Transform the Time Series Data into PyTorch Tensors

The final step involves converting the time series data into PyTorch tensors. LSTM models use PyTorch tensors as input and output. Here is an example of how to transform time series data into PyTorch tensors:

import torch

# Convert the time series data to PyTorch tensors
train_data = torch.FloatTensor(train_data).unsqueeze(1)
test_data = torch.FloatTensor(test_data).unsqueeze(1)

The code converts NumPy arrays into PyTorch tensors from the normalized time series data.

PyTorch is imported in the first line.

Using the torch.FloatTensor() method, the second line converts the NumPy array ‘train_data’ to a PyTorch FloatTensor object. By calling the ‘unsqueeze(1)’ method, a new dimension is added to the tensor at position 1. PyTorch expects input data to be in the form of [batch size, sequence length, number of features], and we have only a single feature.

The third line converts and unsqueezes the NumPy array ‘test_data’.

To use the data as input for deep learning models built with PyTorch, the data must be converted to PyTorch tensors. Python’s tensor data structure provides many useful features, including automatic differentiation, GPU acceleration, and compatibility with PyTorch’s neural network modules.

The code provided imports the torch library at the beginning. Using torch.FloatTensor(), we transform the training and testing data into PyTorch tensors. As a final step, we use unsqueeze() to add an additional dimension to the data, aligning it with LSTM’s input shape.

You can use PyTorch to load and preprocess time series data for LSTM time series prediction by following these steps.

The next step is to establish the LSTM model once the data has been loaded and preprocessed. An LSTM (Long Short-Term Memory) is a recurrent neural network (RNN) variant commonly used in time series prediction. LSTMs are designed to handle data sequences with long-term dependencies, making them suitable for time series prediction.

Using PyTorch, the following code snippet illustrates how to define an LSTM model for time series prediction:

import torch
import torch.nn as nn

class LSTM(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size, hidden_size)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, input):
        batch_size = input.size(0)
        hidden = self.init_hidden(batch_size)
        lstm_out, hidden = self.lstm(input.view(len(input), batch_size, -1), hidden)
        output = self.fc(lstm_out[-1])
        return output

    def init_hidden(self, batch_size):
        return (torch.zeros(1, batch_size, self.hidden_size),
                torch.zeros(1, batch_size, self.hidden_size))

LSTM is a long short-term memory (LSTM) neural network that is used for sequence prediction in PyTorch.

This module inherits from the PyTorch ‘nn.Module’ class, which provides basic neural network functionality such as parameter management and backpropagation. The constructor method ‘init()’ initializes the LSTM network with the input, hidden, and output sizes specified. There are two layers in an LSTM network: an LSTM layer and a linear layer. Based on an input of size ‘input_size’, the LSTM layer produces a hidden state of size ‘hidden_size’. In the linear layer, the final hidden state of the LSTM layer is mapped to an output of size ‘output_size’.

This method implements the forward pass of the LSTM network. The ‘init_hidden()’ method is used to initialize the hidden state of the LSTM layer. Following that, it applies the LSTM layer to the input sequence, resulting in an output sequence and a final hidden state. The final hidden state is then passed through the linear layer to produce the output prediction.

‘init_hidden()’ initializes the hidden state of the LSTM layer with zeros, given a batch size.

The code defines a basic LSTM network module for sequence prediction tasks, which can be trained and used for time series predictions.

The provided code establishes an LSTM model using PyTorch’s nn.Module class. There are three layers in the LSTM model: an input layer, an LSTM layer, and an output layer. An input layer receives a data sequence, which is then passed to the LSTM layer to produce a hidden state. In order to generate the predicted value, the hidden state is passed through the output layer.

Init() initializes the LSTM model’s parameters, such as input size, hidden size, and output size. In the forward() function, an input sequence is accepted and a predicted value is returned. An LSTM layer’s hidden state is initialized with init_hidden().

LSTM models are defined by setting the input and output dimensions, determining the number of LSTM layers and hidden units, and specifying the activation function. Using the preprocessed time series data, the model can be trained and tested.

Using the preprocessed time series data, the model is then educated. In this phase, we will establish the training loop, which includes calculating the loss, updating the model’s parameters, and iterating through the training data for a specified number of epochs.

In the example code snippet below, we demonstrate how to train an LSTM model for time series prediction using PyTorch:

import torch.optim as optim

# Define the hyperparameters
input_size = ...
hidden_size = ...
output_size = ...
learning_rate = ...
num_epochs = ...

# Instantiate the model
model = LSTM(input_size, hidden_size, output_size)

# Define the loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=learning_rate)

# Train the model
for epoch in range(num_epochs):
    for i, (inputs, labels) in enumerate(train_loader):
        optimizer.zero_grad()

        # Forward pass
        outputs = model(inputs)
        loss = criterion(outputs, labels)

        # Backward pass and optimization
        loss.backward()
        optimizer.step()

        if (i+1) % 100 == 0:
            print('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
                  .format(epoch+1, num_epochs, i+1, len(train_loader), loss.item()))

The code below shows how to train an LSTM neural network model for time series data using PyTorch. To set up the model, parameters such as input_size, hidden_size, output_size, learning_rate, and num_epochs are defined. Using the hyperparameters, an LSTM model is created. Using PyTorch’s ‘nn.MSELoss()’ class, we define the loss function as the mean square error (MSE), and the optimizer as Adam using the ‘optim.Adam()’ method. In order to train the LSTM model, the training data is batched loaded. The forward pass of the network is calculated, the MSE loss is calculated between the predicted and actual output, and the gradients are calculated. Gradients are then used to update the model parameters. Every 100th iteration, the code prints the loss. As a result of the training loop, a final trained LSTM model can be used to make predictions about new time series data.

In the provided code, we specify the model’s hyperparameters, including input size, hidden size, output size, learning rate, and epoch number. Next, we define the loss function and optimizer for the LSTM model. A mean square error (MSE) is used as the loss function and the Adam optimizer is used to adjust the model’s parameters.

For a predetermined number of epochs, the training loop iterates through the training data. Each epoch, we cycle through the training data batches and perform the following steps:

1. To prevent accumulation from previous iterations, reset the gradients
2. To generate predicted values, run the model forward
3. Calculate the difference between the predicted and actual values
4. To compute the gradients of the loss for the model’s parameters, perform a backward pass
5. Utilize the optimizer to update the model’s parameters

To track training progress, we can display the loss for each iteration.

An LSTM model for time series prediction in PyTorch is trained by defining hyperparameters, loss function, and optimizer, and iterating through the training data.

A test dataset is used to evaluate the model’s performance. To compute the test loss and other evaluation metrics, we will calculate the predictions for the test data and compare them to the actual values.

PyTorch code snippet below shows how to test an LSTM model for time series prediction:

# Set the model to evaluation mode
model.eval()

# Define the test loop
test_loss = 0.0
with torch.no_grad():
    for inputs, labels in test_loader:
        # Forward pass
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        test_loss += loss.item()

    # Compute the evaluation metrics
    avg_test_loss = test_loss / len(test_loader)
    print('Average Test Loss: {:.4f}'.format(avg_test_loss))

It disables dropout and batch normalization layers by setting the LSTM neural network model to evaluation mode using the ‘model.eval()’ method. A test loop is then defined to evaluate the model’s performance on the test set. A zero value is assigned to the variable ‘test_loss’. With PyTorch’s ‘torch.no_grad()’ context manager, the loop iterates over the test data, applying the LSTM network to each input sequence and computing the MSE loss. In the ‘test_loss += loss.item()’ line, the loss of each batch is added together. Using the ‘print()’ method, the average test loss is calculated by dividing the total test loss by the number of batches. By computing the average test loss, this code evaluates the performance of the trained LSTM model on the test set.

In the provided code, the eval() method is used to set the model to evaluation mode. PyTorch relies on this information to avoid modifying parameters during testing. As a next step, we define the test loop, which iterates through the test data and executes the following steps:

1. Conduct a forward pass through the model to generate predicted values
2. Calculate the loss between the predicted values and the actual values
3. Accumulate the test loss over all the test data batches

As a result of completing the test loop, we calculate the average test loss by dividing the accumulated test loss by the number of test data batches. To assess the model’s performance on the test dataset, we can display the average test loss.

For a more comprehensive evaluation, we can also calculate other evaluation metrics, such as root mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination (R-squared).

Testing an LSTM model for time series prediction in PyTorch involves setting the model to evaluation mode and iterating through the test data to determine predictions and evaluation metrics.

The next step is to generate predictions using the trained LSTM model on new, unseen data, after training an LSTM model in PyTorch and assessing its performance on a test dataset. Input sequences are fed into a trained LSTM model, which produces the corresponding output sequence.

This code snippet shows how to make predictions using an LSTM model in PyTorch:

# Set the model to evaluation mode
model.eval()

# Initialize the input sequence for prediction
inputs = torch.tensor(test_data[-input_size:].reshape(1, input_size, 1), dtype=torch.float32)

# Generate predictions for the next num_predictions time steps
predictions = []
with torch.no_grad():
    for i in range(num_predictions):
        # Forward pass
        outputs = model(inputs)

        # Save the prediction
        predictions.append(outputs.item())

        # Update the input sequence for the next time step
        inputs = torch.cat((inputs[:, 1:, :], outputs.reshape(1, 1, 1)), axis=1)

# Print the predicted sequence
print('Predicted Sequence:', predictions)

Using a trained LSTM neural network model, this code predicts future time steps of a time series. Using ‘model.eval()’, we set the LSTM model to evaluation mode, which disables dropout and batch normalization layers. The LSTM network is then initialized with the last ‘input_size’ data points from the test data, reshaped to match the expected input shape. Predicted values are stored in the ‘predictions’ list. A loop is used to generate ‘num_predictions’ predictions within the ‘torch.no_grad()’ context. In each iteration, the LSTM model uses the ‘inputs’ tensor as input and uses the ‘model(inputs)’ method to predict the output. By using the ‘predictions.append(outputs.item())’ line, the predicted value is added to the ‘predictions’ list. After removing the oldest value, adding the predicted value, and reshaping to match the expected input shape of the LSTM network, the ‘inputs’ tensor is updated for the next time step. The predictions list contains the predicted values for the next num_predictions time steps after the loop. The ‘print()’ method is used to print the predicted sequence. This code shows how to use a trained LSTM model to generate such predictions based on past observations of a time series.

We set the model to evaluation mode using the eval() method in the provided code. To prepare the input sequence for prediction, we select the last input_size data points from the test dataset and reshape them into the model’s input format. Following that, we iteratively input the sequence into the model and update the input sequence with the predicted output for the next num_predictions time steps.

To visualize the predicted values, the predictions list can be printed or plotted. Time series prediction inherently contains uncertainty, so the predicted values might not perfectly align with the actual values.

In summary, when making predictions using an LSTM model for time series prediction in PyTorch, the input sequence is initialized, the trained model is fed the input sequence iteratively, and the input sequence is updated with the predicted output for each time step.

Visualizing the results of LSTM model testing and generating predictions in PyTorch is crucial for understanding the model’s performance. We will compare the predicted values against the actual values in this phase.

The example code snippet below demonstrates how to visualize the test results for LSTM time series prediction using PyTorch:

# Set the model to evaluation mode
model.eval()

# Generate predictions for the test dataset
predictions = []
with torch.no_grad():
    for inputs, _ in test_loader:
        # Forward pass
        outputs = model(inputs)

        # Save the predictions
        predictions += outputs.squeeze().tolist()

# Convert the predictions and actual values to numpy arrays
predictions = np.array(predictions)
actuals = test_targets.numpy()

# Plot the predicted and actual values
plt.plot(actuals, label='Actuals')
plt.plot(predictions, label='Predictions')
plt.xlabel('Time')
plt.ylabel('Value')
plt.legend()
plt.show()

This code snippet sets the LSTM model to evaluation mode and generates predictions for the test dataset. For performance and memory optimization, torch.no_grad() is used to disable gradient calculation. Iterates through the test_loader, performs a forward pass, and saves predictions. For visualization, predictions and actual values are converted to numpy arrays. Last but not least, it plots both the actual and predicted values against time, displaying the resulting graph using Matplotlib.

A model is first set to evaluation mode using the eval() method in the provided code. Predictions for the test dataset are then generated by iterating through the test data and executing a forward pass through the model to produce the predicted values.

After converting the predicted and actual values to numpy arrays, they are plotted using the matplotlib library. Blue represents the actual value, while orange represents the predicted value. Time is represented on the x-axis, and value is represented on the y-axis.

Visualizing the predicted and actual values in PyTorch allows us to assess how closely they align and evaluate the LSTM model’s performance. A model performs well if the predicted values closely match the actual values. Model tuning may be necessary if significant discrepancies exist between predicted and actual values.

Thanks for reading folks, see you in the next one!