Predicting Stock Prices with Deep Learning Models: An Evaluation of LSTM, CNN, and ConvLSTM for Next Week’s Market Trends

In this article, we will discuss how to evaluate the performance of different deep learning models, specifically LSTM, CNN, and ConvLSTM models, on stock price prediction. We will train and test these models using a large dataset of S&P500 stock prices, and then evaluate them using various metrics, such as ROC AUC, precision, recall, and F1 score. By comparing the performance of these models, we can determine which architecture is the most effective for predicting stock prices.

We will also discuss the importance of feature selection and engineering in the prediction of stock prices. While we used a simple set of features in our previous article, we can greatly improve the accuracy of our models by including more types of features, such as technical indicators, macroeconomic variables, and news sentiment analysis.

Ultimately, the goal of this article is to provide a practical guide to developing and evaluating deep learning models for stock price prediction, with a focus on feature engineering and architecture selection. By the end of this article, readers should have a good understanding of how to evaluate the performance of different deep learning models and how to select and engineer features that are relevant for predicting stock prices.

Data Collection and Preparation

1. Collecting Stock Tickers

Before we can start predicting stock prices, we need to collect the stock data that we will be using. In the case of the S&P 500, this means collecting data for all the companies that are included in the index. One way to do this is to manually look up the tickers for each company, but this can be time-consuming and error-prone. Instead, we can automate this process using Python and the Wikipedia API.

Installing the Required Libraries

To get started, we need to install the libraries we will be using. Open up your terminal or command prompt and enter the following command:

pip install wikipedia pandas
pip install yfinance

This will install the wikipedia library, which we will use to query the Wikipedia API, and the pandas library, which we will use to store and manipulate the data.

Querying the Wikipedia API

To get a list of all the companies in the S&P 500, we can query the S&P 500 page on Wikipedia and parse the table of companies. Here’s some example code that does this:

import wikipedia
import pandas as pd

# Set the Wikipedia page title and section header
page_title = "List_of_S%26P_500_companies"
section = 0
# Query the Wikipedia API for the page content
page = wikipedia.page(page_title)
html = page.html()
# Use pandas to parse the HTML table into a DataFrame
df = pd.read_html(html, header=0, index_col=0)[section]

This code retrieves the page content for the S&P 500 page on Wikipedia, and then uses the pd.read_html() function from the pandas library to parse the HTML table into a DataFrame. The header=0 argument specifies that the first row of the table should be used as the column headers, and the index_col=0 argument specifies that the first column should be used as the row labels.

Extracting the Tickers

Now that we have the DataFrame of S&P 500 companies, we can extract the tickers for each company. Here’s some example code that does this:

tickers = df.index.tolist()

This code simply extracts the row labels (which correspond to the tickers) from the DataFrame and converts them to a list.

2. Collecting Stock Price Data

Now that we have a list of stock tickers, we can use the Yahoo Finance API to collect historical stock price data for each ticker. Here’s some example code that does this:

import yfinance as yf

# Set the date range for the historical data
start_date = "2010-01-01"
end_date = "2023-02-20"
# Loop over the tickers and collect the historical data
data = {}
for ticker in tickers:
    try:
        # Download the historical data for the ticker
        ticker_data = yf.download(ticker, start=start_date, end=end_date)
        
        # Store the data in a dictionary
        data[ticker] = ticker_data
    except Exception as e:
        print(f"Error downloading data for {ticker}: {e}")

This code uses the yfinance library to download historical stock price data for each ticker. The start and end arguments specify the date range for the data. The loop downloads the data for each ticker and stores the data in a dictionary, with the ticker symbol as the key and the data as the value. We also catch any errors that occur during the download and print a message to the console.

Handling Missing Data

It’s possible that some tickers will have missing data, either because the company went bankrupt, the stock was delisted, or for some other reason. We can check for missing data using the following code:

# Remove tickers with missing data
for ticker in list(data.keys()):
    if data[ticker].empty:
        print(ticker)
        data.pop(ticker)

This code checks if each ticker in the data dictionary is empty, and if not, deletes it.

Preprocessing the Data

Before we can use the historical stock price data to train a deep learning model, we need to preprocess it. Here are some common preprocessing steps:

• Removing NaN values: If the data contains NaN (not a number) values, we need to remove them or fill them in with some other value. We can use the fillna() method of a pandas DataFrame to fill in NaN values with the previous day's closing price, for example.
• Splitting the data into training and test sets: We need to split the data into a training set and a test set, so that we can train the model on one set of data and test it on another set. We can use the train_test_split() function from the sklearn library to do this.
• Normalizing the data: We want to scale the data so that all the features (e.g., opening price, closing price, volume) have similar ranges. We can do this by subtracting the mean and dividing by the standard deviation.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Remove NaN values
for ticker in data:
    data[ticker] = data[ticker].fillna(method="ffill")
# Split the data into training and test sets
train_data = {}
test_data = {}
for ticker in data:
    ticker_data = data[ticker]
    train_data[ticker], test_data[ticker] = train_test_split(ticker_data, test_size=0.2, shuffle=False)
# Normalize the training data
scaler = StandardScaler()
for ticker in train_data:
    train_data[ticker] = pd.DataFrame(scaler.fit_transform(train_data[ticker]), 
                                 columns=train_data[ticker].columns, 
                                 index=train_data[ticker].index)
# Normalize the test data using the same scaler object
for ticker in test_data:
    test_data[ticker] = pd.DataFrame(scaler.transform(test_data[ticker]), 
                                columns=test_data[ticker].columns, 
                                index=test_data[ticker].index)

This code uses the fillna() method to fill in any NaN values with the previous day's closing price, the train_test_split() function to split the data into training and test sets and the StandardScaler() object to normalize the data. The training data is stored in the train_data dictionary and the test data is stored in the test_data dictionary, with the ticker symbol as the key. The shuffle=False argument ensures that the data is not shuffled before splitting, which is important for time series data.

Now that we have collected and preprocessed the data, we can move on to training and evaluating different deep learning models for S&P 500 stock price prediction.

3. Defining the Objective

The objective of our deep learning models will be to predict whether the price of a given stock in the S&P 500 index one week from now will be higher than the current price. This is a common objective in financial prediction tasks, as investors often want to know whether a stock is likely to increase in value over the short term.

To accomplish this, we’ll need to create a binary label for each data point indicating whether the price one week from now is higher than the current price. We can do this by creating a new column in the data frames for each ticker called Target, which will contain a 1 if the price one week from now is higher than the current price and a 0 otherwise. Here's some example code that does this:

for ticker in train_data:
    train_data[ticker]['Target'] = (train_data[ticker]['Close'].shift(-5) > train_data[ticker]['Close']).astype(int)
for ticker in test_data:
    test_data[ticker]['Target'] = (test_data[ticker]['Close'].shift(-5) > test_data[ticker]['Close']).astype(int)

This code first loops over the training data and creates a new column called Target for each ticker in the data. It does this by shifting the Close column back by 5 rows (since we're predicting the price one week from now), comparing the shifted Close column to the original Close column, and then converting the resulting boolean values to integers (1 for True and 0 for False).

The code then does the same thing for the test data.

Now that we have our labels, we can move on to defining and training our deep learning models.

Defining and Training Deep Learning Models

With our data preprocessed and our objective defined, we can now move on to defining and training our deep learning models.

In this article, we’ll be comparing the performance of three different types of deep learning models: a Long Short-Term Memory (LSTM) model, a Convolutional Neural Network (CNN) model, and a combination of the two called a ConvLSTM model.

For example purposes we will select N tickers from all the data to make the code run faster

# Select N tickers for example purposes
n_ExampleSymbols = 20

train_data = dict(list(train_data.items())[:n_ExampleSymbols])
test_data = dict(list(test_data.items())[:n_ExampleSymbols])

LSTM Model

An LSTM is a type of recurrent neural network (RNN) that is well-suited for time series data like stock prices. It is able to “remember” information from previous time steps, which makes it particularly useful for predicting future values in a time series.

Here’s the code that defines and trains an LSTM model:

from keras.models import Sequential
from keras.layers import LSTM, Dense

def build_lstm_model(n_features):
    model = Sequential()
    model.add(LSTM(50, input_shape=(n_features, 1)))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
lstm_models = {}
for ticker in train_data:
    X_train = train_data[ticker].drop(['Target'], axis=1)
    y_train = train_data[ticker]['Target']
    n_features = X_train.shape[1]
    X_train = X_train.values.reshape((X_train.shape[0], X_train.shape[1], 1))
    lstm_models[ticker] = build_lstm_model(n_features)
    lstm_models[ticker].fit(X_train, y_train, epochs=50, verbose=0)

This code defines a function called build_lstm_model that takes the number of features in the input data as an argument and returns a compiled Keras model. The model has a single LSTM layer with 50 units and a dense output layer with a sigmoid activation function. The model is compiled with binary cross-entropy loss and the Adam optimizer.

The code then loops over each ticker in the training data and trains an LSTM model for that ticker. It does this by first separating the input data (X_train) from the target labels (y_train). It then reshapes the input data to have a third dimension of size 1, which is required for the LSTM layer.

The code then builds the LSTM model using the build_lstm_model function and trains it on the input data and labels using the fit method.

CNN Model

A CNN is a type of neural network that is commonly used for image classification tasks, but can also be used for time series data by treating the time dimension as a spatial dimension. By using convolutional filters to extract features from the time series, a CNN can be trained to identify patterns in the data that are relevant for predicting future values.

Here’s the code that defines and trains a CNN model:

from keras.layers import Conv1D, MaxPooling1D, Flatten

def build_cnn_model(n_features):
    model = Sequential()
    model.add(Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(n_features, 1)))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Flatten())
    model.add(Dense(50, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
cnn_models = {}
for ticker in train_data:
    X_train = train_data[ticker].drop(['Target'], axis=1)
    y_train = train_data[ticker]['Target']
    n_features = X_train.shape[1]
    X_train = X_train.values.reshape((X_train.shape[0], X_train.shape[1], 1))
    cnn_models[ticker] = build_cnn_model(n_features)
    cnn_models[ticker].fit(X_train, y_train, epochs=100, verbose=0)

This code defines a function called build_cnn_model that takes the number of features in the input data as an argument and returns a compiled Keras model. The model has a single convolutional layer with 64 filters, a kernel size of 2, and a ReLU activation function. It also has a max pooling layer with a pool size of 2 to downsample the output of the convolutional layer. The flattened output is then passed through two fully connected layers with 50 and 1 units, respectively. The model is compiled with binary cross-entropy loss and the Adam optimizer.

The code then loops over each ticker in the training data and trains a CNN model for that ticker. It does this in a similar way to the LSTM model by first separating the input data and labels, reshaping the input data to have a third dimension of size 1, building the CNN model using the build_cnn_model function, and training the model using the fit method.

ConvLSTM Model

A ConvLSTM model combines the convolutional and LSTM architectures by adding a convolutional layer before the LSTM layer. This allows the model to learn both spatial and temporal features from the time series data.

Here’s the code that defines and trains a ConvLSTM model:

from keras.layers import ConvLSTM2D

def build_convlstm_model(n_features):
    model = Sequential()
    model.add(ConvLSTM2D(filters=64, kernel_size=(1,2), activation='relu', input_shape=(1, 1, n_features, 1)))
    model.add(Flatten())
    model.add(Dense(50, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
convlstm_models = {}
for ticker in train_data:
    X_train = train_data[ticker].drop(['Target'], axis=1)
    y_train = train_data[ticker]['Target']
    n_features = X_train.shape[1]
    X_train = X_train.values.reshape((X_train.shape[0], 1, 1, n_features, 1))
    convlstm_models[ticker] = build_convlstm_model(n_features)
    convlstm_models[ticker].fit(X_train, y_train, epochs=100, verbose=0)

This code defines a function called build_convlstm_model that takes the number of features in the input data as an argument and returns a compiled Keras model. The model has a single ConvLSTM2D layer with 64 filters, a kernel size of (1,2), and a ReLU activation function. The output is then flattened and passed through two fully connected layers with 50 and 1 units, respectively. The model is compiled with binary cross-entropy loss and the Adam optimizer.

The code then loops over each ticker in the training data and trains a ConvLSTM model for that ticker. It does this in a similar way to the LSTM and CNN models by first separating the input data and labels, reshaping the input data to have a five-dimensional shape, building the ConvLSTM model using the build_convlstm_model function, and training the model using the fit method.

4. Model Evaluation

Once we have trained the deep learning models, we can evaluate their performance on the test data. Here’s the code for evaluating the LSTM, CNN, and ConvLSTM models:

import numpy as np
from sklearn.metrics import roc_auc_score, accuracy_score, recall_score, f1_score

def evaluate(model, X_test, y_test):
    y_pred = model.predict(X_test)
    auc_score = roc_auc_score(y_test, y_pred)
    y_pred = [1 if p > 0.5 else 0 for p in y_pred]
    precision = accuracy_score(y_test, y_pred)
    recall = recall_score(y_test, y_pred)
    f1 = f1_score(y_test, y_pred)
    return auc_score, precision, recall, f1

lstm_aucs = []
lstm_precisions = []
lstm_recalls = []
lstm_f1s = []
for ticker in test_data:
    X_test = test_data[ticker].drop(['Target'], axis=1)
    y_test = test_data[ticker]['Target']
    n_features = X_test.shape[1]
    X_test = X_test.values.reshape((X_test.shape[0], X_test.shape[1], 1))
    lstm_auc, lstm_precision, lstm_recall, lstm_f1 = evaluate(lstm_models[ticker], X_test, y_test)
    lstm_aucs.append(lstm_auc)
    lstm_precisions.append(lstm_precision)
    lstm_recalls.append(lstm_recall)
    lstm_f1s.append(lstm_f1)

cnn_aucs = []
cnn_precisions = []
cnn_recalls = []
cnn_f1s = []
for ticker in test_data:
    X_test = test_data[ticker].drop(['Target'], axis=1)
    y_test = test_data[ticker]['Target']
    n_features = X_test.shape[1]
    X_test = X_test.values.reshape((X_test.shape[0], X_test.shape[1], 1))
    cnn_auc, cnn_precision, cnn_recall, cnn_f1 = evaluate(cnn_models[ticker], X_test, y_test)
    cnn_aucs.append(cnn_auc)
    cnn_precisions.append(cnn_precision)
    cnn_recalls.append(cnn_recall)
    cnn_f1s.append(cnn_f1)

convlstm_aucs = []
convlstm_precisions = []
convlstm_recalls = []
convlstm_f1s = []
for ticker in test_data:
    X_test = test_data[ticker].drop(['Target'], axis=1)
    y_test = test_data[ticker]['Target']
    n_features = X_test.shape[1]
    X_test = X_test.values.reshape((X_test.shape[0], 1, 1, n_features, 1))
    convlstm_auc, convlstm_precision, convlstm_recall, convlstm_f1 = evaluate(convlstm_models[ticker], X_test, y_test)
    convlstm_aucs.append(convlstm_auc)
    convlstm_precisions.append(convlstm_precision)
    convlstm_recalls.append(convlstm_recall)
    convlstm_f1s.append(convlstm_f1)
print('LSTM ROC AUC Score: {:.2f}, Precision: {:.2f}%, F1 Score: {:.2f}'.format(np.mean(lstm_aucs), np.mean(lstm_precision) * 100, np.mean(lstm_f1s)))
print('CNN ROC AUC Score: {:.2f}, Precision: {:.2f}%, F1 Score: {:.2f}'.format(np.mean(cnn_aucs),  np.mean(cnn_precision) * 100, np.mean(cnn_f1s)))
print('ConvLSTM ROC AUC Score: {:.2f},  Precision: {:.2f}%, F1 Score: {:.2f}'.format(np.mean(convlstm_aucs), np.mean(convlstm_precision) * 100, np.mean(convlstm_f1s)))

The code evaluates the performance of the trained models using various metrics such as ROC AUC score, precision, recall, and F1 score.

The evaluate function takes the trained model, X_test and y_test as input and returns the calculated metrics as output. The X_test and y_test are the test data and its corresponding target values for a particular ticker.

The for loops iterate over the test_data dictionary, which contains the test data for each ticker, and extracts the test data and target values for each ticker. Then, it uses the evaluate function to calculate the metrics for each ticker and stores them in separate lists for each model.

After evaluating the performance of each model on all tickers, the mean values of the metrics are calculated using np.mean and printed to the console. The output shows the mean ROC AUC score, precision, and F1 score for each of the three models: LSTM, CNN, and ConvLSTM. Below are the results obtained:

LSTM ROC AUC Score: 0.61, Precision: 56.14%, F1 Score: 0.37
CNN ROC AUC Score: 0.59, Precision: 63.16%, F1 Score: 0.31
ConvLSTM ROC AUC Score: 0.58,  Precision: 66.67%, F1 Score: 0.37

5. Conclussion

In conclusion, we have explored the use of deep learning models, namely LSTM, CNN, and ConvLSTM, to predict stock price movement. We used stock price data as the only feature, and our objective was to predict whether the price one week from now is higher than the current price.

Our results showed that the LSTM model achieved the highest ROC AUC score, with the CNN and ConvLSTM models performing slightly worse. The precision and F1 scores were highest for the ConvLSTM model, with the LSTM model and CNN model close behind. Overall, all three models performed reasonably well, indicating that deep learning models can be useful tools for predicting stock price movement.

However, it is worth noting that the models’ performance could be further improved by including more types of features, such as financial ratios, news sentiment analysis, or macroeconomic indicators. Additionally, different combinations of features could potentially result in even better performance.

In summary, while our results show that deep learning models can be effective for predicting stock price movement, there is still room for improvement. The addition of more features could be one way to achieve this, and it will be interesting to see how future research in this area continues to evolve.

Become a Medium member today and enjoy unlimited access to thousands of Python guides and Data Science articles! For just $5 a month, you’ll have access to exclusive content and support me as a writer. Sign up now using my link and I’ll earn a small commission at no extra cost to you.

More content at PlainEnglish.io.

Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.

Interested in scaling your software startup? Check out Circuit.