Exploring Trend Detection with 1-Minute Data in the First 15 Minutes of Each Day (Part 5)

In the fifth of our trend detection series, we will explore the application of a Genetic Algorithm (GA) to identify trends in financial markets. Genetic Algorithms are heuristic optimization techniques inspired by the process of natural selection in genetics.

We will explain how Genetic Algorithm works and, using the baseline methods, will try to improve our trend detection.

Understanding Genetic Algorithms

Genetic Algorithms work based on the principles of evolution and survival of the fittest. The algorithm starts with a population of potential solutions represented as chromosomes. Each chromosome contains a set of genes representing possible solutions to the problem. In our case, each gene represents a weight assigned to a feature in the trend detection model.

The key steps of the Genetic Algorithm are as follows:

1. Initialization: The algorithm starts by creating a random population of potential solutions (chromosomes) with random gene values.
2. Evaluation (Fitness Function): Each chromosome’s fitness is evaluated by a fitness function that measures how well the corresponding solution performs in the problem domain. In our case, the fitness function uses the F1-Score as the evaluation metric to assess the trend detection model’s performance.
3. Selection: The fittest chromosomes are selected to form a mating pool based on their fitness scores. The selection process is usually biased towards higher fitness scores to promote better solutions.
4. Crossover: Pairs of chromosomes from the mating pool are selected for crossover (recombination). Crossover involves exchanging genetic information between chromosomes to create new offspring.
5. Mutation: After crossover, some genes in the offspring may undergo mutation, introducing random changes to the solution. Mutation helps introduce diversity into the population.
6. Replacement: The offspring, along with some of the unchanged parent chromosomes, form the next generation population.
7. Termination: The process is repeated for several generations, allowing better solutions to emerge in subsequent generations. The algorithm terminates when a stopping criterion is met, such as reaching a maximum number of generations.

Before I continue sharing all the information, if you enjoy reading my articles, please hit the follow button — Diego Degese

Implementing trend detection with Genetic Algorithms

In order to ensure consistent data processing, we will employ the same baseline methods and dataset used in Part 1. By utilizing the optimization capabilities of the Genetic Algorithm, we aim to find the optimal weights for the features, enhancing the model’s ability to identify and distinguish different market trends effectively.

Let’s explore the code implementation of trend detection with Genetic Algorithms.

Importing Required Libraries and defining some constants

from sklearn.metrics import f1_score
from tqdm import tqdm
import numpy as np
import pygad

# Constants
SOLUTIONS = 30
GENERATIONS = 50

We start by importing the necessary libraries. The f1_score function from sklearn.metrics will be used to evaluate the model's performance. We also import tqdm to display a progress bar during the Genetic Algorithm execution. The numpy library is used for numerical computations and pygad is a Python library for implementing Genetic Algorithms.

Defining the method to predict the values

def get_predicted_values(solution, features):

    pred_y = np.clip(np.dot(features, solution), 0, 1)
    pred_y = np.where(pred_y > 0.5, 1, 0)
    return pred_y

We define the function get_predicted_values to obtain the model's predictions using the solution (weights) obtained from the Genetic Algorithm. This function transforms the dot product of the features and the solution into trend predictions.

Defining the Fitness Function

# Define fitness function to be used by the PyGAD instance
def fitness_func(self, solution, sol_idx):

    global train_x, train_y

    pred_y = get_predicted_values(solution, train_x)
    result = f1_score(train_y, pred_y, average='binary', pos_label=1) + \
             f1_score(train_y, pred_y, average='binary', pos_label=0)

    return result

The fitness_func is the fitness function that measures the model's performance. It calculates the F1-Score by comparing the predicted trend values with the actual trend values in the training dataset.

Training the Genetic Algorithm Model

# Train Model
with tqdm(total=GENERATIONS) as pbar:
    ga_instance = pygad.GA(
        num_generations=GENERATIONS,
        num_parents_mating=5,
        fitness_func=fitness_func,
        sol_per_pop=SOLUTIONS,
        num_genes=BARS,
        gene_space={'low': -1, 'high': 1},
        random_seed=42,
        on_generation=lambda _: pbar.update(1),
    )

    # Run the Genetic Algorithm
    ga_instance.run()

We create a Genetic Algorithm instance (ga_instance) with several configuration parameters, including the number of generations (num_generations), the number of parents for mating (num_parents_mating), and the fitness function to be used. The sol_per_pop parameter specifies the number of solutions (chromosomes) in the population.

We then execute the Genetic Algorithm using the run() method. The algorithm will iterate through the specified number of generations, optimizing the weights for the trend detection model.

Evaluating and Saving the Genetic Algorithm Results

# Set the best weights
solution, _, _ = ga_instance.best_solution()

# Predict and save train values
pred_y = get_predicted_values(solution, train_x)
save_result(train_y, pred_y, 'ga.train.csv.gz')

# Predict and save validation values
pred_y = get_predicted_values(solution, val_x)
save_result(val_y, pred_y, 'ga.val.csv.gz')

# Predict, show and save test values
pred_y = get_predicted_values(solution, test_x)
show_result(test_y, pred_y)
save_result(test_y, pred_y, 'ga.test.csv.gz')

After running the Genetic Algorithm, we extract the best solution (weights) obtained by the algorithm. We then use this solution to predict the trends for the training, validation, and test datasets, following the same process as in the previous parts.

The results are saved in separate CSV files for further analysis.

Genetic Algorithm Results

********************* RESULT TEST **********************
* Confusion Matrix (Top: Predicted - Left: Real)
[[149 100]
 [ 97 183]]
* Classification Report
              precision    recall  f1-score   support

           0       0.61      0.60      0.60       249
           1       0.65      0.65      0.65       280

    accuracy                           0.63       529
   macro avg       0.63      0.63      0.63       529
weighted avg       0.63      0.63      0.63       529

For the trend category ‘DOWN’ (class 0):

• The precision is approximately 61%, indicating that out of all the instances classified as ‘DOWN’, 61% of them are true negative predictions.
• The recall is approximately 60%, indicating that the model correctly captures 60% of all actual ‘DOWN’ trends.
• The F1-Score for the ‘DOWN’ category is around 60%, representing the harmonic mean of precision and recall.

For the trend category ‘UP’ (class 1):

• The precision is approximately 65%, indicating that out of all the instances classified as ‘UP’, 65% of them are true positive predictions.
• The recall is approximately 65%, indicating that the model correctly captures 65% of all actual ‘UP’ trends.
• The F1-Score for the ‘UP’ category is around 65%, representing the harmonic mean of precision and recall.

The overall accuracy of the Genetic Algorithm model is approximately 63%, which means that it correctly predicted the trend (either ‘UP’ or ‘DOWN’) in about 63% of the test dataset instances.

The Genetic Algorithm model demonstrates a balanced performance in predicting both upward and downward trends. It has slightly higher precision and F1-Score for the ‘UP’ category, indicating its effectiveness in identifying profitable upward trends.

Conclusion

In this fifth part, we explored the application of a Genetic Algorithm to enhance trend detection in financial markets. By implementing the Genetic Algorithm approach, we aimed to optimize the model’s feature weights for more accurate trend identification.

Genetic Algorithms are powerful optimization techniques inspired by natural selection, making them well-suited for solving complex problems.

In a future part of our series, we will compare the performance of the Genetic Algorithm trend detection with the previously explored models. Using the F1-Score as our evaluation metric, we aim to identify the most accurate and reliable approach for detecting trends in financial markets.

If you enjoy my work, please support me on Medium by becoming a member through my referral link, and consider giving it a clap as a small gesture of motivation. Thank you!

Download the full source code and the colab notebook of this article from here

Twitter / X: https://twitter.com/diegodegese LinkedIn: https://www.linkedin.com/in/ddegese Github: https://github.com/crapher

Disclaimer: Investing in the stock market involves risk and may not be suitable for all investors. The information provided in this article is for educational purposes only and should not be construed as investment advice or a recommendation to buy or sell any particular security. Always do your own research and consult with a licensed financial advisor before making any investment decisions. Past performance is not indicative of future results.

A Message from InsiderFinance

Thanks for being a part of our community! Before you go:

• 👏 Clap for the story and follow the author 👉
• 📰 View more content in the InsiderFinance Wire
• 📚 Take our FREE Masterclass
• 📈 Discover Powerful Trading Tools