How to Write your Grid Search Function in Python

Summary

The article presents a Python function using itertools.product to efficiently generate hyperparameter combinations for grid search without relying on scikit-learn's GridSearchCV.

Abstract

The web content introduces a Pythonic approach to performing grid search for machine learning model hyperparameter tuning. The author, Simon Hawe, emphasizes the utility of the itertools.product function to create a flexible and concise grid search implementation. This method eliminates the need for nested loops and adapts easily to additional parameters. The solution involves generating the Cartesian product of hyperparameter values, which is then converted into a dictionary for straightforward model initialization. The article concludes by suggesting scenarios where this approach is beneficial, such as when working with custom models or wanting to demonstrate advanced Python skills.

Opinions

The author is enthusiastic about sharing Python coding tips for machine learning and developer efficiency.
There is a preference for native Python solutions over using scikit-learn's GridSearchCV in certain scenarios, such as when working with custom code or for demonstrating coding proficiency.
The itertools.product function is highly regarded by the author for its ability to avoid inflexible nested loops and to easily extend the search to more parameters.
The author believes that mapping the grid search results to a dictionary simplifies the usage of the grid search function.
The article promotes the idea that writing concise code, like the three-line grid search function, is a valuable skill.
A suggestion is made for readers to contact the author via LinkedIn for further discussion, indicating a willingness to engage with the community.
The author endorses an AI service, ZAI.chat, as a cost-effective alternative to ChatGPT Plus (GPT-4), highlighting its affordability and similar capabilities.

Introduction

I am back trying to write something here at medium about coding, machine learning, and Python. Probably, I am the only one being excited about that but at least there is someone :)

As I don’t have too much time, I decided to (hopefully) write a series explaining tiny Python functions that could make your daily developer lives easier. Are you ready and have about one minute? Then let's get started!

The Problem

Imagine you want to test a set of hyperparameters to train your model. One way of doing this is using a grid search. In a grid search, you create every possible combination of the parameters that you want to try out. For all those combinations, you train your model and run some scoring method to know how well the trained model performs given the set of parameters.

When you are a Python user, the obvious choice to do that is using scikit-learn and more specifically the class called GridsearchCV. However, you might not like scikit-learn (which I doubt), or you have written some custom code that doesn’t adhere to the scikit-learn interface, or you just want to do something completely different. For those cases, I have a solution for you written in pure Python. With that, you don’t have to write many nested for loops and don’t have to change your code when you want to extend your search to more parameters.

The Solution

Now, without a lot of talking, let's jump directly into the solution

What did I do here? First, to avoid nested and inflexible for loops, I used the product function that is available from the awesome itertools module. What that does is building an iterable that returns the cartesian product of all iterables you are passing in. Sounds crazy. As an example, if you pass four iterables each containing two items to product, you will get back 2⁴=16 tuples each of size 4. Maybe doesn’t sound better, so just try it out.

Second, for convenience, I don’t yield the plain tuples containing the concert values, but create a dictionary that maps the input keys to the respective individual value. That should simplify the usage of the grid_parameters function.

Finally, how do you use it? You have to define the parameters you want to try out in a dictionary. The keys/names should ideally match the arguments you would pass to the __init__ function of your model. They don’t have to, but that will make your life just easier. The values of your dictionary must be iterables, so lists or tuples or something similar containing all the values you want to try out. Now, you just pass that dictionary to the grid_parameters function, and you iterate over it. After that, it is up to you what you want to do with your grid points. Easy, isn’t it? And only 3 lines of code. Yeah!

Conclusion

In this very short post, I showed you a compact and easy-to-use native Python implementation to generate grid points that you can use to perform a grid search. The main reason for using this is either when you’ve built a model that doesn’t implement the scikit learn estimator Apis, or when you just want to show off your Python skills :-)

Thank you for following this post. As always, feel free to contact me for questions, comments, or suggestions either here or via LinkedIn.

How to Write your Grid Search Function in Python

Itertools is all you need

Introduction

The Problem

The Solution

Conclusion