Build a Pre-commit Workflow to Check and Fix Your Python Code Automatically
Improve your code quality and accelerate your code review process
Git can run custom scripts, which are the so-called hooks, automatically when certain actions occur.
For example, the pre-commit, commit-msg, and pre-push hooks are triggered when a commit is made, when a commit message is saved, and before a commit is pushed to the remote repository, respectively.
Among these Git hooks, the most important one is pre-commit which can perform static code checking and fix trivial and time-consuming issues automatically. Thanks to pre-commit, the code reviewers can save a lot of time in code review and focus more efficiently on the important logic changes of the code.
We can write a pre-commit script, which is just a regular executable script, from scratch. However, there will be an issue when the same hook is used in several repositories. We would need to duplicate and maintain the same piece of code in different places, which is not very convenient.
Luckily, most of the best industry-standard linters such as black, isort, and pylint provide some Git hooks that can be used directly, with customizable arguments. These Git hooks can be installed with a special Python library called pre-commit. Yes, it’s the same name as the pre-commit Git hook! It is a framework for managing and maintaining multi-language pre-commit hooks.
In this post, we will introduce the most commonly used pre-commit hooks installable with the pre-commit package manager. The workflow we are going to build can perform the most common static code checks for Python code. It can improve your code quality dramatically and make code review much easier for your team.
As the pre-commit package manager is a Python library, we can install it with pip. You can install it in your system’s library or in a virtual environment, depending on your own use case.
$ pip install pre-commitNow the pre-commit command should be available and we can start to create the configuration file for it.
$ pre-commit --version
pre-commit 2.18.1The configurations for the pre-commit package manager are saved in a file named .pre-commit-config.yaml. You can create it and open it in vim or your favorite editor such as VS Code.
pre-commit-hooks
Let’s first add some most commonly used hooks available in the pre-commit-hooks repository.
repos is the top-level key for .pre-commit-config.yaml and defines a list of repository mappings. Every repo is a GitHub repository where some Git hooks are defined. If you open the corresponding repository, you will find a file called .pre-commit-hooks.yaml which defines the hooks available. The hooks are identified by ids and include other metadata as well such as name, description, types, arguments, etc.
In this example, some most commonly used hooks from the pre-commit-hooks repository are demonstrated. The ids of them are pretty self-explanatory of their functions. If you want to learn more about each hook, you can check them in .pre-commit-hooks.yaml in the repository.
Especially, the arguments for the hooks can be specified with the args key. In this example, it is specified that committing directly to the develop and main branches are not allowed.
autoflake
autoflake removes unused imports and variables from Python code. It looks trivial but it is really handy in practice as you may accidentally leave some unused variables and imports in your code, which can become messy over time.
The arguments are simple. It’s specified that unused variables and imports will be removed in place. You can find more arguments for autoflake on its GitHub page.
flake8
Flake8 is a wrapper around PyFlakes, pycodestyle, Ned Batchelder’s McCabe script. It is a great tool for checking your Python code against PEP8 coding style and common programming errors. You can install it as an extension in VS Code and configure it as demonstrated in this post. However, with a pre-commit hook for flake8, your code can be automatically checked before it’s committed.
According to PEP8, the max line length and max doc length are specified to be 79 and 72, respectively. Besides, errors E203 and W503 are ignored which means “Whitespace before ‘:’ ” and “Line break occurred before a binary operator”, respectively. You will encounter these errors when you use black for formatting as will be introduced soon.
black
If you have more complex settings for black, you add them in pyproject.toml as demonstrated in this link.
isort
isort is a Python library to sort imports alphabetically and get them automatically separated into sections and by type. isort can organize your messy imports and order them according to PEP8. Besides, you will have a consistent style in your team and everybody can be happy. Please check this post for a more detailed introduction to isort if needed.
The line length is specified to be 79 to make it consistent with other linters as demonstrated above. Besides, importantly, you must specify the src option and add the path to your source code.
Otherwise, your local libraries cannot be ordered correctly. Also, it should be noted that the option is src_paths in .isort.cfg and pyproject.toml. Check this link if you need to have more control over isort.
pylint
Pylint is an automatic Python code analysis tool that can detect programming errors and provides suggestions for code changes according to PEP8. It is a great tool that can improve the quality of your Python code dramatically.
Pylint is quite strict by default, which is good in most cases but can be overkill sometimes, especially for legacy code. Normally you don’t want to introduce too much refactoring to your legacy code. In this case, you can disable the warnings that would cause major refactoring.
For example, many functions or methods would require some common arguments even if they are not used, especially for some API frameworks like Falcon. We can specify the option --disable=unused-argument to disable this type of warning. You can find the error code in the output of pylint. For an exhaustive list, please check this link.
mypy
Mypy is a linter that performs static type checking for Python code. Please check this post if you are new to Python-type annotation. It should be noted that mypy only checks the code that has type annotations added explicitly, and won’t check those that don’t have typing added.
By default, mypy does not support the typing for third-party libraries and you would need to add them to additional_dependencies. Besides, a common argument is --ignore-missing-imports, which is useful when some libraries don’t have type stubs available.
pydocstyle
pydocstyle is a linter for checking Python docstrings according to PEP 257. It is a good tool to make your docstrings look nicer and have some standards in your team. For example, it requires the first line of the docstring to be a summary line and must start with an imperative verb and end with a dot.
The default settings should suffice in most cases.
After the hooks are added in .pre-commit-config.yaml, you need to install them to make them effective. You can run the following command to install the pre-commit hooks:
$ pre-commit installWhen the command is run, a script named pre-commit will be created in folder .git/hooks. It will be run automatically whenever you make a new commit.
In this post, we have introduced a set of pre-commit hooks including black, isort, mypy, pylint, etc which can automatically check and fix your Python code to make it error-free and more professional. You can either use the hooks directly or fine-tune them to make them suit the requirement of your team.
Related article:
