Maintain Clean Python Code With Black and GitHub Actions

Nobody wants a messy codebase; few have the patience to clean it

Writing code is hard enough, but ensuring it's well-formatted and easy to read can be even more challenging.

My coding skills have improved a lot in a decade. But the majority of them are not some fancy API usage or anything. It's how I format the code.

In my earliest years, I've coded for the outcome. Of course, that's every programmer's ultimate goal. But as I progressed, I understood that good coding is much more than simply getting things done.

It wasn't easy to share my code with others and not get a truckload of questions to explain them. Other programmers found digesting my code very challenging, even with supporting documents and inline comments.

A programmer's job is not over with a working code.

Beyond documentation, there's something that makes good code great. And the programming community hasn't failed to find what it is.

It's how we format our code.

Python, my favorite programming language, has the easiest syntax. But that doesn't warrant everyone who reads your Python code will understand it.

For this reason, we have a style guide known as PEP 8. This standard brings every programmer to code in the same style. If done correctly, a new programmer can spend less time figuring out which line is what.

Here's a sample code. The first is how I'd have coded it early in my career. It gets the job done. It reads a file and trains a model.

In the above image, you can see only half of my code fits into the screen. I’d have to scroll to the right and left to read and understand the code. There are unwanted blank lines and whitespace everywhere, while there’s no blank line where there needs to be one.

And this is how it would look like after implementing the style guide. This one is easier to understand, isn’t it?

But there's an issue. Remembering the style guide and forcing myself to code it this way takes a lot of work. I still wanted to spend more time getting the job done.

It would be helpful if someone else took care of the code formatting. That's where Black comes in. Black is a Python package to format your code to a predefined style guide in a single command. This guide can be PEP 8, or you can tweak it to your organizational version.

Modern code editors usually come with support for document formatting. For instance, in VSCode, you can right-click on the editor and click on code formatting. If Black is installed, it'll format your code at once.

All these improvements haven't stopped there. You can automate code formatting with pre-commit hooks and not worry about it at the time of coding. When you're committing a change, it will format all your python files. I've discussed that in a previous post.

The last missing piece of the puzzle is this. Even pre-commit hooks run on the developers' computers. When working as a team, you still rely on them for formatting.

What if you can do it centrally regardless of who the developer is and how they did it?

That's the focus of this post. We leverage GitHub Actions to do this.

When developers commit to the central repository, their code will automatically align with the organizational standards. GitHub actions will trigger a workflow whenever we push changes to the repository. We can configure it to run a Black formatting command.

Set up Black on GitHub Actions to Automatically Format Python Code

To use black to format your Python code when committing changes using GitHub Actions, you can follow these steps:

First, ensure that black is installed in your project. You can do this by adding black to the requirements.txt file in your project or by running pip install black in your project's virtual environment.

Next, create a new file in your project's .github/workflows directory called format.yml. This file will contain the configuration for your GitHub Actions workflow.

In the format.yml file, you can define a workflow that runs black on your code whenever you commit changes to your repository. Here's an example workflow that does this:

name: Format code

on:
  push:
    branches: [ master ]

jobs:
  format:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Format code with black
        run: |
          pip install black
          black .
      - name: Commit changes
        uses: EndBug/add-and-commit@v4
        with:
          author_name: ${{ github.actor }}
          author_email: ${{ github.actor }}@users.noreply.github.com
          message: "Format code with black"
          add: "."
          branch: ${{ github.ref }}

This workflow will run whenever you push changes to the master branch of your repository.

It installs black, and then runs the black command on the entire project (the . at the end of the black command specifies the current directory). Finally, it commits the new changes to the same branch.

Once you have defined your workflow, you can commit the changes to your repository and push them to GitHub. When you do this, the workflow will run automatically, formatting your code with black.

Excluding and including files and folders for formatting

There's an issue with the above configuration. It goes through all the files and folders in our repository and tries to format them. Sometimes, we want to avoid that happening for specific files.

Black has options to configure its formatting behavior.

In the following example, we've configured black to ignore any files inside the ref folder.

- name: Format code with black
  run: |
    pip install black
    black . --exclude="env/*"

This will run the black command on all files in the current directory, except for those in the env folder.

By default, Black will format all .py, .pyi, and .ipynb files. Alternatively, you can specify the list of files to include directly using the --include flag, like this:

- name: Format code with black
  run: |
    pip install black
    black --include="\.py" .

This will run the black command only on files with a .py extension in the current directory.

Yes, you can use both the --include and --exclude flags together to specify a more complex pattern of files to include or exclude.

- name: Format code with black
  run: |
    pip install black
    black --include="\.py" --exclude="env/*" .

This will run the black command on all files with a .py extension in the current directory, except for those in the env folder.

Note that the --include flag takes precedence over the --exclude flag, so if a file matches both patterns, it will be included. Thus be extra careful when you include patterns. Blindely including patterns, like in the above example, would cause black to format files in your env folder too.

You can specify multiple patterns for both --include and --exclude by separating them with a comma, like this:

- name: Format code with black
  run: |
    pip install black
    black --include="\.py,\.pyi" --exclude="env/*,tests/*" .

This will run the black command on all files with a .py or .pyi extension in the current directory, except for those in the env or tests folders.

Another helpful way to manage files to exclude is your .gitignore file. You may have one in your project already. If a file matches any patterns in .gitignore, those files will be ignored for formatting.

If you need to use both .gitignore and exclude behaviors, according to the official docs, you need to use --extend-exclude instead of --exclude.

Extending the workflow with more clean code tools

Black is one of the essential tools for formatting Python code. It takes care of a lot of things. But we do have other important stuff too.

One aspect of clean coding is logically sorting imports. But once again, you don't have to worry about this. You can use the package isort with your GitHub workflow to handle this automatically.

Removing redundant or unused variables is another quality of a good codebase. Many IDEs nowadays automatically highlight such new variables. But autoflake8 could automatically remove them as well.

Here's the finished GitHub Actions configuration:

name: Format code

on:
  push:
    branches: [ master ]

jobs:
  format:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Format code with black
        run: |
          pip install black
          black .
      - name: Sort imports with isort
        run: |
          pip install isort
          isort .
      - name: Remove unused imports with autoflake
        run: |
          pip install autoflake
          autoflake --in-place --remove-all-unused-imports --remove-unused-variables --recursive .
      - name: Commit changes
        uses: EndBug/add-and-commit@v4
        with:
          author_name: ${{ github.actor }}
          author_email: ${{ github.actor }}@users.noreply.github.com
          message: "Format code with black"
          add: "."
          branch: ${{ github.ref }}

When things don't go the way we want

Regardless of all our effort, there may be instances where these automated code refactoring can fail.

In a good development environment, this should be avoided. Most developers test their code locally before pushing it to Git because it allows them to identify any bugs or issues with the code before deployment.

Testing locally also allows developers to make necessary changes and ensure that the code functions correctly before it is pushed live, saving time and resources in the long run.

Because of this, issues such as typos and other minor things, usually why code refactoring fails, won't be an issue. But when they happen, we need to be prepared.

The below config will send an email notification whenever our code refactoring fails.

- name: Notify errors
  if: failure()
  uses: dawidd6/action-send-mail@v2
  with:
    server_address: smtp.gmail.com
    server_port: 465
    username: ${{ secrets.EMAIL_USERNAME }}
    password: ${{ secrets.EMAIL_PASSWORD }}
    subject: "Format code with black"
    body: "Format code with black failed"
    to: ${{ secrets.EMAIL_TO }}
    from: ${{ secrets.EMAIL_FROM }}
    content_type: text/plain

The above extension to the configuration uses the custom GitHub action created by Dawid Dziurla.

Conclusion

Nobody wants messy code, but a few have the patience to clean it up.

Every programming community has a style guide to ensure the code is readable. But that's the first hurdle in solving the problem. We need to make code formatting effortless for every developer to use often. Packages like Black does that.

But making it effortless doesn't mean everyone uses it. Pre-commit hooks automate the process but still rely on the developer. This post has presented a way to do the formatting centrally automatically.

Black with GitHub Actions can ensure the code is always formatted. You can centrally dictate the style guide. It can be PEP 8 or your custom one. And when they fail for any reason, you can also get notified.

I hope this helps.

Thanks for reading, friend! Say Hi to me on LinkedIn, Twitter, and Medium.

Not a Medium member yet? Please use this link to become a member because, at no extra cost for you, I earn a small commission for referring you.