Summary

This article provides guidance on managing external dependencies in a Python project using git submodules and Pipenv.

Abstract

The article titled "How to Effectively Manage “git submodule” Dependencies in Your Project" offers a detailed approach to managing dependencies with git submodules and Pipenv. It emphasizes the importance of strict version control for external dependencies, illustrating how to integrate a submodule into a project's Pipfile and Pipfile.lock using an example dependency named tool_kit. The author outlines the steps to add a submodule to a project, including the use of the git submodule add command, and how to update the Pipfile.lock with the submodule's transitive dependencies. The article also provides a sample setup.py file for creating a submodule and mentions the generation of a pyproject.toml file for installing dependencies. The author encourages readers to engage with the content by clapping and subscribing for more articles.

Opinions

The author endorses the use of Pipenv for package management, suggesting it simplifies dependency management.
The inclusion of a tool_kit submodule is presented as a practical example, implying its usefulness in real-world projects.
The article positions the combination of git submodules and Pipenv as an effective method for managing project dependencies.
By advising readers to add the Pipfile, Pipfile.lock, and .gitmodules files to their projects, the author suggests that this practice facilitates easier collaboration and setup for other users.
The author's request for claps and subscription implies a belief in the value and quality of the provided content.

How to Effectively Manage “git submodule” Dependencies in Your Project

Manage “git submodule” dependencies in your project using Pipenv.

Managing external dependencies, with strict control over versions becomes simpler when using git submodules. In a recent project, I needed to do precisely this. I used Pipenv as my package manager and the submodule entry in my project’s Pipfile was as follows:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
loguru = "*"
requests = "*"
tool_kit = {editable = true, path = "./submodules/tool_kit"}

[dev-packages]
pytest = "*"
requests-mock = "*"

[requires]
python_version = "3.9"

tool_kit is an example of an external dependency.

If you are looking to add one or more git submodules to your project, this article will serve as a good reference.

Prerequisites

Python 3.9, pip==21.1.3 and Pipenv==2021.5.29 or later

Add The Required Submodule to Your Pipfile

My primary project had a subdirectory named submodules that contained all external dependencies and libraries needed by my application.

I added the git submodule, tool_kit, to my project as follows:

git submodule add /path/to/tool_kit.git submodules/tool_kit

Executing this command created a .gitmodules file as follows:

[submodule "submodules/tool_kit"]
 path = submodules/tool_kit
 url = /path/to/tool_kit.git

Running the following command updated my project’s Pipfile.lock with the transitive dependencies for the submodule:

pipenv install --keep-outdated -e ./submodules/tool_kit

Adding the Pipfile, Pipfile.lock and the .gitmodules files to your project will make it simple for other users to clone and use your application.

Looking To Creating Your Own Submodule?

Be sure to correctly set up your project to avoid painful dependency issues.

Here is a sample setup.py file for our example project tool_kit:

import setuptools

setuptools.setup(
    name="tool_kit",
    version="0.1",
    packages=["tool_kit"],
    install_requires=[
        "requests",
        "pandas",
        "loguru",
        "pyarrow",
        "croniter",
    ],
    python_requires=">=3.9",
)

Note: Running pipenv install — keep-outdated -e ./submodules/tool_kit will generate a generic pyproject.toml file in your submodule’s directory with a =40.8.0", "wheel"] build-backend = "setuptools.build_meta:__legacy__"">de>[build-system] tag (if not already present). This file is used to install all dependencies needed by setup.py.

Thanks for reading. If you liked this article, please leave (up to 50) claps and consider subscribing to get the best of my articles delivered straight to you.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Check out our Community Discord and join our Talent Collective.