Summary

The article outlines the use of BFG, an open-source utility, to remove sensitive credentials from a Git repository's history.

Abstract

The article addresses the common issue of inadvertently committing sensitive information, such as passwords or secrets, into a Git repository. It discusses the limitations of traditional methods like git-filter-branch and emphasizes the ease of using BFG for this purpose. BFG is a user-friendly tool written in Scala that simplifies the process of cleaning up repositories by removing both credentials and large files. The article provides a step-by-step guide on how to use BFG to identify and replace sensitive data in a repository's history, including the creation of a password file to specify the credentials to be removed, the manual removal of credentials from the latest commit, and the final steps to prune and push the cleaned repository. The author also cautions that once credentials are committed, they should be considered compromised and advises changing them immediately.

Opinions

The author suggests that using pre-commit hooks can prevent the accidental committing of secrets but acknowledges that it's not foolproof.
BFG is presented as a superior alternative to git-filter-branch due to its simplicity and effectiveness in cleaning up repositories.
The author emphasizes the importance of handling sensitive data with care and suggests that once exposed, credentials should be treated as compromised.
A strong recommendation is made for readers to avoid direct commits to the master branch in real-world scenarios, contrary to the example provided in the article for simplicity.
The article implies that despite the availability of tools like BFG, there is no substitute for vigilance and prompt action when credentials are accidentally exposed in version control systems.

The Easiest Way To Remove Checked In Credentials From A Git Repo

Using An Open Source Utility — BFG

Sometime you may get into a situation where someone might accidentally or unknowingly end up committing credentials, passwords, secrets, etc. in a git repository. In one of my previous articles, I had talked about how to prevent committing secrets using pre-commit hooks. Even by taking such precautions if the credentials get committed then there is no easy way to remove these credentials. If you just remove the credentials or the file itself and commit again, the credentials can be seen in the history easily.

There are other options like git-filter-branch but you need real good git expertise to use it. In this article, we will be using an open-source utility that will help us achieve this.

Introduction to BFG

BFG is a repo cleaner, open-source utility. It is written in Scala. Apart from credentials clean up, it can also help us remove large blobs from the Git. You can download BFG JAR and it is ready to use —

wget https://repo1.maven.org/maven2/com/madgag/bfg/1.13.0/bfg-1.13.0.jar

chmod +x bfg-1.13.0.jar

Clone Repo

Next, you need to clone the repo from which you need to remove the credentials. Here I am using my sample repo in which I had password checked in into application.properties file.

git clone https://[email protected]/tadeshpande/MyFirstProject/_git/MyFirstProject

For safety, it is recommended to copy your repo before running BFG command.

cp -R MyFirstProject MyFirstProjectCopy

Create a Password File

We need to create a file in which we need to give the password string to be matched into the actual repo. BFG uses this string to be removed from the Git Repo.

vi password.txt

Run BFG Command To Check Passwords

Now, you need to run the BFG to first check if there are matching strings as given in the password file.

java -jar bfg-1.13.0.jar --replace-text passwords.txt MyFirstProject

This will NOT remove the password as it is in the HEAD . You will see the message as shown below

Manually Remove the Credentials

As we can see in the message, we need to manually remove the credentials and check-in first.

vi MyFirstProject/src/main/resources/application.properties

Remove the password and save the file.

git add src/main/resources/application.properties

git commit -m "Updated application.properties"

git push origin master

This will remove the password from the file but will still stay in history. Here I am directly committing to the master branch. but you should never do that in the actual world.

Run BFG To Remove Password

Now, its time to run the BFG to actually remove the password from the git history.

java -jar bfg-1.13.0.jar --replace-text passwords.txt MyFirstProject

Push the Git Repo

Now, we just need to prune the git reference logs and we are good to push.

git reflog expire --expire=now --all && git gc --prune=now --aggressive

and then

git push --all --force

Here we need to do force push as remote repo will not allow merging. Usually, the command refuses to update a remote ref that is not an ancestor of the local ref used to overwrite it. This flag disables these checks, and can cause the remote repository to lose commits; use it with care.

Now if you go back to your repo and look for commit history, you will see the password text will be replaced with ***Removed***.

Likewise, you can also delete credentials files using the BFG command.

Please keep in mind, if you see any instance of credentials/files getting checked in, consider them as exposed and change those right a way. None of the above methods will be able to stop any misuse if the credentials were already copied somewhere.

Hey, if you enjoyed this story, check out Medium Membership! Just $5/month!

Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium.

Become a Medium Member Now!