How to Completely Remove a Sensitive File from Git History (BFG Repo Cleaner)

You know that sinking feeling. The one where you push a commit, sip your coffee, and then freeze as you realize you just uploaded your AWS root keys or a production database password to a public repository. Your stomach drops. You feel the heat rise in your neck.

We’ve all been there. It’s the “rite of passage” for developers that nobody wants.

Your instinct screams: “Delete the file and push again!” Stop. Do not do that.

Git is a time machine. Creating a new commit to delete a file only hides it in the present; the secret remains immortalized in the past, accessible to anyone who clones your repo or knows how to browse the history. In this guide, we aren’t just going to “clean” your repo; we are going to perform a forensic sterilization using the “Nuclear Method” and the BFG Repo-Cleaner.

STOP! Read This Before You Run git rm (The Triage Phase)

Immediate Action Required: If you are reading this because you just leaked a credential, take your hands off the keyboard. Do not try to fix the Git history yet.

Step 0 is Revocation, not Deletion.

Here is the hard truth that most tutorials miss: The moment that commit hit the remote server, it was scraped. Bots patrol GitHub constantly. According to the 2025 State of Secrets Sprawl report by GitGuardian, a staggering 70% of leaked secrets remain active and valid long after the initial leak. That means developers are scrubbing their history but failing to rotate the actual keys.

The “Private Repo” Safety Myth
“But my repo is private, so I’m safe, right?” Wrong. The same report reveals that 35% of private repositories contain hardcoded secrets, with critical passwords appearing three times more frequently than in public code. Private repos are often less guarded, making them a prime target for lateral movement attacks. Treat this leak as if it were on a billboard in Times Square.

The “Nuclear” Method: Why We Choose BFG Over filter-branch

For years, the standard advice was to use git filter-branch. In 2025, using git filter-branch is like trying to mow a lawn with a pair of scissors. It is painfully slow, complex, and prone to corrupting your repo.

Why BFG Repo-Cleaner Wins
We choose BFG for one reason: Speed under pressure. When you are panicking, you don’t want to write complex Python scripts (which git-filter-repo often requires). You want a simple command that works fast. BFG is written in Scala and leverages multi-threading to run 10x to 1000x faster than standard Git tools.

Step-by-Step: How to Completely Remove the File (The Execution)

This is the surgical part. Follow these steps exactly. I’ve used this method to salvage repos that were over 5GB in size without losing the project history.

Step 1: The Mirror Clone

You need a fresh copy of your repository. Not a standard clone, but a mirror clone. This pulls down every single reference, branch, and tag—effectively the entire database.

git clone --mirror https://github.com/yourusername/your-repo.git
cd your-repo.git

    Pro Tip: Before you touch anything, create a tarball backup of this directory.

tar czf backup-repo.tar.gz .

If you mess up the rewrite, you can extract this and start over.

# Pre-Flight Safety Check
if [ "$(git config --get core.bare)" != "true" ]; then
    echo "❌ STOP: This is not a mirror clone!"
elif [ ! -f "../backup-repo.tar.gz" ]; then
    echo "⚠️ WARNING: No backup found in parent directory."
else
    echo "✅ SYSTEM GREEN: Ready for BFG."
fi

Step 2: The BFG Execution

You have two options here. Most people just delete the file, but I recommend “Text Replacement” if you want to keep the build artifacts intact (like config files) but strip the actual secret.

Option A: Nuke the file entirely (Easiest)
Use this if the file itself (like id_rsa or passwords.txt) should never have existed.

# Download BFG first (requires Java)
java -jar bfg.jar --delete-files id_rsa

Option B: The “Surgical” Replace (Best for Config Files)
If you deleted a file like config.json that the app needs to run, deleting it from history breaks every old build. Instead, use BFG to find the password inside the file and replace it with the text ***REMOVED***.

# Create a file listing the secrets to remove
echo "super_secret_password_123" > secrets.txt

# Run BFG to replace the text in history
java -jar bfg.jar --replace-text secrets.txt

Step 3: The “Double Tap” Garbage Collection

BFG updates the commits, but the old dirty data is still hanging around in Git’s “Reflog” (the trash can that hasn’t been emptied). We need to force Git to expire these logs immediately.

# Strip out the dirty references
git reflog expire --expire=now --all
git gc --prune=now --aggressive

Running --aggressive is vital here. It tells Git to invest extra CPU cycles to repack the entire repository, ensuring the file size actually drops.

Step 4: The Force Push

Once verified, push the rewritten history back to the remote.

git push --force

The “Dirty Secret” of Git: Why “Deleted” Isn’t “Gone”

Here is where the other tutorials stop, and where your security risk actually begins. You pushed the clean history, so you’re safe, right? Wrong.

Distributed systems like GitHub are designed for availability, not privacy. They cache everything.

1. The “Cached View” Vulnerability
Even though the commit is gone from the main branch, if an attacker has the SHA-1 hash of the old commit (perhaps from a browser history, a Slack link, or a CI log), they can type that URL directly into GitHub. GitHub will happily serve the cached view of that commit, secret included. The object still exists on the server’s disk; it’s just “orphaned.”

2. The Pull Request (PR) Problem
This is the nastiest surprise. GitHub creates read-only references for Pull Requests (refs/pull/...). BFG usually cannot rewrite these because they are controlled by GitHub. If you opened a PR with the secret, that diff is often immutable.

The Solution?
You must manually trigger a cleanup on the server side. You cannot do this yourself. You must contact GitHub Support.

Troubleshooting the Force Push (When Git Fights Back)

Sometimes, the git push --force command fails. It’s frustrating, but usually fixable. Here are the most common blockers I’ve encountered in the wild:

The Team Protocol: Preventing “Zombie” Leaks

You cleaned the repo. You cleared the cache. Then, your coworker Bob comes back from vacation.

Bob runs git pull.

Git sees Bob’s history (which still has the secret) and your new history (which doesn’t) as “diverged.” It helpfully merges them together. Boom. The secret is back, and now the history is a tangled mess of duplicate commits.

The “Burn It Down” Order
You cannot “fix” your teammates’ local repos. You must order them to destroy them.

Send this message to your team:

    “I have performed a history rewrite to remove a security vulnerability.

1. STOP: Do not pull or push.
2. DELETE: Delete your local repository folder entirely.
3. CLONE: Re-clone the repository from fresh.

Any work you had not pushed is lost. Sorry, but security comes first.”

Verification: How to Prove It’s Gone

Paranoia is a virtue in DevOps. Before you declare victory, verify the eradication.

Use the “Pickaxe” command to search the entire history for the string you just deleted:

git log -S "super_secret_password_123" --source --all

If this returns nothing, the text is gone from the commit diffs. Next, verify the pack size to ensure the binary blobs are gone:

git count-objects -vH

You should see a significant drop in the “size-pack” metric compared to before the operation.

Future-Proofing: Automated Defense in 2025

The average cost of a data breach is now $4.4 million according to the IBM Cost of a Data Breach Report. You cannot afford to rely on human memory to not commit keys.

We are entering the era of AI-driven security. The GitHub MCP (Model Context Protocol) Server is changing the game. It connects AI agents directly to your repo to perform tasks, but it also integrates new push protections that scan for secrets before they leave your machine.

Your Defense Stack for 2025:

• Local: Install Talisman or pre-commit hooks. These reject commits locally if they look like keys.
• Platform: Enable “Secret Scanning” and “Push Protection” in GitHub settings. It’s free for public repos and vital for private ones.
• Workflow: Never use .env files without immediately adding them to .gitignore. Ideally, use a secrets manager (like 1Password or Vault) to inject environment variables at runtime, so the file never exists on disk to begin with.

Final Thoughts

Cleaning Git history is like doing surgery on a beating heart. It’s risky, stressful, and necessary. But once you’ve run the BFG, rotated your keys, and scrubbed the caches, you can breathe easier.

Just remember: The only secret that is truly safe is the one that was never committed. Stay paranoid, stay safe.