proof of concept script for checking the size of staged git commits and rejecting based on individual file or overall total

.rubocop.yml

# DHH's house style — https://github.com/rails/rubocop-rails-omakase
inherit_gem:
  rubocop-rails-omakase: rubocop.yml

AllCops:
  TargetRubyVersion: 3.4

README.md

Don't let big files sneak into your repo

Binaries, vendored tarballs, an accidental node_modules, that 40 MB training CSV someone swore they'd .gitignore — they all get committed the same way: by accident, by someone in a hurry, and nobody notices until the clone takes four minutes and the history is poisoned forever.

Git history is forever. The cheapest place to stop a fat commit is before it happens. That's all this is: a pre-commit hook that weighs your staged changes and refuses the commit if they're too heavy.

There are two flavors. Pick the one that matches how strict you want to be.

git-diff-size-check.rb is the opinionated one. No single file may grow by more than 1 MiB, and the whole commit may not exceed 4 MiB. Break either rule and it tells you exactly which file blew the budget, then walks away. Use this when you want a firm hand.

git-diff-size-check-total-only.rb only cares about the bottom line. Stay under 4 MB in total and it doesn't care how you got there. Use this when a single large-but-legitimate file is fine, but a commit full of them isn't.

Using it

The scripts are executable and self-contained — Ruby and git, nothing else. Drop one in as your pre-commit hook:

ln -s ../../git-diff-size-check.rb .git/hooks/pre-commit

That's it. Now every git commit gets weighed first. Reject the commit, trim the fat, commit again. The size limits are constants at the top of each file; if 4 MB is the wrong number for you, it's the wrong number for about ten seconds, because that's how long it takes to change it.

Where this came from

The original idea and code are Joe Miller's — credit where it's due, that's the gist this one was forked from. What changed here:

• It diffs the staged index against HEAD, the way a pre-commit hook is supposed to, instead of comparing the working tree to a branch called master that your repo may not even have.
• Renames and copies no longer crash it. The old code only knew A, M, and D; anything else handed it a nil and the whole thing fell over. Now every status is accounted for.
• The very first commit in a fresh repo works, because there's no HEAD yet and the script knows to fall back to the empty tree instead of exploding.

Same spirit, fewer sharp edges.

git-diff-size-check-total-only.rb

#!/usr/bin/env ruby

MAX_DIFF_SIZE_MB = 4  # MB

def bytes_to_mb(bytes)
  bytes.to_f / (1024*1024)
end

total_diff_bytes = 0

# Empty-tree SHA: used as the base on the very first commit, when HEAD does not
# yet point at anything to diff against.
EMPTY_TREE = '4b825dc642cb6eb9a060e54bf8d69288fbee4904'
base = system('git rev-parse --verify -q HEAD > /dev/null') ? 'HEAD' : EMPTY_TREE

def blob_size(hash)
  return 0 if hash.nil? || hash =~ /\A0+\z/
  %x(git cat-file -s "#{hash}").to_i
end

# 'git diff-index --cached <base>' lists the staged changes (the index), so this
# runs after a 'git add' and before 'git commit'. We use git-cat-file to
# determine the delta of each file from the previous commit.
#
# Raw output format (the leading ':' rides on the first field):
#   :prev-mode new-mode prev-sha new-sha status<TAB>path[<TAB>new-path]
# 'status' is one char plus an optional similarity score for R/C (e.g. R100);
# renames/copies carry TWO tab-separated paths.
# example:
# :000000 100644 0000000000000000000000000000000000000000 9e0f96a2... A	1mb.file
# :100644 100644 257cc5642cb1a054... 9fcf2fbda9a5d884... M	README.md
IO.popen([ 'git', 'diff-index', '--cached', base ]).each do |l|
  meta, * = l.chomp.split("\t")
  _, _, prev_hash, new_hash, status = meta.split
  case status[0]
  when 'A', 'C' # Added new file, or new file created by a copy
    delta_bytes = blob_size(new_hash)
  when 'M', 'T', 'R' # Modified, type-changed, or renamed (content may differ)
    delta_bytes = blob_size(new_hash) - blob_size(prev_hash)
  when 'D' # Deleted file
    delta_bytes = blob_size(prev_hash)
  else
    next # Unmerged ('U') or unknown status: nothing meaningful to measure
  end
  total_diff_bytes += delta_bytes
end

diff_mb = bytes_to_mb(total_diff_bytes)
if diff_mb > MAX_DIFF_SIZE_MB
  $stderr.printf "Commit rejected, diff size is %0.1f MB. Limit is %0.1f MB\n", diff_mb, MAX_DIFF_SIZE_MB
  exit 1
end

$stderr.printf "Commit OK, diff size is %0.1f MB", diff_mb
exit 0

git-diff-size-check.rb

#!/usr/bin/env ruby

MAX_FILE_DELTA = 1048576
MAX_TOTAL_DELTA = 4194304

total_delta_bytes = 0
errors = []

# Empty-tree SHA: used as the base on the very first commit, when HEAD does not
# yet point at anything to diff against.
EMPTY_TREE = '4b825dc642cb6eb9a060e54bf8d69288fbee4904'
base = system('git rev-parse --verify -q HEAD > /dev/null') ? 'HEAD' : EMPTY_TREE

def blob_size(hash)
  return 0 if hash.nil? || hash =~ /\A0+\z/
  %x(git cat-file -s "#{hash}").to_i
end

# 'git diff-index --cached <base>' lists the staged changes (the index), so this
# runs after a 'git add' and before 'git commit'. We use git-cat-file to
# determine the delta of each file from the previous commit.
#
# Raw output format (the leading ':' rides on the first field):
#   :prev-mode new-mode prev-sha new-sha status<TAB>path[<TAB>new-path]
# 'status' is one char plus an optional similarity score for R/C (e.g. R100);
# renames/copies carry TWO tab-separated paths (old, then new).
# example:
# :000000 100644 0000000000000000000000000000000000000000 9e0f96a2... A	1mb.file
# :100644 100644 257cc5642cb1a054... 9fcf2fbda9a5d884... M	README.md
IO.popen([ 'git', 'diff-index', '--cached', base ]).each do |l|
  meta, path, new_path = l.chomp.split("\t")
  _, _, prev_hash, new_hash, status = meta.split
  filename = new_path || path
  case status[0]
  when 'A', 'C' # Added new file, or new file created by a copy
    delta_bytes = blob_size(new_hash)
  when 'M', 'T', 'R' # Modified, type-changed, or renamed (content may differ)
    delta_bytes = blob_size(new_hash) - blob_size(prev_hash)
  when 'D' # Deleted file
    delta_bytes = blob_size(prev_hash)
  else
    next # Unmerged ('U') or unknown status: nothing meaningful to measure
  end

  if delta_bytes > MAX_FILE_DELTA
    errors << "File '#{filename}' changed size is +#{delta_bytes} bytes. This is larger than the #{MAX_FILE_DELTA} bytes limit."
  end

  total_delta_bytes += delta_bytes
end

if total_delta_bytes > MAX_TOTAL_DELTA
  errors << "Total size of the new commit is +#{total_delta_bytes} bytes. This is larger than the #{MAX_TOTAL_DELTA} byte limit"
end

if errors.length > 0
  puts 'Commit rejected due to size of new or modified files! Errors in this commit:'
  puts
  puts errors.join("\n")
  exit(1)
end
puts "OK. Total commit size is #{total_delta_bytes} bytes."