Clean up the bloated Backblaze bzfileids.dat

README.md

Place in /Library/Backblaze.bzpkg/bzdata/bzbackup, and run with:

ruby purge_nonexistent.rb

It will process bzfileids.dat placing any file that exists in bzfileids.dat-found and any missing file in bzfileids.dat-missing. You can then back up your original file and replace it with bzfileids.dat-found.

My original file was almost 1.4GB. It had grown so large that the backup would cause all my fans spin and it never seemed to complete. Backblaze support suggested I delete my entire backup with them and start over. After running this script, my new bzfileids.dat file is 301MB. Still huge, but about 1/5th the size. The backup seemed to go much more smoothly.

purge_nonexistent.rb

#!/usr/bin/env ruby
# vim: ft=ruby
# 

File.open('./bzfileids.dat').each_with_index do |line, index|
  split = line.match(/^([a-z0-9_-]+)\s+(.+)$/)
  if split
    file_id = split[1]
    path = split[2]

    if !File.exists?(path)
      File.open('./bzfileids.dat-missing', 'a+b') do |f|
        f.puts line
        f.close
      end
    else
      File.open('./bzfileids.dat-found', 'a+b') do |f|
        f.puts line
        f.close
      end
    end
  else
    puts "#{index} Invalid line found: #{line}"
  end
end