Dedup posters

gistfile1.rb

require "digest/md5"
require "set"

found = Set.new

def md5sum(filename)
  content = File.read(filename)
  Digest::MD5.hexdigest(content)
end

Poster.all.each do |poster|
  # Not sure how this could happen, but anyways..
  poster.destroy if poster.image.nil? or poster.image.path.blank?
  
  md5sum = md5sum(poster.image.path)
  if found.include?(md5sum)
    poster.destroy
  else
    found << md5sum
  end
end

/cc @timsjoberg

poster.destroy unless File.file?(poster.image.path)

Not sure if the above is necessary, but maybe add it anyway. Otherwise it looks good

No, I believe the file is actually there. There is nothing stopping dups. open-uri downloads them with a random filename. Content is what matters.