sha1-bug.rb

require 'digest/sha1'

def buf_sha file, buf_size
  d = Digest::SHA1.new
  buf = String.new

  open(file) do |io|
    d.update buf while io.read(buf_size, buf)
  end

  d
end

file = 'data'
MiB = 1024 ** 2

# uncomment the line below to generate the data file
# `dd if=/dev/urandom of=#{file} bs=#{MiB} count=550`

puts "Digest::SHA1#file:\t#{Digest::SHA1.file(file)}"

buf_size = 512 * MiB
puts "updated < 512 MiB:\t#{buf_sha(file, buf_size - 1)}"
puts "updated = 512 MiB:\t#{buf_sha(file, buf_size)}"
puts "updated = file size:\t#{buf_sha(file, File.size(file))}"

i think it might have something with mmap and the maximun size of an the number used for the offset, i never looked into it tho

On Tue, Sep 27, 2011 at 2:27 PM, Ed Summers < ***@***.***>wrote: color me confused ## Reply to this email directly or view it on GitHub: https://gist.github.com/735465

## franco

At any rate I've got a merge request for your bagit library which I think addresses large file support.