Ruby 2.4 hash performance benchmark

00_Comment.md

This is a comment for blog post about improved Ruby 2.4 performance http://blog.redpanthers.co/behind-scenes-hash-table-performance-ruby-2-4/ The post describes improvements in Ruby 2.4 hash implementation and suggests 3x performance improvements.

The article proves that the benchmark presented at the beginning is misleading and suggests invalid conclusions. Since in 2.4 Hash stores list of its entries in sequential array, so the hash.values operation is immediate -- just need to copy the array. So there is significant performance gain comparing to 2.3, but it has nothing to do with performance of usual hash operations: insert, find, delete. Results of the presented benchmark cannot be used to predict performance gain of the other hash operations.

I repeated the experiment (like good physicists did in good old times recalled by Dijkstra) and added some more to test performance of the other operations. The benchmark and results can be found here: https://gist.github.com/wrzasa/6b456f73012ce98ae6feb6aaa4ba933e You can check if it's representative and verify results yourself.

Conclusions are that:

• there is noticeable performance gain for keys and values method, but nowhere near 3 times; more like 1.8 to 2 times;
• there is noticeable performance gain for find and insert operations, but nowhere near 3 times; more like 1.2 to 1.3 times;
• delete operation in 2.4 is actually slower then it was in 2.3 (about 1.1 times).

Bechmark

#! /usr/bin/env ruby

require 'benchmark'

puts "Ruby version: #{RUBY_VERSION}"

h = {}
1000.times do |i|
  h[i] = i.to_s
end

count = 50_000_000

indexes = count.times.map { rand(1000) }

h2 = {}
Benchmark.bm do |x| 
  x.report("values") { 500_000.times { h.values } }
  x.report("keys  ") { 500_000.times { h.keys } }
  x.report("find  ") { indexes.each { |i| h[i] } }
  x.report("insert") { indexes.each { |i| h2[i] = i.to_s } }
  x.report("delete") { indexes.each { |i| h.delete(i) } }
end

Results for 2.3

Ruby version: 2.3.1
       user     system      total        real
values  3.410000   0.140000   3.550000 (  3.554780)
keys    3.290000   0.000000   3.290000 (  3.293044)
find    4.520000   0.000000   4.520000 (  4.522883)
insert 41.110000   0.010000  41.120000 ( 41.151679)
delete  4.440000   0.000000   4.440000 (  4.444386)

Results for 2.4

Ruby version: 2.4.0
       user     system      total        real
values  1.800000   0.170000   1.970000 (  1.976577)
keys    1.370000   0.160000   1.530000 (  1.522444)
find    3.650000   0.000000   3.650000 (  3.652276)
insert 31.820000   0.010000  31.830000 ( 31.869769)
delete  5.190000   0.000000   5.190000 (  5.200268)