Measuring different Java interface implementation strategies using JRuby.

comparator.rb

require 'java'
require 'benchmark'
java_import "java.util.Comparator"
java_import "java.util.Collections"

# dummy Comparable object
class O
  include Comparable
  attr_reader :id
  def initialize(id); @id = id; end
  def <=>(other); id <=> other.id; end
end

# Java interface implemetation using mixin
class NaturalComparator
  include Comparator
  def compare(a, b); a <=> b; end
end

puts("initializing structures")

STRINGS_COUNT = 4000000
CHARS = ('a'..'z').to_a.freeze
STRINGS = (1..STRINGS_COUNT).map{CHARS.shuffle[0, 4].join.freeze}.freeze
OBJECTS = STRINGS.map{|s| O.new(s)}.freeze

# since Java Collections::sort mutates the passed collection, Ruby's sort! is used
# and sorting is done on dup'ed collections.

def ruby_sort(collection, &comparator)
  comparator ? collection.sort!(&comparator) : collection.sort!
end

def java_sort_with_comparator_block(collection, &comparator)
  comparator ? Collections::sort(collection, Comparator.impl(&comparator)) : Collections::sort(collection)
end

def java_sort_with_comparator_class(collection, comparator)
  Collections::sort(collection, comparator)
end

benchmarks = [
  lambda{|x| x.report("Ruby default comparator on strings") {ruby_sort(STRINGS.dup)}},
  lambda{|x| x.report("Ruby default comparator on objects") {ruby_sort(OBJECTS.dup)}},

  lambda{|x| x.report("Ruby block comparator on strings")   {ruby_sort(STRINGS.dup){|a, b| a <=> b}}},
  lambda{|x| x.report("Ruby block comparator on objects")   {ruby_sort(OBJECTS.dup){|a, b| a <=> b}}},

  lambda{|x| x.report("Java default comparator on strings") {java_sort_with_comparator_block(STRINGS.dup)}},

  lambda{|x| x.report("Java block comparator on strings")   {java_sort_with_comparator_block(STRINGS.dup){|method, a, b| a <=> b}}},
  lambda{|x| x.report("Java block comparator on objects")   {java_sort_with_comparator_block(OBJECTS.dup){|method, a, b| a <=> b}}},

  lambda{|x| x.report("Java class comparator on strings")   {java_sort_with_comparator_class(STRINGS.dup, NaturalComparator.new)}},
  lambda{|x| x.report("Java class comparator on objects")   {java_sort_with_comparator_class(OBJECTS.dup, NaturalComparator.new)}},
]

benchmarks.each do |b|
  puts
  Benchmark.bmbm(&b)
end

output

# on MBP 13r, 2.8Ghz Core i7, 16GB
# jruby 1.7.10 (1.9.3p392) 2014-01-09 c4ecd6b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_11-b21 [darwin-x86_64]

ruby --server -J-Xmx6G -J-Xms6G  -J-Djruby.compile.mode=FORCE -J-Djruby.jit.threshold=0 -Xcompile.invokedynamic=true comparator.rb

initializing structures

Rehearsal ----------------------------------------------------------------------
Ruby default comparator on strings   7.840000   0.010000   7.850000 (  7.781000)
------------------------------------------------------------- total: 7.850000sec

                                         user     system      total        real
Ruby default comparator on strings   7.980000   0.010000   7.990000 (  7.892000)

Rehearsal ----------------------------------------------------------------------
Ruby default comparator on objects  16.780000   0.030000  16.810000 ( 16.545000)
------------------------------------------------------------ total: 16.810000sec

                                         user     system      total        real
Ruby default comparator on objects  16.290000   0.020000  16.310000 ( 16.366000)

Rehearsal --------------------------------------------------------------------
Ruby block comparator on strings  19.070000   0.250000  19.320000 ( 18.885000)
---------------------------------------------------------- total: 19.320000sec

                                       user     system      total        real
Ruby block comparator on strings  17.030000   0.020000  17.050000 ( 17.080000)

Rehearsal --------------------------------------------------------------------
Ruby block comparator on objects  28.530000   0.040000  28.570000 ( 28.698000)
---------------------------------------------------------- total: 28.570000sec

                                       user     system      total        real
Ruby block comparator on objects  27.570000   0.040000  27.610000 ( 27.723000)

Rehearsal ----------------------------------------------------------------------
Java default comparator on strings   7.900000   0.030000   7.930000 (  7.634000)
------------------------------------------------------------- total: 7.930000sec

                                         user     system      total        real
Java default comparator on strings   8.520000   0.090000   8.610000 (  8.634000)

Rehearsal --------------------------------------------------------------------
Java block comparator on strings  51.290000   0.140000  51.430000 ( 51.449000)
---------------------------------------------------------- total: 51.430000sec

                                       user     system      total        real
Java block comparator on strings  52.750000   0.270000  53.020000 ( 53.039000)

Rehearsal --------------------------------------------------------------------
Java block comparator on objects  32.220000   0.100000  32.320000 ( 32.301000)
---------------------------------------------------------- total: 32.320000sec

                                       user     system      total        real
Java block comparator on objects  31.750000   0.060000  31.810000 ( 31.927000)

Rehearsal --------------------------------------------------------------------
Java class comparator on strings  40.580000   0.120000  40.700000 ( 40.868000)
---------------------------------------------------------- total: 40.700000sec

                                       user     system      total        real
Java class comparator on strings  39.900000   0.210000  40.110000 ( 40.300000)

Rehearsal --------------------------------------------------------------------
Java class comparator on objects  19.020000   0.020000  19.040000 ( 19.036000)
---------------------------------------------------------- total: 19.040000sec

                                       user     system      total        real
Java class comparator on objects  18.850000   0.020000  18.870000 ( 18.974000)

results.md

Description

Summarized benchmark results of different Java interface implementation strategies using JRuby.

The idea is to see the performance impact of using a custom java.util.Comparator interface implemented in JRuby compared to the default native sort in Ruby and Java. The Ruby sort! method and the Java Collections::sort method are tested on both strings and comparable objects collections.

Tests

• Ruby default comparator: using the sort! method with default comparator.
• Ruby block comparator: passing a block comparator to the sort! method.
• Java default comparator: using the Java Collections::sort method with default comparator.
• Java block comparator: passing a block comparator to the Java Collections::sort method using the closure conversion interface implementation of java.util.Comparator.
• Java class comparator: passing a class comparator to the Java Collections::sort method using the module mixin interface implementation of java.util.Comparator.

Results summary

| Comparator | Strings | Objects --- | --- | --- | --- 1 | Ruby default comparator | 7.85s | 16.31s 2 | Ruby block comparator | 17.05s | 27.61s 3 | Java default comparator | 7.93s | N/A⁽¹⁾ 4 | Java block comparator | 51.43s | 31.81s 5 | Java class comparator | 40.11s | 18.87s

Notes:

1. Java cannot sort non-primitive JRuby objects without a custom comparator

Observations

• (1 & 3) As expected both Ruby sort! and Java Collections::sort on string collection using the default comparator yield similar performance.
• (4 & 5) Curiously using Java Collections::sort on a collection of comparable Ruby objects is significantly faster than on Ruby strings.
• (4 & 5) Using the module mixin interface implementation for a class comparator is significantly faster than using closure conversion interface implementation for the block comparator.
• (1 & 5) Very interesting: both Ruby sort! using default comparator on comparable objects and Java Collections::sort using a class comparator on comparable objects has similar performance.