Pure Ruby MessagePack decoding speed

Explanation.md

For fun I'm writing a MessagePack encoder/decoder in pure Ruby to see how close to the speed of the native C and Java implementations I can get. Currently it's just 20% slower than the native C extension for decoding (haven't implemented encoding yet), and twice as slow as native Java, when running in the latest Java 1.7.

Below are some benchmarks run in both MRI and JRuby, comparing the native decoders for both platforms with my pure Ruby implementation with and without InvokeDynamic in JRuby.

The MessagePack decoder code is here: https://github.com/iconara/msgpack-pure-ruby/blob/647bcf74f8b059e5e31cc620f81eeb39b73393a4/lib/msgpack.rb

The benchmark code is here: https://github.com/iconara/msgpack-pure-ruby/blob/647bcf74f8b059e5e31cc620f81eeb39b73393a4/spec/benchmark.rb

The benchmark outputs start with the output of java -version and then runs a series of benchmarks with one million repetitions of decoding the same chunk of bytes. The lines beginning with * show the Ruby version used and additional info.

All of the tests were run in OS X 10.8.4.

Java 1.6.0u51

java version "1.6.0_51"
Java(TM) SE Runtime Environment (build 1.6.0_51-b11-457-11M4509)
Java HotSpot(TM) 64-Bit Server VM (build 20.51-b01-457, mixed mode)


* jruby-1.7.4 native java msgpack

Rehearsal ----------------------------------------------
msgpack      7.150000   0.220000   7.370000 (  6.993000)
------------------------------------- total: 7.370000sec

                 user     system      total        real
msgpack      6.800000   0.200000   7.000000 (  6.761000)

========================================================

* jruby-1.7.4 pure ruby msgpack

Rehearsal ----------------------------------------------
msgpack     20.360000   0.180000  20.540000 ( 20.166000)
------------------------------------ total: 20.540000sec

                 user     system      total        real
msgpack     20.210000   0.170000  20.380000 ( 20.203000)

========================================================

Java 1.7.0_40-b40

java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b40)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b55, mixed mode)


* jruby-1.7.4 native java msgpack

Rehearsal ----------------------------------------------
msgpack      6.080000   0.160000   6.240000 (  3.626000)
------------------------------------- total: 6.240000sec

                 user     system      total        real
msgpack      2.270000   0.020000   2.290000 (  2.244000)

========================================================

* jruby-1.7.4 pure ruby msgpack with -Xcompile.invokedynamic=true

Rehearsal ----------------------------------------------
msgpack     11.080000   0.170000  11.250000 (  8.310000)
------------------------------------ total: 11.250000sec

                 user     system      total        real
msgpack      5.980000   0.040000   6.020000 (  5.946000)

========================================================

* jruby-1.7.4 pure ruby msgpack with -X+C -J-XX:+TieredCompilation -Xcompile.invokedynamic=true

Rehearsal ----------------------------------------------
msgpack      9.250000   0.190000   9.440000 (  6.353000)
------------------------------------- total: 9.440000sec

                 user     system      total        real
msgpack      5.270000   0.030000   5.300000 (  5.183000)

========================================================

* jruby-1.7.4 pure ruby msgpack with -Xcompile.invokedynamic=false

Rehearsal ----------------------------------------------
msgpack      9.300000   0.140000   9.440000 (  7.576000)
------------------------------------- total: 9.440000sec

                 user     system      total        real
msgpack      6.550000   0.020000   6.570000 (  6.477000)

========================================================

* jruby-head (9422f547) pure ruby msgpack with -X+C -J-XX:+TieredCompilation -Xcompile.invokedynamic=true

Rehearsal ----------------------------------------------
msgpack      9.600000   0.200000   9.800000 (  6.186000)
------------------------------------- total: 9.800000sec

                 user     system      total        real
msgpack      4.690000   0.020000   4.710000 (  4.597000)

========================================================

* jruby-head (9422f547) pure ruby msgpack with -Xcompile.invokedynamic=false

Rehearsal ----------------------------------------------
msgpack      9.910000   0.150000  10.060000 (  8.050000)
------------------------------------ total: 10.060000sec

                 user     system      total        real
msgpack      7.110000   0.020000   7.130000 (  7.047000)

========================================================

MRI

* ruby-2.0.0-p195 native msgpack

Rehearsal ----------------------------------------------
msgpack      3.930000   0.220000   4.150000 (  4.185830)
------------------------------------- total: 4.150000sec

                 user     system      total        real
msgpack      3.910000   0.210000   4.120000 (  4.175436)

========================================================

* ruby-2.0.0-p195 pure ruby msgpack

Rehearsal ----------------------------------------------
msgpack     17.530000   0.040000  17.570000 ( 17.599123)
------------------------------------ total: 17.570000sec

                 user     system      total        real
msgpack     17.250000   0.030000  17.280000 ( 17.313037)

========================================================