Ruby system call benchmark

.gitignore

.bundle/

README.md

Comparison between the following different possibilities for calling an external process from ruby:

• Backticks `
• system (aka. Kernel.system)
• IO.popen
• Open3.popen3
• Open4::popen4

exec is excluded as it replaces the current process (which is the ruby process), which means we don't return to the ruby process, thus there's no benchmark.

The benchmarking is done with benchmark-ips.

Gemfile

source 'https://rubygems.org'

gem 'benchmark-ips'
gem 'open4'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    benchmark-ips (2.1.0)
    open4 (1.3.4)

PLATFORMS
  ruby

DEPENDENCIES
  benchmark-ips
  open4

main.rb

require 'benchmark/ips'
require 'open3'
require 'open4'

Benchmark.ips do |x|
  # without capturing the output into variables

  x.report('backticks (no capture)') do
    `echo 1 > /dev/null`
  end

  x.report('system (no capture)') do
    system('echo 1 > /dev/null')
  end

  x.report('io#popen (no capture)') do
    IO.popen('echo 1 > /dev/null').close
  end

  x.report('open3#popen3 (no capture)') do
    Open3.popen3('echo 1 > /dev/null')[1].close
  end

  x.report('open4#popen4 (no capture)') do
    Open4::popen4('echo 1 > /dev/null')[2].close
  end

  # with capturing the output into variables

  x.report('backticks (capture)') do
    stdout = `echo 1 > /dev/null`
  end

  x.report('system (capture)') do
    success = system('echo 1 > /dev/null')
  end

  x.report('io#popen (capture)') do
    stdout = IO.popen('echo 1 > /dev/null')
    stdout.read
  end

  x.report('open3#popen3 (capture)') do
    stdin, stdout, stderr = Open3.popen3('echo 1 > /dev/null')
    stdout.read
  end

  x.report('open4#popen4 (capture)') do
    pid, stdin, stdout, stderr = Open4::popen4('echo 1 > /dev/null')
    stdout.read
  end

  x.compare!
end

output

-- ruby 2.1.5p273

  Comparison:
    backticks (no capture):     489.3 i/s
    io#popen (no capture):      488.2 i/s - 1.00x slower
    system (no capture):        449.2 i/s - 1.09x slower
    open3#popen3 (no capture):  398.2 i/s - 1.23x slower
    system (capture):           344.5 i/s - 1.42x slower
    backticks (capture):        332.1 i/s - 1.47x slower
    open3#popen3 (capture):     321.7 i/s - 1.52x slower
    io#popen (capture):         282.8 i/s - 1.73x slower
    open4#popen4 (no capture):  280.9 i/s - 1.74x slower
    open4#popen4 (capture):     238.7 i/s - 2.05x slower


-- ruby 1.9.3p484

  Comparison:
    open4#popen4 (no capture):  312.6 i/s
    system (no capture):        308.8 i/s - 1.01x slower
    open3#popen3 (no capture):  297.7 i/s - 1.05x slower
    io#popen (no capture):      281.3 i/s - 1.11x slower
    backticks (no capture):     268.8 i/s - 1.16x slower
    system (capture):           255.0 i/s - 1.23x slower
    open3#popen3 (capture):     245.9 i/s - 1.27x slower
    io#popen (capture):         240.8 i/s - 1.30x slower
    open4#popen4 (capture):     223.6 i/s - 1.40x slower
    backticks (capture):        221.3 i/s - 1.41x slower


-- ruby 2.2.0preview2

  Comparison:
    open3#popen3 (no capture): 905.9 i/s
    io#popen (capture):        612.0 i/s - 1.48x slower
    backticks (no capture):    554.6 i/s - 1.63x slower
    system (no capture):       531.1 i/s - 1.71x slower
    open3#popen3 (capture):    524.5 i/s - 1.73x slower
    io#popen (no capture):     499.9 i/s - 1.81x slower
    backticks (capture):       489.6 i/s - 1.85x slower
    system (capture):          484.8 i/s - 1.87x slower
    open4#popen4 (no capture): 263.9 i/s - 3.43x slower
    open4#popen4 (capture):    197.6 i/s - 4.59x slower


-- rbx-2.2.10

  Comparison:
    backticks (capture):       380.2 i/s
    backticks (no capture):    379.6 i/s - 1.00x slower
    io#popen (no capture):     173.4 i/s - 2.19x slower
    io#popen (capture):        166.8 i/s - 2.28x slower
    system (capture):          87.3 i/s - 4.36x slower
    open4#popen4 (no capture): 85.2 i/s - 4.46x slower
    system (no capture):       80.8 i/s - 4.71x slower
    open3#popen3 (no capture): 70.9 i/s - 5.36x slower
    open3#popen3 (capture):    62.4 i/s - 6.09x slower
    open4#popen4 (capture):    60.3 i/s - 6.31x slower