Benchmarks

The basic example uses only a single route at /hello_world and return Hello world. These benchmarks are extremely simple and should not be used as a decision factor for any of the tools presented (unless you are building hello world apps ;). I am just using them as performance guideline when working on Dynamo.

What happens if we add more routes?

This benchmark also measures what happens if you add 10, 100 and 1000 /hello_world/#{i} routes and try to match the last /hello_world/1000 route.

What happens if I am serving a file?

We also check the behavior of the app when it has to serve a file with 100kb of size.

Results

Results were obtained by running the examples below in a Macbook Pro 2011 2.3GHz dual-core Intel Core i5.

No customization was done to the underlying VM/Engine/Webserver besides passing the "production" flag to all libraries, i.e.

We are solely measuring the out-of-the-box experience. This means Elixir is running on both cores, while the other solutions aren't. This is one of the several benefits of using the Erlang VM.

req/s average for 8000 requests, concurrency 40 (ab -n 8000 -c 40) (more is better):

   Routes     | Dynamo/Elixir | SinatraS/Ruby 1.9.2 | Express/node.js 0.4.7
      1       |    9135.83    |      3833.54        |      4730.74
   10 + 1     |    9105.35    |      3561.69        |      4635.8
  100 + 1     |    9103.91    |      2472.51        |      4171.94
 1000 + 1     |    9043.72    |       598.47        |      1904.71
 100kb file   |    6087.99    |      2088.66        |      1734.17

Notes

Thanks to the Erlang VM, Dynamo performs very well in the basic hello world case. Indeed, Dynamo is in very early stage and the throughput will reduce by a couple dozens req/s as we add more basic features. Regardless, very good results.

It is also important to notice that increasing the number of routes increases the CPU usage which will block on single process, non threaded solutions like Node.js, affecting throughput. Also the performance of the algorithm used in the routes matching matters in the results above.

Code

defmodule MyApp do
  use Dynamo::Routes

  get "/hello_world" do
    "Hello World"
  end
end

Dynamo::Cowboy.run MyApp

var app = require('express').createServer();

app.get('/hello_world', function(req, res){
  res.send('hello world');
});

app.listen(3000);

require 'sinatra/base'
require 'sinatra/synchrony'

class HelloWorld < Sinatra::Base
  register Sinatra::Synchrony

  get '/hello_world' do
    'Hello World'
  end
end