RingoJs comes with a HTTP Server and basic utilities to deal with JSGI. Unless you are building something very specific, you will want to use packages to ease web development. With this benchmark I try to shed some light on how much those (beta quality) packages for web development degrade RingoJs' performance. The benchmark tasks defined by this benchmark are simple but can give us an upper limit for the performance achievable in real world projects.
I am using the TechEmpower FrameworkBenchmarks. This benchmark tests web development frameworks by running multiple, standardized tasks at varying concurrency levels. The four tasks are outlined in more detail in the results section.
I benchmarked Ringo twice to see how a pure JSGI application, written just with the Ringo standard library, performs compared to an application utilizing all the libraries we recommend for convenient web development. Those two test runs are called ringojs and ringojs-convenient:
ringojsis using Ringo's standard library only and talks directly to the database with JDBCringojs-convenientis using additional modules to get an ORM, template rendering and URL routing
The get an upper limit for the ringojs performance I benchmarked servlet. servlet uses JSP as a template engine and talks directly to the database with JDBC.
nodejs is benchmarked to see how we do against the gorilla in the ServerSide market. I also tested express, which seems to be the go-to web framework for NodeJs. nodejs talks directly to the database and express uses the ORM Sequelize. Both use mustache to render their templates.
Summary of the frameworks I tested:
Framework ORM Templates Webframework Language/Engine
-------------------------------------------------------------------------------------
servlet - - - Java
nodejs - mustache - JavaScript V8
express sequelize mustache express JavaScript V8
ringojs - mustache - JavaScript Rhino
ringojs-convenient ringo-sqlstore reinhardt stick JavaScript Rhino
servlet is going to be the fastest. It's pure Java and has the least amount of layers down to the actual TCP streams. ringojs-convenient and express play in the same performance league; the both have ORMs and ringojs-convenient also adds a template engine. ringojs versus nodejs is going to be interesting. ringojs draws power from the mature Java library world but it's all scripted with the comparatively slow Rhino JavaScript engine.
In my setup, there are four types of tasks and each task gets one test run for each concurrency level (64 and 128) except for the database multiple query test, which is run once for each query count (5, 10, 15, 20) at the maximum concurreny. Each test run is run for one minute (with warmups and pauses between the tests and everything (If such questions arise in your head, please consult the source before you talk to me)).
I run the benchmark on two EC2 m1.large instances. MySQL database.
Concurrency
Responses/sec 64 128
--------------------------------
servlet 8,656 8,735
ringojs 4,162 4,315
nodejs 2,496 2,457
express 1,645 1,598
ringo-convenient 1,462 1,491
Max responses/sec
-------------------
servlet ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 8,735
ringojs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 4,315
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 2,457
express ▇▇▇▇▇▇▇▇▇ 1,645
ringojs-convenient ▇▇▇▇▇▇▇▇ 1,491
Latency in ms AVG STDEV
--------------------------------
servlet 15 17
ringojs 29 22
nodejs 54 19
ringo-convenient 64 64
express 124 49
Average Latency
-------------------
servlet ▇▇▇▇▇▇ 15
ringojs ▇▇▇▇▇▇▇▇▇▇▇ 29
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 54
ringo-convenient ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 64
express ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 124
Return the result of a database query as JSON
random_id = random(1, 10000)
world = World.find(random_id)
return json.encode(world)
Concurrency
Responses/sec 64 128
--------------------------------
servlet 4,737 4,688
ringojs 1,115 1,112
ringo-convenient 869 888
Max responses/sec
------------------
servlet: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 4,737
ringojs: ▇▇▇▇▇▇▇▇▇▇▇ 1,115
ringojs-convenient: ▇▇▇▇▇▇▇▇▇ 888
Latency in ms AVG STDEV
--------------------------------
servlet 27 7
ringo-convenient 106 93
ringojs 133 115
Average Latency
------------------
servlet ▇▇▇▇▇▇▇▇▇▇ 27
ringo-convenient ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 106
ringojs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 133
Render a template with the result of a database query; the database result is modified before it gets passed to the template
fortunes = Fortune.all()
fortunes.add( { id: 0, message: 'Added at request time' } )
template.render(fortunes)
Queries/Request
Responses/sec 5 10 15 20
------------------------------------------------------
servlet 2,447 1,295 895 670
ringojs 1,763 1,081 754 586
nodejs 650 359 243 186
express 388 204 140 106
ringo-convenient 353 194 128 92
Responses/sec for 5 queries
--------------------------
servlet ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇2,447
ringojs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇1,763
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇ 650
express ▇▇▇▇▇▇▇ 388
ringojs-convenient ▇▇▇▇▇▇▇ 353
Latency in ms AVG STDEV
---------------------------------
servlet 189 60
ringojs 218 67
nodejs 784 261
ringo-convenient 1,340 287
express 1,940 701
Errors timeout
---------------------------------
express 338
ringojs-convenient 19
Average Latency
--------------------------
servlet ▇▇▇▇ 189
ringojs ▇▇▇▇▇ 218
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 784
ringo-convenient ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1,340
express ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1,940
Return the result of 5, 10, 15 and 20 queries as JSON
number_of_queries = get("queries")
worlds = []
for i = 0; i < number_of_queries; i++
random_id = random(1, 10000)
worlds[i] = World.find(random_id)
json.encode(worlds)
Concurrency
Responses/sec 64 128
----------------------------------
servlet 28,371 28,528
nodejs 10,985 11,070
express 7,653 7,357
ringojs 6,592 7,093
ringo-convenient 5,745 5,760
Max responses/sec (servlet not shown)
------
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 11,070
express ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 7,653
ringojs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 7,093
ringojs-convenient ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 5,760
Latency in ms AVG STDEV
----------------------------------
servlet 5 2
nodejs 15 6
ringojs 21 30
express 22 14
ringo-convenient 28 38
Average Latency
------------------------
servlet ▇▇▇▇▇▇▇▇ 5
nodejs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 15
ringojs ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 21
express ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 22
ringo-convenient ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 28
Stringify a small object, which must be newly instantiated for each request
obj = { message : "Hello, World!" }
json.encode(obj)
All tests were served by Ringo's embedded web server, which is a JSGI wrapping around the Jetty project. There was no other web server (apache, ngix) in front of Ringo. For the ringojs-convenient benchmark I added the following packages to get an ORM, template rendering and URL routing (bascially, this turns it into a loosely coupled web framework):
-
Stick generally helps with composing JSGI applications; URL routing, middleware support and parameter parsing.
-
ringo-sqlstore is an ORM with some powerful caching concepts borrowed from Helma but reimplemented in pure JavaScript on top of JDBC.
-
Reinhard is a JavaScript port of the Django template engine.
The benchmark was run with tag oberhamsi-round1
Application code per framework tested: