Skip to content

Instantly share code, notes, and snippets.

@mglinski
Last active August 29, 2015 14:25
Show Gist options
  • Save mglinski/0dd0033bf2834cb7d66f to your computer and use it in GitHub Desktop.
Save mglinski/0dd0033bf2834cb7d66f to your computer and use it in GitHub Desktop.

Setup

AWS c4.xlarge instance with 32gb EBS, io1 store type with 960 IOPS

Latest 64bit AMI Linux

Default php56 installed with YUM, including opcache,fpm,mbstring

Default nginx installed with YUM, with some configurations

  Ensure that php-fpm has process priority
  Use epool
  2 workers

PHP-FPM setup with TCP connections over port 9000

  Static server setup, 5 children

PHP, NGINX - All logging disabled

Composer deps all installed and then autoloader generated with -o optimizations

Opcache Settings

These are very important for getting max preformance:

opcache.save_comments=0
opcache.enable_file_override=1
opcache.memory_consumption=1024
opcache.interned_strings_buffer=64
opcache.max_accelerated_files=160000
opcache.revalidate_freq=0
opcache.validate_timestamps=0
opcache.fast_shutdown=1
opcache.enable_cli=0

Testing Methodology

Framework installs are current latest stable versions of each framework.

Frameworks are installed from composer with 0 modifications. Slim has a custom app file that is included below to load the app classes and do the exact same thing that lumen is doing in its routes.php file.

Tests are executed with AB using the command strings listed below:

Lumen

ab -t 20 -c 100 -k http://127.0.0.1/

Slim

ab -t 20 -c 100 -k http://127.0.0.1/slim.php

Code Being Run

Lumen app is executing the following code (default in app/Http/routes.php:

<?php
$app->get('/', function () use ($app) {
    return $app->welcome();
});

Slim app is written as below (modified version of the hello world app to make the view data identical in length):

<?php
require '../vendor/autoload.php';

$app = new \Slim\Slim();
$app->get('/', function () {
    echo '<!DOCTYPE html>
            <html>
            <head>
                <title>Lumen</title>

                <link href="//fonts.googleapis.com/css?family=Lato:100" rel="stylesheet" type="text/css">

                <style>
                    body {
                        margin: 0;
                        padding: 0;
                        width: 100%;
                        height: 100%;
                        color: #B0BEC5;
                        display: table;
                        font-weight: 100;
                        font-family: "Lato";
                    }

                    .container {
                        text-align: center;
                        display: table-cell;
                        vertical-align: middle;
                    }

                    .content {
                        text-align: center;
                        display: inline-block;
                    }

                    .title {
                        font-size: 96px;
                        margin-bottom: 40px;
                    }

                    .quote {
                        font-size: 24px;
                    }
                </style>
            </head>
            <body>
                <div class="container">
                    <div class="content">
                        <div class="title">Lumen.</div>
                    </div>
                </div>
            </body>
            </html>
        ';
});
$app->run();

Results

Lumen:

  Requests per second:    2696.89 [#/sec] (mean)
  Time per request:       37.080 [ms] (mean)
  Complete requests:      50000
  Failed requests:        0

Slim:

  Requests per second:    2189.81 [#/sec] (mean)
  Time per request:       45.666 [ms] (mean)
  Complete requests:      43797
  Failed requests:        0

Notes

AB has a built in default of 50000 requests when using a time based test. IMO, this does not significantly effect the Lumen result, which does hit this limit.

My Interpertation

Lumen on hardware that emphasizes CPU workloads is 18.80~% faster then Slim Framework v2 when rendering the exact same view data (taken from lumen).

This generally means nothing outside of epeen boasting though. Average request times differ by 7ms. While not insignificant, your code is likely to be much less preformant then these base classes are(due to you actually doing something), and the design of a framework relative to what you want to do with it is much more important this these type of benchmarks when picking a framework.

[[email protected] bench]# ab -t 20 -c 100 -k http://127.0.0.1/
This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests
Server Software: nginx
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /
Document Length: 1454 bytes
Concurrency Level: 100
Time taken for tests: 18.540 seconds
Complete requests: 50000
Failed requests: 0
Keep-Alive requests: 0
Total transferred: 80450000 bytes
HTML transferred: 72700000 bytes
Requests per second: 2696.89 [#/sec] (mean)
Time per request: 37.080 [ms] (mean)
Time per request: 0.371 [ms] (mean, across all concurrent requests)
Transfer rate: 4237.60 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 3.6 0 31
Processing: 1 35 13.0 36 159
Waiting: 1 34 12.3 36 158
Total: 4 37 11.3 37 159
Percentage of the requests served within a certain time (ms)
50% 37
66% 40
75% 43
80% 45
90% 51
95% 56
98% 61
99% 64
100% 159 (longest request)
[[email protected] bench]# ab -t 20 -c 100 -k http://127.0.0.1/slim.php
This is ApacheBench, Version 2.3 <$Revision: 1638069 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Finished 43797 requests
Server Software: nginx
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /slim.php
Document Length: 1454 bytes
Concurrency Level: 100
Time taken for tests: 20.000 seconds
Complete requests: 43797
Failed requests: 0
Keep-Alive requests: 0
Total transferred: 69336983 bytes
HTML transferred: 63686654 bytes
Requests per second: 2189.81 [#/sec] (mean)
Time per request: 45.666 [ms] (mean)
Time per request: 0.457 [ms] (mean, across all concurrent requests)
Transfer rate: 3385.53 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 4.1 0 40
Processing: 2 44 13.6 45 122
Waiting: 2 43 12.9 45 122
Total: 5 46 11.7 46 128
Percentage of the requests served within a certain time (ms)
50% 46
66% 48
75% 50
80% 52
90% 59
95% 67
98% 73
99% 76
100% 128 (longest request)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment