Skip to content

Instantly share code, notes, and snippets.

@AdityaSoni19031997

AdityaSoni19031997/README-python-framework-benchmark.md

Forked from nhymxu/README-python-framework-benchmark.md
Created January 5, 2023 18:07
Show Gist options
  • Save AdityaSoni19031997/a2505ca5acbc3ba73e27b59e8995fdae to your computer and use it in GitHub Desktop.
Save AdityaSoni19031997/a2505ca5acbc3ba73e27b59e8995fdae to your computer and use it in GitHub Desktop.
Download ZIP
Flask vs Falcon vs FastAPI benchmark
Raw
README-python-framework-benchmark.md 
gunicorn run:app --workers=9
gunicorn run:app --workers=9 --worker-class=meinheld.gmeinheld.MeinheldWorker

Macbook Pro 2015 Python 3.7

Framework Server Req/s Max latency +/- Stdev
FastAPI uvicorn 15326.66 252.88ms 90.65%
hypercorn 8709.82 130.41ms 78.14%
gunicorn + uvicorn 9283.99 171.82ms 86.06%
FastAPI - async uvicorn 20473.65 142.84ms 85.13%
hypercorn 10436.90 61.90ms 71.20%
gunicorn + uvicorn 11485.82 122.48ms 77.98%
Falcon gunicorn 812.16 401.51ms 99.21%
gunicorn + meinheld 91454.62 134.42ms 91.90%
Falcon cython gunicorn 815.64 523.73ms 98.46%
gunicorn + meinheld 96540.80 128.47ms 90.44%
Flask gunicorn 823.77 314.30ms 99.21%
gunicorn + meinheld 24746.28 46.84ms 75.44%
gunicorn + gevent 6703.45 104.37ms 97.91%
Raw
falcon-run.py
import falcon
class ThingsResource(object):
def on_get(self, req, resp):
"""Handles GET requests"""
resp.status = falcon.HTTP_200 # This is the default status
resp.body = ('pong')
# falcon.API instances are callable WSGI apps
app = falcon.API()
# Resources are represented by long-lived class instances
things = ThingsResource()
# things will handle all requests to the '/things' URL path
app.add_route('/ping', things)
Raw
flask-run.py
from flask import Flask
app = Flask(__name__)
@app.route("/ping")
def ping():
return "pong"
Raw
result1.txt
wrk --duration 20s --threads 1 --connections 100 http://localhost:8000/ping
--------------------------------
Falcon cython
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 7.14ms 3.68ms 51.96ms 73.50%
Req/Sec 9.33k 1.48k 12.44k 82.65%
185796 requests in 20.05s, 27.46MB read
Requests/sec: 9266.09
Transfer/sec: 1.37MB
--------------------------------
Falcon cython + meinheld
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.36ms 1.75ms 43.63ms 96.45%
Req/Sec 85.32k 10.13k 95.71k 82.00%
1697213 requests in 20.03s, 257.36MB read
Requests/sec: 84737.10
Transfer/sec: 12.85MB
--------------------------------
Falcon
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 6.97ms 3.36ms 48.88ms 72.74%
Req/Sec 9.51k 1.43k 12.42k 80.10%
189142 requests in 20.06s, 27.96MB read
Requests/sec: 9430.63
Transfer/sec: 1.39MB
--------------------------------
Falcon + meinheld
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.52ms 2.17ms 54.69ms 95.91%
Req/Sec 80.24k 13.45k 92.23k 81.50%
1596695 requests in 20.04s, 242.11MB read
Requests/sec: 79676.11
Transfer/sec: 12.08MB
--------------------------------
Flask + meinheld
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.57ms 2.29ms 52.72ms 87.86%
Req/Sec 29.47k 1.87k 30.87k 88.00%
586316 requests in 20.02s, 93.38MB read
Requests/sec: 29285.76
Transfer/sec: 4.66MB
--------------------------------
Flask
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 10.54ms 4.23ms 67.09ms 82.06%
Req/Sec 7.93k 1.03k 9.36k 70.05%
157485 requests in 20.01s, 24.48MB read
Requests/sec: 7872.26
Transfer/sec: 1.22MB
--------------------------------
Flask + pypy3.6-7.1.1
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 28.93ms 14.93ms 241.74ms 66.13%
Req/Sec 3.30k 1.49k 9.65k 83.92%
66034 requests in 20.02s, 10.26MB read
Requests/sec: 3298.62
Transfer/sec: 525.07KB
--------------------------------
Falcon + pypy3.6-7.1.1
--
Running 20s test @ http://localhost:8000/ping
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 9.11ms 10.42ms 146.96ms 83.78%
Req/Sec 9.21k 5.13k 23.23k 55.78%
184084 requests in 20.08s, 27.21MB read
Requests/sec: 9168.11
Transfer/sec: 1.36MB
Raw
result2.txt
wrk --duration 20s --threads 10 --connections 200 http://localhost:8000/ping
--------------------------------
Falcon cython
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 16.30ms 60.85ms 1.68s 98.87%
Req/Sec 1.14k 249.32 2.06k 71.67%
225481 requests in 20.03s, 33.33MB read
Socket errors: connect 0, read 0, write 0, timeout 14
Requests/sec: 11257.30
Transfer/sec: 1.66MB
--------------------------------
Falcon cython + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.14ms 2.76ms 48.32ms 82.92%
Req/Sec 7.05k 3.73k 22.49k 83.59%
1407587 requests in 20.08s, 213.44MB read
Requests/sec: 70082.22
Transfer/sec: 10.63MB
--------------------------------
Falcon
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 16.00ms 50.71ms 1.67s 98.66%
Req/Sec 1.10k 275.07 1.86k 68.70%
218841 requests in 20.06s, 32.35MB read
Requests/sec: 10908.97
Transfer/sec: 1.61MB
--------------------------------
Falcon + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.59ms 3.95ms 79.25ms 90.61%
Req/Sec 6.99k 3.49k 21.57k 76.60%
1391987 requests in 20.03s, 211.07MB read
Requests/sec: 69499.75
Transfer/sec: 10.54MB
--------------------------------
Flask + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.55ms 7.00ms 87.67ms 74.76%
Req/Sec 2.63k 1.24k 7.20k 69.95%
523834 requests in 20.05s, 83.43MB read
Requests/sec: 26124.87
Transfer/sec: 4.16MB
--------------------------------
Flask
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 21.15ms 61.75ms 1.68s 98.64%
Req/Sec 823.05 209.44 1.34k 74.05%
163927 requests in 20.05s, 25.48MB read
Socket errors: connect 0, read 0, write 0, timeout 17
Requests/sec: 8174.99
Transfer/sec: 1.27MB
Raw
result3.txt
uvicorn run:app --workers 9
hypercorn run:app --workers 9
gunicorn run:app --workers=9 -k uvicorn.workers.UvicornWorker
gunicorn run:app --workers=9
gunicorn run:app --workers=9 --worker-class=meinheld.gmeinheld.MeinheldWorker
wrk --duration 20s --threads 10 --connections 200 http://localhost:8000/ping
------------------------------
FastAPI + uvicorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 17.13ms 19.24ms 252.88ms 90.65%
Req/Sec 1.54k 219.20 2.29k 69.35%
307068 requests in 20.03s, 38.07MB read
Socket errors: connect 0, read 85, write 0, timeout 0
Requests/sec: 15326.66
Transfer/sec: 1.90MB
------------------------------
FastAPI + uvicorn - async
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 10.55ms 7.91ms 142.84ms 85.13%
Req/Sec 2.06k 611.43 4.66k 63.60%
410659 requests in 20.06s, 50.91MB read
Requests/sec: 20473.65
Transfer/sec: 2.54MB
------------------------------
FastAPI + hypercorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 25.37ms 18.92ms 130.41ms 78.14%
Req/Sec 0.88k 342.42 1.91k 72.30%
174472 requests in 20.03s, 22.30MB read
Requests/sec: 8709.82
Transfer/sec: 1.11MB
------------------------------
FastAPI + hypercorn - async
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 19.60ms 10.87ms 61.90ms 71.20%
Req/Sec 1.05k 359.66 2.04k 60.55%
209097 requests in 20.03s, 26.72MB read
Requests/sec: 10436.90
Transfer/sec: 1.33MB
------------------------------
FastAPI + gunicorn + uvicorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 26.29ms 24.72ms 171.82ms 86.06%
Req/Sec 0.94k 484.69 2.06k 56.80%
186557 requests in 20.09s, 23.13MB read
Socket errors: connect 0, read 36, write 0, timeout 0
Requests/sec: 9283.99
Transfer/sec: 1.15MB
------------------------------
FastAPI + gunicorn + uvicorn - async
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 19.25ms 14.96ms 122.48ms 77.98%
Req/Sec 1.15k 291.98 2.04k 60.30%
230221 requests in 20.04s, 28.54MB read
Socket errors: connect 0, read 76, write 0, timeout 0
Requests/sec: 11485.82
Transfer/sec: 1.42MB
------------------------------
Falcon + gunicorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 21.58ms 33.17ms 401.51ms 99.21%
Req/Sec 572.84 260.23 1.08k 74.64%
16279 requests in 20.04s, 2.41MB read
Socket errors: connect 0, read 412, write 6, timeout 0
Requests/sec: 812.16
Transfer/sec: 122.93KB
------------------------------
Falcon + gunicorn + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.90ms 4.33ms 134.42ms 91.90%
Req/Sec 9.20k 1.25k 15.58k 78.15%
1838302 requests in 20.10s, 278.75MB read
Socket errors: connect 0, read 39, write 0, timeout 0
Requests/sec: 91454.62
Transfer/sec: 13.87MB
------------------------------
Falcon cython + gunicorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 23.35ms 47.76ms 523.73ms 98.46%
Req/Sec 608.04 290.92 1.06k 69.66%
16387 requests in 20.09s, 2.42MB read
Socket errors: connect 0, read 357, write 4, timeout 0
Requests/sec: 815.64
Transfer/sec: 123.46KB
------------------------------
Falcon cython + gunicorn + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.67ms 3.67ms 128.47ms 90.44%
Req/Sec 9.70k 1.38k 14.16k 66.70%
1931630 requests in 20.01s, 292.90MB read
Socket errors: connect 0, read 44, write 0, timeout 0
Requests/sec: 96540.80
Transfer/sec: 14.64MB
------------------------------
Flask + gunicorn
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 23.87ms 25.11ms 314.30ms 99.21%
Req/Sec 521.89 193.32 0.88k 76.62%
16528 requests in 20.06s, 2.57MB read
Socket errors: connect 0, read 341, write 5, timeout 0
Requests/sec: 823.77
Transfer/sec: 131.13KB
------------------------------
Flask + gunicorn + meinheld
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.20ms 4.24ms 46.84ms 75.44%
Req/Sec 2.49k 565.34 3.98k 77.55%
495311 requests in 20.02s, 78.89MB read
Socket errors: connect 0, read 46, write 0, timeout 0
Requests/sec: 24746.28
Transfer/sec: 3.94MB
------------------------------
Flask + gunicorn + gevent
--
Running 20s test @ http://localhost:8000/ping
10 threads and 200 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.49ms 2.73ms 104.37ms 97.91%
Req/Sec 1.31k 553.40 3.36k 60.99%
134625 requests in 20.08s, 21.57MB read
Socket errors: connect 0, read 0, write 0, timeout 23
Requests/sec: 6703.45
Transfer/sec: 1.07MB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment