Skip to content

Instantly share code, notes, and snippets.

@juntatalor
Last active October 24, 2018 13:42
Show Gist options
  • Save juntatalor/409dcfcd455cd4903b7ea56be6d8ea38 to your computer and use it in GitHub Desktop.
Save juntatalor/409dcfcd455cd4903b7ea56be6d8ea38 to your computer and use it in GitHub Desktop.
Async file write with tornado
import os
from uuid import uuid4
from tornado import web
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from tornado.iostream import PipeIOStream
IOLoop.configure('tornado.platform.asyncio.AsyncIOMainLoop')
class SafeFileIOStream:
def __init__(self, fname):
self.fname = fname
def __enter__(self):
# Create file
os.open(self.fname, os.O_CREAT)
# Create stream
fd = os.open(self.fname, os.O_WRONLY)
self.stream = PipeIOStream(fd)
return self.stream
def __exit__(self, exc_type, exc_val, exc_tb):
# Close stream
self.stream.close()
class TestHandler(web.RequestHandler):
async def get(self):
fname = '/tmp/' + uuid4().hex
with SafeFileIOStream(fname) as stream:
await stream.write(b'hello stranger')
self.write('success')
app_handlers = [
(r'/test/?', TestHandler),
]
application = web.Application(app_handlers)
# application.listen(8000, '0.0.0.0')
if __name__ == '__main__':
server = HTTPServer(application)
server.bind(8000)
server.start(0)
loop = IOLoop.current()
loop.start()
@juntatalor
Copy link
Author

ab -c 8 -n 8000 http://127.0.0.1:8000/test/

This is ApacheBench, Version 2.3 <$Revision: 1748469 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 800 requests
Completed 1600 requests
Completed 2400 requests
Completed 3200 requests
Completed 4000 requests
Completed 4800 requests
Completed 5600 requests
Completed 6400 requests
Completed 7200 requests
Completed 8000 requests
Finished 8000 requests

Server Software: TornadoServer/4.4.1
Server Hostname: 127.0.0.1
Server Port: 8000

Document Path: /test/
Document Length: 7 bytes

Concurrency Level: 8
Time taken for tests: 9.722 seconds
Complete requests: 8000
Failed requests: 0
Total transferred: 1608000 bytes
HTML transferred: 56000 bytes
Requests per second: 822.86 #/sec
Time per request: 9.722 ms
Time per request: 1.215 [ms](mean, across all concurrent requests)
Transfer rate: 161.52 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 8
Processing: 2 9 10.3 8 548
Waiting: 2 9 10.2 8 548
Total: 2 10 10.4 8 550

Percentage of the requests served within a certain time (ms)
50% 8
66% 9
75% 10
80% 11
90% 14
95% 19
98% 28
99% 36
100% 550 (longest request)

@sebrestin
Copy link

This may seem async, because the server is using multiple processes, thus it executes requests in parallel. If you limit the processes to one (server.start(1)), the server will be sync and not asyc. Also, try using bigger files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment