Last active
October 24, 2018 13:42
-
-
Save juntatalor/409dcfcd455cd4903b7ea56be6d8ea38 to your computer and use it in GitHub Desktop.
Async file write with tornado
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
from uuid import uuid4 | |
from tornado import web | |
from tornado.httpserver import HTTPServer | |
from tornado.ioloop import IOLoop | |
from tornado.iostream import PipeIOStream | |
IOLoop.configure('tornado.platform.asyncio.AsyncIOMainLoop') | |
class SafeFileIOStream: | |
def __init__(self, fname): | |
self.fname = fname | |
def __enter__(self): | |
# Create file | |
os.open(self.fname, os.O_CREAT) | |
# Create stream | |
fd = os.open(self.fname, os.O_WRONLY) | |
self.stream = PipeIOStream(fd) | |
return self.stream | |
def __exit__(self, exc_type, exc_val, exc_tb): | |
# Close stream | |
self.stream.close() | |
class TestHandler(web.RequestHandler): | |
async def get(self): | |
fname = '/tmp/' + uuid4().hex | |
with SafeFileIOStream(fname) as stream: | |
await stream.write(b'hello stranger') | |
self.write('success') | |
app_handlers = [ | |
(r'/test/?', TestHandler), | |
] | |
application = web.Application(app_handlers) | |
# application.listen(8000, '0.0.0.0') | |
if __name__ == '__main__': | |
server = HTTPServer(application) | |
server.bind(8000) | |
server.start(0) | |
loop = IOLoop.current() | |
loop.start() |
This may seem async, because the server is using multiple processes, thus it executes requests in parallel. If you limit the processes to one (server.start(1)), the server will be sync and not asyc. Also, try using bigger files.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ab -c 8 -n 8000 http://127.0.0.1:8000/test/
This is ApacheBench, Version 2.3 <$Revision: 1748469 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 800 requests
Completed 1600 requests
Completed 2400 requests
Completed 3200 requests
Completed 4000 requests
Completed 4800 requests
Completed 5600 requests
Completed 6400 requests
Completed 7200 requests
Completed 8000 requests
Finished 8000 requests
Server Software: TornadoServer/4.4.1
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /test/
Document Length: 7 bytes
Concurrency Level: 8
Time taken for tests: 9.722 seconds
Complete requests: 8000
Failed requests: 0
Total transferred: 1608000 bytes
HTML transferred: 56000 bytes
Requests per second: 822.86 #/sec
Time per request: 9.722 ms
Time per request: 1.215 [ms](mean, across all concurrent requests)
Transfer rate: 161.52 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 8
Processing: 2 9 10.3 8 548
Waiting: 2 9 10.2 8 548
Total: 2 10 10.4 8 550
Percentage of the requests served within a certain time (ms)
50% 8
66% 9
75% 10
80% 11
90% 14
95% 19
98% 28
99% 36
100% 550 (longest request)