Skip to content

Instantly share code, notes, and snippets.

@oconnor663
Created September 2, 2015 00:17
Show Gist options
  • Save oconnor663/08c081904264043e55bf to your computer and use it in GitHub Desktop.
Save oconnor663/08c081904264043e55bf to your computer and use it in GitHub Desktop.
reading and writing from an os.pipe() in asyncio
#! /usr/bin/python
import asyncio
import os
@asyncio.coroutine
def do_writing(writer):
for i in range(1, 4):
writer.write(("stuff " + str(i)).encode())
yield from asyncio.sleep(1)
writer.close()
@asyncio.coroutine
def do_reading(reader):
while not reader.at_eof():
some_bytes = yield from reader.read(2 ** 16)
print("here's what we got:", some_bytes)
@asyncio.coroutine
def main():
read_fd, write_fd = os.pipe()
reader = asyncio.StreamReader()
read_protocol = asyncio.StreamReaderProtocol(reader)
read_transport, _ = yield from loop.connect_read_pipe(
lambda: read_protocol, os.fdopen(read_fd))
write_protocol = asyncio.StreamReaderProtocol(asyncio.StreamReader())
write_transport, _ = yield from loop.connect_write_pipe(
lambda: write_protocol, os.fdopen(write_fd, 'w'))
writer = asyncio.StreamWriter(write_transport, write_protocol, None, loop)
loop.create_task(do_writing(writer))
yield from do_reading(reader)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main())
finally:
loop.close()
@mightymercado
Copy link

mightymercado commented Mar 10, 2020

Is there any way to do this when a parent process spawns a child process with multiprocessing.Process and the child process runs an event loop. How do I connect via OS Pipes? The code you wrote assumes event-loop environment, I wanna know if it's possible to perform write on a non-event loop environment? Like this:

writer = os.fdopen(w, 'w')
write.write('message')

It doesn't seem to work...

@oconnor663
Copy link
Author

Your question is a little too broad for me to understand it. If you put up a complete gist, and provided an explanation how its behavior was different from what you expected, that would help me give you better advice.

@mightymercado
Copy link

I have solved it and pasted a solution at my gist: https://gist.github.com/mightymercado/4efba1f070a6ba6526c3e237f0eb0443
Basically, what you have there is just one process. What I needed was communicating OS pipe with two processes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment