Erlang and Named Pipes

The intention of this post is to provide a solution (with examples) to a somewhat uncommon issue in Erlang. I hate searching for answers to the same problems over and over, and I had a hard time finding answers to this particular problem, so I wrote it all down once I figured it out. If one day you decide to read and write data through fifo's (named pipes) and then decide you want to read or write the other end of the pipe from Erlang, this post is for you.

The Problem

I wanted to read and write to a fifo from a C/C++ app and have an Erlang app communicate over the other end of that fifo. Put simply, Erlang doesn't really support what I was trying to do. You cannot just file:open/2 a fifo, or any special device for that matter, in Erlang and expect it to work. This is documented in Erlang's FAQ.

The Solution! ... ???

In that FAQ it offers two links that you'd imagine have some reasonable solution to the answer, but both posts do not really give you a good solution, they just further state the problem.

So, next you think, "hey, I'll Google this and surely someone has figured it out". Indeed, maybe people have figured it out and suggest using open_port() to solve all your woes. But alas even this post and this post take more of the "use open_port and that should work for you" stance. While this points you in the right direction, it isn't really helpful if you aren't familiar with ports.

So yes, open_port() is the solution, but getting it to work is non-obvious (at least to me) initially.

The Solution!

First lets setup what we want to do. In bash, we might do the following to read/write to a fifo.

Using just a shell

Create fifo

➜  cd /tmp
➜  mkfifo test.pipe
➜  test -p test.pipe && echo "It worked"
It worked

Reading from a fifo blocks, so you need a writer and reader to test that it is working

Shell 1

➜  cat test.pipe
... # blocking

Shell 2

➜  echo "Hey now" > test.pipe
# returns immediately

Meanwhile back in Shell 1

➜  cat test.pipe
Hey now
➜ # It now returns and closes its end of the pipe

In Erlang

Okay, so we know what is supposed to happen in the shell, now lets try it in Erlang. If you are looking for solid examples online, there really isn't any that deal directly with named pipes, and the one other port example I could find in Erlang's documentation didn't work with fifo's. All we know is "use open_port", cool story.

Instead of walking through all the terrible ways I've failed trying different things, I'll get right to the correct answer. TL;DR, none of the errors I got back were super helpful, just a lot of badsig or bad argument errors that are hinted at in the port_command/2 documentation.

To Read

First, make sure you created that pipe

➜  cd /tmp
➜  mkfifo test.pipe

Then open your Erlang shell

➜  cd /tmp
➜  erl
Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] [dtrace]

Eshell V6.3  (abort with ^G)
1>

Okay, lets attempt to open test.pipe

1> Fifo = open_port("test.pipe", [eof]).
#Port<0.470>

Next, we cannot read from this like a file (see the FAQ linked above), so instead we need to use receive

1> Fifo = open_port("test.pipe", [eof]).
#Port<0.470>
2> receive
2>   {Fifo, {data, Data}} ->
2>     io:format("Got some data: ~p~n", [Data])
2>   end.
%% This is blocking, waiting for data

Back to your shell

➜  echo "Hello Erlang" > test.pipe
➜

Erlang now returns

1> Fifo = open_port("test.pipe", [eof]).
#Port<0.470>
2> receive
2>   {Fifo, {data, Data}} ->
2>     io:format("Got some data: ~p~n", [Data])
2>   end.
Got some data: "Hello Erlang\n"
ok
3>

Okay, sweet, that wasn't so hard. That is of course once you know you need to use receive and to know what to match against. Next up, lets write some stuff from Erlang.

To Write

Much like the previous example, make sure you actually have a fifo first.

To speed through, here's the bash side waiting to read

➜ cat test.pipe
➜ # ... blocking

In Erlang

Eshell V6.3  (abort with ^G)
1> Fifo = open_port("test.pipe", [eof]).
#Port<0.470>
2> port_command(Fifo, <<"Hello bash\n">>).
true
3> port_command(Fifo, <<"Howzit?\n">>).
true
4>

Back to Shell

➜ cat test.pipe
Hello bash
Howzit?

So that also works.

Some advice from my mistakes, what you pass into port_command is very important. The newline in <<"Hello bash\n">> is very important. I was leaving it off when sending data and nothing was working. My esteemed boss Andrew suggested using flush in the Erlang shell to flush out the mailbox, but that didn't work. He realized a simple \n was needed. It is also worth noting that passing in the [eof] flag to open_port keeps the port open, whereas if you just used the shell and echo'd, it'd close the port after it sent its line.

I know this wasn't a mind bending problem or solution, but one of those frustrating things you don't want to go searching for late at night when you just want to read and write from a damn pseudo-file.

binarytemple-external/read_write_fifo_erlang.md