Bidirectional named pipes on Linux

ruby-talk
1

I've got two programs which need to communicate (which eliminates
un-named pipes). Information needs to be sent in both directions such
that one program (say it's called the master) sends a couple of bytes
to and then waits for a response from the other program (say it's called the
slave) and so on. It looks like if you open the pipe (or FIFO file) from
both sides in read/write mode that the pipe is opened non-blocking so what
ends up happening is that the side which sends first just keeps sending and
doesn't wait for a response.

....yeah, I should probably be using sockets, but this is a sort of
'legacy' system.

phil

2

Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such

The mere fact that 2 independent programs need to communicate does not eliminate un-named pipes. A third program may be written to create un-named pipes, fork 2 processes, redirect file descriptors accordingly in both of them and then exec your programs in those processes. It may also be possible to exec one program in the original process.

that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response.

....yeah, I should probably be using sockets, but this is a sort of 'legacy' system.

phil

How do you specify that FIFO file to your programs? Do they know it by name and open it explicitly or do they expect certain file descriptor to point to the file on program startup?

You may end up with 2 FIFOs, as if you use only one FIFO a process that wrote to the pipe and then started reading from it will compete with the other process to which the data was intended. I would also think about using a pty for this as it is truly bi-directional.

If you need more help, please provide details on your programs.

Gennady.

3

In C, you'd use something like this:

fcntl(fd, F_SETFL,
         fcntl(fd, F_GETFL) &
         ~O_NONBLOCK);

In Ruby, the following:

require 'io/nonblock'
fh = open('/tmp/myfifo')
fh.nonblock = false

...which just hides the fcntl call and nasty constants in a nicer
interface. On my test machine, though, it looks like the FIFO opens in
blocking mode by default, at least if opened for writing.

Any of that help?

Lennon

4

what about

   -------- ---- ---------
   read_end fifo write_end
   -------- ---- ---------
   master 0 slave
   slave 1 master

-a

···

On Wed, 16 Jun 2004, Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates
un-named pipes). Information needs to be sent in both directions such
that one program (say it's called the master) sends a couple of bytes
to and then waits for a response from the other program (say it's called the
slave) and so on. It looks like if you open the pipe (or FIFO file) from
both sides in read/write mode that the pipe is opened non-blocking so what
ends up happening is that the side which sends first just keeps sending and
doesn't wait for a response.

...yeah, I should probably be using sockets, but this is a sort of
'legacy' system.

phil

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen

===============================================================================

5

Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response.

...yeah, I should probably be using sockets, but this is a sort of 'legacy' system.

phil

Phil,

I'm doing this exact same thing on a linux system so that a ruby process and a C process can communicate. It works perfectly, here is generally how I did it. First, pipes are one way. You can have one end reading and one end writing at the same time, but both sides may not write simultaneously and both ends may not read simultaneously. Pipes are strictly fifos, that's why the command to make named ones is mkfifo. One way in, one way out. So you can either make two pipes, one for each direction or switch who is reading and who is writing, this is difficult if you ask me.

Next, just write to it as if it is a file. The data will stay in the pipe until the other end opens it and then it spills out all over if you don't catch it. It can only be read out once. So here is how I think you should do it..

Make two fifos, one for each direction. The process that sends first sends data down the out fifo and calls select on the in fifo. Select will cause it to block until it is ready to read. The other process starts out selecting on the incoming fifo and when the data finally comes in it can send the response on the outgoing fifo.

Try to think of it like two guys with those vacuum tubes sending messages in canisters. There is just one canister and the tubes are one way. The guy puts the canister in the pipe that goes out and then sits looking at the other pipe waiting for a response.

I hope this helps. I know its not bi-directional, but it really can't be.

-Scott

6

It looks like if you open the pipe (or FIFO file) from
both sides in read/write mode that the pipe is opened non-blocking so what
ends up happening is that the side which sends first just keeps sending and
doesn't wait for a response.

Umm, if I understand things aright, if you open a named pipe in read/write, mode do a write then a read, I'm afraid you just chatting to yourself!

eg.
mkfifo foofi
ruby -e 'open("foofi", "r+") {|i| i.syswrite( "as");puts i.sysread(2)}'
as

Solution you need to open _TWO_ named pipes, one for reading, one for writing.

Also note I used sysread and syswrite others buffering and screw up your life.

....yeah, I should probably be using sockets, but this is a sort of
'legacy' system.

So now you have two IO's to watch, you can use a "select" to cope with that, _or_ Threads. Did you know (Matz may have changed it but this is the way it was when I last looked) all the threading stuff all wends it way down into a single select lurking deep within the bowels of ruby?

ie. Ruby Thread based app === "select" event based app in a very deep sense.

Ruby threading is just simpler.

John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter@tait.co.nz
New Zealand

The universe is absolutely plastered with the dashed lines exactly one
space long.

···

On Thu, 17 Jun 2004, Phil Tomson wrote:

7

Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates
un-named pipes). Information needs to be sent in both directions such

The mere fact that 2 independent programs need to communicate does not
eliminate un-named pipes. A third program may be written to create
un-named pipes, fork 2 processes, redirect file descriptors accordingly
in both of them and then exec your programs in those processes. It may
also be possible to exec one program in the original process.

That's an idea, but in this case it probably won't work. One of the
programs is usually started much earlier than the other and stays up much
longer (the other program might be invoked and finish several times while
the first one is running).

that one program (say it's called the master) sends a couple of bytes
to and then waits for a response from the other program (say it's called the
slave) and so on. It looks like if you open the pipe (or FIFO file) from
both sides in read/write mode that the pipe is opened non-blocking so what
ends up happening is that the side which sends first just keeps sending and
doesn't wait for a response.

....yeah, I should probably be using sockets, but this is a sort of
'legacy' system.

phil

How do you specify that FIFO file to your programs? Do they know it by
name and open it explicitly or do they expect certain file descriptor to
point to the file on program startup?

By name.

You may end up with 2 FIFOs, as if you use only one FIFO a process that
wrote to the pipe and then started reading from it will compete with the
other process to which the data was intended. I would also think about
using a pty for this as it is truly bi-directional.

Yes, that's the conclusion that I've come to: I need two FIFO files.

If you need more help, please provide details on your programs.

Sorry 'bout that. The current code is in C so i didn't want to post it
here (also it doesn't work). I'm only going to be using Ruby on one side
just to do a bit of debugging...

Phil

···

In article <40D0D1CA.4080408@tonesoft.com>, Gennady <gfb@tonesoft.com> wrote:

8

In article <5d4c612404061616185967e8c6@mail.gmail.com>,

In C, you'd use something like this:

fcntl(fd, F_SETFL,
         fcntl(fd, F_GETFL) &
         ~O_NONBLOCK);

Hmmm... are you sure about that negated O_NONBLOCK (~O_NONBLOCK)? I'm not
sure what those constants are defined as in the respective .h file, but
depending on what they are that may not work (and may not work 'badly'
:-).

In Ruby, the following:

require 'io/nonblock'
fh = open('/tmp/myfifo')
fh.nonblock = false

..which just hides the fcntl call and nasty constants in a nicer
interface. On my test machine, though, it looks like the FIFO opens in
blocking mode by default, at least if opened for writing.

Any of that help?

I'll take a look at io/nonblock, but right now I think I actually need
two FIFOs.

(BTW: This is something I'm converting from Windows C code - apparently
they allow bidirectional pipes on Windows).

Phil

···

Lennon Day-Reynolds <rcoder@gmail.com> wrote:

9

In article <5d4c612404061616185967e8c6@mail.gmail.com>, Lennon Day-Reynolds
wrote:

In C, you'd use something like this:

fcntl(fd, F_SETFL,
         fcntl(fd, F_GETFL) &
         ~O_NONBLOCK);

In Ruby, the following:

require 'io/nonblock'
fh = open('/tmp/myfifo')
fh.nonblock = false

..which just hides the fcntl call and nasty constants in a nicer
interface. On my test machine, though, it looks like the FIFO opens in
blocking mode by default, at least if opened for writing.

Do you meah

  fh.nonblock = true

perhaps :slight_smile:

10

Actually, the original command for making fifos (as well as other types of special files) is mknod (it is in /etc on some systems, in /bin or /sbin on others):

mknod <fifo file name> p

:wink:

Sincerely,
Gennady Bystritsky

···

On Jun 16, 2004, at 22:48, Scott Rubin wrote:

Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response. ...yeah, I should probably be using sockets, but this is a sort of 'legacy' system.
phil

Phil,

I'm doing this exact same thing on a linux system so that a ruby process and a C process can communicate. It works perfectly, here is generally how I did it. First, pipes are one way. You can have one end reading and one end writing at the same time, but both sides may not write simultaneously and both ends may not read simultaneously. Pipes are strictly fifos, that's why the command to make named ones is mkfifo. One way in, one way out. So you can

11

In article <40d0f617$1@buckaroo.cs.rit.edu>,

···

Scott Rubin <slr2777@cs.rit.edu> wrote:

Phil Tomson wrote:

I've got two programs which need to communicate (which eliminates
un-named pipes). Information needs to be sent in both directions such
that one program (say it's called the master) sends a couple of bytes
to and then waits for a response from the other program (say it's called the
slave) and so on. It looks like if you open the pipe (or FIFO file) from
both sides in read/write mode that the pipe is opened non-blocking so what
ends up happening is that the side which sends first just keeps sending and
doesn't wait for a response.

...yeah, I should probably be using sockets, but this is a sort of
'legacy' system.

phil

Phil,

I'm doing this exact same thing on a linux system so that a ruby process
and a C process can communicate. It works perfectly, here is generally
how I did it. First, pipes are one way. You can have one end reading
and one end writing at the same time, but both sides may not write
simultaneously and both ends may not read simultaneously. Pipes are
strictly fifos, that's why the command to make named ones is mkfifo.
One way in, one way out. So you can either make two pipes, one for each
direction or switch who is reading and who is writing, this is difficult
if you ask me.

Next, just write to it as if it is a file. The data will stay in the
pipe until the other end opens it and then it spills out all over if you
don't catch it. It can only be read out once. So here is how I think you
should do it..

Make two fifos, one for each direction. The process that sends first
sends data down the out fifo and calls select on the in fifo. Select
will cause it to block until it is ready to read. The other process
starts out selecting on the incoming fifo and when the data finally
comes in it can send the response on the outgoing fifo.

Try to think of it like two guys with those vacuum tubes sending
messages in canisters. There is just one canister and the tubes are one
way. The guy puts the canister in the pipe that goes out and then sits
looking at the other pipe waiting for a response.

You wouldn't happen to have the code which shows the select call, would
you?

Phil

12

Actually, since it sounded like Phil was trying to force the writer to
wait for the reader, my assumption was that he wanted to force
blocking writes.

After a bit more experimentation, though, the problem looks like it is
probably caused by buffering, rather than async writes. You could try
or'ing the O_SYNC to the flags for your open() call, or adding it
after the fact with fcntl().

Lennon