# Bidirectional named pipes on Linux **URL:** \ **Category:** ruby-talk\ **Created:** [16 June 2004 22:13 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160 "2004-06-16T22:13:31Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Ptkwt](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@Ptkwt](https://rubytalk.org/u/Ptkwt)\ **Post date:** [16 June 2004 22:13 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/1 "2004-06-16T22:13:31Z")
I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response. ....yeah, I should probably be using sockets, but this is a sort of 'legacy' system. phil ---
**Author:** ![Gennady1](https://avatars.discourse-cdn.com/v4/letter/g/a4c791/32.png) [@Gennady1](https://rubytalk.org/u/Gennady1)\ **Post date:** [16 June 2004 23:03 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/2 "2004-06-16T23:03:30Z")
Phil Tomson wrote: > I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such The mere fact that 2 independent programs need to communicate does not eliminate un-named pipes. A third program may be written to create un-named pipes, fork 2 processes, redirect file descriptors accordingly in both of them and then exec your programs in those processes. It may also be possible to exec one program in the original process. > that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response. > > ....yeah, I should probably be using sockets, but this is a sort of 'legacy' system. > > phil How do you specify that FIFO file to your programs? Do they know it by name and open it explicitly or do they expect certain file descriptor to point to the file on program startup? You may end up with 2 FIFOs, as if you use only one FIFO a process that wrote to the pipe and then started reading from it will compete with the other process to which the data was intended. I would also think about using a pty for this as it is truly bi-directional. If you need more help, please provide details on your programs. Gennady. ---
**Author:** ![Lennon\_Day-Reynolds1](https://avatars.discourse-cdn.com/v4/letter/l/5f8ce5/32.png) [@Lennon\_Day-Reynolds1](https://rubytalk.org/u/Lennon_Day-Reynolds1)\ **Post date:** [16 June 2004 23:18 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/3 "2004-06-16T23:18:06Z")
In C, you'd use something like this: > fcntl(fd, F\_SETFL, >          fcntl(fd, F\_GETFL) & >          ~O\_NONBLOCK); In Ruby, the following: > require 'io/nonblock' > fh = open('/tmp/myfifo') > fh.nonblock = false ...which just hides the fcntl call and nasty constants in a nicer interface. On my test machine, though, it looks like the FIFO opens in blocking mode by default, at least if opened for writing. Any of that help? Lennon ---
**Author:** ![Ara.T.Howard3](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Ara.T.Howard3](https://rubytalk.org/u/Ara.T.Howard3)\ **Post date:** [16 June 2004 23:53 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/4 "2004-06-16T23:53:25Z")
what about    -------- ---- ---------    read\_end fifo write\_end    -------- ---- ---------    master 0 slave    slave 1 master -a > **···** > > On Wed, 16 Jun 2004, Phil Tomson wrote: > > > I've got two programs which need to communicate (which eliminates > > un-named pipes). Information needs to be sent in both directions such > > that one program (say it's called the master) sends a couple of bytes > > to and then waits for a response from the other program (say it's called the > > slave) and so on. It looks like if you open the pipe (or FIFO file) from > > both sides in read/write mode that the pipe is opened non-blocking so what > > ends up happening is that the side which sends first just keeps sending and > > doesn't wait for a response. > > > > ...yeah, I should probably be using sockets, but this is a sort of > > 'legacy' system. > > > > phil > > # -- > > > EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov > > PHONE :: 303.497.6469 > > A flower falls, even though we love it; and a weed grows, even though we do > > not love it. --Dogen > > =============================================================================== ---
**Author:** ![Scott\_Rubin](https://avatars.discourse-cdn.com/v4/letter/s/91b2a8/32.png) [@Scott\_Rubin](https://rubytalk.org/u/Scott_Rubin)\ **Post date:** [17 June 2004 05:48 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/5 "2004-06-17T05:48:30Z")
Phil Tomson wrote: > I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response. > > ...yeah, I should probably be using sockets, but this is a sort of 'legacy' system. > > phil Phil, I'm doing this exact same thing on a linux system so that a ruby process and a C process can communicate. It works perfectly, here is generally how I did it. First, pipes are one way. You can have one end reading and one end writing at the same time, but both sides may not write simultaneously and both ends may not read simultaneously. Pipes are strictly fifos, that's why the command to make named ones is mkfifo. One way in, one way out. So you can either make two pipes, one for each direction or switch who is reading and who is writing, this is difficult if you ask me. Next, just write to it as if it is a file. The data will stay in the pipe until the other end opens it and then it spills out all over if you don't catch it. It can only be read out once. So here is how I think you should do it.. Make two fifos, one for each direction. The process that sends first sends data down the out fifo and calls select on the in fifo. Select will cause it to block until it is ready to read. The other process starts out selecting on the incoming fifo and when the data finally comes in it can send the response on the outgoing fifo. Try to think of it like two guys with those vacuum tubes sending messages in canisters. There is just one canister and the tubes are one way. The guy puts the canister in the pipe that goes out and then sits looking at the other pipe waiting for a response. I hope this helps. I know its not bi-directional, but it really can't be. -Scott ---
**Author:** ![John\_Carter](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@John\_Carter](https://rubytalk.org/u/John_Carter)\ **Post date:** [17 June 2004 22:04 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/6 "2004-06-17T22:04:57Z")
> It looks like if you open the pipe (or FIFO file) from > both sides in read/write mode that the pipe is opened non-blocking so what > ends up happening is that the side which sends first just keeps sending and > doesn't wait for a response. Umm, if I understand things aright, if you open a named pipe in read/write, mode do a write then a read, I'm afraid you just chatting to yourself! eg. mkfifo foofi ruby -e 'open("foofi", "r+") {|i| i.syswrite( "as");puts i.sysread(2)}' as Solution you need to open \_TWO\_ named pipes, one for reading, one for writing. Also note I used sysread and syswrite others buffering and screw up your life. > ....yeah, I should probably be using sockets, but this is a sort of > 'legacy' system. So now you have two IO's to watch, you can use a "select" to cope with that, \_or\_ Threads. Did you know (Matz may have changed it but this is the way it was when I last looked) all the threading stuff all wends it way down into a single select lurking deep within the bowels of ruby? ie. Ruby Thread based app === "select" event based app in a very deep sense. Ruby threading is just simpler. John Carter Phone : (64)(3) 358 6639 Tait Electronics Fax : (64)(3) 359 4632 PO Box 1645 Christchurch Email : john.carter@tait.co.nz New Zealand The universe is absolutely plastered with the dashed lines exactly one space long. > **···** > > On Thu, 17 Jun 2004, Phil Tomson wrote: ---
**Author:** ![Ptkwt](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@Ptkwt](https://rubytalk.org/u/Ptkwt)\ **Post date:** [17 June 2004 00:08 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/7 "2004-06-17T00:08:25Z")
> Phil Tomson wrote: > > > I've got two programs which need to communicate (which eliminates > > un-named pipes). Information needs to be sent in both directions such > > The mere fact that 2 independent programs need to communicate does not > eliminate un-named pipes. A third program may be written to create > un-named pipes, fork 2 processes, redirect file descriptors accordingly > in both of them and then exec your programs in those processes. It may > also be possible to exec one program in the original process. That's an idea, but in this case it probably won't work. One of the programs is usually started much earlier than the other and stays up much longer (the other program might be invoked and finish several times while the first one is running). > > that one program (say it's called the master) sends a couple of bytes > > to and then waits for a response from the other program (say it's called the > > slave) and so on. It looks like if you open the pipe (or FIFO file) from > > both sides in read/write mode that the pipe is opened non-blocking so what > > ends up happening is that the side which sends first just keeps sending and > > doesn't wait for a response. > > > > ....yeah, I should probably be using sockets, but this is a sort of > > 'legacy' system. > > > > phil > > How do you specify that FIFO file to your programs? Do they know it by > name and open it explicitly or do they expect certain file descriptor to > point to the file on program startup? By name. > You may end up with 2 FIFOs, as if you use only one FIFO a process that > wrote to the pipe and then started reading from it will compete with the > other process to which the data was intended. I would also think about > using a pty for this as it is truly bi-directional. Yes, that's the conclusion that I've come to: I need two FIFO files. > If you need more help, please provide details on your programs. Sorry 'bout that. The current code is in C so i didn't want to post it here (also it doesn't work). I'm only going to be using Ruby on one side just to do a bit of debugging... Phil > **···** > > In article \<40D0D1CA.4080408@tonesoft.com\>, Gennady \ wrote: ---
**Author:** ![Ptkwt](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@Ptkwt](https://rubytalk.org/u/Ptkwt)\ **Post date:** [17 June 2004 00:08 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/8 "2004-06-17T00:08:25Z")
In article \<5d4c612404061616185967e8c6@mail.gmail.com\>, > In C, you'd use something like this: > > > fcntl(fd, F\_SETFL, > >          fcntl(fd, F\_GETFL) & > >          ~O\_NONBLOCK); Hmmm... are you sure about that negated O\_NONBLOCK (~O\_NONBLOCK)? I'm not sure what those constants are defined as in the respective .h file, but depending on what they are that may not work (and may not work 'badly' :-). > In Ruby, the following: > > > require 'io/nonblock' > > fh = open('/tmp/myfifo') > > fh.nonblock = false > > ..which just hides the fcntl call and nasty constants in a nicer > interface. On my test machine, though, it looks like the FIFO opens in > blocking mode by default, at least if opened for writing. > > Any of that help? I'll take a look at io/nonblock, but right now I think I actually need two FIFOs. (BTW: This is something I'm converting from Windows C code - apparently they allow bidirectional pipes on Windows). Phil > **···** > > Lennon Day-Reynolds \ wrote: ---
**Author:** ![Tim\_Sutherland1](https://avatars.discourse-cdn.com/v4/letter/t/8e7dd6/32.png) [@Tim\_Sutherland1](https://rubytalk.org/u/Tim_Sutherland1)\ **Post date:** [17 June 2004 00:18 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/9 "2004-06-17T00:18:24Z")
In article \<5d4c612404061616185967e8c6@mail.gmail.com\>, Lennon Day-Reynolds wrote: > In C, you'd use something like this: > > > fcntl(fd, F\_SETFL, > >          fcntl(fd, F\_GETFL) & > >          ~O\_NONBLOCK); > > In Ruby, the following: > > > require 'io/nonblock' > > fh = open('/tmp/myfifo') > > fh.nonblock = false > > ..which just hides the fcntl call and nasty constants in a nicer > interface. On my test machine, though, it looks like the FIFO opens in > blocking mode by default, at least if opened for writing. Do you meah   fh.nonblock = true perhaps 🙂 ---
**Author:** ![Gennady1](https://avatars.discourse-cdn.com/v4/letter/g/a4c791/32.png) [@Gennady1](https://rubytalk.org/u/Gennady1)\ **Post date:** [17 June 2004 16:26 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/10 "2004-06-17T16:26:11Z")
Actually, the original command for making fifos (as well as other types of special files) is mknod (it is in /etc on some systems, in /bin or /sbin on others): mknod \ p 😉 Sincerely, Gennady Bystritsky > **···** > > On Jun 16, 2004, at 22:48, Scott Rubin wrote: > > > Phil Tomson wrote: > > > > > I've got two programs which need to communicate (which eliminates un-named pipes). Information needs to be sent in both directions such that one program (say it's called the master) sends a couple of bytes to and then waits for a response from the other program (say it's called the slave) and so on. It looks like if you open the pipe (or FIFO file) from both sides in read/write mode that the pipe is opened non-blocking so what ends up happening is that the side which sends first just keeps sending and doesn't wait for a response. ...yeah, I should probably be using sockets, but this is a sort of 'legacy' system. > > > phil > > > > Phil, > > > > I'm doing this exact same thing on a linux system so that a ruby process and a C process can communicate. It works perfectly, here is generally how I did it. First, pipes are one way. You can have one end reading and one end writing at the same time, but both sides may not write simultaneously and both ends may not read simultaneously. Pipes are strictly fifos, that's why the command to make named ones is mkfifo. One way in, one way out. So you can ---
**Author:** ![Ptkwt](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@Ptkwt](https://rubytalk.org/u/Ptkwt)\ **Post date:** [17 June 2004 17:23 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/11 "2004-06-17T17:23:27Z")
In article \<40d0f617$1@buckaroo.cs.rit.edu\>, > **···** > > Scott Rubin \ wrote: > > > Phil Tomson wrote: > > > > > I've got two programs which need to communicate (which eliminates > > > un-named pipes). Information needs to be sent in both directions such > > > that one program (say it's called the master) sends a couple of bytes > > > to and then waits for a response from the other program (say it's called the > > > slave) and so on. It looks like if you open the pipe (or FIFO file) from > > > both sides in read/write mode that the pipe is opened non-blocking so what > > > ends up happening is that the side which sends first just keeps sending and > > > doesn't wait for a response. > > > > > > ...yeah, I should probably be using sockets, but this is a sort of > > > 'legacy' system. > > > > > > phil > > > > Phil, > > > > I'm doing this exact same thing on a linux system so that a ruby process > > and a C process can communicate. It works perfectly, here is generally > > how I did it. First, pipes are one way. You can have one end reading > > and one end writing at the same time, but both sides may not write > > simultaneously and both ends may not read simultaneously. Pipes are > > strictly fifos, that's why the command to make named ones is mkfifo. > > One way in, one way out. So you can either make two pipes, one for each > > direction or switch who is reading and who is writing, this is difficult > > if you ask me. > > > > Next, just write to it as if it is a file. The data will stay in the > > pipe until the other end opens it and then it spills out all over if you > > don't catch it. It can only be read out once. So here is how I think you > > should do it.. > > > > Make two fifos, one for each direction. The process that sends first > > sends data down the out fifo and calls select on the in fifo. Select > > will cause it to block until it is ready to read. The other process > > starts out selecting on the incoming fifo and when the data finally > > comes in it can send the response on the outgoing fifo. > > > > Try to think of it like two guys with those vacuum tubes sending > > messages in canisters. There is just one canister and the tubes are one > > way. The guy puts the canister in the pipe that goes out and then sits > > looking at the other pipe waiting for a response. > > You wouldn't happen to have the code which shows the select call, would > you? > > Phil ---
**Author:** ![Lennon\_Day-Reynolds1](https://avatars.discourse-cdn.com/v4/letter/l/5f8ce5/32.png) [@Lennon\_Day-Reynolds1](https://rubytalk.org/u/Lennon_Day-Reynolds1)\ **Post date:** [17 June 2004 02:43 UTC](https://rubytalk.org/t/bidirectional-named-pipes-on-linux/12160/12 "2004-06-17T02:43:07Z")
Actually, since it sounded like Phil was trying to force the writer to wait for the reader, my assumption was that he wanted to force blocking writes. After a bit more experimentation, though, the problem looks like it is probably caused by buffering, rather than async writes. You could try or'ing the O\_SYNC to the flags for your open() call, or adding it after the fact with fcntl(). Lennon