# Nonblocking IO read

**URL:** <https://rubytalk.org/t/nonblocking-io-read/32545>\
**Category:** ruby-talk\
**Created:** [31 October 2006 22:55 UTC](https://rubytalk.org/t/nonblocking-io-read/32545 "2006-10-31T22:55:15Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Robert\_James](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@Robert\_James](https://rubytalk.org/u/Robert_James)\
**Post date:** [31 October 2006 22:55 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/1 "2006-10-31T22:55:15Z")

</div>

How can I perform a nonblocking IO read? That is, read whatever is  
there to read and return. If there's nothing yet, just return.

Failing that, is there a way to manually check if there's anything to  
read or the read will block?

(Note that I'm developing for both 'nix and Windows.)

---

<div class="post-metadata">

**Author:** ![Ara.T.Howard6](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@Ara.T.Howard6](https://rubytalk.org/u/Ara.T.Howard6)\
**Post date:** [31 October 2006 23:23 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/2 "2006-10-31T23:23:18Z")

</div>

it can't be done. search the archives, this sort of thing almost always  
indicates a design flaw. for instance - what will your program do if there is  
no input?

regards.

-a

> **···**
>
> On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> 
> > How can I perform a nonblocking IO read? That is, read whatever is  
> > there to read and return. If there's nothing yet, just return.
> > 
> > Failing that, is there a way to manually check if there's anything to  
> > read or the read will block?
> > 
> > (Note that I'm developing for both 'nix and Windows.)
> 
> --  
> my religion is very simple. my religion is kindness. -- the dalai lama

---

<div class="post-metadata">

**Author:** ![Francis\_Cianfrocca](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Francis\_Cianfrocca](https://rubytalk.org/u/Francis_Cianfrocca)\
**Post date:** [1 November 2006 05:30 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/3 "2006-11-01T05:30:07Z")

</div>

Are you having an interactive conversation with the process created by  
popen4? Is that why you need nbio, because you need to poll more than one  
stream simultaneously?

You can get the file descriptors associated with stdout and stderr (they'll  
be 1 and 2 unless you did something unusual), set them nonblocking, and  
select them for reading with a short timeout.

> **···**
>
> On 10/31/06, srobertjames@gmail.com \<srobertjames@gmail.com\> wrote:
> 
> > How can I perform a nonblocking IO read? That is, read whatever is  
> > there to read and return. If there's nothing yet, just return.
> > 
> > Failing that, is there a way to manually check if there's anything to  
> > read or the read will block?
> > 
> > (Note that I'm developing for both 'nix and Windows.)

---

<div class="post-metadata">

**Author:** ![Tom\_Pollard](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@Tom\_Pollard](https://rubytalk.org/u/Tom_Pollard)\
**Post date:** [1 November 2006 00:08 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/4 "2006-11-01T00:08:14Z")

</div>

> > How can I perform a nonblocking IO read? That is, read whatever is there to read and return. If there's nothing yet, just return.

That's what the Unix read() function does ("man 2 read"), available in Ruby as IO#sysread(). The ri documentation doesn't explain much about how to use Ruby's sysread, unfortunately.

> > Failing that, is there a way to manually check if there's anything to read or the read will block?
> > 
> > (Note that I'm developing for both 'nix and Windows.)

The select() function (Kernel#select, in Ruby) is the standard way to do IO multiplexing under Unix. You give it a list of file handles on which you're waiting to be able to read or write, and it returns when one or more of them is ready. Windows provides a select() function, too, but it only supports sockets (not pipes or ordinary file IO.) The Win32 API generally doesn't provide non-blocking IO methods, because it's assumed you'll use threads if you want to do something else while waiting for IO.

> it can't be done. search the archives, this sort of thing almost always  
> indicates a design flaw. for instance - what will your program do if there is  
> no input?

Check other IO channels, wait for other sorts of events, do work - whatever. Where do come from asserting that the use of non-blocking IO represents a "design flaw"?

Tom

> **···**
>
> On Oct 31, 2006, at 6:23 PM, ara.t.howard@noaa.gov wrote:
> 
> > On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:

---

<div class="post-metadata">

**Author:** ![Robert\_James](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@Robert\_James](https://rubytalk.org/u/Robert_James)\
**Post date:** [1 November 2006 00:15 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/5 "2006-11-01T00:15:13Z")

</div>

I'm running an external command (using popen4 - yes, it is available on  
Windows also). I capture stdout and stderr. "what will your program do  
if there is no input?" - I assume you mean output - Well, I hope there  
is no stderr output. If there is, I'd like to capture it and feed it  
to Logger.

Likewise, if there is a problem, and there's no stdout, then I don't  
want to wait for ever - I want to throw an exception.

> **···**
>
> ara.t.howard@noaa.gov wrote:
> 
> > On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> > 
> > \> How can I perform a nonblocking IO read? That is, read whatever is  
> > \> there to read and return. If there's nothing yet, just return.  
> > \>  
> > \> Failing that, is there a way to manually check if there's anything to  
> > \> read or the read will block?  
> > \>  
> > \> (Note that I'm developing for both 'nix and Windows.)
> > 
> > it can't be done. search the archives, this sort of thing almost always  
> > indicates a design flaw. for instance - what will your program do if there is  
> > no input?

---

<div class="post-metadata">

**Author:** ![Bjorn\_Borud](https://avatars.discourse-cdn.com/v4/letter/b/e68b1a/32.png) [@Bjorn\_Borud](https://rubytalk.org/u/Bjorn_Borud)\
**Post date:** [1 November 2006 02:30 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/6 "2006-11-01T02:30:08Z")

</div>

[ara.t.howard@noaa.gov]

> it can't be done. search the archives, this sort of thing almost  
> always indicates a design flaw. for instance - what will your  
> program do if there is no input?

nonblocking IO is not a design-flaw; it is something that is in common  
use and is supported by most languages with a decent socket API. on  
some OS'es this is even available for file IO (although widespread  
language support seems to lag behind).

Java for example got proper nonblocking socket IO in 1.4 (see  
JSR-000051 "New I/O APIs for the JavaTM Platform").

for examples of use you might want to check out the Reactor pattern  
and other patterns for concurrent programming.

-Bjørn

---

<div class="post-metadata">

**Author:** ![Vidar\_Hokstad](https://avatars.discourse-cdn.com/v4/letter/v/aca169/32.png) [@Vidar\_Hokstad](https://rubytalk.org/u/Vidar_Hokstad)\
**Post date:** [5 November 2006 10:10 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/7 "2006-11-05T10:10:14Z")

</div>

I have a messaging middleware server running right now that is  
processing millions of messages a day. It's written in Ruby, and does  
\_all\_ it's work in a single thread using IO multiplexing with select().

Not only can it be done - it is fairly easy (~700 lines for the entire  
app, including db persistence support etc.). Doing the same thing with  
threads and blocking IO, on the other hand, would fall apart horribly  
due to the way Ruby does threading. Forking also wouldn't be an option  
as the processes would still need to actually exchange those messages.

In fact, internally, Ruby does all it's IO using non-blocking IO  
exactly because the threading model would cause everything to block  
otherwise. Incidentally that's also one of the reasons why using  
threads + blocking IO performs extremely badly in Ruby once the number  
of threads gets beyond a certain level, because it causes Ruby to call  
select() far too often.

Vidar

> **···**
>
> ara.t.howard@noaa.gov wrote:
> 
> > On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> > 
> > \> How can I perform a nonblocking IO read? That is, read whatever is  
> > \> there to read and return. If there's nothing yet, just return.  
> > \>  
> > \> Failing that, is there a way to manually check if there's anything to  
> > \> read or the read will block?  
> > \>  
> > \> (Note that I'm developing for both 'nix and Windows.)
> > 
> > it can't be done. search the archives, this sort of thing almost always  
> > indicates a design flaw. for instance - what will your program do if there is  
> > no input?

---

<div class="post-metadata">

**Author:** ![Robert\_James](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@Robert\_James](https://rubytalk.org/u/Robert_James)\
**Post date:** [1 November 2006 00:20 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/8 "2006-11-01T00:20:11Z")

</div>

In the interests of promoting creativity, I'll ask the question without  
any prior assumptions:

I'm using popen4 (great gem, btw) to run an external command and  
capture stdout and stderr. If one (or both) of those is empty, I don't  
want to hang. Just keep on moving. In fact, stderr will normally be  
empty.

How could I do this?

(I'm supporting 'nix and Windows)

---

<div class="post-metadata">

**Author:** ![Ara.T.Howard6](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@Ara.T.Howard6](https://rubytalk.org/u/Ara.T.Howard6)\
**Post date:** [1 November 2006 00:33 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/9 "2006-11-01T00:33:05Z")

</div>

> > > How can I perform a nonblocking IO read? That is, read whatever is there to read and return. If there's nothing yet, just return.
> 
> That's what the Unix read() function does ("man 2 read"), available in Ruby as IO#sysread(). The ri documentation doesn't explain much about how to use Ruby's sysread, unfortunately.

no, that blocks. try this:

&nbsp;&nbsp;&nbsp;harp:~ \> ruby -e 'STDIN.sysread(1)'

it blocks. the OP wants a non-blocking call. eg

&nbsp;&nbsp;&nbsp;harp:~ \> ruby -r io/nonblock -e 'STDIN.nonblock{ STDIN.sysread(1) }'  
&nbsp;&nbsp;&nbsp;-e:1:in `sysread': Resource temporarily unavailable (Errno::EAGAIN)  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from -e:1  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /home/ahoward//lib/ruby/1.8/io/nonblock.rb:19:in `nonblock'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from -e:1

> The select() function (Kernel#select, in Ruby) is the standard way to do IO multiplexing under Unix. You give it a list of file handles on which you're waiting to be able to read or write, and it returns when one or more of them is ready. Windows provides a select() function, too, but it only supports sockets (not pipes or ordinary file IO.) The Win32 API generally doesn't provide non-blocking IO methods, because it's assumed you'll use threads if you want to do something else while waiting for IO.

right. but ruby's thread are green and, on windows, block the entire process  
when waiting on IO so not an option here.

> Check other IO channels, wait for other sorts of events, do work - whatever. Where do come from asserting that the use of non-blocking IO represents a "design flaw"?

because nine times out of ten code has nothing to do if no input is available.  
if it does that's another story, but then there are basically two options:

- use select or an event abstraction in one giant loop. if you do this you are  
going to block anyhow, only based on all inputs, not just one.

- use threads, only you can't if you're wanting ruby code to run nb in windows.

i'm not saying there is never a case for nbio - i use it myself - just that,  
unless a developer has a good reason to be doing something else it's not  
required and, as a search of ruby-talk will show, that is often the case.

regards.

-a

> **···**
>
> On Wed, 1 Nov 2006, Tom Pollard wrote:
> 
> > On Oct 31, 2006, at 6:23 PM, ara.t.howard@noaa.gov wrote:
> > 
> > > On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> 
> --  
> my religion is very simple. my religion is kindness. -- the dalai lama

---

<div class="post-metadata">

**Author:** ![Ara.T.Howard6](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@Ara.T.Howard6](https://rubytalk.org/u/Ara.T.Howard6)\
**Post date:** [1 November 2006 00:35 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/10 "2006-11-01T00:35:29Z")

</div>

so, basically you want to timeout on a read right? in unix you can just use

&nbsp;&nbsp;&nbsp;require 'timeout'

&nbsp;&nbsp;&nbsp;Timeout::timeout(n) do  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Open4.popen4 ...  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...  
&nbsp;&nbsp;&nbsp;end

but, this is implimented via a thread so i think the read is going to block  
your entire process in windows... you might try it.

-a

> **···**
>
> On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> 
> > I'm running an external command (using popen4 - yes, it is available on  
> > Windows also). I capture stdout and stderr. "what will your program do if  
> > there is no input?" - I assume you mean output - Well, I hope there is no  
> > stderr output. If there is, I'd like to capture it and feed it to Logger.
> > 
> > Likewise, if there is a problem, and there's no stdout, then I don't want to  
> > wait for ever - I want to throw an exception.
> 
> --  
> my religion is very simple. my religion is kindness. -- the dalai lama

---

<div class="post-metadata">

**Author:** ![John\_Carter](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@John\_Carter](https://rubytalk.org/u/John_Carter)\
**Post date:** [1 November 2006 02:42 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/11 "2006-11-01T02:42:45Z")

</div>

At a very deep level (last time I looked) thread animation and IO met  
and became the same thing in a giant select lurking in the core of the  
ruby Interpreter.

ie. Use select / spin another thread amounted cosmically to the same  
thing. Choose whatever is easiest for you.

John Carter Phone : (64)(3) 358 6639  
Tait Electronics Fax : (64)(3) 359 4632  
PO Box 1645 Christchurch Email : john.carter@tait.co.nz  
New Zealand

"We have more to fear from  
&nbsp;&nbsp;The Bungling of the Incompetent  
&nbsp;&nbsp;Than from the Machinations of the Wicked." (source unknown)

> **···**
>
> On Wed, 1 Nov 2006, Tom Pollard wrote:
> 
> > The select() function (Kernel#select, in Ruby) is the standard way to do IO multiplexing under Unix.

---

<div class="post-metadata">

**Author:** ![Bjorn\_Borud](https://avatars.discourse-cdn.com/v4/letter/b/e68b1a/32.png) [@Bjorn\_Borud](https://rubytalk.org/u/Bjorn_Borud)\
**Post date:** [1 November 2006 02:45 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/12 "2006-11-01T02:45:05Z")

</div>

[Tom Pollard \<tomp@earthlink.net\>]

> The select() function (Kernel#select, in Ruby) is the standard way to  
> do IO multiplexing under Unix. You give it a list of file handles on  
> which you're waiting to be able to read or write, and it returns when  
> one or more of them is ready. Windows provides a select() function,  
> too, but it only supports sockets (not pipes or ordinary file IO.)

if memory serves, a select(2) on a file descriptors representing a  
(disk) file only had defined semantics when used to wait on mandatory  
locking and is not used for ensuring that a read(2) or write(2) will  
not block. (I might be in error since it has been a while since I  
looked into it).

-Bjørn

---

<div class="post-metadata">

**Author:** ![Ara.T.Howard6](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@Ara.T.Howard6](https://rubytalk.org/u/Ara.T.Howard6)\
**Post date:** [1 November 2006 03:28 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/13 "2006-11-01T03:28:03Z")

</div>

> [ara.t.howard@noaa.gov]  
> \>  
> \> it can't be done. search the archives, this sort of thing almost  
> \> always indicates a design flaw. for instance - what will your  
> \> program do if there is no input?
> 
> nonblocking IO is not a design-flaw; it is something that is in common  
> use and is supported by most languages with a decent socket API. on  
> some OS'es this is even available for file IO (although widespread  
> language support seems to lag behind).
> 
> Java for example got proper nonblocking socket IO in 1.4 (see  
> JSR-000051 "New I/O APIs for the JavaTM Platform").

but it's archaic compared to event driven frameworks like

&nbsp;&nbsp;&nbsp;[libevent](http://www.monkey.org/~provos/libevent/)  
&nbsp;&nbsp;&nbsp;[http://liboop.ofb.net/](http://liboop.ofb.net/)

in addition it's a hack in the c lib of many oses.

> for examples of use you might want to check out the Reactor pattern  
> and other patterns for concurrent programming.

afaik the reactor pattern is a synchronous pattern

&nbsp;&nbsp;&nbsp;[artima - Comparing Two High-Performance I/O Design Patterns](http://www.artima.com/articles/io_design_patterns2.html)

not unlike the model of libevent and liboop - which are both synchronous...  
am i missing something?

-a

> **···**
>
> On Wed, 1 Nov 2006, Bjorn Borud wrote:  
> --  
> my religion is very simple. my religion is kindness. -- the dalai lama

---

<div class="post-metadata">

**Author:** ![Robert\_James](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@Robert\_James](https://rubytalk.org/u/Robert_James)\
**Post date:** [1 November 2006 01:30 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/14 "2006-11-01T01:30:09Z")

</div>

Not very attractive...

Back to the original question: there is no simple way to read  
nonblocking, or see if any output is available ?!?!

> **···**
>
> ara.t.howard@noaa.gov wrote:
> 
> > On Wed, 1 Nov 2006 srobertjames@gmail.com wrote:
> > 
> > \> I'm running an external command (using popen4 - yes, it is available on  
> > \> Windows also). I capture stdout and stderr. "what will your program do if  
> > \> there is no input?" - I assume you mean output - Well, I hope there is no  
> > \> stderr output. If there is, I'd like to capture it and feed it to Logger.  
> > \>  
> > \> Likewise, if there is a problem, and there's no stdout, then I don't want to  
> > \> wait for ever - I want to throw an exception.
> > 
> > so, basically you want to timeout on a read right? in unix you can just use
> > 
> > &nbsp;&nbsp;&nbsp;require 'timeout'
> > 
> > &nbsp;&nbsp;&nbsp;Timeout::timeout(n) do  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Open4.popen4 ...  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...  
> > &nbsp;&nbsp;&nbsp;end
> > 
> > but, this is implimented via a thread so i think the read is going to block  
> > your entire process in windows... you might try it.
> > 
> > -a  
> > --  
> > my religion is very simple. my religion is kindness. -- the dalai lama

---

<div class="post-metadata">

**Author:** ![Tom\_Pollard](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@Tom\_Pollard](https://rubytalk.org/u/Tom_Pollard)\
**Post date:** [1 November 2006 02:37 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/15 "2006-11-01T02:37:29Z")

</div>

Have you tried simply reading from them? When the command terminates, that should close those pipes on the command's end. At that point, a simple read on the stdout handle, for instance, will return whatever output there was and then eof. If there's no output, the read should return immediately. Did you try that yet? Did you try the (Windows-oriented) example from the 'ri Open4' page? If you're working in Unix you could try

&nbsp;&nbsp;&nbsp;status = POpen4::poen4('cat -') do |stdout, stderr, stdin|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;stdin.puts 'hello world'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;stdin.close  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;puts "stdout: #{stdout.read.strip}"  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;puts "stderr: #{stderr.read.strip}"  
&nbsp;&nbsp;&nbsp;end

Anyway, you would only need nonblocking IO if you wanted to read bits of the stderr stream before the command exited, but that doesn't sound like what you're want.

Tom

> **···**
>
> On Oct 31, 2006, at 7:20 PM, srobertjames@gmail.com wrote:
> 
> > In the interests of promoting creativity, I'll ask the question without  
> > any prior assumptions:
> > 
> > I'm using popen4 (great gem, btw) to run an external command and  
> > capture stdout and stderr. If one (or both) of those is empty, I don't  
> > want to hang. Just keep on moving. In fact, stderr will normally be  
> > empty.
> > 
> > How could I do this?

---

<div class="post-metadata">

**Author:** ![Francis\_Cianfrocca](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Francis\_Cianfrocca](https://rubytalk.org/u/Francis_Cianfrocca)\
**Post date:** [1 November 2006 05:42 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/16 "2006-11-01T05:42:04Z")

</div>

Ara, I'm curious to know why you think the usage of nbio is archaic or  
indicative of a design flaw. I wrote the EventMachine library in order to  
make certain kinds of Ruby programs easier to write, and it uses nbio  
pervasively. I'm fairly sure libevent does, also. EM works with almost any  
kind of descriptor, except for certain things on Windows (like \_pipe()  
descriptors, which are nonselectable, and some of the native Windows objects  
that don't support a socket-like API).

Up until recently (late May 2006), Ruby didn't have complete support for  
nbio on all descriptor types, which is one of the reasons that EventMachine  
includes a compiled C++ extension. EM interoperates perfectly well with Ruby  
threads,\* which is another important reason to use nbio.

> **···**
>
> On 10/31/06, ara.t.howard@noaa.gov \<ara.t.howard@noaa.gov\> wrote:
> 
> > On Wed, 1 Nov 2006, Bjorn Borud wrote:
> > 
> > \> [ara.t.howard@noaa.gov]  
> > \> \>  
> > \> \> it can't be done. search the archives, this sort of thing almost  
> > \> \> always indicates a design flaw. for instance - what will your  
> > \> \> program do if there is no input?  
> > \>  
> > \> nonblocking IO is not a design-flaw; it is something that is in common  
> > \> use and is supported by most languages with a decent socket API. on  
> > \> some OS'es this is even available for file IO (although widespread  
> > \> language support seems to lag behind).  
> > \>  
> > \> Java for example got proper nonblocking socket IO in 1.4 (see  
> > \> JSR-000051 "New I/O APIs for the JavaTM Platform").
> > 
> > but it's archaic compared to event driven frameworks like
> 
> ---  
> \*Except for certain cases on Windows where Ruby code that accesses standard  
> descriptors blocks all other Ruby threads. But this is not an EM issue,

---

<div class="post-metadata">

**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\
**Post date:** [1 November 2006 16:56 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/17 "2006-11-01T16:56:42Z")

</div>

> In the interests of promoting creativity, I'll ask the question without  
> any prior assumptions:
> 
> I'm using popen4 (great gem, btw) to run an external command and  
> capture stdout and stderr. If one (or both) of those is empty, I don't  
> want to hang. Just keep on moving. In fact, stderr will normally be  
> empty.
> 
> How could I do this?
> 
> (I'm supporting 'nix and Windows)

Windows will be a problem. Admittedly, I haven't tried  
ruby 1.8.5 yet, which has new nonblock\_\* methods. However,  
my expectation is that you'll only get nonblocking behavior  
on windows from sockets, not from pipes.

On Windows, calling select() on a pipe, always returns immediately with "data ready to read", regardless if there's  
any data there or not.

This has been the bane of my existence on Windows ruby for  
5 or 6 years. I do IPC on Windows ruby using TCP over  
loopback, instead of pipes, in order to get nonblocking  
semantics. (That still doesn't help for reading from the  
console, though... (search the archives for 'kbhit' for a  
partial solution there...))

One of these years, I'd like to chat with a Windows guru  
and ask how he/she would recommend making a select() that  
works on both sockets and pipes on Windows. Ruby could  
\*really\* use one.

Regards,

Bill

> **···**
>
> From: \<srobertjames@gmail.com\>

---

<div class="post-metadata">

**Author:** ![Gabriele\_Marrone](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@Gabriele\_Marrone](https://rubytalk.org/u/Gabriele_Marrone)\
**Post date:** [1 November 2006 17:27 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/18 "2006-11-01T17:27:02Z")

</div>

I thought Ruby internally uses non-blocking I/O in order to avoid that a green thread reading something blocks every other thread: am I wrong?  
Or is this true just under unix?

> **···**
>
> Il giorno 01/nov/06, alle ore 01:33, ara.t.howard@noaa.gov ha scritto:
> 
> > > The select() function (Kernel#select, in Ruby) is the standard way to do IO multiplexing under Unix. You give it a list of file handles on which you're waiting to be able to read or write, and it returns when one or more of them is ready. Windows provides a select() function, too, but it only supports sockets (not pipes or ordinary file IO.) The Win32 API generally doesn't provide non-blocking IO methods, because it's assumed you'll use threads if you want to do something else while waiting for IO.
> > 
> > right. but ruby's thread are green and, on windows, block the entire process  
> > when waiting on IO so not an option here.

---

<div class="post-metadata">

**Author:** ![Bjorn\_Borud](https://avatars.discourse-cdn.com/v4/letter/b/e68b1a/32.png) [@Bjorn\_Borud](https://rubytalk.org/u/Bjorn_Borud)\
**Post date:** [2 November 2006 09:00 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/19 "2006-11-02T09:00:10Z")

</div>

[ara.t.howard@noaa.gov]

> \>  
> \> Java for example got proper nonblocking socket IO in 1.4 (see  
> \> JSR-000051 "New I/O APIs for the JavaTM Platform").
> 
> but it's archaic compared to event driven frameworks like
> 
> &nbsp;&nbsp;&nbsp;[libevent](http://www.monkey.org/~provos/libevent/)  
> &nbsp;&nbsp;&nbsp;[http://liboop.ofb.net/](http://liboop.ofb.net/)

you are comparing apples and oranges here. you have to look at where  
these things fit into the big picture. the above mentioned libraries  
provide abstractions that are used to provide a more convenient  
programming model, and at least in the case of libevent, provide an  
abstraction that insulates code from having to deal with supporting  
the various readiness selection APIs that exist. they do not replace  
the underlying OS interfaces upon which they are built.

JSR-000051 does more or less the same thing, but with a slightly  
different goal in mind. the primary goal is to provide the programmer  
with access to asynchronous network IO in a platform-agnostic  
manner. the emphasis is on providing a sensible abstraction that works  
well with the various operating system APIs that exist -- exposing a  
reasonable common subset of functionality that one can expect to be  
able to support on a variety of platforms. JSR-000051 provides the  
primitives on top of which you would implement IO frameworks that  
provide conventient programming models.

for a language, providing sensible APIs for primitives is more  
important than imposing a particular programming model. given a set  
of sensible primitives that can be widely supported, whatever higher  
level frameworks one wishes to create can be built atop that.

> in addition it's a hack in the c lib of many oses.

I am not sure I understand exactly what you mean. the way I see it  
there are three distinct levels to networking APIs:

&nbsp;&nbsp;1. OS API, that is, the system calls

&nbsp;&nbsp;2. standardized system libraries, like libc, java.io, java.util.nio  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;etc, which provide access to IO primitives and possibly abstract  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;away the underlying OS APIs

&nbsp;&nbsp;3. higher level IO frameworks that provide more convenient programming  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;models

libevent would overlap with both 2 and 3 in this case since its  
mission is \*both\* to abstract away the underlying OS interfaces \*and\*  
provide a convenient programming model. Java's NIO is what you'd find  
in 2 and a typical Reactor pattern implementation would be in 3.

(if you use low-level socket APIs (ie the types of functions  
documented in section 2 of the UNIX manual pages) on UNIX you will  
find yourself using a mix of 1 and 2 if you use C/C++ since you use  
wrappers in libc to perform system calls, but this is just a very,  
very thin convenience layer on top of the system calls. if this is  
confusing then just forget I mentioned it :-).

> \> for examples of use you might want to check out the Reactor pattern  
> \> and other patterns for concurrent programming.
> 
> afaik the reactor pattern is a synchronous pattern

no, the Reactor pattern is mainly used to implement asynchronous IO  
and is not really anything new. I've both written and seen variations  
of this pattern in a multitude of languages since I started writing  
networking code in the early 90s and the only thing that has really  
changed is that we've gotten better at classifying these types of  
patterns, give them better definitions, and give them names. (when I  
started writing networking software in the early 90s, "patterns"  
wasn't commonly part of the programmers vocabulary).

> &nbsp;&nbsp;&nbsp;[artima - Comparing Two High-Performance I/O Design Patterns](http://www.artima.com/articles/io_design_patterns2.html)
> 
> not unlike the model of libevent and liboop - which are both synchronous...  
> am i missing something?

you are missing an "a" in front of "synchronous" 🙂

-Bjørn

---

<div class="post-metadata">

**Author:** ![Ryan\_Davis1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/ryan_davis1/32/1848_2.png) [@Ryan\_Davis1](https://rubytalk.org/u/Ryan_Davis1)\
**Post date:** [1 November 2006 02:20 UTC](https://rubytalk.org/t/nonblocking-io-read/32545/20 "2006-11-01T02:20:39Z")

</div>

in io/wait:

--- IO#ready?  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;returns non-nil if input available without blocking, or nil.

also: ri IO.read\_nonblock

> **···**
>
> On Oct 31, 2006, at 5:30 PM, S. Robert James wrote:
> 
> > Not very attractive...
> > 
> > Back to the original question: there is no simple way to read  
> > nonblocking, or see if any output is available ?!?!

[Next page](https://rubytalk.org/t/nonblocking-io-read/32545.md?page=2)
