Select on solaris

Jeff_Putsch · 21 August 2002 18:53

Howdy,

I’m trying to use select() with Open3.popen3 on Solaris 8 with ruby 1.6.7.
If it matters ruby was built on Solaris 2.6.

It seems that select() is claiming file descriptors are ready whether or
not they are.

The simple program I’m trying as a test is:

#!/usr/sepp/bin/ruby

require ‘open3’

def run_command
[stdin, out, err] = Open3.popen3(“cat /etc/hosts”)
while not (r=select([out,err])).nil?
yield r.inspect
end
end

run_command { |l|
p l
}

When run, the output is:

$ ruby /tmp/select_test.rb
[#IO:0x496a8, #IO:0x495d0]
“out(65280): #\n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): # Internet host table\n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): #\n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): 127.0.0.1\tlocalhost \n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): 172.17.100.124\tblue.mxim.com blue\n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): \tparis.mxim.com loghost\n”
[#IO:0x496a8, #IO:0x495d0]
“out(65280): 172.17.100.5\tparis.mxim.com loghost\n”
[#IO:0x496a8, #IO:0x495d0]
""
[#IO:0x496a8, #IO:0x495d0]
""
[#IO:0x496a8, #IO:0x495d0]
""
[#IO:0x496a8, #IO:0x495d0]

… repeat forever …

This does not seem right. First, there is never any data on the error
object, so why is select saying there is? Second once the end of the
file is reached, why is select continuing to claim both descriptors
have data? I’m sure I’m doing something wrong…

Eventually I’d like the “run_command” method to detect that
the external program has exited and check the return code (via $?),
raising an exception if neccessary. That way I can process output one
line at a time, but still check return codes from the external program.

Any insights, suggestions, or help will be greatly appreciated.

Thanks,

Jeff.

···

–
Jeff Putsch Email: putsch@mxim.com
Maxim Integrated Products Office: (503)547-2037
High Frequency CAD Engineering

Jeff_Putsch · 21 August 2002 19:18

Sorry, I botched the simple program. Here are two simple examples
illustrating my confusion:

#!/usr/sepp/bin/ruby

def run_command
f = IO.popen(“cat /etc/hosts”)

while not (r=select([f])).nil?
  p r
  p r[0][0].gets.to_s
end

end

run_command

and

#!/usr/sepp/bin/ruby

require ‘open3’

def run_command
(stdin, out, err) = Open3.popen3(“cat /etc/hosts”)

while not (r=select([out,err])).nil?
  p r
  p "out: "+r[0][0].gets.to_s
  p "err: "+r[0][1].gets.to_s
end

end

run_command

In both cases select returns that data is ALWAYS ready. The first example
gives this output:

[[#IO:0x4a848], , ]
“#\n”
[[#IO:0x4a848], , ]
“# Internet host table\n”
[[#IO:0x4a848], , ]
“#\n”
[[#IO:0x4a848], , ]
“127.0.0.1\tlocalhost \n”
[[#IO:0x4a848], , ]
“172.17.100.124\tblue.mxim.com blue\n”
[[#IO:0x4a848], , ]
“\tparis.mxim.com loghost\n”
[[#IO:0x4a848], , ]
“172.17.100.5\tparis.mxim.com loghost\n”
[[#IO:0x4a848], , ]
“”
[[#IO:0x4a848], , ]
“”
[[#IO:0x4a848], , ]
“”

The second is similar, but shows the “err” never has any data.

So why is select doing this? What am I doing wrong?

Once I get select working, I’d like to be using the Open3.popen3() version
and detecting when the input data is done (a time out may not work), then
check the return code of the program called by Open3.popen3().

Any advice and/or pointers will be greatly appreciated.

Thanks,

Jeff.

···

On Thu, Aug 22, 2002 at 03:53:51AM +0900, Jeff Putsch wrote:

The simple program I’m trying as a test is:

#!/usr/sepp/bin/ruby

require ‘open3’

def run_command
[stdin, out, err] = Open3.popen3(“cat /etc/hosts”)
while not (r=select([out,err])).nil?
yield r.inspect
end
end

run_command { |l|
p l
}

–
Jeff Putsch Email: putsch@mxim.com
Maxim Integrated Products Office: (503)547-2037
High Frequency CAD Engineering

Reimer_Behrends · 21 August 2002 23:13

What you describe is actually normal behavior for select().

If select() returns a ready file descriptor and read() returns zero
bytes, this indicates an EOF condition (think of EOF as a zero-length
block of data that can be read).

In Ruby, testing for io.eof? should also work to distinguish between
real data and an EOF.

[…]

		Reimer Behrends

···

Jeff Putsch (putsch@mxim.com) wrote:

Howdy,

I’m trying to use select() with Open3.popen3 on Solaris 8 with ruby 1.6.7.
If it matters ruby was built on Solaris 2.6.

It seems that select() is claiming file descriptors are ready whether or
not they are.

James_F_Hranicky · 21 August 2002 23:54

I ran into something like the programming in C: it seemed that select
simply didn’t work on pipes.

I was doing this:

writer -> proc1 -> proc2 -> ... -> procN -> reader

Basically, forking an arbitrary # of procs in a pipeline, and getting
the writer on one end and the reader at the other. I found what I
had to do was make the writer and reader non blocking, use read()
and write() and catch EAGAIN when the operations failed. It looked
like this:

if (fcntl(reader, F_SETFL, O_NONBLOCK) < 0) 
    exit(fprintf(stderr, "fcntl(reader, F_SETFL, O_NONBLOCK): %s\n", strerror(errno)));

if (fcntl(writer, F_SETFL, O_NONBLOCK) < 0) 
    exit(fprintf(stderr, "fcntl(reader, F_SETFL, O_NONBLOCK): %s\n", strerror(errno)));

while (1) {               
if ((from_stdin_bytes = read(0, to_pipe_buf, BUFSIZE)) < 0)
        exit(fprintf(stderr, "read from stdin %d: (%d) %s\n", 0, errno, strerror(errno)));
if (to_pipe_bytes == 0)
        break;
     while ((to_pipe_bytes = write(writer, to_pipe_buf, from_stdin_bytes)) < 0) {
        if (errno == EAGAIN) {
            if ((from_pipe_bytes = read(reader, from_pipe_buf, BUFSIZE)) < 0) { 
                if (errno == EAGAIN) { 
                    continue;
                } else { 
                    exit(fprintf(stderr, "read from pipe %d: (%d) %s\n", 
		     reader, errno, strerror(errno)));
                }
            } else { 
                if (write(1, from_pipe_buf, from_pipe_bytes) < 0) { 
                    exit(fprintf(stderr, "write to stdout: %s\n", strerror(errno)));
                }
                continue;
            }
        } else { 
            exit(fprintf(stderr, "write to pipe: %s\n", strerror(errno)));
        }
    }
}
close(writer);
tv.tv_sec = 0;
tv.tv_usec = 10;
numreads = 0;
while (1) {
    if (numreads == MAXREADS) { 
        char *msg = "Error: reached max # reads\n";
        write(2, msg, strlen(msg));
        break;
    }
    numreads++; 
    if ((from_pipe_bytes = read(reader, from_pipe_buf, BUFSIZE)) < 0) { 
        if (errno == EAGAIN) { 
            select(32, NULL, NULL, NULL, &tv); /* pause for 1/100th of a 
					      sec to allow other procs to finish */
            continue;
        } else { 
            exit(fprintf(stderr, "read from pipe %d: (%d) %s\n", reader, errno,
		     strerror(errno)));
        }
    } else if (from_pipe_bytes == 0) { 
        break;
    } else { 
        numreads = 0;
        if (write(1, from_pipe_buf, from_pipe_bytes) < 0) { 
            exit(fprintf(stderr, "write to stdout: %s\n", strerror(errno)));
        }
        continue;
    }
}

The kicker was the select() in the read loop after the writer had been closed.
I had to add a tiny bit of delay to ensure all the procs in the pipeline
finished.

Anyway, this worked for me – I’m sure other Unix folks on the list can add
more insight.

Jim

···

On Thu, 22 Aug 2002 03:53:51 +0900 “Jeff Putsch” putsch@mxim.com wrote:

It seems that select() is claiming file descriptors are ready whether or
not they are.

The simple program I’m trying as a test is:

This does not seem right. First, there is never any data on the error
object, so why is select saying there is? Second once the end of the
file is reached, why is select continuing to claim both descriptors
have data? I’m sure I’m doing something wrong…

Jeff_Putsch · 21 August 2002 21:17

djberge@qwest.com writes:

From: jeff Putsch [maito:putsch@mxim.com]
This does not seem right. First, there is never any data on the error
object, so why is select saying there is? Second once the end of the
file is reached, why is select continuing to claim both descriptors
have data? I’m sure I’m doing something wrong…

err is in IO object, it’s not the data contained within the object.

I know err is an IO object. My choice of words was cleary poor. The
issue I’m having is #select takes an array of IO objects and should
wait for data to become available on them. It then (should) returns an
array of objects that are ready.

The behavior I’m observing is that it ALWAYS returns, claiming all my
IO objects have data ready on them (even after an EOF on the IO
object). The reality is not all the objects do. In the example(s) I posted
the “err” object should not have data at all, so it should not be in the
array of ready object, yet it is.

That, hopefully, better explains what I am seeing and why I need advice
on solving the problem.

Thanks,

Jeff.

···

-----Original Message-----
–
Jeff Putsch Email: putsch@mxim.com
Maxim Integrated Products Office: (503)547-2037
High Frequency CAD Engineering

Jeff_Putsch · 21 August 2002 23:20

Thanks,

I had just figured this out when I got your email. The root of my problems
is I was selecting on file descriptors that had reached eof.

Making sure I only selected on file descriptors that had not reached eof
solved my problem.

Jeff.

···

On Thu, Aug 22, 2002 at 08:13:50AM +0900, Reimer Behrends wrote:

What you describe is actually normal behavior for select().

If select() returns a ready file descriptor and read() returns zero
bytes, this indicates an EOF condition (think of EOF as a zero-length
block of data that can be read).

–
Jeff Putsch Email: putsch@mxim.com
Maxim Integrated Products Office: (503)547-2037
High Frequency CAD Engineering

Topic		Replies	Views
Select on solaris ruby-talk	0	104	21 August 2002
Select on solaris ruby-talk	6	127	2 September 2002
Select() ruby-talk	5	89	23 April 2003
Select? ruby-talk	3	117	22 August 2002
Select strange behavier ruby-talk	21	131	20 May 2003

Select on solaris

Related topics