Oddities with select

I'm trying to use Ruby to talk to an network application, and noticed something odd. Using ethereal, I can see that the server is returning a newline to the client, however my Ruby code, calling select([socket], [], [], 0.1) repeatedly for 5 seconds never seems to know that there's data to be read.

I've tried to make sure the socket is non-blocking with:

@sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

But that seems to have no effect.

Is there something I'm missing?

I haven't tried the same code in another language to see if the problem may be there. At the moment though, this seems broken.

Any ideas?

Ben

I'm trying to use Ruby to talk to an network application, and noticed
something odd. Using ethereal, I can see that the server is returning a
newline to the client, however my Ruby code, calling select([socket],
, , 0.1) repeatedly for 5 seconds never seems to know that there's
data to be read.

I've tried to make sure the socket is non-blocking with:

@sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

But that seems to have no effect.

Is there something I'm missing?

don't you want to block? i mean, isn't that why you are using select? to do
nothing until data arrives? if it IS blocking your code should sleep untill
data becomes available.

also, i'm not sure what the buffering semantics of select are - but i'm sure
there are some - might want to check that out too...

I haven't tried the same code in another language to see if the problem
may be there. At the moment though, this seems broken.

Any ideas?

Ben

-a

···

On Wed, 9 Jun 2004, Ben Giddings wrote:
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen

===============================================================================

Ben Giddings <bg-rubytalk@infofiend.com> writes:

I'm trying to use Ruby to talk to an network application, and noticed
something odd. Using ethereal, I can see that the server is returning a
newline to the client, however my Ruby code, calling select([socket],
, , 0.1) repeatedly for 5 seconds never seems to know that there's
data to be read.

This works for me, unless I am misunderstanding what you are trying
to do:

---- test_server.rb ------
#!/usr/bin/ruby -w

require 'socket'
a = TCPServer.new("localhost", 9999).accept
loop do
  a.puts
  sleep 3
end

···

--------------------------

and then

---- test_client.rb ------
#!/usr/bin/ruby -w

require 'socket'

a = TCPSocket.new("localhost", 9999)
loop do
  if select([a], , , 0.1)
      p a.read(1)
  end
end
--------------------------

Having already started the server, the client prints out a "\n" as
soon as it is started, and then every 3 seconds thereafter. Is it
possible that your code is actually getting held up somewhere else,
so that it's not even hitting the select()? For example, if I were to
change the "a.read(1)" in the client there to just "a.read", then it
would get stuck there until the socket was closed.

Ara.T.Howard wrote:

don't you want to block? i mean, isn't that why you are using select? to do
nothing until data arrives? if it IS blocking your code should sleep untill
data becomes available.

Actually, no. What I want to do is know that a command was accepted (indicated by a blank line), unfortunately the code that I'm talking to doesn't always do the right thing, so I want to bail out eventually if it isn't going to happen.

What I want to do is: "Try to read until you get a blank line, for a maximum of N seconds"

I know that some commands will go quickly, giving me a blank line in under 200ms, and others will take a few seconds to complete, but if they're taking 500ms and say 4.5s respectively, I want to stop waiting.

also, i'm not sure what the buffering semantics of select are - but i'm sure
there are some - might want to check that out too...

Hmm, ok... I thought select just told you when there was data available, and either blocked or didn't, based on the timeout parameter you give it. That's why I thought that I could essentially poll it with a low timeout value, and when data became availble, I could use 'recv' to read it.

Ben

why not something along the lines of this

   buf = ''
   t = Thread.new{ loop{ buf << io.getc } }
   sleep n_seconds
   t.kill

   case buf
     when %r/^$/o
       raise "command not accepted"
     when %r/^\s+$/o

···

On Wed, 9 Jun 2004, Ben Giddings wrote:

Ara.T.Howard wrote:

don't you want to block? i mean, isn't that why you are using select?
to do
nothing until data arrives? if it IS blocking your code should sleep
untill
data becomes available.

Actually, no. What I want to do is know that a command was accepted
(indicated by a blank line), unfortunately the code that I'm talking to
doesn't always do the right thing, so I want to bail out eventually if
it isn't going to happen.

What I want to do is: "Try to read until you get a blank line, for a
maximum of N seconds"

       #
       # handle command
       #
     else
       raise "bad response <#{ buf.inspect }>"
   end

??

I know that some commands will go quickly, giving me a blank line in
under 200ms, and others will take a few seconds to complete, but if
they're taking 500ms and say 4.5s respectively, I want to stop waiting.

also, i'm not sure what the buffering semantics of select are - but i'm
sure
there are some - might want to check that out too...

Hmm, ok... I thought select just told you when there was data available, and
either blocked or didn't, based on the timeout parameter you give it.
That's why I thought that I could essentially poll it with a low timeout
value, and when data became availble, I could use 'recv' to read it.

Ben

no - you are correct. ignore me! it's just that you don't know how MUCH data
is available - only that it is, or that it's eof.

-a
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen

===============================================================================

Ara.T.Howard wrote:

why not something along the lines of this

  buf = ''
  t = Thread.new{ loop{ buf << io.getc } }
  sleep n_seconds
  t.kill

  case buf
    when %r/^$/o
      raise "command not accepted"
    when %r/^\s+$/o
      #
      # handle command
      #
    else
      raise "bad response <#{ buf.inspect }>"
  end

What I'm actually using is pretty close to that:

    def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
     retval = ""

     timed_out = false

     begin
       timeout(timeout) do
         while true
           if select([@sock], , , 0.1)
             retval += @sock.recv(256)
             if stop_pattern
               if retval =~ stop_pattern
                 break
               end
             end
           end
         end
       end
     rescue Timeout::Error
       timed_out = true
     end

# if timed_out
# puts "Timed out"
# raise
# end

     retval
   end

The problem is that although I've seen a newline go over the network to my PC, it is never spotted by the Ruby app, which sits there doing the 'select' forever.

Your code looks slightly different, but I think it would do the exact same thing.

For some reason, the newline goes over the network, but doesn't make it far enough (in 5s) that the 'select()' call sees it.

Ben

What I'm actually using is pretty close to that:

   def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
    retval = ""

    timed_out = false

    begin
      timeout(timeout) do
        while true
          if select([@sock], , , 0.1)
            retval += @sock.recv(256)
            if stop_pattern
              if retval =~ stop_pattern
                break
              end
            end
          end
        end
      end
    rescue Timeout::Error
      timed_out = true
    end

# if timed_out
# puts "Timed out"
# raise
# end

    retval
  end

The problem is that although I've seen a newline go over the network to
my PC, it is never spotted by the Ruby app, which sits there doing the
'select' forever.

Your code looks slightly different, but I think it would do the exact
same thing.

For some reason, the newline goes over the network, but doesn't make it
far enough (in 5s) that the 'select()' call sees it.

Ben

didn't you say something about your socket being in non-blocking mode? if so,
isn't this just a busy loop since select will keep returning 'true - a read
will not block':

   man 2 select

     ...
     three independent sets of descriptors are watched. those listed in
            readfds will be watched to see if characters become available for read-
            ing (more precisely, to see if a read will not block - in particular, a
            file descriptor is also ready on end-of-file)
     ...

if not in non-blocking mode, perhaps your pattern is bad? what happens if you
do this:

<snip>

        while true
          if select([@sock], , , 0.1)
            retval += @sock.recv(256)

               p retval
</snip>

??

do you see retval growing?

-a

···

On Wed, 9 Jun 2004, Ben Giddings wrote:
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen

===============================================================================

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<40C720A2.4070403@infofiend.com>...
<snip>

    def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
     retval = ""

     timed_out = false

     begin
       timeout(timeout) do
         while true
           if select([@sock], , , 0.1)
             retval += @sock.recv(256)
             if stop_pattern
               if retval =~ stop_pattern
                 break
               end
             end
           end
         end
       end
     rescue Timeout::Error
       timed_out = true
     end

<snip>

A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], , ] == true returns false (I
suppose).

Hth

Chris

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<40C720A2.4070403@infofiend.com>...
<snip>

    def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
     retval = ""

     timed_out = false

     begin
       timeout(timeout) do
         while true
           if select([@sock], , , 0.1)
             retval += @sock.recv(256)
             if stop_pattern
               if retval =~ stop_pattern
                 break
               end
             end
           end
         end
       end
     rescue Timeout::Error
       timed_out = true
     end

<snip>

A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], , ] == true returns false (I
suppose).

Hth

Chris

didn't you say something about your socket being in non-blocking mode? if so,
isn't this just a busy loop since select will keep returning 'true - a read
will not block':

I'm pretty sure that the timeout parameter is what tells select whether or not to block. I think the nonblocking flag only affects reading type operations, like 'recv' or 'read'. If you're using blocking sockets, they wait until there's input, if you're using nonblocking IO, they return immediately, either with data or without. Select is used (I think generally with blocking IO) to test to see if there's any point in attempting a read, to avoid blocking when no data will be read.

If there's no serial traffic, it should return my socket if there's input to be read, otherwise it should return nil.

if not in non-blocking mode, perhaps your pattern is bad? what happens if you
do this:

<snip>

        while true
          if select([@sock], , , 0.1)
            retval += @sock.recv(256)

              p retval
</snip>

??

do you see retval growing?

No, in those cases when there's some data there but 'select' doesn't see it, I don't see anything being appended to 'retval' and I get an empty string back.

Thanks for the ideas though,

Ben

···

On Jun 9, 2004, at 11:43, Ara.T.Howard wrote:

On Wed, 9 Jun 2004, Ben Giddings wrote:

Well since select() returns anArray or nil, if it returns 'nil', nil evaluates to false in conditionals, but anything else, other than 'false', evaluates to true. So if it returns an array of arrays, or an array, or even an empty array, it will evaluate to true:

irb(main):001:0> var = nil
=> nil
irb(main):002:0> if var then puts "#{var.inspect} is true(ish)" end
=> nil
irb(main):003:0> var = false
=> false
irb(main):004:0> if var then puts "#{var.inspect} is true(ish)" end
=> nil
irb(main):005:0> var = true
=> true
irb(main):006:0> if var then puts "#{var.inspect} is true(ish)" end
true is true(ish)
=> nil
irb(main):007:0> var =
=>
irb(main):008:0> if var then puts "#{var.inspect} is true(ish)" end
is true(ish)
=> nil
irb(main):009:0> require 'socket'
=> true
irb(main):010:0> sock = TCPSocket.new('localhost', 80)
=> #<TCPSocket:0x23d4dc>
irb(main):011:0> var = sock
=> #<TCPSocket:0x23d4dc>
irb(main):012:0> if var then puts "#{var.inspect} is true(ish)" end
#<TCPSocket:0x23d4dc> is true(ish)
=> nil
irb(main):013:0> var = [[sock], , ]
=> [[#<TCPSocket:0x23d4dc>], , ]
irb(main):014:0> if var then puts "#{var.inspect} is true(ish)" end
[[#<TCPSocket:0x23d4dc>], , ] is true(ish)
=> nil
irb(main):015:0> [[sock], , ] == true
=> false

Ben

···

On Jun 9, 2004, at 18:13, Chris Reay wrote:

     begin
       timeout(timeout) do
         while true
           if select([@sock], , , 0.1)
             retval += @sock.recv(256)
             if stop_pattern
               if retval =~ stop_pattern
                 break
               end
             end
           end
         end
       end
     rescue Timeout::Error
       timed_out = true
     end

A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], , ] == true returns false (I
suppose).

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<F5EE0C89-BA7F-11D8-8F24-00039381462A@infofiend.com>...
<snip>

irb(main):015:0> [[sock], , ] == true
=> false

Ben

There you are. Suck it and see.

Hth

Chris

···

On Jun 9, 2004, at 18:13, Chris Reay wrote:

Chris Reay wrote:

irb(main):015:0> [[sock], , ] == true
=> false

There you are. Suck it and see.

Right... but how does that change anything?

Ben

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<40C87DCC.5050309@infofiend.com>...

Chris Reay wrote:
>>irb(main):015:0> [[sock], , ] == true
>>=> false
>>

Wow. This *is* hard yakka.

if select([@sock], , , 0.1)
  @sock.recv()
  etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

We have established the following:

(a) select() returns nil or anArray (from a RTFM).
(b) nil != true (axiom).
(c) anArray != true. (See your irb snippet above).

Thus you are requesting the block never to be called. Your program is
working perfectly, and the socket is brimming with bytes :slight_smile:

Try this (and pardon my ruby, I've forgotten most of the syntax):

while true
  sockArr = select([@sock], , , 0.1)
  if !sockArr.nil? and sockArr.at(0).include?(@sock)
    @sock.recv(256)
    etc, etc.

Pardon my didactic tone too. I hope this helps.

Yours

Chris

$ ruby -e "p [[nil], , ] == true"
false

$ ruby -e "p 'true' if [[nil], , ]"
"true"

martin

···

Chris Reay <mrchameleon@hotmail.com> wrote:

(c) anArray != true. (See your irb snippet above).

I'm going to assume you really misunderstand and aren't trolling, because even if you are trolling, maybe this explanation will come in useful for someone else.

Chris Reay wrote:

if select([@sock], , , 0.1)
  @sock.recv()
  etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

Actually no. In Ruby, as I showed in my previous message, "if _____" doesn't test to see if "_____" is true or false, it check to see if it is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is nil, then the conditional is executed. Otherwise, whether _____ is the boolean 'true' or whether it is any other non-nil, non-false object, the conditional is executed.

Conditional execution truth table:

parameter | conditional executed

···

------------------------------------
false | no
true | yes
nil | no
anything else | yes

And note, that 'anything else' includes some things that may surprise people used to programming in C, including the number 0, an empty string, an empty array, etc.

What I'm asking Ruby to do, is to execute that bit of code if the result of calling select() is not nil.

We have established the following:

(a) select() returns nil or anArray (from a RTFM).
(b) nil != true (axiom).

Right

(c) anArray != true. (See your irb snippet above).

Right, however

anArray =

if anArray
   puts "This is printed because anArray is non-nil and not false"
else
   puts "This is not printed."
end

Thus you are requesting the block never to be called. Your program is
working perfectly, and the socket is brimming with bytes :slight_smile:

I'm confused by your talking about the block. There is a block in that bit of code, but it isn't inside the conditional.

In this section of code:

            if select([@sock], , , 0.1)
              retval += @sock.recv(256)
              if stop_pattern
                if retval =~ stop_pattern
                  break
                end
              end
            end

This bit:

              retval += @sock.recv(256)
              if stop_pattern
                if retval =~ stop_pattern
                  break
                end
              end

Will get executed if "select([@sock], , , 0.1)" returns something which isn't nil, and isn't the boolean 'false'. The documentation of 'select' says that it returns nil, or an array. If it returns an array, that

Try this (and pardon my ruby, I've forgotten most of the syntax):

while true
  sockArr = select([@sock], , , 0.1)
  if !sockArr.nil? and sockArr.at(0).include?(@sock)
    @sock.recv(256)
    etc, etc.

That code will work, but, as I explained above, it isn't really necessary. Since the only possible non-nil return value of "select([@sock], , , 0.1)" is a set of arrays including "@sock". I don't really need to look at the return value. I know that if select() returns non-nil, my socket will not block if I try to read it.

I hope that convinces you. If not, try playing around with if, and select() and see if you can figure out what I'm talking about.

I think this does point out a need in Ruby for a 'to_bool' operator. Obviously, this kind of test already happens behind the scenes when it comes to conditionals.

All you'd need is the equivalent of:

class Object
   def to_bool
     true
   end
end

class NilClass
   def to_bool
     false
   end
end

class TrueClass
   def to_bool
     true
   end
end

class FalseClass
   def to_bool
     false
   end
end

If nothing else, this would help explain to people why "if 0" takes the 'true' branch. Just point out that 0.to_bool is true

Any opinions? Should I make an RCR?

Ben

Ben Giddings wrote:

I'm going to assume you really misunderstand and aren't trolling, because even if you are trolling, maybe this explanation will come in useful for someone else.

Chris Reay wrote:

if select([@sock], , , 0.1)
  @sock.recv()
  etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

Actually no. In Ruby, as I showed in my previous message, "if _____" doesn't test to see if "_____" is true or false, it check to see if it is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is nil, then the conditional is executed. Otherwise, whether _____ is the

You mistyped here --------> is not executed. ...

···

boolean 'true' or whether it is any other non-nil, non-false object, the conditional is executed.

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<40C9E3EC.9090806@infofiend.com>...

I'm going to assume you really misunderstand and aren't trolling,
because even if you are trolling, maybe this explanation will come in
useful for someone else.

Of course I'm not trolling; don't be so touchy.

Conditional execution truth table:

parameter | conditional executed
------------------------------------
false | no
true | yes
nil | no
anything else | yes

And note, that 'anything else' includes some things that may surprise
people used to programming in C, including the number 0, an empty
string, an empty array, etc.

Well, that does surprise me! But I'm often enough wrong to be used to
it. Now I don't understand why your program doesn't work, but mine
does. I'll think about it.

Remember The PP's tip #26 :slight_smile:

Gennady wrote:

Ben Giddings wrote:

I'm going to assume you really misunderstand and aren't trolling, because even if you are trolling, maybe this explanation will come in useful for someone else.

Chris Reay wrote:

if select([@sock], , , 0.1)
  @sock.recv()
  etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

Actually no. In Ruby, as I showed in my previous message, "if _____" doesn't test to see if "_____" is true or false, it check to see if it is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is nil, then the conditional is executed. Otherwise, whether _____ is the

You mistyped here --------> is not executed. ...

Yup. :slight_smile:

Good thing I overexplained elsewhere.

Ben

Ben Giddings <bg-rubytalk@infofiend.com> wrote in message news:<40C9E7F5.20607@infofiend.com>...

Gennady wrote:
> Ben Giddings wrote:
>
>> I'm going to assume you really misunderstand and aren't trolling,
>> because even if you are trolling, maybe this explanation will come in
>> useful for someone else.
>>

I replied to this a moment ago, and then did the ad hoc experiment
that I should have done before opening my virtual mouth earlier -
sorry 'bout that.

I tested this:

require "socket"

svr = TCPServer.new("localhost", 4567)
sock = svr.accept
printf "Accepted conn\n"
while true
  if select([sock], , , 3.0)
    printf("%s\n", sock.recv(256))
  end
end

I sent some "hello world" messages to it from a TCPSocket. It works
perfectly.

Sorry to appear tardy, tedious or trollish; but what gives?