Problem with sockets

Hi
I've just started using ruby and thought I'd do some experimenting
with TCPSocket and TCPServer

I've got a small problem, in that the program I've written runs fine
on Linux, but runs with a lag on windows. By this I mean that when you
type into the client on std input in Linux, what you type is
automatically displayed.
However in windows nothing happens until you type and send something
else. Like it's caught in a buffer and needs to be flushed. I've tried
flushing the output stream as you would in Java, but nothing seems to
work
If anyone could give me some advice I would be eternally grateful.
I've searched around the net, but to no avail.

Client

require 'socket'
require 'thread'

class Client

  def initialize

    host = 'localhost'
    port = 40000

    s = TCPSocket.new(host, 40000)

    t = Thread.new do

    while(true)

    line = s.gets

  if(line == nil)
  puts("Program is exiting")
  break
  end #of if

$stdout.flush

     puts "Received: #{line}"

$stdout.flush

    end #of while

    s.close

    end #of thread

    while(true)

    puts("About to run stdin")

    line = $stdin.gets

    s.puts(line)

  end #of while
  puts 'QUIT'

  t.join

  s.close

  Process.exit

      end #of con

end #of class

st = Client.new

Server

require 'socket'
require 'thread'

class ServerThread

  def initialize

    t = TCPServer.new('localhost', 40000)

    while(true)

    go = t.accept

    s = Thread.new(go) do |v|

  while(true)

# data = "none"

    data = v.gets

    v.write(data)

  if(data == nil)
    puts("From inside if")
      self.closeSocket(v)

    end #of if

    end # while

      end #of thread

end #end of while

    s.join

      end #of constructor

  def closeSocket(sock)

    puts("This is closeSocket")

  sock.close

  end #of method

            end #of class

s = ServerThread.new

Thanks Ian

STDIN.gets blocks hard on windows - that is, it blocks at the "c level" which blocks all running green threads.

The result is this 'lag' behavior - in fact what is happening normally is that you're not getting any running threads or socket reads except in the time between gets returning a value, and being called again.

If you avoid $stdin, you will find that the flow is smooth.

Oh, and $stdout blocks too...

There's a video here that sort of demonstrates this in a visual way:

http://blog.ra66i.org/wp-content/network_rotator_fun.swf

One side is an echo server, which reverses the string it echos, and the other is a generator, which generates strings with a moving # in them. "I" and "O" have meaning local to the respective application. The right hand side is the generator, the left hand side is the reversing echo server.

Those were actually built on top of EM, but it's the same issue you see.

···

On 4 Jan 2008, at 06:00, ian@frogcottage.demon.co.uk wrote:

Hi
I've just started using ruby and thought I'd do some experimenting
with TCPSocket and TCPServer

I've got a small problem, in that the program I've written runs fine
on Linux, but runs with a lag on windows. By this I mean that when you
type into the client on std input in Linux, what you type is
automatically displayed.
However in windows nothing happens until you type and send something
else. Like it's caught in a buffer and needs to be flushed. I've tried
flushing the output stream as you would in Java, but nothing seems to
work
If anyone could give me some advice I would be eternally grateful.
I've searched around the net, but to no avail.

Client

require 'socket'
require 'thread'

class Client

def initialize

   host = 'localhost'
   port = 40000

   s = TCPSocket.new(host, 40000)

   t = Thread.new do

   while(true)

   line = s.gets

  if(line == nil)
  puts("Program is exiting")
  break
  end #of if

$stdout.flush

    puts "Received: #{line}"

$stdout.flush

   end #of while

   s.close

   end #of thread

   while(true)

   puts("About to run stdin")

   line = $stdin.gets

   s.puts(line)

end #of while
puts 'QUIT'

t.join

  s.close

  Process.exit

     end #of con

end #of class

st = Client.new

Server

require 'socket'
require 'thread'

class ServerThread

def initialize

   t = TCPServer.new('localhost', 40000)

   while(true)

   go = t.accept

   s = Thread.new(go) do |v|

while(true)

# data = "none"

   data = v.gets

   v.write(data)

if(data == nil)
   puts("From inside if")
     self.closeSocket(v)

   end #of if

   end # while

     end #of thread

The generic answer that EM would provide is: "don't use threads at all."
Unfortunately, at the current time, there is a problem in EM's console
handler that makes event-driven keyboard input unreliable on Windows. We
ought to solve this problem asap.

···

On Jan 4, 2008 10:01 AM, James Tucker <jftucker@gmail.com> wrote:

STDIN.gets blocks hard on windows - that is, it blocks at the "c
level" which blocks all running green threads.

The result is this 'lag' behavior - in fact what is happening normally
is that you're not getting any running threads or socket reads except
in the time between gets returning a value, and being called again.

If you avoid $stdin, you will find that the flow is smooth.

Oh, and $stdout blocks too...

There's a video here that sort of demonstrates this in a visual way:

http://blog.ra66i.org/wp-content/network_rotator_fun.swf

One side is an echo server, which reverses the string it echos, and
the other is a generator, which generates strings with a moving # in
them. "I" and "O" have meaning local to the respective application.
The right hand side is the generator, the left hand side is the
reversing echo server.

Those were actually built on top of EM, but it's the same issue you see.

AFAIK, there was a patch for 1.8 that solved this, but couldn't find
it.

Is related to rb_w32_select (win32/win32.c).

Maybe someone can dig ruby-core mails :wink:

Regards,

···

On 4 ene, 13:14, Francis Cianfrocca <garbageca...@gmail.com> wrote:

The generic answer that EM would provide is: "don't use threads at all."
Unfortunately, at the current time, there is a problem in EM's console
handler that makes event-driven keyboard input unreliable on Windows. We
ought to solve this problem asap.

--
Luis Lavena

------=_Part_26026_29096075.1199459699490
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

STDIN.gets blocks hard on windows - that is, it blocks at the "c
level" which blocks all running green threads.

The result is this 'lag' behavior - in fact what is happening normally
is that you're not getting any running threads or socket reads except
in the time between gets returning a value, and being called again.

If you avoid $stdin, you will find that the flow is smooth.

Oh, and $stdout blocks too...

There's a video here that sort of demonstrates this in a visual way:

http://blog.ra66i.org/wp-content/network_rotator_fun.swf

One side is an echo server, which reverses the string it echos, and
the other is a generator, which generates strings with a moving # in
them. "I" and "O" have meaning local to the respective application.
The right hand side is the generator, the left hand side is the
reversing echo server.

Those were actually built on top of EM, but it's the same issue you see.

The generic answer that EM would provide is: "don't use threads at all."

The video there doesn't use threads. That's a completely "native" EM stack that uses the protocol code I showed you a few weeks ago. In fact, it's the demo application I'll be releasing with that protocol.

With or without threads, the problem the OP was describing actually comes up anyway on Windows, as the console IO code could really do with being made non-blocking, like it is on *nix. My understanding is though, that this is non-trivial.

Unfortunately, at the current time, there is a problem in EM's console
handler that makes event-driven keyboard input unre

lol, :slight_smile: - Is that me or you, or a joke? (the premature termination).

Hmm, Luis got it, looks like it was me, scary. *slaps* Mail.app.

···

On 4 Jan 2008, at 11:14, Francis Cianfrocca wrote:

On Jan 4, 2008 10:01 AM, James Tucker <jftucker@gmail.com> wrote:

Hi thanks for the answers. I removed the stdin line and replaced it
with gets, and it still has the same error.
Can I take it Ruby Sockets don't work very well with windows?
Big disadvantage if that's true, as it's so often compared to Java,
and it's claimed that it's multi platform.
I'm not sure what all the "EM" stuff means, is that some windows
thing?

Thanks again, but I'm still non the wiser
Ian

···

On 4 Jan, 15:35, James Tucker <jftuc...@gmail.com> wrote:

On 4 Jan 2008, at 11:14, Francis Cianfrocca wrote:

> ------=_Part_26026_29096075.1199459699490
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline

> On Jan 4, 2008 10:01 AM, James Tucker <jftuc...@gmail.com> wrote:

>> STDIN.gets blocks hard on windows - that is, it blocks at the "c
>> level" which blocks all running green threads.

>> The result is this 'lag' behavior - in fact what is happening
>> normally
>> is that you're not getting any running threads or socket reads except
>> in the time between gets returning a value, and being called again.

>> If you avoid $stdin, you will find that the flow is smooth.

>> Oh, and $stdout blocks too...

>> There's a video here that sort of demonstrates this in a visual way:

>>http://blog.ra66i.org/wp-content/network_rotator_fun.swf

>> One side is an echo server, which reverses the string it echos, and
>> the other is a generator, which generates strings with a moving # in
>> them. "I" and "O" have meaning local to the respective application.
>> The right hand side is the generator, the left hand side is the
>> reversing echo server.

>> Those were actually built on top of EM, but it's the same issue you
>> see.

> The generic answer that EM would provide is: "don't use threads at
> all."

The video there doesn't use threads. That's a completely "native" EM
stack that uses the protocol code I showed you a few weeks ago. In
fact, it's the demo application I'll be releasing with that protocol.

With or without threads, the problem the OP was describing actually
comes up anyway on Windows, as the console IO code could really do
with being made non-blocking, like it is on *nix. My understanding is
though, that this is non-trivial.

> Unfortunately, at the current time, there is a problem in EM's console
> handler that makes event-driven keyboard input unre

lol, :slight_smile: - Is that me or you, or a joke? (the premature termination).

Hmm, Luis got it, looks like it was me, scary. *slaps* Mail.app.

"EM" is EventMachine: http://rubyforge.org/projects/eventmachine/

···

On Jan 4, 2008 4:34 PM, <ian@frogcottage.demon.co.uk> wrote:

Hi thanks for the answers. I removed the stdin line and replaced it
with gets, and it still has the same error.
Can I take it Ruby Sockets don't work very well with windows?
Big disadvantage if that's true, as it's so often compared to Java,
and it's claimed that it's multi platform.
I'm not sure what all the "EM" stuff means, is that some windows
thing?

Thanks again, but I'm still non the wiser
Ian

> > ------=_Part_26026_29096075.1199459699490
> > Content-Type: text/plain; charset=ISO-8859-1
> > Content-Transfer-Encoding: 7bit
> > Content-Disposition: inline

> >> STDIN.gets blocks hard on windows - that is, it blocks at the "c
> >> level" which blocks all running green threads.

> >> The result is this 'lag' behavior - in fact what is happening
> >> normally
> >> is that you're not getting any running threads or socket reads except
> >> in the time between gets returning a value, and being called again.

> >> If you avoid $stdin, you will find that the flow is smooth.

> >> Oh, and $stdout blocks too...

> >> There's a video here that sort of demonstrates this in a visual way:

> >>http://blog.ra66i.org/wp-content/network_rotator_fun.swf

> >> One side is an echo server, which reverses the string it echos, and
> >> the other is a generator, which generates strings with a moving # in
> >> them. "I" and "O" have meaning local to the respective application.
> >> The right hand side is the generator, the left hand side is the
> >> reversing echo server.

> >> Those were actually built on top of EM, but it's the same issue you
> >> see.

> > The generic answer that EM would provide is: "don't use threads at
> > all."

> The video there doesn't use threads. That's a completely "native" EM
> stack that uses the protocol code I showed you a few weeks ago. In
> fact, it's the demo application I'll be releasing with that protocol.

> With or without threads, the problem the OP was describing actually
> comes up anyway on Windows, as the console IO code could really do
> with being made non-blocking, like it is on *nix. My understanding is
> though, that this is non-trivial.

> > Unfortunately, at the current time, there is a problem in EM's console
> > handler that makes event-driven keyboard input unre

> lol, :slight_smile: - Is that me or you, or a joke? (the premature termination).

> Hmm, Luis got it, looks like it was me, scary. *slaps* Mail.app.

Hi thanks for the answers. I removed the stdin line and replaced it
with gets, and it still has the same error.

I/O operations are blocking thread in 1.8 (this under Windows)

Can I take it Ruby Sockets don't work very well with windows?

I commented that was a patch for rb_w32_select on ruby-core or
rubyforge tracker but couldn't find it.

I think like most of the patches hanging there, they get ignored.

Maybe I should start learning japanesse and hang on ruby-dev :wink:

Regards,

···

On 4 ene, 19:30, i...@frogcottage.demon.co.uk wrote:

On 4 Jan, 15:35, James Tucker <jftuc...@gmail.com> wrote:
> On 4 Jan 2008, at 11:14, Francis Cianfrocca wrote:
> > On Jan 4, 2008 10:01 AM, James Tucker <jftuc...@gmail.com> wrote:

--
Luis Lavena

Ian,

Read slowly:

STDIN.gets blocks hard on windows - that is, it blocks at the "c
level" which blocks all running green threads.

The result is this 'lag' behavior - in fact what is happening
normally
is that you're not getting any running threads or socket reads except
in the time between gets returning a value, and being called again.

If you avoid $stdin, you will find that the flow is smooth.

Oh, and $stdout blocks too...

There's a video here that sort of demonstrates this in a visual way:

http://blog.ra66i.org/wp-content/network_rotator_fun.swf

With or without threads, the problem the OP was describing actually
comes up anyway on Windows, as the console IO code could really do
with being made non-blocking, like it is on *nix. My understanding is
though, that this is non-trivial.

Hi thanks for the answers. I removed the stdin line and replaced it
with gets, and it still has the same error.

gets *is* $stdin.gets.

Can I take it Ruby Sockets don't work very well with windows?

Sockets work fine.

Console I/O is where the problem is.

Big disadvantage if that's true, as it's so often compared to Java,
and it's claimed that it's multi platform.

I have production (threaded) apps that do not read from $stdin running over windows sockets under reasonable load, with no problems whatsoever. Ruby has been a great multi-platform solution for us, with the same apps running on *nix, Windows and OS X with no platform issues.

Thanks again, but I'm still non the wiser

Remove your user input altogether, and *generate* the data instead of reading it from input, and you will no longer see any issues with the 'smoothness' of your runtime.

···

On 4 Jan 2008, at 17:34, ian@frogcottage.demon.co.uk wrote:

On 4 Jan, 15:35, James Tucker <jftuc...@gmail.com> wrote:

On 4 Jan 2008, at 11:14, Francis Cianfrocca wrote:

On Jan 4, 2008 10:01 AM, James Tucker <jftuc...@gmail.com> wrote:

Ian

I know it has no bearing to your current point, but this is unaccurate.

Kernel.gets will read from a file if there's something specified on the
command line (ARGV).

I've been bitten by this recently : I had a small script asking
questions from the command line ; then, I tried to add somes switches to
it, and I wondered why I had the message "in `gets': No such file or
directory - -t (Errno::ENOENT)"...

Fred

···

Le 5 janvier 2008 à 11:58, James Tucker a écrit :

On 4 Jan 2008, at 17:34, ian@frogcottage.demon.co.uk wrote:

Hi thanks for the answers. I removed the stdin line and replaced it
with gets, and it still has the same error.

gets *is* $stdin.gets.

--
Full-size scottish highland pipes are loud enough to fall into the
category of 'weaponry', especially if they're played poorly, but it's an
analogue of heavy metal if they're played well.
                                           (Gary S. Callison in the SDM)

Hi, thanks for the reply. I read it very slowly.

The problem is the way that ruby deals with the ConsoleIO, which is in
effect a problem with ruby, from my point of view.
So, how can I "generate" data from a user, do they have to put it in a
file first?
Or do I have to call a .cmd script from inside ruby?
It seems to me, like I'm having to jump through hoops just to do a
basically simple thing, which is fair enough, but they don't tell you
that do they.
Anyway thanks for the advice, I was intrigued as to why I couldn't get
it to work.
I'll give it a go in Python, and see if that works "out of the box",
just out of curiosity.
Ian

···

On 5 Jan, 10:58, James Tucker <jftuc...@gmail.com> wrote:

Ian,

Read slowly:

On 4 Jan 2008, at 17:34, i...@frogcottage.demon.co.uk wrote:

> On 4 Jan, 15:35, James Tucker <jftuc...@gmail.com> wrote:
>> On 4 Jan 2008, at 11:14, Francis Cianfrocca wrote:
>>> On Jan 4, 2008 10:01 AM, James Tucker <jftuc...@gmail.com> wrote:

>>>> STDIN.gets blocks hard on windows - that is, it blocks at the "c
>>>> level" which blocks all running green threads.

>>>> The result is this 'lag' behavior - in fact what is happening
>>>> normally
>>>> is that you're not getting any running threads or socket reads
>>>> except
>>>> in the time between gets returning a value, and being called again.

>>>> If you avoid $stdin, you will find that the flow is smooth.

>>>> Oh, and $stdout blocks too...

>>>> There's a video here that sort of demonstrates this in a visual
>>>> way:

>>>>http://blog.ra66i.org/wp-content/network_rotator_fun.swf

>> With or without threads, the problem the OP was describing actually
>> comes up anyway on Windows, as the console IO code could really do
>> with being made non-blocking, like it is on *nix. My understanding is
>> though, that this is non-trivial.

> Hi thanks for the answers. I removed the stdin line and replaced it
> with gets, and it still has the same error.

gets *is* $stdin.gets.

> Can I take it Ruby Sockets don't work very well with windows?

Sockets work fine.

Console I/O is where the problem is.

> Big disadvantage if that's true, as it's so often compared to Java,
> and it's claimed that it's multi platform.

I have production (threaded) apps that do not read from $stdin running
over windows sockets under reasonable load, with no problems
whatsoever. Ruby has been a great multi-platform solution for us, with
the same apps running on *nix, Windows and OS X with no platform issues.

> Thanks again, but I'm still non the wiser

Remove your user input altogether, and *generate* the data instead of
reading it from input, and you will no longer see any issues with the
'smoothness' of your runtime.

> Ian

Kernel.gets will read from a file if there's something specified on the
command line (ARGV).

Right, ARGF. In fact, it's all of ARGF in order isn't it? Something like that.

I've been bitten by this recently : I had a small script asking
questions from the command line ; then, I tried to add somes switches to
it, and I wondered why I had the message "in `gets': No such file or
directory - -t (Errno::ENOENT)"...

Yup.

···

On 5 Jan 2008, at 07:55, F. Senault wrote:

Le 5 janvier 2008 à 11:58, James Tucker a écrit :

Hi, thanks for the reply. I read it very slowly.

Thank you.

The problem is the way that ruby deals with the ConsoleIO, which is in
effect a problem with ruby, from my point of view.

I don't disagree. The problem was, you were still talking about sockets. :wink:

So, how can I "generate" data from a user, do they have to put it in a
file first?

Well if you really need data from the user then you'll need to find out how to do a non-blocking read from STDIN on Windows. This may or may not work in the popular Highline lib. I haven't yet found a need to resolve this long term problem myself, and as such I don't know how to yet. It's possible that Daniel Berger might know more, or that one of his Win32 projects can help you.

Or do I have to call a .cmd script from inside ruby?
It seems to me, like I'm having to jump through hoops just to do a
basically simple thing, which is fair enough, but they don't tell you
that do they.

Again, I don't disagree.

Anyway thanks for the advice, I was intrigued as to why I couldn't get
it to work.
I'll give it a go in Python, and see if that works "out of the box",
just out of curiosity.

It may well. It's important to note that the only problem is with STDIN when it's reading from the Console, it seems to work just fine reading from file (although I haven't done parallel file access under win32 that requires significant threading), and it works just fine with sockets.

It's a shame to loose a user to this, but it doesn't surprise me.

···

On 5 Jan 2008, at 21:45, ian@frogcottage.demon.co.uk wrote: