Considering Ruby For a Networking Application

ruby-talk
1

I'm going to be working on a fairly basic networking application shortly. This is low level socket work. (I cannot use a nicety like DRB sadly.)

I'm very familiar with this kind of stuff in Perl, but have done very little socket work in Ruby. I need something that can reliably work with around 50 users sending light activity. There is the possibility of a large write to them here and there and that cannot block the process.

I know a little from past questions here:

http://rubyurl.com/miq

However, I think I remember someone saying Ruby 1.8.4 is better with nonblocking sockets. So, can some networking guru take a stab at these questions:

1. I need to use threads *and* nonblocking sockets to be safe, right?
2. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need to do low level reads and writes?
3. Do I need to rescue Errno::EWOULDBLOCK?
4. This isn't going to work on Windows, right? (That's okay in this case. Just making sure.)

Thanks for the help.

James Edward Gray II

2

James, you might want to look for "EventMachine".

It seems interesting.

-austin

···

On 5/5/06, James Edward Gray II <james@grayproductions.net> wrote:

I'm going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

3

James Edward Gray II wrote:

However, I think I remember someone saying Ruby 1.8.4 is better with
nonblocking sockets. So, can some networking guru take a stab at
these questions:

I haven't noticed any difference between 1.8.3 and 1.8.4.

1. I need to use threads *and* nonblocking sockets to be safe, right?

No. The whole point of select() and non-blocking sockets is to de-multiplex I/O.

2. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need
to do low level reads and writes?

I would guess these would be "unsafe" as would any network I/O function that depends on specific separators. Would gets() effectively block on a non-blocking socket if the EOL separator never arrives?
*shrug* I don't know.

3. Do I need to rescue Errno::EWOULDBLOCK?

Yes.

4. This isn't going to work on Windows, right? (That's okay in this
case. Just making sure.)

Sure it work on Windows.

Here's a pretty good example. No threads. Single select client/server code that uses non-blocking sockets.

http://sourcery.dyndns.org/svn/teensymud/release/tmud-2.9.0/lib/network/

···

--
J. Lambert

4

I'm going to be working on a fairly basic networking application shortly. This is low level socket work. (I cannot use a nicety like DRB sadly.)

I'm very familiar with this kind of stuff in Perl, but have done very little socket work in Ruby. I need something that can reliably work with around 50 users sending light activity. There is the possibility of a large write to them here and there and that cannot block the process.

I know a little from past questions here:

http://rubyurl.com/miq

However, I think I remember someone saying Ruby 1.8.4 is better with nonblocking sockets. So, can some networking guru take a stab at these questions:

1. I need to use threads *and* nonblocking sockets to be safe, right?
2. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need to do low level reads and writes?
3. Do I need to rescue Errno::EWOULDBLOCK?
4. This isn't going to work on Windows, right? (That's okay in this case. Just making sure.)

I'm sorry I don't have definitive answers. I'm going to
need to find out for sure pretty soon, myself. But . . . .

From looking at the changes in the ruby sources between 1.8.2
and 1.8.4, it _looks_ as though nonblocking sockets are pretty
well supported now. (Even on Windows!)

It would appear nonblocking is handled properly internally
whether you use ruby threads or not. (I.e. from the perspective
of the underlying mechanisms, the main ruby thread is just as
much a thread as any other ruby thread.)

In 1.8.4, you can:

  require 'io/nonblock'

This adds methods to the IO class, allowing:

  some_socket.nonblock = true

In the past, I've done all my ruby socket programming using the lower level send() and recv() methods. However, again from
reading the source (io.c and ext/openssl/lib/openssl/buffering.rb)
it looks like IO#readpartial or IO#sysread (but *not* IO#read)
may be the new favored approach to dealing with nonblocking
descriptors in general.

(In Windows, I'm pretty sure only sockets have any chance of
working with nonblocking semantics. Whereas in Unix, I'd presume support for any file descriptor. So using IO#sysread
and IO#syswrite, now that these methods apparently are nonblocking-savvy, should be more general than using send() and recv().)

If you look at ext/openssl/lib/openssl/buffering.rb, you'll
see what looks like a fairly generic module for building a buffered read and readpartial, as well as write, on top of
sysread and syswrite. I note that the lowest level read and
write methods here both rescue Errno::EAGAIN, and retry. I'm
a little surprised by that - because it appears that EAGAIN
is already being handled along with EWOULDBLOCK in io.c
anyway. But I haven't really studied the code in io.c, just
perused it.

Hmm..........

My sense of it--at least, this is what I plan to try in my
first test--is:

1. require 'io/nonblock' and set 'nonblock = true' on all
   my sockets.

2. use a separate ruby thread to handle I/O for each socket.

3. use IO#readpartial to read... (note that the comments in
   io.c say that IO#readpartial can block regardless of the nonblocking flag... but I'm pretty sure this means
   block the ruby thread, NOT the native thread...!)

4. use IO#syswrite to write... (I think, with 1.8.4 and a
   nonblocking socket, ruby _appears_ to be handling
   EWOULDBLOCK internally, so that we can do a large write
   but not block the whole process.)
   - But maybe we need to rescue Errno::EAGAIN for some
     reason, as seen in openssl/buffering.rb

With luck, this will work on Unix and Windows (sockets only
on windows, any file descriptor on unix), and never block
the whole native thread ruby runs in.

If so, we've got a reasonable solution for TCP. UDP had its
own unique blocking issues (related to linux kernels breaking
POSIX semantics for select/recv for packets with bad UDP
checksums.... or something like that.... I dunno if this has
changed since 1.8.2 or not...)

Or, yeah, maybe that EventMachine library... <grin>

Regards,

Bill

···

From: "James Edward Gray II" <james@grayproductions.net>

5

James, your application needs to support about 50 users with the occasional
large write. Is the application a client app or a server app. It's funny to
re-read your message, because I just started designing a
presence-dissemination system, which will have very similar requirements to
yours! (The network protocol will work a lot like BGP4, except that the
"routing tables" are actually the IM-presence status of users.) And this
will be both a client and a server.

···

On 5/5/06, James Edward Gray II <james@grayproductions.net> wrote:

I'm going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

I'm very familiar with this kind of stuff in Perl, but have done very
little socket work in Ruby. I need something that can reliably work
with around 50 users sending light activity. There is the
possibility of a large write to them here and there and that cannot
block the process.

I know a little from past questions here:

http://rubyurl.com/miq

However, I think I remember someone saying Ruby 1.8.4 is better with
nonblocking sockets. So, can some networking guru take a stab at
these questions:

1. I need to use threads *and* nonblocking sockets to be safe, right?
2. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need
to do low level reads and writes?
3. Do I need to rescue Errno::EWOULDBLOCK?
4. This isn't going to work on Windows, right? (That's okay in this
case. Just making sure.)

Thanks for the help.

James Edward Gray II

6

James Edward Gray II wrote:

4. This isn't going to work on Windows, right? (That's okay in this
case. Just making sure.)

Sure it work on Windows.

If nonblocking works on Windows, it's a post-1.8.2 development...
(But I'm stoked if it does, now.)

Regards,

Bill

···

From: "Jon A. Lambert" <jlsysinc@alltel.net>

7

This is a terrific find. Thank you so much!

James Edward Gray II

···

On May 5, 2006, at 10:28 PM, Austin Ziegler wrote:

On 5/5/06, James Edward Gray II <james@grayproductions.net> wrote:

I'm going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

James, you might want to look for "EventMachine".

8

Thank you for the very detailed breakdown. It was quite helpful.

James Edward Gray II

···

On May 5, 2006, at 11:42 PM, Bill Kelly wrote:

I'm sorry I don't have definitive answers. I'm going to
need to find out for sure pretty soon, myself. But . . . .

9

Yeah, this is pretty much how I did it in Perl. Thank you from the example though.

James Edward Gray II

P.S. I need to play with your MUD sometime. Looks like a blast!

···

On May 5, 2006, at 11:13 PM, Jon A. Lambert wrote:

Here's a pretty good example. No threads. Single select client/server code that uses non-blocking sockets.

http://sourcery.dyndns.org/svn/teensymud/release/tmud-2.9.0/lib/network/

10

A great deal depends on what you really need to accomplish. Are you doing a
plain-vanilla protocol handler for some standard server protocol? If your
performance and scalability requirements are low, then the easiest thing is
probably to use blocking i/o with a thread per socket. If you use blocking
i/o, you're better off multiplexing the i/o with a select loop. (My
preference in these cases is ALMOST ALWAYS to avoid a thread pool for a
range of reasons, but most people disagree with me ;-).) If you use
nonblocking i/o, then you really want to avoid multithreading because you'll
need to poll in each one. Ruby can handle it because its implementation of
select is integrated with its internal thread scheduler. (If you try to run
select on a native thread, you'll have major problems with Ruby threads.)

If your process has to do work on other threads, meaning this isn't a pure
server application, then
the calculations change. UDP is really quite a lot easier than TCP, but it
always seems to take some getting used to. So the major questions are: tell
us more about the nature of the application, and tell us what kind of
performance and scalability you need. And additionally, do you need
encryption?

If you decide to look at EventMachine, I can support you if you like. Just
email me or ask questions here. It runs on Windows, works correctly with
Ruby threads, and supports encryption.

11

I'm writing a server.

James Edward Gray II

···

On May 8, 2006, at 2:21 PM, Francis Cianfrocca wrote:

James, your application needs to support about 50 users with the occasional
large write. Is the application a client app or a server app.

12

If your performance and scalability requirements are low, then the easiest thing is
probably to use blocking i/o with a thread per socket. If you use blocking
i/o, you're better off multiplexing the i/o with a select loop.

But then a large write could still block the entire process, since Ruby's threads aren't native. Right?

If you decide to look at EventMachine, I can support you if you like.

I believe this is going to be my first choice. I haven't had time to play with it yet, but it looks very nice.

Are there API docs available anywhere for EventMachine?

James Edward Gray II

···

On May 8, 2006, at 12:12 PM, Francis Cianfrocca wrote:

13

Writes can block your process whether they are large or small- it all
depends on what is happening in your kernel at that particular point in
time. However, if a socket has just selected writable, then it generally
will be able to take a write that is the size of an ethernet packet (1400
bytes or so). (Of course the sharper readers will point out that there are
also good reasons to avoid such small writes, especially if you don't turn
off the Nagle algorithm.) But you're asking a good question. This is one of
the reasons why blocking i/o on threads can make a lot of sense if your app
has relatively light requirements. You may find this odd but I worry more
about accepts blocking than writes. An accepting socket can select readable
(meaning a new connection is available), but be empty by the time you get
around to reading it (like if there was a network error that caused a
pending connection to reset). This can be a real nasty.

If you download the eventmachine gem and install the rdocs, the API docs
will all be there, and there is sample code too.

···

On 5/8/06, James Edward Gray II <james@grayproductions.net> wrote:

On May 8, 2006, at 12:12 PM, Francis Cianfrocca wrote:

> If your performance and scalability requirements are low, then the
> easiest thing is
> probably to use blocking i/o with a thread per socket. If you use
> blocking
> i/o, you're better off multiplexing the i/o with a select loop.

But then a large write could still block the entire process, since
Ruby's threads aren't native. Right?

> If you decide to look at EventMachine, I can support you if you like.

I believe this is going to be my first choice. I haven't had time to
play with it yet, but it looks very nice.

Are there API docs available anywhere for EventMachine?

James Edward Gray II

14

If your performance and scalability requirements are low, then the easiest thing is
probably to use blocking i/o with a thread per socket. If you use blocking
i/o, you're better off multiplexing the i/o with a select loop.

But then a large write could still block the entire process, since Ruby's threads aren't native. Right?

Judging from the results of my test program on Linux and
Windows, it appears nonblocking writes are supported in
1.8.4.

ruby 1.8.4 (2005-12-24) [i686-linux]
ruby 1.8.4 (2005-12-24) [i386-mswin32]

Interestingly, while the Linux version behaved as expected,
displaying blocking characteristics until I explicitly set
nonblock=true, the windows version appears to sport some
sort of always-on nonblocking buffered implementation behind
the scenes now.

On Windows, even if I never set nonblock=true, I could syswrite
40 megs to the socket and #syswrite would return immediately!
(Subsequent syswrites would then appear to block until the
data from the previous huge syswrite was consumed; but just the
ruby thread performing the syswrite was blocked, never the whole
process. So anyway there's some kind of dynamic buffering going
on behind the scenes in the windows implementation--but in any
case, it never blocked the whole process. Excellent.)

In my test program (attached) I'm catching just about every
exception under the sun, including EAGAIN, which I saw being
done in lib/openssl/buffering.rb. However, the only exception I actually saw raised was EOFError. (I never tried
to force an EPIPE condition on the write.)

Anyway, for what it's worth, it appears ruby 1.8.4 now
definitely supports nonblocking socket I/O on both Linux
and Windows!! Yay!

I'll still be looking at EventMachine though. :slight_smile:

Regards,

Bill

nonblock_test.rb (1.71 KB)

···

From: "James Edward Gray II" <james@grayproductions.net>

On May 8, 2006, at 12:12 PM, Francis Cianfrocca wrote:

15

Forgive me if this is answered in the documentation, I have had time to look: Is EventMachine multi-threaded? I'm mainly wondering if I need to synchronize the methods that socket data arriving.

James Edward Gray II

···

On May 8, 2006, at 2:09 PM, Francis Cianfrocca wrote:

If you download the eventmachine gem and install the rdocs, the API docs
will all be there, and there is sample code too.

16

This is some awesome information. Thanks Bill!

I believe I am going to try EventMachine first though as it looks quite nice to work with.

James Edward Gray II

···

On May 8, 2006, at 4:24 PM, Bill Kelly wrote:

Judging from the results of my test program on Linux and
Windows, it appears nonblocking writes are supported in
1.8.4.

17

From what you said, this code doesn't do what you think it should do:

# hmm, io/nonblock refuses to work on windows because
# of missing GETFL :frowning: Just fake it...
if RUBY_PLATFORM =~ /mswin32/
  class IO
    def nonblock=(nb)
      fcntl(Fcntl::F_SETFL, nb ? Fcntl::O_NONBLOCK : 0)
    end
  end
end

That's because on Windows, this isn't the way to set sockets nonblocking.
Microsoft reinvented a lot of wheels in Windows, and this is one of 'em. Try
this C code instead (I'll leave it to you to turn it into Ruby):

unsigned long one = 1;
ioctlsocket (the_socket_descriptor, FIONBIO, &one);

This will return non-zero in case of error (I've never seen it return an
error).
Best
-francis

···

On 5/8/06, Bill Kelly <billk@cts.com> wrote:

From: "James Edward Gray II" <james@grayproductions.net>
>
> On May 8, 2006, at 12:12 PM, Francis Cianfrocca wrote:
>
>> If your performance and scalability requirements are low, then the
>> easiest thing is
>> probably to use blocking i/o with a thread per socket. If you use
>> blocking
>> i/o, you're better off multiplexing the i/o with a select loop.
>
> But then a large write could still block the entire process, since
> Ruby's threads aren't native. Right?

Judging from the results of my test program on Linux and
Windows, it appears nonblocking writes are supported in
1.8.4.

ruby 1.8.4 (2005-12-24) [i686-linux]
ruby 1.8.4 (2005-12-24) [i386-mswin32]

Interestingly, while the Linux version behaved as expected,
displaying blocking characteristics until I explicitly set
nonblock=true, the windows version appears to sport some
sort of always-on nonblocking buffered implementation behind
the scenes now.

On Windows, even if I never set nonblock=true, I could syswrite
40 megs to the socket and #syswrite would return immediately!
(Subsequent syswrites would then appear to block until the
data from the previous huge syswrite was consumed; but just the
ruby thread performing the syswrite was blocked, never the whole
process. So anyway there's some kind of dynamic buffering going
on behind the scenes in the windows implementation--but in any
case, it never blocked the whole process. Excellent.)

In my test program (attached) I'm catching just about every
exception under the sun, including EAGAIN, which I saw being
done in lib/openssl/buffering.rb. However, the only
exception I actually saw raised was EOFError. (I never tried
to force an EPIPE condition on the write.)

Anyway, for what it's worth, it appears ruby 1.8.4 now
definitely supports nonblocking socket I/O on both Linux
and Windows!! Yay!

I'll still be looking at EventMachine though. :slight_smile:

Regards,

Bill

18

Quoting garbagecat10@gmail.com, on Tue, May 09, 2006 at 04:09:20AM +0900:

has relatively light requirements. You may find this odd but I worry more
about accepts blocking than writes. An accepting socket can select readable
(meaning a new connection is available), but be empty by the time you get
around to reading it (like if there was a network error that caused a
pending connection to reset). This can be a real nasty.

Stevens describes this problem, but I thought he also described the fix,
set the socket to non-blocking so the accept() won't block.

doesn't this work? is this a ruby-specific problem?

Sam

19

I'm glad you asked. Eventmachine is single-threaded. That's an essential
part of its design. (More for performance and scalability than to remove any
need for synchronization, which is very difficult between native threads and
Ruby threads. (Well, it's actually impossible. Eventmachine does it through
a mechanism that is unrelated to threads.) There's an EventMachine wiki that
has a page on the "Reactor model" which explains this in more detail.
http://eventmachine.rubyforge.org/wiki/wiki.pl

However, if you want to implement a thread-pool in Ruby threads to handle
your workload, that's perfectly compatible with Eventmachine, and your
threads will run concurrently with Eventmachine. This might make sense if
the processing you have to do on behalf of your clients involves something
like writing to a database, where there is a system latency that you can't
hand off to Eventmachine (because DBMS client libraries are usually not
asynchronous).

···

On 5/8/06, James Edward Gray II <james@grayproductions.net> wrote:

On May 8, 2006, at 2:09 PM, Francis Cianfrocca wrote:

> If you download the eventmachine gem and install the rdocs, the API
> docs
> will all be there, and there is sample code too.

Forgive me if this is answered in the documentation, I have had time
to look: Is EventMachine multi-threaded? I'm mainly wondering if I
need to synchronize the methods that socket data arriving.

James Edward Gray II

20

From what you said, this code doesn't do what you think it should do:

# hmm, io/nonblock refuses to work on windows because
# of missing GETFL :frowning: Just fake it...
if RUBY_PLATFORM =~ /mswin32/
  class IO
    def nonblock=(nb)
      fcntl(Fcntl::F_SETFL, nb ? Fcntl::O_NONBLOCK : 0)
    end
  end
end

That's because on Windows, this isn't the way to set sockets nonblocking.
Microsoft reinvented a lot of wheels in Windows, and this is one of 'em. Try
this C code instead (I'll leave it to you to turn it into Ruby):

unsigned long one = 1;
ioctlsocket (the_socket_descriptor, FIONBIO, &one);

Right, except that in win32/win32.c, ruby corrects microsoft's
misfeatures for us, by making fcntl() on win32-ruby shunt to
ioctlsocket behind the scenes:

int
fcntl(int fd, int cmd, ...)
{
//[...]

    if (arg & O_NONBLOCK) {
        ioctlArg = 1;
    }
    else {
        ioctlArg = 0;
    }
    RUBY_CRITICAL({
        ret = ioctlsocket(sock, FIONBIO, &ioctlArg);
        if (ret == -1) {
            errno = map_errno(WSAGetLastError());
        }
    });

Regards,

Bill

···

From: "Francis Cianfrocca" <garbagecat10@gmail.com>