Using poll/kqueue/etc. instead of select

For a testing project, I've built a little SIP library that emulates SIP
terminals (phones, if you like). Currently, it runs as two threads, a
listener that owns all of the terminals' sockets, selects on them, does
a little bit of processing of input, and maintains the input queue, and
a "main" thread does everything else, particularly sending requests.

Unfortunately, under NetBSD, I've now run into the file descriptor limit
on select (256 file handles), and I'll likely be hitting the Linux one
(1024 file handles) sooner rather than later.

Well, 256 is not the real limit on NetBSD, and when it stopped working
(essentially, ignoring input on a handle) over 256, it looked like an
FD_SETSIZE issue, so I tried bumping up the NetBSD one as per the manual
page, by redefining FD_SETSIZE=1024 in CFLAGS, but that didn't seem to
fix the issue.

Obviously, select is not what I want to be using here anyway; I should
be using kqueue, poll, or whatever.

My question is, for those of us who need it, what are our options?
It looks to me that if I write a bit of C support code to use, say,
poll, I can no longer use Ruby threads at all. Should I just move to a
single-thread, event driven model, and use custom C code for dealing
with the I/O? Should I try to replace the Ruby interpreter's use of
select with something else? Are there any plans to change the Ruby
interpreter at some point to support options other than select?

cjs

···

--
Curt Sampson <cjs@starling-software.com> +81 90 7737 2974
Mobile sites and software consulting: http://www.starling-software.com

Look at the Ruby/EventMachine project. It supports both kqueue and epoll,
easily breaks through the 1024-descriptor limit, and naturally supports a
non-threaded programming model. If your SIP implementation is UDP-based,
you'll find it much easier than the threaded approach. If you're doing SIP
over TCP, use the included LineText2 protocol module.

At one point there was a project afoot to support a full-featured SIP stack
using EventMachine, but priorities changed as they often do.

···

On Feb 4, 2008 7:58 PM, Curt Sampson <cjs@cynic.net> wrote:

For a testing project, I've built a little SIP library that emulates SIP
terminals (phones, if you like). Currently, it runs as two threads, a
listener that owns all of the terminals' sockets, selects on them, does
a little bit of processing of input, and maintains the input queue, and
a "main" thread does everything else, particularly sending requests.

Unfortunately, under NetBSD, I've now run into the file descriptor limit
on select (256 file handles), and I'll likely be hitting the Linux one
(1024 file handles) sooner rather than later.

Well, 256 is not the real limit on NetBSD, and when it stopped working
(essentially, ignoring input on a handle) over 256, it looked like an
FD_SETSIZE issue, so I tried bumping up the NetBSD one as per the manual
page, by redefining FD_SETSIZE=1024 in CFLAGS, but that didn't seem to
fix the issue.

Obviously, select is not what I want to be using here anyway; I should
be using kqueue, poll, or whatever.

My question is, for those of us who need it, what are our options?
It looks to me that if I write a bit of C support code to use, say,
poll, I can no longer use Ruby threads at all. Should I just move to a
single-thread, event driven model, and use custom C code for dealing
with the I/O? Should I try to replace the Ruby interpreter's use of
select with something else? Are there any plans to change the Ruby
interpreter at some point to support options other than select?