[ANN] Eventmachine 0.8.0 released with epoll, breaks Ruby file-descriptor limit

The Eventmachine team is pleased to announce the release of version 0.8.0,
available now on RubyForge as a tarball and as a gem.

Eventmachine is a framework for writing fast, scalable network-aware
programs (clients, servers, or both) in Ruby or C/C++, without requiring the
use of threads. EM is used today in a variety of production systems
including Web, mail, and LDAP/RADIUS servers, SNMP agents, and custom
protocol handlers. It's also used in Kirk Haines's Swiftiply, a fast
clustering proxy for Rails and other web frameworks.

The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels.
This enables Eventmachine programs to break through Ruby's limit of 1024
file and socket descriptors per process. In performance tests designed to
simulate busy web servers, Eventmachine with epoll exhibits the behavior you
would expect: with very large numbers (>20,000) of quiet connections and a
few active ones, there is no noticeable degradation of responsiveness or
total throughput, compared to cases with only a few connections.

We expect the next major EM release to be called 0.9.0. It will add
Erlang-like features intended to make it simple for EM-based programs to
send messages to each other, or to other processes. A typical use case for
this capability is a "service" web site that must make calls to other web
sites, web services, or other network services in order to fulfill requests
from its own clients.

Thanks to all for your support of the EM project, and especially to Tony
Arcieri, who tirelessly bugged me for weeks to add the epoll support, and to
Kirk for helping to shake it out.

Any online documentation available? It sounds like an interesting
project but I can't seem to find any documentation covering how it can
be used.

Sincerely,
Anthony Eden

···

On 7/11/07, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

The Eventmachine team is pleased to announce the release of version 0.8.0,
available now on RubyForge as a tarball and as a gem.

Eventmachine is a framework for writing fast, scalable network-aware
programs (clients, servers, or both) in Ruby or C/C++, without requiring the
use of threads. EM is used today in a variety of production systems
including Web, mail, and LDAP/RADIUS servers, SNMP agents, and custom
protocol handlers. It's also used in Kirk Haines's Swiftiply, a fast
clustering proxy for Rails and other web frameworks.

The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels.
This enables Eventmachine programs to break through Ruby's limit of 1024
file and socket descriptors per process. In performance tests designed to
simulate busy web servers, Eventmachine with epoll exhibits the behavior you
would expect: with very large numbers (>20,000) of quiet connections and a
few active ones, there is no noticeable degradation of responsiveness or
total throughput, compared to cases with only a few connections.

We expect the next major EM release to be called 0.9.0. It will add
Erlang-like features intended to make it simple for EM-based programs to
send messages to each other, or to other processes. A typical use case for
this capability is a "service" web site that must make calls to other web
sites, web services, or other network services in order to fulfill requests
from its own clients.

Thanks to all for your support of the EM project, and especially to Tony
Arcieri, who tirelessly bugged me for weeks to add the epoll support, and to
Kirk for helping to shake it out.

--
Cell: 808 782-5046
Current Location: Melbourne, FL

Just tried to compile EventMachine on a SPARC Solaris box here at
work. g++ does not like the compiler flags that are meant for the
SUNWspro compilers :confused:

We are using the Blastwave packages for our Ruby install. That package
was compiled using the SUNWspro compilers, and so extconf.rb is
grabbing the CFLAGS from there. -KPIC makes g++ very unhappy.

I wish I was writing to say I have a patch for you, but that is not the case.

Any pointers from rubyland on how to get the correct CFLAGS into extconf.rb?

CONFIG["CFLAGS"] = " -O2 -fPIC" # <-- doesn't seem to do the trick

I always end up with the CFLAGS meant for the SUNWspro compiler.

Blessings,
TwP

···

On 7/11/07, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

The Eventmachine team is pleased to announce the release of version 0.8.0,
available now on RubyForge as a tarball and as a gem.

Eventmachine is a framework for writing fast, scalable network-aware
programs (clients, servers, or both) in Ruby or C/C++, without requiring the
use of threads. EM is used today in a variety of production systems
including Web, mail, and LDAP/RADIUS servers, SNMP agents, and custom
protocol handlers. It's also used in Kirk Haines's Swiftiply, a fast
clustering proxy for Rails and other web frameworks.

The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels.
This enables Eventmachine programs to break through Ruby's limit of 1024
file and socket descriptors per process. In performance tests designed to
simulate busy web servers, Eventmachine with epoll exhibits the behavior you
would expect: with very large numbers (>20,000) of quiet connections and a
few active ones, there is no noticeable degradation of responsiveness or
total throughput, compared to cases with only a few connections.

We expect the next major EM release to be called 0.9.0. It will add
Erlang-like features intended to make it simple for EM-based programs to
send messages to each other, or to other processes. A typical use case for
this capability is a "service" web site that must make calls to other web
sites, web services, or other network services in order to fulfill requests
from its own clients.

Thanks to all for your support of the EM project, and especially to Tony
Arcieri, who tirelessly bugged me for weeks to add the epoll support, and to
Kirk for helping to shake it out.

Francis Cianfrocca wrote:

The 0.8.0 release adds automatic support for epoll, on Linux 2.6 kernels.
This enables Eventmachine programs to break through Ruby's limit of 1024
file and socket descriptors per process. In performance tests designed to
simulate busy web servers, Eventmachine with epoll exhibits the behavior you
would expect: with very large numbers (>20,000) of quiet connections and a
few active ones, there is no noticeable degradation of responsiveness or
total throughput, compared to cases with only a few connections.

This is great news! I just gave the epoll feature a try and it seemed to work without problems. I remember that several people tried to use epoll for their extensions before, but all of them gave eventually up, because using anything else than select clashed with Ruby internals' use of select for almost anything. What did you do differently? Didn't you have any problems?

···

--
Florian Frank

There's an extensive Rdoc in the package, with sample code. There's also a
Wiki on Rubyforge that has material that is dated but still correct and
useful. There will shortly be a dedicated EventMachine web site as well.

···

On 7/11/07, Anthony Eden <anthonyeden@gmail.com> wrote:

Any online documentation available? It sounds like an interesting
project but I can't seem to find any documentation covering how it can
be used.

Sincerely,
Anthony Eden

$CFLAGS = " -O2 -fPIC"

···

On 12 juil. 07, at 05:06, Tim Pease wrote:

Any pointers from rubyland on how to get the correct CFLAGS into extconf.rb?

--
Luc Heinrich

Well, it was a very extensive piece of code to write, requiring changes in
almost every of part of EM's reactor core. Getting the interoperability with
Ruby threads was challenging but we got it to work. I haven't heard of any
serious problems with this epoll implementation. It's been in the EM source
tree for nearly a month now, and we deliberately waited that length of time
before putting it into a formal release in case problems turned up.

···

On 7/12/07, Florian Frank <flori@nixe.ping.de> wrote:

This is great news! I just gave the epoll feature a try and it seemed to
work
without problems. I remember that several people tried to use epoll for
their
extensions before, but all of them gave eventually up, because using
anything
else than select clashed with Ruby internals' use of select for almost
anything.
What did you do differently? Didn't you have any problems?

Thanks. That did the trick. Now on to the other compilation problems.

TwP

···

On 7/12/07, Luc Heinrich <luc@honk-honk.com> wrote:

On 12 juil. 07, at 05:06, Tim Pease wrote:

> Any pointers from rubyland on how to get the correct CFLAGS into
> extconf.rb?

$CFLAGS = " -O2 -fPIC"

Fixed the compilation errors on Solaris. I filed a bug with project's
bug tracker, and attached a patch file.

http://rubyforge.org/tracker/index.php?func=detail&aid=12228&group_id=1555&atid=6058

Blessings,
TwP

···

On 7/12/07, Tim Pease <tim.pease@gmail.com> wrote:

On 7/12/07, Luc Heinrich <luc@honk-honk.com> wrote:
> On 12 juil. 07, at 05:06, Tim Pease wrote:
>
> > Any pointers from rubyland on how to get the correct CFLAGS into
> > extconf.rb?
>
> $CFLAGS = " -O2 -fPIC"
>

Thanks. That did the trick. Now on to the other compilation problems.

Patch accepted and applied. Thanks, Tim.
-francis

···

On 7/12/07, Tim Pease <tim.pease@gmail.com> wrote:

Fixed the compilation errors on Solaris. I filed a bug with project's
bug tracker, and attached a patch file.

http://rubyforge.org/tracker/index.php?func=detail&aid=12228&group_id=1555&atid=6058

Blessings,
TwP