[ANN] EventMachine 0.7.2 has been released

ruby-talk
1

All:
The Ruby/EventMachine team released version 0.7.2 last week. The release has
been stable and hasn't turned up any significant problems. For Windows
users, there is a binary gem that was built on a VC6 tool-chain, and
includes openssl support. If anyone turns up problems working with this
binary gem, let us know. We're amenable to releasing it in multiple flavors
if necessary.

A raft of usability features has been added since the 0.7.0 release last
November. The most notable change is that we've cleaned up the licensing
text that appears in the source files comprising the release. This has
confused a great many people, for which I apologize. EventMachine is
dual-licensed under either the GPL or Ruby's license, your choice. This has
been true since the product was released over a year ago, and has been
memorialized in a variety of communications on public fora since then. But
until this release, the text in many of the source files has reflected only
the GPL flavor.

The most notable new features are support for an internal thread pool
(EventMachine#defer), and for the Deferrable class, which behaves much like
Twisted's "Deferred," but with a Ruby-esque (read, easier-to-use) API. As
one example of its capabilities, EM's Deferrable can be used to generate web
page fragments (including multipart/mixed pages) from long-running external
operations. It can generate many pages simultaneously, without blocking and
without threads.

The new release also contains some new helper classes to assist people who
need to write their own network protocols (for example, to communicate with
legacy systems that define one-off protocols). There is also an evented HTTP
server which has extremely fast performance with excellent scalability. It's
designed to support a range of applications requiring dynamically-generated
content. We'll shortly be releasing evented versions of some other
protocols, including SMTP, LDAP, and XML-stanzas. (The latter is intended
for use with RSS or XMPP streams.)

It's been great to hear from all of the people who have chosen to use and
trust EM for critical applications. A special shout-out to Kirk Haines, who
wrote the extremely clever and interesting Swiftiply, a clustering-proxy
server for web applications which has already given some sites a big speed
boost: http://swiftiply.swiftcore.org.

We've started planning for a pure-Java version of EventMachine. This will
enable applications written to the current EM API to run unmodified on
JRuby. The intention is that the Java implementation will live alongside the
C++ one, and they will stay matched in terms of features. (The current C++
based implementation will be supported indefinitely.)

Finally, I'd like to acknowledge Selem Delul and his mentor Pat Eyler, who
are doing a very interesting SoC project. It will allow you to write evented
protocol handlers using Ragel grammars.

Thanks, everyone, and keep those feature requests and bug reports coming.
-francis

2

All, I was asked to provide a description of what the EventMachine package
does. Here's the project README file:

EventMachine implements a fast, single-threaded engine for arbitrary network

communications. It's extremely easy to use in Ruby. EventMachine wraps all
interactions with IP sockets, allowing programs to concentrate on the
implementation of network protocols. It can be used to create both network
servers and clients. To create a server or client, a Ruby program only needs
to specify the IP address and port, and provide a Module that implements the
communications protocol. Implementations of several standard network
protocols
are provided with the package, primarily to serve as examples. The real goal
of EventMachine is to enable programs to easily interface with other
programs
using TCP/IP, especially if custom protocols are required.

A Ruby program uses EventMachine by registering the addresses and ports of
network servers and clients, and then entering an event-handling loop.
EventMachine contains glue code in Ruby which will execute callbacks to
user-supplied code for all significant events occurring in the clients
and servers. These events include connection acceptance, startup,
data-receipt,
shutdown, and timer events. Arbitrary processing can be performed by user
code
during event callbacks, including sending data to one or more remote network
peers, startup and shutdown of network connections, and installation of new
event handlers.

The EventMachine implements a very familiar model for network programming.
It emphasizes: 1) the maximum possible isolation of user code from network
objects like sockets; 2) maximum performance and scalability; and 3) extreme
ease-of-use for user code. It attempts to provide a higher-level interface
than similar projects which expose a variety of low-level event-handling
and networking objects to Ruby programs.

The design and implementation of EventMachine grows out of nearly ten years
of experience writing high-performance, high-scaling network server
applications.
We have taken particular account of the challenges and lessons described as
the "C10K problem" by Dan Kegel and others.

EventMachine consists of an extension library written in C++ (which can be
accessed from languages other than Ruby), and a Ruby module which can be
dropped
into user programs. The current implementation uses the <tt>select(2)</tt>
system call
and may be used on a range of Unix-like systems. A future version of
EventMachine
for Linux will use <tt>epoll(4)</tt> and will thus require a Linux 2.6kernel.
Here's a fully-functional echo server written with EventMachine:

      require 'rubygems'
      require 'eventmachine'

      module EchoServer
        def receive_data data
          send_data ">>>you sent: #{data}"
          close_connection if data =~ /quit/i
        end
      end

      EventMachine::run {
        EventMachine::start_server "192.168.0.100", 8081, EchoServer

···

On 5/29/07, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

3

We have taken particular account of the challenges and lessons described
as
the "C10K problem" by Dan Kegel and others.

...

into user programs. The current implementation uses the <tt>select(2)</tt>
system call
and may be used on a range of Unix-like systems. A future version of
EventMachine
for Linux will use <tt>epoll(4)</tt> and will thus require a Linux
2.6kernel.

Ok, I have one question (I have the same question for Twisted, but haven't
asked them about it...): why not use libevent? It seems much more
developer-friendly than having to write your own implementations of epoll,
kqueue, poll, select, etc. Why isn't it more widely used?

4

I can't speak for Twisted. As far as Ruby is concerned, there is a problem
with libevent and libraries like it, in that it's a challenge to integrate
with Ruby because of how Ruby's thread scheduler works. The problem was easy
enough to work around in EM because we wrote the whole stack.

A minor issue is that libevent is a very complex library full of features,
with a large API intended for use by low-level programmers. That's entirely
suitable for a certain class of tasks, but EM is aimed at the other end of
the scale: it seeks to wrap up the complexity of network programming and
hiding it behind as simple and high-level an API as possible.

Even more to the point, the goal of EM is to provide a complete framework
for highly-scalable, fast programs that are network-aware. Just handling
descriptor-level I/O is a small part of the charter. You'll soon start
seeing releases of full-featured handlers for a variety of standard
protocols, as well as higher-level support for applications that make use of
those protocols.

Speaking for myself personally, libevent is a first-rate piece of work. But
I've been writing network programs for quite a few years now, and I find
using my own knowledge to be more developer-friendly than using libevent
;-).

···

On 5/29/07, tsuraan <tsuraan@gmail.com> wrote:

Ok, I have one question (I have the same question for Twisted, but
haven't
asked them about it...): why not use libevent? It seems much more
developer-friendly than having to write your own implementations of epoll,
kqueue, poll, select, etc. Why isn't it more widely used?

5

I can't speak for Twisted. As far as Ruby is concerned, there is a problem
with libevent and libraries like it, in that it's a challenge to integrate
with Ruby because of how Ruby's thread scheduler works. The problem was
easy
enough to work around in EM because we wrote the whole stack.

Ok, that makes sense.

A minor issue is that libevent is a very complex library full of features,

with a large API intended for use by low-level programmers. That's
entirely
suitable for a certain class of tasks, but EM is aimed at the other end of
the scale: it seeks to wrap up the complexity of network programming and
hiding it behind as simple and high-level an API as possible.

Even more to the point, the goal of EM is to provide a complete framework
for highly-scalable, fast programs that are network-aware. Just handling
descriptor-level I/O is a small part of the charter. You'll soon start
seeing releases of full-featured handlers for a variety of standard
protocols, as well as higher-level support for applications that make use
of
those protocols.

Those will still run on a select (or epoll or kqueue) based core though,
right? Or are you going to make use of non-fd events such as signals?

Speaking for myself personally, libevent is a first-rate piece of work. But

I've been writing network programs for quite a few years now, and I find
using my own knowledge to be more developer-friendly than using libevent
;-).

I was just thinking that libevent looks like a really nice way to get a
high-performance cross-platform mainloop. Twisted has a dozen different
"reactors" to handle poll, select, kqueue, epoll,
whatever-the-heck-windows-uses, etc. I would think riding on top of a
library like libevent as an abstraction above those would be nice. I've
never used libevent though, and from what I've read on its API, it does look
like you have to write your entire program on around it; it hadn't occurred
to me that integration with it would be made difficult by the fact that Ruby
isn't written around libevent.

6

Those will still run on a select (or epoll or kqueue) based core though,
right? Or are you going to make use of non-fd events such as signals?

The EM reactor core currently includes an I/O processor and a timer
processor. We'll be adding a signal reactor very shortly. I started adding a
file I/O processor, but in this age of really advanced kernel-VM and disk
controllers, I'm not sure it's worth it. I'm undecided whether we should add
keyboard and mouse events, not because that's a bad idea, but because
Twisted did it and they messed up a lot of things when they did. Then they
took their multiple implementations of the same things and started calling
them features. :wink:

I was just thinking that libevent looks like a really nice way to get a
high-performance cross-platform mainloop. Twisted has a dozen different
"reactors" to handle poll, select, kqueue, epoll,
whatever-the-heck-windows-uses, etc. I would think riding on top of a
library like libevent as an abstraction above those would be nice. I've
never used libevent though, and from what I've read on its API, it does
look
like you have to write your entire program on around it; it hadn't
occurred
to me that integration with it would be made difficult by the fact that
Ruby
isn't written around libevent.

Give EM a try. I think you'll find that, like Ruby itself, it takes care of
a lot of the low-level stuff that other tools force you to think about, and
lets you just get on with coding your application.

···

On 5/30/07, tsuraan <tsuraan@gmail.com> wrote: