Acgi - a fastcgi alternative?

ruby-talk
1

web rubyists-

i have been thinking quite abit about the things i would like to see in a web
app server for ruby, some of the things which have sprung to mind have been :

  • simplified persistence

    • normal ruby scoping rules
    • ability to cache in memory (process space) or on disk

  • fast execution times

    • on par with fastcgi or mod_ruby

  • simplicity

    • just say no to jboss

  • compatibility

    • well, at least with *nix cgi capable web server

  • familiarity

    • standard ruby cgi api

  • flexibility

    • use whatever template library you like
    • use CGI tag methods
    • whatever…

  • ‘installlessness’

    • i hate modules

  • self-motivated

    • process has ability to do work even when no requests are pending

so. i think i may have the beginnings of something along those lines. it’s
not much, so don’t laugh, but here is the design :

  • a simple C program cgi ‘responder’ (FAST) which delegates cgi requests to
    a persistent backend ruby cgi server, communcating via named pipes (fifos)

that’s it.

i have a simple demo of this (massively prototypical and certainly quite
buggy) available at :

http://eli.fsl.noaa.gov/lib/ruby/acgidemo/

baby docs live at :

http://eli.fsl.noaa.gov/lib/ruby/acgidemo/doc/

it’s simple enough that by looking at the files

http://eli.fsl.noaa.gov/lib/ruby/acgidemo/index.c
http://eli.fsl.noaa.gov/lib/ruby/acgidemo/acgi.rb
http://eli.fsl.noaa.gov/lib/ruby/acgidemo/index.rb

you should have a complete understanding of the idea. if you’ve worked with
fastcgi, you’ll see the design immeadiately.

my next plans are to implement process pooling and session affinity (not
needed now because there is only one instance!) and then i’ll have, in
essence, all the funtionality of fastcgi with none of the headaches.

i’m very curious to know what other may think of this approach, and if any any
would like to contribute to work on such a project. my main concern is
whether people think this design is worth pursuing not considering impl.

-a

ps. ab showed this approach to be roughly 10 times faster that an equivalent
ruby cgi program. that’s fast enough IMHO, especially considering that any
current cgi program could be ported in about 1 minute.

pss. the example program is running at

http://eli.fsl.noaa.gov/lib/ruby/acgidemo/index.cgi

···

–

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

2

I always wanted to ask this question:
why people are developing socket connected backends instead of using
fifo’s?
I see you’re using htme, would you mind explaining your choice?

···

il Mon, 24 Mar 2003 21:54:33 +0000, ahoward ahoward@fsl.noaa.gov ha scritto::

  • a simple C program cgi ‘responder’ (FAST) which delegates cgi requests to
    a persistent backend ruby cgi server, communcating via named pipes (fifos)

that’s it.

3

I don’t see how this differs much from fastcgi, if you use the ‘cgi-fcgi’
program supplied with the fcgi library (it’s a small C program which is
invoked as a ‘normal’ CGI, which in turn passes on the request to one of a
pool of FastCGI processes). That fixes the ‘no external modules, no
configuration of Apache’ issue.

FastCGI is working very well for me, and I’ll certainly persist with it -
there’s a lot of development already behind it, and the ability to run perl
CGIs under the same API is extremely useful.

I am just in the process of updating the ruby-fcgi module (programs now run
either from the command-line, as normal CGI, or as FastCGI with no changes!)
I’ll post it tomorrow when I’ve finished working out why FreeBSD appears to
be mangling signals…

Regards,

Brian.

···

On Tue, Mar 25, 2003 at 07:20:18AM +0900, ahoward wrote:

  • a simple C program cgi ‘responder’ (FAST) which delegates cgi requests to
    a persistent backend ruby cgi server, communcating via named pipes (fifos)
4

htme means “these” , sorry :slight_smile:

···

il Mon, 24 Mar 2003 22:22:45 GMT, gabriele renzi surrender_it@remove.yahoo.it ha scritto::

5

what’s htme?

i’m using fifos because

  • simple
  • fast (data never hit’s disk)

in addition, shell commands may use them to communicate with the backend! if
you look in the makefile you’ll see that i do a reload by simply

echo reload > acgi/env

which causes the backend to do

exec $0

pretty simple.

i’m not opposed to using sockets and, as some may have noticed, my approach
using fifo’s may not be totally workable (though it might be). mainly i used
them because they’re easy and fast.

-a

···

On Mon, 24 Mar 2003, gabriele renzi wrote:

I always wanted to ask this question: why people are developing socket
connected backends instead of using fifo’s? I see you’re using htme, would
you mind explaining your choice?

–

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

6

I don’t see how this differs much from fastcgi, if you use the ‘cgi-fcgi’
program supplied with the fcgi library (it’s a small C program which is
invoked as a ‘normal’ CGI, which in turn passes on the request to one of a
pool of FastCGI processes). That fixes the ‘no external modules, no
configuration of Apache’ issue.

  • session affinity
    • global data does not good without it (i’m aware of the patch btw., but
      it definitely ups complexity)
  • simplicity
    • application reloads of pools are not fun.

FastCGI is working very well for me, and I’ll certainly persist with it -
there’s a lot of development already behind it, and the ability to run perl
CGIs under the same API is extremely useful.

i agree with you. however, my take is the benefits/complexity ratio is simply
not there for fastcgi and that this is why more people have not adopted it. i
have been using it myself and really like it…

I am just in the process of updating the ruby-fcgi module (programs now run
either from the command-line, as normal CGI, or as FastCGI with no changes!)
I’ll post it tomorrow when I’ve finished working out why FreeBSD appears to
be mangling signals…

i spent quite a few hours mucking with that too. signals are evil IMHO and
should be steered clear of if simplicity and robustness are really desired.
i’m not saying it can’t be done - just that it’s one of those slipperly slopes
like threads and continuations.

i will definitely check out your patches to fcgi and give them a try.

-a

···

On Tue, 25 Mar 2003, Brian Candler wrote:

–

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

7

FastCGI is working very well for me, and I’ll certainly persist with it -
there’s a lot of development already behind it, and the ability to run perl
CGIs under the same API is extremely useful.

i agree with you. however, my take is the benefits/complexity ratio is simply
not there for fastcgi and that this is why more people have not adopted it. i
have been using it myself and really like it…

I’m not sure why it hasn’t taken off so much. Many CGI application writers
use PHP, and PHP support is noticeably missing from www.fastcgi.com, so they
are forced to use mod_php. Perlers will likely use mod_perl if their
webserver has it compiled in already, which is often the case (more than
mod_fastcgi anyway)

lets me run persistent C programs (e.g. sqwebmail), Ruby programs and Perl
programs. Furthermore I get better visibility, since each application runs
as a separate process, and therefore I can see it in ‘top’, attach a ktrace
to it, etc.

Also, an application can be restarted automatically if its source file has
changed (although not the libraries it depends on):

FastCgiConfig -autoUpdate

I am just in the process of updating the ruby-fcgi module (programs now run
either from the command-line, as normal CGI, or as FastCGI with no changes!)
I’ll post it tomorrow when I’ve finished working out why FreeBSD appears to
be mangling signals…

i spent quite a few hours mucking with that too. signals are evil IMHO and
should be steered clear of if simplicity and robustness are really desired.
i’m not saying it can’t be done - just that it’s one of those slipperly slopes
like threads and continuations.

I don’t think it’s that bad. The confusion arose because Ruby’s “traps” sit
on top of OS “signals” and aren’t just the same thing. Ruby installs its own
handlers for the majority of signals, and it marks them with SA_RESTART so
that interrupted system calls are automatically restarted. When a signal
occurs, it takes action based on its internal trap table. So when you change
handlers using ‘trap’ it doesn’t actually change the signal handler
installed in the OS.

When you understand it, it’s a nice abstraction though. You can protect
critical code with

 trap('SIGUSR1') { flag = true }
 ... critical code
 trap('SIGUSR1','DEFAULT')
 break if flag

then a USR1 which happens outside this block will be handled immediately,
raising an exception, but if it happens inside this block it will be
deferred until the end of the critical section.

Regards,

Brian.

···

On Tue, Mar 25, 2003 at 08:00:49AM +0900, ahoward wrote:
From my point of view, FastCGI is the simpler alternative. One Apache module

8

FastCGI is working very well for me, and I’ll certainly persist with it -
there’s a lot of development already behind it, and the ability to
run perl
CGIs under the same API is extremely useful.

i agree with you. however, my take is the benefits/complexity ratio is
simply
not there for fastcgi and that this is why more people have not adopted
it. i
have been using it myself and really like it…

I’m not sure why it hasn’t taken off so much. Many CGI application writers
use PHP, and PHP support is noticeably missing from www.fastcgi.com, so they
are forced to use mod_php. Perlers will likely use mod_perl if their
webserver has it compiled in already, which is often the case (more than
mod_fastcgi anyway)

what’s wrong with mod_php? i think concerning simplicity, mod_php is the
benchmark…
many users can update their scripts without having trouble with others, not
needing
webserver restarts etc.

i once read, mod_php works with aggressive cleaning methods to achieve
this. on the
other hand it’s still quite fast. since ruby interpreter is so flexible
(morphable) by the
hands of the user, how can one be safe,
that the next request will be answered by the same (regarding the premises)
interpreter?

From my point of view, FastCGI is the simpler alternative. One Apache module
lets me run persistent C programs (e.g. sqwebmail), Ruby programs and Perl
programs. Furthermore I get better visibility, since each application runs
as a separate process, and therefore I can see it in ‘top’, attach a ktrace
to it, etc.

Also, an application can be restarted automatically if its source file has
changed (although not the libraries it depends on):

FastCgiConfig -autoUpdate

what is a library? anything stated in “require”?

regards,
robert

···

At 22:45 25.03.03 +0900, you wrote:

On Tue, Mar 25, 2003 at 08:00:49AM +0900, ahoward wrote:

9

i agree with you. however, my take is the benefits/complexity ratio is
simply
not there for fastcgi and that this is why more people have not adopted
it. i
have been using it myself and really like it…

I’m not sure why it hasn’t taken off so much. Many CGI application writers
use PHP, and PHP support is noticeably missing from www.fastcgi.com, so they
are forced to use mod_php. Perlers will likely use mod_perl if their
webserver has it compiled in already, which is often the case (more than
mod_fastcgi anyway)

what’s wrong with mod_php? i think concerning simplicity, mod_php is the
benchmark…
many users can update their scripts without having trouble with others, not
needing
webserver restarts etc.

I didn’t say there was anything wrong with mod_php - only trying to guess
why mod_fastcgi is not as popular as it might be.

i once read, mod_php works with aggressive cleaning methods to achieve
this. on the
other hand it’s still quite fast. since ruby interpreter is so flexible
(morphable) by the
hands of the user, how can one be safe,
that the next request will be answered by the same (regarding the premises)
interpreter?

Well, it will be the same interpreter, unless Ruby recompiles itself :slight_smile:

Typically you would not want a CGI application to morph during its lifetime,
since it will be handling requests from many different clients. It can of
course create a separate object representing each session, and those objects
could morph, but unless you can save them persistently you are not going to
be able to change code without breaking all user sessions.

Remember that objects of singleton classes cannot be serialized.

Also, an application can be restarted automatically if its source file has
changed (although not the libraries it depends on):

FastCgiConfig -autoUpdate

what is a library? anything stated in “require”?

Yep, that’s what I meant. Apache can check the timestamp of the FCGI program
it is running, but doesn’t know how to check the dependencies.

Regards,

Brian.

···

On Wed, Mar 26, 2003 at 12:09:30AM +0900, Robert Wagner wrote: