Web APIs

I’ve been working with web programming for a while now, and I’ve yet to
find a totally satisfactory API. I like using literate URLs, and no
system so far supports that well.

I’d like to open a discussion about what everyone would like to see in a
web API, so that perhaps such a thing may exist.

Let me sum up some existing ways it’s done:

CGI: It’s fairly raw, an undercooked and certainly not object-oriented
protocol for seeing what the HTTP server (assumably) got, and a
limited way to respond. You have to know about the deployment
environment, or introspect from the provided environment (which is
often inconsistent) about how you were called.

FCGI is CGI hacked to be fast – replace stdin and stdout with sockets
so they can be opened more than once, and make the app persistent.
That’s about it.

WEBrick and other servlet APIs: They are far more object-oriented, and
support design-by-composition of elements. WEBrick makes assumptions
that it is the only server process involved, so integrating it with
another server involves heavyweight proxy rewriting, ala Apache’s
mod_proxy.

mod_ruby and the like let you see any information you want about the
request, but has similar shortcomings to the rest, and it’s specific
to Apache.

I’d like to create a solution that works for a lot of applications, and
under a lot of web servers, though that would mean writing API modules
for each server, or translators for some other framework.

Some needed features:

  • URL portability – if a script lives in a server’s namespace at
    /application/feed, and you need to move it to /apllication2/feed, it
    shouldn’t have to refer to everything using absolute paths nor
    relative ones when at all possible. It should, at the least, be able
    to ask the API for a URL like:

    #{thisscript}/path
    #{thisserver}/path
    #{thisrequest}/path

to facilitate more aware applications. In addition, where possible, it
would be nice to have rewrite support be bidirectional, so if there’s
a rewrite for /foo/bar to /baz/bar, if the app requests a path that
would ordinarily return /baz/bar, the server could rewrite to /foo/bar
so the user sees only clean URLs.

  • Registry or inspection of namespace – I’d love an application to be
    able to find out what other valid paths are within the server, so that
    there’s less guesswork in making a non-brittle way of linking
    applications together. Being able to say
    "server.application[‘wiki’].url" would be really cool, though perhaps
    too simplistic.

  • Object orientation – Represent all of this in good, clear OO style.
    WEBrick excels here more than the other solutions, but it’s still
    pretty tentative in comparison to what it could be.

  • Streamability – templating has to be optional, and preferably
    separate, so that one could, say, write a streaming video-over-http
    server, or a realtime chat system.

  • Support for more than GET and POST methods. It should be possible to
    write a WebDAV server using the API, without having to modify the
    webserver configuration.

  • Filtering of requests, so I could define a handler to intercept all
    text/http documents served by the server, and apply some
    transformation to them. This would aid writing applications by
    aggregation rather than making applications monolithic. It brings to
    mind the useful unix concepts of filters and pipes.

There’s probably a lot more, but I just want to start the conversation
(and if neccesary, we can form a mailing list just for such a thing) so
we can stop re-inventing the wheel and make something exceptional.

Ari

Hello Aredridel,

I’ve been working with web programming for a while now, and I’ve yet to
find a totally satisfactory API. I like using literate URLs, and no
system so far supports that well.

Do we really need another one :frowning: ???

I’m sick about having 100 different solutions with each one having its
highlights and drawbacks but none of them competitive to J2EE, Zope or
even PHP. Okay ruby does not have native threads so there can’t be
anything competitive to J2EE, but maybe we can get something more
stable - that would be a huge improvement.

And most important, it is time for the community to decide about 2 or
3 standard solutions and send the rest of all this 80% unusable frameworks
to hell. To much competition is not a healthy situation.

···


Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

I’d like to create a solution that works for a lot of applications, and
under a lot of web servers, though that would mean writing API modules
for each server, or translators for some other framework.

I think this is the idea behind narf (http://www.narf-lib.org). One of
its main strong points is that applications developed with narf can
easily be unit-tested.

Some needed features:

  • URL portability – if a script lives in a server’s namespace at
    /application/feed, and you need to move it to /apllication2/feed, it
    shouldn’t have to refer to everything using absolute paths nor
    relative ones when at all possible. It should, at the least, be able
    to ask the API for a URL like:

What options are there besides absolute paths and relative paths?

Paul

···

On Fri, May 28, 2004 at 03:59:55AM +0900, Aredridel wrote:

  • URL portability – if a script lives in a server’s namespace at
    /application/feed, and you need to move it to /apllication2/feed,
    it shouldn’t have to refer to everything using absolute paths nor
    relative ones when at all possible. It should, at the least, be able
    to ask the API for a URL like:

    #{thisscript}/path
    #{thisserver}/path
    #{thisrequest}/path
    to facilitate more aware applications. In addition, where possible,
    it would be nice to have rewrite support be bidirectional, so if there’s
    a rewrite for /foo/bar to /baz/bar, if the app requests a path that
    would ordinarily return /baz/bar, the server could rewrite to /foo/bar
    so the user sees only clean URLs.

On the surface, this seems like a pretty simple thing, although my brain
isn’t coughing up any good real life examples of why this would be useful.
Can you help my poor brain with one?

  • Registry or inspection of namespace – I’d love an application
    to be able to find out what other valid paths are within the server,
    so that there’s less guesswork in making a non-brittle way of linking
    applications together. Being able to say
    “server.application[‘wiki’].url” would be really cool, though perhaps
    too simplistic.

These seem somewhat related to the URL portability, and is a cool idea. In
fact, I can see enough reason in my own work for this one right now that I
might build this into Iowa. I’d love to be able to register components in
some centralized sort of registry that would exist outside of the umbrella
of any one application process.

So, to borrow from your example, maybe I could do something like this:

new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
yield new_page

or, if I wanted to embed a direct link to the wiki within my output, in the
template I could do something like this:

Jump To The HTML Templates Wiki Page

— (in the code file)

def wikilink
new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
new_page.url
end

That would be very, very neat if it would work regardless of whether the
wiki and the application that wanted to connect to it was being managed
under the same overarching process or a different one.

The wiki application, when it registered itself, would have to tell the
registry what parameters (such as ‘page’) it accepts, and could provide a
method to call to determine the URL to call to invoke it, if it was more
complicated than the superclass method could handle (such as if it takes a
parameter on the query string to define what wiki page to display). That
could be a very neat way to link applications that would work even if one
changed the location of an application. After changing the location, the
application would just reregister itself in the new place, and everything
that cared about it could still find it. That would be very, very, very
cool for me. Very very very cool.

  • Streamability – templating has to be optional, and preferably
    separate, so that one could, say, write a streaming video-over-http
    server, or a realtime chat system.

This is interesting. I don’t think it is necessarily related specifically
to templating, though, but rather more generally to a buffered versus an
unbuffered response. For example, with Iowa one can skip using a template
simply by making the “template” be a single call to a method that returns
the actual content. Currently there is no option to have that content be
returned in an unbuffered way, however. It all gets collected into a buffer
and is then returned in one large chunk.

If I were to implement a mechanism that allowed the content to be sent back
to the front end as it was generated, one could then stream that content,
regardless of whether there was any sort of a “template” involved in its
creation or not.

Not that one would probably want to do this under Iowa until we either
have native threads or I implement some sort of a multiple process backend
so that one wasn’t trying to send 15 streams out of seperate threads sharing
the same process, but…

  • Support for more than GET and POST methods. It should be
    possible to write a WebDAV server using the API, without having to
    modify the webserver configuration.

This seems like it would mostly be a function of whatever front end is
receiving the HTTP requests. Whatever API/system is being used to generate
content shouldn’t have any restrictions at all regarding what kind of
request that it is. GET or POST of COPY or MKCOL or STRAIGHTONTILMORNING
should all work if the frontend accepts them.

  • Filtering of requests, so I could define a handler to intercept all
    text/http documents served by the server, and apply some
    transformation to them. This would aid writing applications by
    aggregation rather than making applications monolithic. It brings
    to mind the useful unix concepts of filters and pipes.

That’s a powerful tool. The one thing that a command line has going for it
is that it is easy to specify the order of what pipes into what. Handling
the interactions of ordering with a system that intercepts responses adds a
wrinkle that would need to be thought about carefully in order to find a
solution that is flexible without being difficult.

There’s probably a lot more, but I just want to start the conversation
(and if neccesary, we can form a mailing list just for such a thing)
so we can stop re-inventing the wheel and make something exceptional.

I’d be interested in this, regardless of where the discussion takes place.
This is a fascinating topic, and I know that I can definitely use some of
this to make Iowa a more powerful tool.

Kirk Haines

···

On Fri, 28 May 2004 03:59:55 +0900, Aredridel wrote

Lothar Scholz wrote:

Hello Aredridel,

I’ve been working with web programming for a while now, and I’ve yet to
find a totally satisfactory API. I like using literate URLs, and no
system so far supports that well.

Do we really need another one :frowning: ???

I’m sick about having 100 different solutions with each one having its
highlights and drawbacks but none of them competitive to J2EE, Zope or
even PHP. Okay ruby does not have native threads so there can’t be
anything competitive to J2EE, but maybe we can get something more
stable - that would be a huge improvement.

And most important, it is time for the community to decide about 2 or
3 standard solutions and send the rest of all this 80% unusable frameworks
to hell. To much competition is not a healthy situation.

I think we could use at least one or 2 more to keep things interesting. :wink:

Really, i had my issues with cgi, but mostly, they are addressed in the
latest version. I tend to think that those building the apis for such
know more than i and so tho i may quibble over little stuff , i’m
grateful to have it at all. So when i get smart enuf to write my own api
that is uber oo, i might.

That is the neat thing about this stuff. For example, i ran into trouble
posting remote xml with xmlhttp. The xml was getting munged by cgi…an
equal sign in the first node was disapearing. Anyway, i went into cgi,
scared just opening it, and found that cgi used stdin. This was exciting
to me, fairly new, you know, to getting into the real code. Anyway i
dropped cgi, grabbed stdin, and all was good in my world. So i wrote my
own cgi class, feel free to use it.

class MyCGI
def get
$stdin
end
end

ok id didnt do that -I’m being facetious but it’s neat that i could fix
the problem by looking at the source…so if i need the api the way i
need the api i can make it so and if someone else likes it, that is good
too.

peace,:paul

Hello Aredridel,

I’ve been working with web programming for a while now, and I’ve yet to
find a totally satisfactory API. I like using literate URLs, and no
system so far supports that well.

Do we really need another one :frowning: ???

Well, we don’t have many. I don’t want to write another Rails, another
IOWA, or another Borges, I’m more keen to polish up a really good API
that such would build on – and probably re-implement the connection to
the webserver process (should there be one) instead of basing it on the
ugly, hard-to-use and semantically-ill-defined CGI spec.

I’m sick about having 100 different solutions with each one having its
highlights and drawbacks but none of them competitive to J2EE, Zope or
even PHP. Okay ruby does not have native threads so there can’t be
anything competitive to J2EE, but maybe we can get something more
stable - that would be a huge improvement.

Me too. I’ve not used J2EE (though I’d love to hear a summary of what
it does well), but I’ve used PHP and looked at Zope a lot for ideas.
They’re too high level for what I’m thinking, less of a platform to
build on than a platform to code for.

And most important, it is time for the community to decide about 2 or
3 standard solutions and send the rest of all this 80% unusable frameworks
to hell. To much competition is not a healthy situation.

Yes – I’d love to see the best-of-the-breed polished up and made
useable, and let the rest go.

What I’d really like to see is, optimally, the WEBrick API extended to
be extremely and generally useful, and separate from the specifics of
its HTTPServer.

Part of the problem that a lot of systems have is that the web server
itself doesn’t give enough information, or gives it in a hard-to-use
way. (the CGI variables REQUEST_URI, SCRIPT_NAME, SCRIPT_FILENAME, and
PATH_INFO come to mind). It’s hard to build an excellent application
when you have to keep writing a routine to guess the URL the user
entered to get to the request you’re handling, and it’s also hard to
write good applications when the framework ties your hands: When you
can’t, for example, send a redirect to the browser (or if you can, you
can’t choose between codes 301, 302 and 307) – that’s bad.

I’d love to see a framework that merges the best of WEBrick, FastCGI,
ISAPI modules and CGI (and I mean the protocol, not cgi.rb), and puts it
under a clean, simple umbrella that doesn’t hide HTTP from you, just
abstracts it as it was meant to be.

I want the moon, don’t I?

Ari

···

On Fri, May 28, 2004 at 04:21:30AM +0900, Lothar Scholz wrote:

  • URL portability – if a script lives in a server’s namespace at
    /application/feed, and you need to move it to /apllication2/feed, it
    shouldn’t have to refer to everything using absolute paths nor
    relative ones when at all possible. It should, at the least, be able
    to ask the API for a URL like:

What options are there besides absolute paths and relative paths?

Lookups, and inter-app communication.

Web contact-info app links to account information:

:account_link => App.server.apps['accounting'].ask('url-of', contact.id)

Hypothetical, but very useful for modular systems. It also works by
analogy to the search engine for humans finding things.

I’m investigating NARF, too. Looks interesting – I had written it off
since it has a templating library (yet another).

Ari

  • URL portability – if a script lives in a server’s namespace at
    /application/feed, and you need to move it to /apllication2/feed,
    it shouldn’t have to refer to everything using absolute paths nor
    relative ones when at all possible. It should, at the least, be able
    to ask the API for a URL like:

    #{thisscript}/path
    #{thisserver}/path
    #{thisrequest}/path
    to facilitate more aware applications. In addition, where possible,
    it would be nice to have rewrite support be bidirectional, so if there’s
    a rewrite for /foo/bar to /baz/bar, if the app requests a path that
    would ordinarily return /baz/bar, the server could rewrite to /foo/bar
    so the user sees only clean URLs.

On the surface, this seems like a pretty simple thing, although my brain
isn’t coughing up any good real life examples of why this would be useful.
Can you help my poor brain with one?

Just pragmatism: It’s easier to do with the app cooperating than with a
proxy solution. Proxies might understand HTML, but I use a lot of RDF
and things like that… it’s easy to get wrong.

It’s good for URL migration, and for filtering namespaces – if I
implement an app that, say, is a web-based editor, I can see it being
very useful to “reparent” the entire namespace transparently into a
sub-space –

if /foo.html and /bar.html are the normal namespace, it would be nice to
implement a filter so that /baz/foo.html maps to the original foo.html,
it would be nice to not have to recode any absolute links in the files.

More complicated scenarios affect relative URLs, too.

  • Registry or inspection of namespace – I’d love an application
    to be able to find out what other valid paths are within the server,
    so that there’s less guesswork in making a non-brittle way of linking
    applications together. Being able to say
    “server.application[‘wiki’].url” would be really cool, though perhaps
    too simplistic.

These seem somewhat related to the URL portability, and is a cool idea. In
fact, I can see enough reason in my own work for this one right now that I
might build this into Iowa. I’d love to be able to register components in
some centralized sort of registry that would exist outside of the umbrella
of any one application process.

So, to borrow from your example, maybe I could do something like this:

new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
yield new_page

Ooh, nice.

or, if I wanted to embed a direct link to the wiki within my output, in the
template I could do something like this:

Jump To The HTML Templates Wiki Page

— (in the code file)

def wikilink
new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
new_page.url
end

That would be very, very neat if it would work regardless of whether the
wiki and the application that wanted to connect to it was being managed
under the same overarching process or a different one.

The wiki application, when it registered itself, would have to tell the
registry what parameters (such as ‘page’) it accepts, and could provide a
method to call to determine the URL to call to invoke it, if it was more
complicated than the superclass method could handle (such as if it takes a
parameter on the query string to define what wiki page to display). That
could be a very neat way to link applications that would work even if one
changed the location of an application. After changing the location, the
application would just reregister itself in the new place, and everything
that cared about it could still find it. That would be very, very, very
cool for me. Very very very cool.

Yeah… that’s what I was thinking.

Making it RPC-based isn’t a bad idea, so it would work across servers
(Aha! Web-enabling web applications! A novel idea!)

  • Streamability – templating has to be optional, and preferably
    separate, so that one could, say, write a streaming video-over-http
    server, or a realtime chat system.

This is interesting. I don’t think it is necessarily related specifically
to templating, though, but rather more generally to a buffered versus an
unbuffered response. For example, with Iowa one can skip using a template
simply by making the “template” be a single call to a method that returns
the actual content. Currently there is no option to have that content be
returned in an unbuffered way, however. It all gets collected into a buffer
and is then returned in one large chunk.

Bingo. I just associate it with templating, because template engines
-all- buffer as far as I can tell.

Even for non-realtime apps, it’s important, because it’s nice to give
the browser something to munch on while the rest is generating. Users
like feedback.

If I were to implement a mechanism that allowed the content to be sent back
to the front end as it was generated, one could then stream that content,
regardless of whether there was any sort of a “template” involved in its
creation or not.

Not that one would probably want to do this under Iowa until we either
have native threads or I implement some sort of a multiple process backend
so that one wasn’t trying to send 15 streams out of seperate threads sharing
the same process, but…

Fifteen wouldn’t be so bad even with the green threads.

Some of the fastest web servers out there use non-blocking IO instead of
threads (which is what Ruby threads do internally.)

  • Support for more than GET and POST methods. It should be
    possible to write a WebDAV server using the API, without having to
    modify the webserver configuration.

This seems like it would mostly be a function of whatever front end is
receiving the HTTP requests. Whatever API/system is being used to generate
content shouldn’t have any restrictions at all regarding what kind of
request that it is. GET or POST of COPY or MKCOL or STRAIGHTONTILMORNING
should all work if the frontend accepts them.

Yeah, it should. Making that a requirement is all I’m doing.

  • Filtering of requests, so I could define a handler to intercept all
    text/http documents served by the server, and apply some
    transformation to them. This would aid writing applications by
    aggregation rather than making applications monolithic. It brings
    to mind the useful unix concepts of filters and pipes.

That’s a powerful tool. The one thing that a command line has going for it
is that it is easy to specify the order of what pipes into what. Handling
the interactions of ordering with a system that intercepts responses adds a
wrinkle that would need to be thought about carefully in order to find a
solution that is flexible without being difficult.

Yeah – However, it can largely be set up by the admin. Let the tools be
linked together in ways not thought of beforehand. Let apps do it
internally, too, with the same mechanism, but present a unified front.

I do it internally in my php-based Wiki. It’s been a great tool. The
Wiki spits out very plain XHTML. My filters transform that into content.
It’s XSLT as web-templating. Just one useful thing.

There’s probably a lot more, but I just want to start the conversation
(and if neccesary, we can form a mailing list just for such a thing)
so we can stop re-inventing the wheel and make something exceptional.

I’d be interested in this, regardless of where the discussion takes place.
This is a fascinating topic, and I know that I can definitely use some of
this to make Iowa a more powerful tool.

Excellent. I aim to keep persuing it.

What triggered the idea in my head as something that needed to be fixed
is how I re-invented the wheel to implement webrick-fcgi – I like the
webrick API, as opposed to the CGI one. Internally, the library takes
the request parsed into variables by the webserver, shipped to the app
via FCGI protocol, and re-assembles it into an HTTP request that’s
hopefully similar to the original, then feed that into WEBrick and let
it parse it into different variables. Hardly optimal.

Then I realized that if mod_proxy could speak over a unix-domain socket,
I could just use WEBrick directly with a little hacking.

That made me realize that CGI variables is a poor, and webrick is a
somewhat better abstraction of HTTP.

Then I started thinking of improvements to HTTP and just to web-app
semantics that would make implementing robust programs easier, and
started off on this tangent.

Ari

···

On Fri, May 28, 2004 at 05:28:30AM +0900, Kirk Haines wrote:

On Fri, 28 May 2004 03:59:55 +0900, Aredridel wrote

It’s good for URL migration, and for filtering namespaces – if I
implement an app that, say, is a web-based editor, I can see it being
very useful to “reparent” the entire namespace transparently into a
sub-space –

if /foo.html and /bar.html are the normal namespace, it would be
nice to implement a filter so that /baz/foo.html maps to the
original foo.html, it would be nice to not have to recode any
absolute links in the files.

K. I think I follow you here.

new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
yield new_page

Ooh, nice.

Yeah. Now, the actual implementation…I’ve got to think about that a lot
more.

Making it RPC-based isn’t a bad idea, so it would work across servers
(Aha! Web-enabling web applications! A novel idea!)

I’m liking the implications of how applications could flexibly interrelate
the more I think about it. Going to be sitting on my butt in the car for
several hours tomorrow. Perfect time to really think about this.

Bingo. I just associate it with templating, because template engines
-all- buffer as far as I can tell.

Even for non-realtime apps, it’s important, because it’s nice to give
the browser something to munch on while the rest is generating. Users
like feedback.

Sometimes. If it only takes a bare fraction of a second to generate the
whole response then it is no big deal. On the other hand if you are waiting
a half a minute while a very large report is generated, it’d sure be a lot
nicer to stream that report out as it is built.

The thing with templating and buffering is that it’s a much easier problem
to solve if one is buffering the entire response.

Fifteen wouldn’t be so bad even with the green threads.

Some of the fastest web servers out there use non-blocking IO
instead of threads (which is what Ruby threads do internally.)

Maybe. I suppose it does depend on the hardware that one is running it on
and on what is actually involved in generating the streams. I posted some
benchmarks in the other fastcgi/webrick thread, but on moderate hardware
(PIII 800Mhz) I can handle mid 30s/second requests with 4+k or so of data
returned per request out of one process (about 160-170k/second total
transfer rate). Larger requests reduce the number per second, but the
throughput goes up. 25/second with a throughput of about 410k/second is the
highest I’ve seen in tests so far. Interestingly (to me, anyway), the
concurrency doesn’t matter much at all in the end results.

So, I suppose what I am saying is that if a person could stream a response
out of Iowa (which is not something that could be implemented before, I’m
guessing, Julyish), Ruby and Iowa probably would provide sufficient
performance to stream at least some types of content to a couple handfuls of
recipients at the same time.

Then I realized that if mod_proxy could speak over a unix-domain
socket, I could just use WEBrick directly with a little hacking.

That made me realize that CGI variables is a poor, and webrick is a
somewhat better abstraction of HTTP.

I’m with you there. I just want an easy to understand abstraction that
gives me access to all of the HTTP headers, both coming and going, and wraps
all of it in some convenience. It is at least half familiarity, but I do
like the way an Apache::Request breaks up a request.

Kirk Haines

···

On Fri, 28 May 2004 07:10:03 +0900, Aredridel wrote

if /foo.html and /bar.html are the normal namespace, it would be
nice to implement a filter so that /baz/foo.html maps to the
original foo.html, it would be nice to not have to recode any
absolute links in the files.

K. I think I follow you here.

Yeah. Just a toy example, so don’t break your brain.

new_page = registry.application(‘wiki’)
new_page.page = ‘HtmlTemplates’
yield new_page

Ooh, nice.

Yeah. Now, the actual implementation…I’ve got to think about that a lot
more.

That’s always the clincher, eh?

Making it RPC-based isn’t a bad idea, so it would work across servers
(Aha! Web-enabling web applications! A novel idea!)

I’m liking the implications of how applications could flexibly interrelate
the more I think about it. Going to be sitting on my butt in the car for
several hours tomorrow. Perfect time to really think about this.

Yeah. I was thinking that other HTTP methods might be best.

LISTAPPS /foo/bar HTTP/1.1

Bingo. I just associate it with templating, because template engines
-all- buffer as far as I can tell.

Even for non-realtime apps, it’s important, because it’s nice to give
the browser something to munch on while the rest is generating. Users
like feedback.

Sometimes. If it only takes a bare fraction of a second to generate the
whole response then it is no big deal. On the other hand if you are waiting
a half a minute while a very large report is generated, it’d sure be a lot
nicer to stream that report out as it is built.

Yeah. In my case, try -several- minutes at the moment.

The thing with templating and buffering is that it’s a much easier problem
to solve if one is buffering the entire response.

Yeah, exactly. I’d like to see one that didn’t, now that we’ve seen the
easy implementations.

Fifteen wouldn’t be so bad even with the green threads.

Some of the fastest web servers out there use non-blocking IO
instead of threads (which is what Ruby threads do internally.)

Maybe. I suppose it does depend on the hardware that one is running it on
and on what is actually involved in generating the streams. I posted some
benchmarks in the other fastcgi/webrick thread, but on moderate hardware
(PIII 800Mhz) I can handle mid 30s/second requests with 4+k or so of data
returned per request out of one process (about 160-170k/second total
transfer rate). Larger requests reduce the number per second, but the
throughput goes up. 25/second with a throughput of about 410k/second is the
highest I’ve seen in tests so far. Interestingly (to me, anyway), the
concurrency doesn’t matter much at all in the end results.

Makes sense.

So, I suppose what I am saying is that if a person could stream a response
out of Iowa (which is not something that could be implemented before, I’m
guessing, Julyish), Ruby and Iowa probably would provide sufficient
performance to stream at least some types of content to a couple handfuls of
recipients at the same time.

Mmm.

Then I realized that if mod_proxy could speak over a unix-domain
socket, I could just use WEBrick directly with a little hacking.

That made me realize that CGI variables is a poor, and webrick is a
somewhat better abstraction of HTTP.

I’m with you there. I just want an easy to understand abstraction that
gives me access to all of the HTTP headers, both coming and going, and wraps
all of it in some convenience. It is at least half familiarity, but I do
like the way an Apache::Request breaks up a request.

Apache::Request isn’t bad, nor is WEBrick::HTTPRequest and Response.

It’s one part of the puzzle.

Ari