[ANN] RWB 0.2.0 (now with extra RubyForge goodness)

RWB 0.1.1 was released last night, but before I put together an
announcement, I ended up putting together a 0.2.0 release too.

This combined release features three new warmup methods:
   warmup(num_runs)
   rand_warmup(num_requests)
   spec_warmup(urls, num_runs)
These methods will exercise your website prior to the recorded tests.

RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
behind a web proxy.

You can grab your copy from rubyforge.org/projects/rwb/ (a couple
of examples are hiding at www.red-bean.com/~pate as well).

Don't forget to send me you comments, suggestions, requests, patches,
etc.

···

--
thanks,
-pate
-------------------------

have you released this as a gem yet?

j.

···

On 11/9/05, pat eyler <pat.eyler@gmail.com> wrote:

RWB 0.1.1 was released last night, but before I put together an
announcement, I ended up putting together a 0.2.0 release too.

This combined release features three new warmup methods:
   warmup(num_runs)
   rand_warmup(num_requests)
   spec_warmup(urls, num_runs)
These methods will exercise your website prior to the recorded tests.

RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
behind a web proxy.

You can grab your copy from rubyforge.org/projects/rwb/ (a couple
of examples are hiding at www.red-bean.com/~pate as well).

Don't forget to send me you comments, suggestions, requests, patches,
etc.

--
thanks,
-pate
-------------------------

--
"Remember. Understand. Believe. Yield! -> http://ruby-lang.org"

Jeff Wood

you guys ever think of explaining wtf you talking about? rwb?? what is it?
:slight_smile:
--r

···

2005/11/9, pat eyler <pat.eyler@gmail.com>:

RWB 0.1.1 was released last night, but before I put together an
announcement, I ended up putting together a 0.2.0 release too.

This combined release features three new warmup methods:
   warmup(num_runs)
   rand_warmup(num_requests)
   spec_warmup(urls, num_runs)
These methods will exercise your website prior to the recorded tests.

RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
behind a web proxy.

You can grab your copy from rubyforge.org/projects/rwb/ (a couple
of examples are hiding at www.red-bean.com/~pate as well).

Don't forget to send me you comments, suggestions, requests, patches,
etc.

--
thanks,
-pate
-------------------------

I took a brief look at the code a week or so ago, and I like it.
However (and this might just be my lack of deep understanding of the
way Ruby deals with sockets), it seems to me that the 'concurrency'
feature doesn't really do much.
Isn't the bulk of the work performed in the concurrent threads being
serialized by the host process? When I look at the timestamps on the
web server, the requests seem to arrive more or less in order.
Am I using a crappy web server log, or is this really how it behaves
in the absence of native threads? I'm probably just expecting
parallelism, and getting concurrency instead.
I slapped together a lame little script that starts 100 instances of
the Ruby interpreter, and that managed to actually make numerous
requests arrive simultaneously at the server. Unfortunately, it used
a truly shocking amount of memory on the client system.

Stated another way; how different are:
tests = RWB::Runner.new(urls, 1000, 5)
and:
tests = RWB::Runner.new(urls, 1000, 1000)
?

Thanks for reading my incoherent babble.
--Wilson.

···

On 11/9/05, pat eyler <pat.eyler@gmail.com> wrote:

RWB 0.1.1 was released last night, but before I put together an
announcement, I ended up putting together a 0.2.0 release too.

This combined release features three new warmup methods:
   warmup(num_runs)
   rand_warmup(num_requests)
   spec_warmup(urls, num_runs)
These methods will exercise your website prior to the recorded tests.

RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
behind a web proxy.

You can grab your copy from rubyforge.org/projects/rwb/ (a couple
of examples are hiding at www.red-bean.com/~pate as well).

Don't forget to send me you comments, suggestions, requests, patches,
etc.

I haven't ... I've been caught up in work related goo. I'm part way
through implementing quartile graphs and will release it as a gem
when I finish that.

-pate

···

On 11/23/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

have you released this as a gem yet?

j.

On 11/9/05, pat eyler <pat.eyler@gmail.com> wrote:
>
> RWB 0.1.1 was released last night, but before I put together an
> announcement, I ended up putting together a 0.2.0 release too.
>
> This combined release features three new warmup methods:
> warmup(num_runs)
> rand_warmup(num_requests)
> spec_warmup(urls, num_runs)
> These methods will exercise your website prior to the recorded tests.
>
> RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
> behind a web proxy.
>
> You can grab your copy from rubyforge.org/projects/rwb/ (a couple
> of examples are hiding at www.red-bean.com/~pate as well).
>
> Don't forget to send me you comments, suggestions, requests, patches,
> etc.
>
> --
> thanks,
> -pate
> -------------------------
>
>

--
"Remember. Understand. Believe. Yield! -> http://ruby-lang.org"

Jeff Wood

--
thanks,
-pate
-------------------------

you guys ever think of explaining wtf you talking about? rwb?? what is it?
:slight_smile:

rwb is the Ruby Web Bench, a Ruby library designed to let you run
performance/load tests against a webserver/website. Using it you
can write scripts like this:

require 'rwb'

urls = RWB::Builder.new()
urls.add_url(50,
        "http://www.example.com/welcome&quot;\)
urls.add_url(25,
        "http://www.example.com/expensive_dynamic_page&quot;\)
search_terms = ['foo+bar', 'bar+baz', 'quux']
urls.add_url_group(10, "http://www.examples.com/search?&quot;,
        search_terms)

tests = RWB::Runner.new(urls, 50_000, 500)
tests.run

tests.sla_levels = [0.5, 0.9, 0.99]
tests.report_header
tests.report_overall([0.25, 0.5, 0.75])
tests.report_urls
tests.report_by_time

Which would create a report something like this:

Concurrency Level: 500
Total Requests: 50000
Total time for testing: 248.407943 secs
Requests per second: 201.281808448452
Mean time per request: 49 msecs
Standard deviation: 44
Overall results:
        Shortest time: 23 msecs
        25.0%ile time: 28 msecs
        50.0%ile time: 29 msecs
        75.0%ile time: 46 msecs
        Longest time: 276 msecs
Results for http://www.example.com/welcome:
        Shortest time: 23 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 123 msecs
        99.0%ile time: 210 msecs
        Longest time: 276 msecs
Results for http://www.example.com/expensive_dynamic_page:
        Shortest time: 23 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 122 msecs
        99.0%ile time: 210 msecs
        Longest time: 255 msecs
Results for http://www.example.com/search?:
        Shortest time: 24 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 120 msecs
        99.0%ile time: 209 msecs
        Longest time: 239 msecs
Results by time:
results for requests 0 - 10000
        Shortest time: 23 msecs
        50.0%ile time: 30 msecs
        90.0%ile time: 61 msecs
        99.0%ile time: 73 msecs
        Longest time: 104 msecs
results for requests 10000 - 20000
        Shortest time: 24 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 93 msecs
        99.0%ile time: 114 msecs
        Longest time: 140 msecs
results for requests 20000 - 30000
        Shortest time: 23 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 129 msecs
        99.0%ile time: 153 msecs
        Longest time: 232 msecs
results for requests 30000 - 40000
        Shortest time: 24 msecs
        50.0%ile time: 29 msecs
        90.0%ile time: 159 msecs
        99.0%ile time: 183 msecs
        Longest time: 203 msecs
results for requests 40000 - 50000
        Shortest time: 23 msecs
        50.0%ile time: 28 msecs
        90.0%ile time: 198 msecs
        99.0%ile time: 231 msecs
        Longest time: 276 msecs

There's also some info in the email you quoted below:

···

On 11/23/05, Robert Wagner <robbie.wilhelm@gmail.com> wrote:

--r

2005/11/9, pat eyler <pat.eyler@gmail.com>:

> You can grab your copy from rubyforge.org/projects/rwb/ (a couple
> of examples are hiding at www.red-bean.com/~pate as well).
>
> Don't forget to send me you comments, suggestions, requests, patches,
> etc.

--
hth,
-pate
-------------------------

I took a brief look at the code a week or so ago, and I like it.
However (and this might just be my lack of deep understanding of the
way Ruby deals with sockets), it seems to me that the 'concurrency'
feature doesn't really do much.

Concurrency may be too strong a word, but I'm not sure what else
to call it. So far RWB has worked well enough for me, but I'd be
happy to work with people who find it's not up to their tasks. My
big aims to this point are to get a working request engine and a
nice, featureful reporting system.

Isn't the bulk of the work performed in the concurrent threads being
serialized by the host process? When I look at the timestamps on the
web server, the requests seem to arrive more or less in order.
Am I using a crappy web server log, or is this really how it behaves
in the absence of native threads? I'm probably just expecting
parallelism, and getting concurrency instead.

Unless someone wants to point out an improvement in my thread
handling, I think that wysiwyg applies.

I slapped together a lame little script that starts 100 instances of
the Ruby interpreter, and that managed to actually make numerous
requests arrive simultaneously at the server. Unfortunately, it used
a truly shocking amount of memory on the client system.

Wow, I'll bet that was huge. I'd be interested in seeing what
native threads do for performance. Could you share your results?

Stated another way; how different are:
tests = RWB::Runner.new(urls, 1000, 5)
and:
tests = RWB::Runner.new(urls, 1000, 1000)
?

The big difference is going to be against a slower webserver
(or against slower dynamic pages). In that case the second
invocation will spawn 1000 threads without weighting for any
of them to finish, while the first will end up waiting at least
some of the time. (At work I've had a couple of requests go
in excess of 3 seconds, so I can really see the difference
with higher levels of (pseudo) concurrancy.

The other difference you'll see (even on a fast site) is in load
generation/testing. Here are two runs for comparison:

Concurrency Level: 5
Total Requests: 1000
Total time for testing: 16.329556 secs
Requests per second: 61.2386521715594
Mean time per request: 81 msecs
Standard deviation: 77
Overall results:
        Shortest time: 48 msecs
        25.0%ile time: 56 msecs
        50.0%ile time: 57 msecs
        75.0%ile time: 59 msecs
        Longest time: 703 msecs

Concurrency Level: 1000
Total Requests: 1000
Total time for testing: 8.444824 secs
Requests per second: 118.415730156129
Mean time per request: 84 msecs
Standard deviation: 186
Overall results:
        Shortest time: 21 msecs
        25.0%ile time: 26 msecs
        50.0%ile time: 34 msecs
        75.0%ile time: 37 msecs
        Longest time: 1103 msecs

You can see that we were able to throw almost
twice as many requests per second against the
server this way, which can be an important measure
when you're talking about how to scale/optimize
a website for a large traffic load.

Thanks for reading my incoherent babble.

Thanks for sharing.

···

On 11/23/05, Wilson Bilkovich <wilsonb@gmail.com> wrote:

--Wilson.

--
thanks,
-pate
-------------------------

> you guys ever think of explaining wtf you talking about? rwb?? what is it?
> :slight_smile:

rwb is the Ruby Web Bench, a Ruby library designed to let you run
performance/load tests against a webserver/website. Using it you
can write scripts like this:

web performance tests... this is very nice.

There's also some info in the email you quoted below:

me thinks: a short "about" should be the first sentence in an announcement

ciao,
--r

···

2005/11/24, pat eyler <pat.eyler@gmail.com>:

On 11/23/05, Robert Wagner <robbie.wilhelm@gmail.com> wrote:

> --r
>
>
> 2005/11/9, pat eyler <pat.eyler@gmail.com>:

> > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
> > of examples are hiding at www.red-bean.com/~pate as well).
> >
> > Don't forget to send me you comments, suggestions, requests, patches,
> > etc.

--
hth,
-pate
-------------------------