Ruby replacement for ab?

ruby-talk
21

If you want more complex interactions than JMeter provides, there's
no reason you can't create WebTest scripts and set off multiple copies
running simultaneously. But no, it's not intrinsically designed for stress
testing.

···

On 12/22/06, khaines@enigo.com <khaines@enigo.com> wrote:

WebTest, from my cursory look, seems to be more oriented at validating the
correctness of a web site/application than stress testing, though. Did I
miss something?

--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com

22

Jason Roelofs wrote:

···

On 12/22/06, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:

Hassan Schroeder wrote:
> On 12/22/06, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
>
>> .... As far as I know, there are no open source tools
>> that have the flexibility in script *capture* these tools have.
>> Essentially, you fire up a script recorder, it fires up a browser, you
>> exercise your app, you close the recorder and it generates a script.
>
> Perhaps you missed my prior post about WebTest and associated
> WebTestRecorder? :slight_smile: Yes, it's Java, but if you're considering
JMeter...
>
> Caveat: I haven't used either of the commercial tools you mentioned,
> so grain of salt, YMMV, etc.
>
Yeah ... I missed it when I first posted but I have just looked at it.
Unfortunately, the recorder apparently only works with Firefox, which is
a deal-killer. However much you may hate IE and Microsoft ...

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given
rabbits fire.

Could you explain why this is a "deal killer"? I would hope that per-browser
functionality testing is a seperate suite from load testing, unless you're
sending different websites depending on the browser type.

Jason

If the application only works with Internet Explorer, I can't record it with Firefox.

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

23

Hassan Schroeder wrote:

WebTest, from my cursory look, seems to be more oriented at validating the
correctness of a web site/application than stress testing, though. Did I
miss something?

If you want more complex interactions than JMeter provides, there's
no reason you can't create WebTest scripts and set off multiple copies
running simultaneously. But no, it's not intrinsically designed for stress
testing.

OK ... let's get our definitions straight.

1. QA/Correctness testing: if I enter request A, does the web app return the correct response? If I buy a gizmo, does the database show one less gizmo available to the next buyer? Can Joe Visitor log in and read Francine User's credit card numbers?

2. Stress test: how gracelessly does the web app behave under an attempted DoS attack?

3. Load and scalability testing: Assuming it returns the correct responses, what kind of realistic user load can the web app and its servers sustain? Where are the bottlenecks? Are they software or hardware? If software, how can I fix them? How can I inform my designers how to do things better the next time? If it's hardware, what kind of hardware? Network? Disk? Server?

Number 3 presupposes you've done Number 1! In fact, the load test scripts must, to be valid, check the data coming back from the application to make sure the response is correct. In any event, SilkPerformer and LoadRunner are specifically designed and evolved to do 3. I didn't dig into Web Test to see if it could be made to do 3 or how much effort would be involved, since it doesn't record IE. Considering how long SilkPerformer and LoadRunner have been around, I rather doubt that Web Test has the kind of "industrial strength" load and scalability testing capabilities.

I realize I sound like a salesman for a couple of rather expensive tools -- if that offends the denizens of this open source list, well, I'm sorry. As I said before, if somebody wants to go out and build an open source tool in Ruby that does industrial strength load and scalability testing, I'll be very encouraging, at least with "moral support". But it's a big job -- it's not as simple as it looks. It's certainly more than I'd be willing to take on as a one-man open source hacker.

···

On 12/22/06, khaines@enigo.com <khaines@enigo.com> wrote:

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

24

I think there is a LARGE amount of room in between ab and an open source, feature complete competitor for SilkPerformer for something that fills a niche and is useful, yet manageable as an open source project.

Kirk Haines

···

On Sat, 23 Dec 2006, M. Edward (Ed) Borasky wrote:

I realize I sound like a salesman for a couple of rather expensive tools -- if that offends the denizens of this open source list, well, I'm sorry. As I said before, if somebody wants to go out and build an open source tool in Ruby that does industrial strength load and scalability testing, I'll be very encouraging, at least with "moral support". But it's a big job -- it's not as simple as it looks. It's certainly more than I'd be willing to take on as a one-man open source hacker.

25

khaines@enigo.com wrote:

I realize I sound like a salesman for a couple of rather expensive tools -- if that offends the denizens of this open source list, well, I'm sorry. As I said before, if somebody wants to go out and build an open source tool in Ruby that does industrial strength load and scalability testing, I'll be very encouraging, at least with "moral support". But it's a big job -- it's not as simple as it looks. It's certainly more than I'd be willing to take on as a one-man open source hacker.

I think there is a LARGE amount of room in between ab and an open source, feature complete competitor for SilkPerformer for something that fills a niche and is useful, yet manageable as an open source project.

Yes ... there is indeed. And Ruby seems to me to be the ideal language to create such tools with. There actually wouldn't be a need at all for load test tools if the application designers were enabled and encouraged to build load testing into the application when it was being designed, rather than as an afterthought a few months before deployment, or even worse, after deployment.

So ... we have test-driven development and we have behavior-driven development and we have "software performance engineering". Why not "benchmark-driven development" too? It should be easy, for example, to write something in RSpec like "this web page should display the shopping cart withing five seconds".

···

On Sat, 23 Dec 2006, M. Edward (Ed) Borasky wrote:

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.