# Web testing with Ruby

**URL:** <https://rubytalk.org/t/web-testing-with-ruby/16321>\
**Category:** ruby-talk\
**Created:** [7 February 2005 18:41 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321 "2005-02-07T18:41:21Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jason\_Sweat](https://avatars.discourse-cdn.com/v4/letter/j/53a042/32.png) [@Jason\_Sweat](https://rubytalk.org/u/Jason_Sweat)\
**Post date:** [7 February 2005 18:41 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/1 "2005-02-07T18:41:21Z")

</div>

Does anyone have suggestions for projects/libraries to web test code  
(HTMLUnit like)?

I had this project bookmarked from a month or so ago on this list:  
[http://www.xpenguin.biz/download/webunit/index-en.html](http://www.xpenguin.biz/download/webunit/index-en.html)

But about 1/2 of the listed "libraries used" are no longer functioning links.

I also looked at watir [http://wtr.rubyforge.org/](http://wtr.rubyforge.org/)  
which looked nice, but unfortunately requires Internet Explorer via a  
COM interface, and I am looking to run this test on a linux server via  
cron.

Are there some other useful web testing libraries people could point me towards?

Thanks,

Regards,  
Jason  
[http://blog.casey-sweat.us/](http://blog.casey-sweat.us/)

---

<div class="post-metadata">

**Author:** ![James\_G\_Britt](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@James\_G\_Britt](https://rubytalk.org/u/James_G_Britt)\
**Post date:** [7 February 2005 19:20 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/2 "2005-02-07T19:20:35Z")

</div>

Take a look at Michael Neumann's WWW::Mechanize. He has made it very  
easy to code up an app to fetch a page, find pertinent content, follow  
links, and so on.

It's on RubyForge, as part of the Wee project

[http://rubyforge.org/projects/wee](http://rubyforge.org/projects/wee)

James

> **···**
>
> On Tue, 8 Feb 2005 03:41:21 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:
> 
> > Does anyone have suggestions for projects/libraries to web test code  
> > (HTMLUnit like)?

---

<div class="post-metadata">

**Author:** ![Tim\_Sutherland1](https://avatars.discourse-cdn.com/v4/letter/t/8e7dd6/32.png) [@Tim\_Sutherland1](https://rubytalk.org/u/Tim_Sutherland1)\
**Post date:** [8 February 2005 05:55 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/3 "2005-02-08T05:55:09Z")

</div>

Selenium: [http://selenium.thoughtworks.com/](http://selenium.thoughtworks.com/)

Works with IE, Mozilla, Safari.

It uses client-side Javascript to control the browser. There is a Ruby  
interface, or you can specify the tests as a bunch of HTML tables that have  
actions and assertions. (There are also interfaces for Java, .NET and  
Python.)

I haven't used the Ruby interface, but started using the table-based one  
today.

> **···**
>
> In article \<5c2874720502071041543561d3@mail.gmail.com\>, Jason Sweat wrote:
> 
> > Does anyone have suggestions for projects/libraries to web test code  
> > (HTMLUnit like)?
> > 
> > I had this project bookmarked from a month or so ago on this list:  
> > [http://www.xpenguin.biz/download/webunit/index-en.html](http://www.xpenguin.biz/download/webunit/index-en.html)
> > 
> > But about 1/2 of the listed "libraries used" are no longer functioning links.
> > 
> > I also looked at watir [http://wtr.rubyforge.org/](http://wtr.rubyforge.org/)  
> > which looked nice, but unfortunately requires Internet Explorer via a  
> > COM interface, and I am looking to run this test on a linux server via  
> > cron.
> > 
> > Are there some other useful web testing libraries people could point me towards?

---

<div class="post-metadata">

**Author:** ![Bret\_Pettichord1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Bret\_Pettichord1](https://rubytalk.org/u/Bret_Pettichord1)\
**Post date:** [1 March 2005 06:36 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/4 "2005-03-01T06:36:19Z")

</div>

Watir has been made to work with Linux and Wine. Details:  
[http://advogato.org/person/superant/diary.html?start=76](http://advogato.org/person/superant/diary.html?start=76)

> **···**
>
> At 12:41 PM 2/7/2005, Jason Sweat wrote:
> 
> > I also looked at watir [http://wtr.rubyforge.org/](http://wtr.rubyforge.org/)  
> > which looked nice, but unfortunately requires Internet Explorer via a  
> > COM interface, and I am looking to run this test on a linux server via  
> > cron.
> 
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_  
> &nbsp;&nbsp;Bret Pettichord  
> &nbsp;&nbsp;www.pettichord.com

---

<div class="post-metadata">

**Author:** ![Jason\_Sweat](https://avatars.discourse-cdn.com/v4/letter/j/53a042/32.png) [@Jason\_Sweat](https://rubytalk.org/u/Jason_Sweat)\
**Post date:** [7 February 2005 19:41 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/5 "2005-02-07T19:41:44Z")

</div>

Thanks, I will have a look at it.

Regards,  
Jason

> **···**
>
> On Tue, 8 Feb 2005 04:20:35 +0900, James G. Britt \<ruby.talk.list@gmail.com\> wrote:
> 
> > On Tue, 8 Feb 2005 03:41:21 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:  
> > \> Does anyone have suggestions for projects/libraries to web test code  
> > \> (HTMLUnit like)?
> > 
> > Take a look at Michael Neumann's WWW::Mechanize. He has made it very  
> > easy to code up an app to fetch a page, find pertinent content, follow  
> > links, and so on.
> > 
> > It's on RubyForge, as part of the Wee project
> > 
> > [http://rubyforge.org/projects/wee](http://rubyforge.org/projects/wee)

---

<div class="post-metadata">

**Author:** ![Its\_Me](https://avatars.discourse-cdn.com/v4/letter/i/958977/32.png) [@Its\_Me](https://rubytalk.org/u/Its_Me)\
**Post date:** [8 February 2005 15:20 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/6 "2005-02-08T15:20:08Z")

</div>

This looks great. Thanks for the pointer!

> **···**
>
> "Tim Sutherland" \<timsuth@ihug.co.nz\> wrote
> 
> > Selenium: [http://selenium.thoughtworks.com/](http://selenium.thoughtworks.com/)

---

<div class="post-metadata">

**Author:** ![Jason\_Sweat](https://avatars.discourse-cdn.com/v4/letter/j/53a042/32.png) [@Jason\_Sweat](https://rubytalk.org/u/Jason_Sweat)\
**Post date:** [8 February 2005 15:35 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/7 "2005-02-08T15:35:54Z")

</div>

I seem to be running into difficulty with mechanize.

$ ruby --version  
ruby 1.9.0 (2005-02-08) [i686-linux]  
$ gem --version  
0.8.1  
$ gem list --local | grep mech  
mechanize (0.1.0)  
$ cat test.rb  
require 'rubygems'  
require 'mechanize'

agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }  
page = agent.get('[https://www.alcoadirect.com/&#39;\](https://www.alcoadirect.com/&#39;%5C))  
p page.forms

$ ruby test.rb  
I, [2005-02-08T09:30:10.092550 #1947] INFO -- : GET:  
[https://www.alcoadirect.com/](https://www.alcoadirect.com/)  
warning: peer certificate won't be verified in this SSL session  
D, [2005-02-08T09:30:14.305853 #1947] DEBUG -- : request-header: accept =\> \*/\*  
D, [2005-02-08T09:30:14.393482 #1947] DEBUG -- : header: last-modified  
: Tue, 08 Feb 2005 00:02:27 GMT  
D, [2005-02-08T09:30:14.393568 #1947] DEBUG -- : header: content-type  
: text/html  
D, [2005-02-08T09:30:14.393607 #1947] DEBUG -- : header: date : Tue,  
08 Feb 2005 15:30:43 GMT  
D, [2005-02-08T09:30:14.393646 #1947] DEBUG -- : header: server : JRun  
Web Server/3.0  
D, [2005-02-08T09:30:14.393684 #1947] DEBUG -- : header:  
transfer-encoding : chunked  
I, [2005-02-08T09:30:14.554417 #1947] INFO -- : status: 200  
/usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize.rb:115:in  
`parse': undefined method `downcase' for nil:NilClass (NoMethodError)  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize.rb:112:in  
`call'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize/parsing.rb:14:in  
`each\_recursive'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize/parsing.rb:13:in  
`each'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/1.9/rexml/element.rb:916:in `each'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/1.9/rexml/xpath.rb:49:in `each'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/1.9/rexml/element.rb:916:in `each'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize/parsing.rb:13:in  
`each\_recursive'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize/parsing.rb:15:in  
`each\_recursive'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;... 136 levels...  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize/parsing.rb:13:in  
`each\_recursive'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize.rb:212:in  
`parse\_html'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from /usr/lib/ruby/gems/1.9/gems/mechanize-0.1.0/lib/mechanize.rb:164:in  
`forms'  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from test.rb:6

Is the HTML parser perhaps dependant on well formed HTML? Is there a  
more appropriate forum for me to raise this issue? Thanks.

Regards,  
Jason

> **···**
>
> On Tue, 8 Feb 2005 04:20:35 +0900, James G. Britt \<ruby.talk.list@gmail.com\> wrote:
> 
> > On Tue, 8 Feb 2005 03:41:21 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:  
> > \> Does anyone have suggestions for projects/libraries to web test code  
> > \> (HTMLUnit like)?
> > 
> > Take a look at Michael Neumann's WWW::Mechanize. He has made it very  
> > easy to code up an app to fetch a page, find pertinent content, follow  
> > links, and so on.
> > 
> > It's on RubyForge, as part of the Wee project
> > 
> > [http://rubyforge.org/projects/wee](http://rubyforge.org/projects/wee)

---

<div class="post-metadata">

**Author:** ![alex\_f\_il](https://avatars.discourse-cdn.com/v4/letter/a/2bfe46/32.png) [@alex\_f\_il](https://rubytalk.org/u/alex_f_il)\
**Post date:** [26 March 2005 20:49 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/8 "2005-03-26T20:49:47Z")

</div>

Look at SWExplorerAutomation  
([http://home.comcast.net/~furmana/SWIEAutomation.htm](http://home.comcast.net/~furmana/SWIEAutomation.htm)). It allows  
visually generate test scripts.

---

<div class="post-metadata">

**Author:** ![James\_G\_Britt](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@James\_G\_Britt](https://rubytalk.org/u/James_G_Britt)\
**Post date:** [8 February 2005 17:44 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/9 "2005-02-08T17:44:10Z")

</div>

> I seem to be running into difficulty with mechanize.

...

> agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }  
> page = agent.get('[https://www.alcoadirect.com/&#39;\](https://www.alcoadirect.com/&#39;%5C))  
> p page.forms

I wonder if the issue is the use of "https" ?

Can you get Mechanize to work with a non-SSL URL?

James

> **···**
>
> On Wed, 9 Feb 2005 00:35:54 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:

---

<div class="post-metadata">

**Author:** ![Jason\_Sweat](https://avatars.discourse-cdn.com/v4/letter/j/53a042/32.png) [@Jason\_Sweat](https://rubytalk.org/u/Jason_Sweat)\
**Post date:** [8 February 2005 18:08 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/10 "2005-02-08T18:08:25Z")

</div>

I don't think https is the issue:  
After creating page I can do

page.inspect  
=\> "#\<WWW::Page:0xb7a0cad0 @code=\"200\", @watch\_for\_set=nil,  
@uri=#\<URI::HTTPS:0x…fdbd0689c URL:[https://www.alcoadirect.com/&gt;](https://www.alcoadirect.com/&gt;),  
@body=\" \\r\\n\<HTML dir=ltr\>\\r\\n\<HEAD\>\\r\\n\<TITLE\>  
...snip...  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\</TD\>\\r\\n \</TR\>\\r\\n \\r\\n  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\</TABLE\>\\r\\n  
\</TD\>\</TR\>\</TABLE\>\</TD\>\</TR\>\</TBODY\>\</TABLE\>\\r\\n\<hr size=1  
width=760\>\\r\\n\</BODY\>\\r\\n\</HTML\>\\r\\n\", @response=#\<Net::HTTPOK  
200 OK readbody=true\>, @cookies=\>"

It gives me the error when I try the page.forms method. Any other ideas?

Regards,  
Jason

> **···**
>
> On Wed, 9 Feb 2005 02:44:10 +0900, James G. Britt \<ruby.talk.list@gmail.com\> wrote:
> 
> > \> agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }  
> > \> page = agent.get('[https://www.alcoadirect.com/&#39;\](https://www.alcoadirect.com/&#39;%5C))  
> > \> p page.forms
> > 
> > I wonder if the issue is the use of "https" ?
> > 
> > Can you get Mechanize to work with a non-SSL URL?

---

<div class="post-metadata">

**Author:** ![James\_G\_Britt](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@James\_G\_Britt](https://rubytalk.org/u/James_G_Britt)\
**Post date:** [8 February 2005 20:11 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/11 "2005-02-08T20:11:05Z")

</div>

> I don't think https is the issue:  
> After creating page I can do
> 
> page.inspect  
> =\> "#\<WWW::Page:0xb7a0cad0 @code=\"200\", @watch\_for\_set=nil,  
> @uri=#\<URI::HTTPS:0x…fdbd0689c URL:[https://www.alcoadirect.com/&gt;](https://www.alcoadirect.com/&gt;),  
> @body=\" \\r\\n\<HTML dir=ltr\>\\r\\n\<HEAD\>\\r\\n\<TITLE\>  
> ...snip...  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\</TD\>\\r\\n \</TR\>\\r\\n \\r\\n  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\</TABLE\>\\r\\n  
> \</TD\>\</TR\>\</TABLE\>\</TD\>\</TR\>\</TBODY\>\</TABLE\>\\r\\n\<hr size=1  
> width=760\>\\r\\n\</BODY\>\\r\\n\</HTML\>\\r\\n\", @response=#\<Net::HTTPOK  
> 200 OK readbody=true\>, @cookies=\>"
> 
> It gives me the error when I try the page.forms method. Any other ideas?

Now this sounds familiar Ithough don';t tink I ever told Mike of my problems )

mechanize.rb, in the parse\_html method, has ths code:

: @root.each\_recursive {|node|  
: name = node.name.downcase  
: case name

If the node does not have a name attribute, then downcase  
expectorates. I think.

Try changing that to  
: name = node.name.to\_s.downcase

or add a nil? test someplace

James

> **···**
>
> On Wed, 9 Feb 2005 03:08:25 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:
> 
> > Jason

---

<div class="post-metadata">

**Author:** ![Jason\_Sweat](https://avatars.discourse-cdn.com/v4/letter/j/53a042/32.png) [@Jason\_Sweat](https://rubytalk.org/u/Jason_Sweat)\
**Post date:** [8 February 2005 20:43 UTC](https://rubytalk.org/t/web-testing-with-ruby/16321/12 "2005-02-08T20:43:10Z")

</div>

Changing all .downcase to .to\_s.downcase worked for me.

Thanks!

Regards,  
Jason

> **···**
>
> On Wed, 9 Feb 2005 05:11:05 +0900, James G. Britt \<ruby.talk.list@gmail.com\> wrote:
> 
> > On Wed, 9 Feb 2005 03:08:25 +0900, Jason Sweat \<jason.sweat@gmail.com\> wrote:
> 
> > Try changing that to  
> > : name = node.name.to\_s.downcase
