Mechanize problem

ruby-talk
1

Hi!
I rarely code but when I try I use Ruby. So this is a beginners problem
:slight_smile:

Im trying to fetch a number from a website and im using mechanize but it
seems to be impossible. Im doing this:

require 'Mechanize'
agent = Mechanize.new
agent.get("http://www.linasmatkasse.se")
agent.page.search('#counter')

This div should contain a value but it doesnt. I get this in irb:

[#<Nokogiri::XML::Element:0x83fe5ff0 name="div"
attributes=[#<Nokogiri::XML::Attr:0x83fde2b4 name="id"
value="counter">]>]

This div contains a number like 26 100 002 when looking at the website.

What is it that im doing wrong?

Any suggestion is appreciated! :slight_smile:

Br
cristian

路路路

--
Posted via http://www.ruby-forum.com/.

2

When you just load the html, that div is empty:

   <div id="counter"></div>

One of the scripts on that page is filling it in.

路路路

On Tue, Dec 11, 2012 at 5:00 PM, cristian cristian <lists@ruby-forum.com> wrote:

Hi!
I rarely code but when I try I use Ruby. So this is a beginners problem
:slight_smile:

Im trying to fetch a number from a website and im using mechanize but it
seems to be impossible. Im doing this:

require 'Mechanize'
agent = Mechanize.new
agent.get("http://www.linasmatkasse.se")
agent.page.search('#counter')

This div should contain a value but it doesnt. I get this in irb:

[#<Nokogiri::XML::Element:0x83fe5ff0 name="div"
attributes=[#<Nokogiri::XML::Attr:0x83fde2b4 name="id"
value="counter">]>]

This div contains a number like 26 100 002 when looking at the website.

What is it that im doing wrong?

Any suggestion is appreciated! :slight_smile:

Br
cristian

--
Posted via http://www.ruby-forum.com/\.

3

The problem is that when you view the page in your browser, your browser
executes some javascript code that sets the value of the counter <div>.
However, when you use mechanize to access the page, mechanize retrieves
a text file containing the lines of javascript code as well as the
html--but mechanize is not a browser, so it cannot execute the
javascript code (nor does mechanize display the colors specified by the
html). All mechanize does is examine the text file for information.

Luckily for you, there are now advanced programs that can retrieve the
text file containing the javascript and html, load the file into a
browser(which executes the javascript), then the program can examine
what is actually displayed in the browser:

require 'watir-webdriver'

b = Watir::Browser.new
b.goto("http://www.linasmatkasse.se")

counter_div = b.div id: 'counter'
puts counter_div.text

--output:--
26 308 814

路路路

--
Posted via http://www.ruby-forum.com/.

4

Thank you both!

Now I understand why my code didnt work and I solved my problem!

Br
cristian

路路路

--
Posted via http://www.ruby-forum.com/.