Mechanize problem "..absolute URL needed (not "").."

ruby-talk
1

Hello All!
This little (crappy :slight_smile: ) code worked fine for a while and stopped
working. I hope somebody out there can give me a hint of what could be
wrong.

The Site table in the db has url in format "http://www.ruby-forum.com"
(without quotes). When I try to puts the url in irb I see the correct
url and the code still writes values to my txt file but doesnt insert
new values to the db.

The error that I get is this:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
607:in `resolve': absolute URL needed (not "") (ArgumentError)
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/h
ttp/agent.rb:223:in `fetch'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize.r
b:440:in `get'
聽聽聽聽聽聽聽聽from Stats_testDB2.rb:41:in `block (2 levels) in <main>'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:508:in `block in each'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `block in each'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `each'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/suppo
rt/lazy_array.rb:411:in `each'
聽聽聽聽聽聽聽聽from
C:/Ruby200/lib/ruby/gems/2.0.0/gems/dm-core-1.2.1/lib/dm-core/colle
ction.rb:505:in `each'
聽聽聽聽聽聽聽聽from Stats_testDB2.rb:39:in `block in <main>'
聽聽聽聽聽聽聽聽from Stats_testDB2.rb:38:in `open'
聽聽聽聽聽聽聽聽from Stats_testDB2.rb:38:in `<main>'

路路路

---

This is my code:

require 'rubygems'
require 'mechanize'
require 'data_mapper'

DataMapper.setup(:default,'sqlite3:\Dropbox\Ruby\Stats\StatsDB.sqlite')

class Site
聽聽include DataMapper::Resource
聽聽property :id, Serial
聽聽property :description, String
聽聽property :url, String

聽聽#has n, :Statistics
end

class Statistic
聽聽include DataMapper::Resource
聽聽property :id, Serial
聽聽property :date, DateTime
聽聽property :number, Integer
聽聽property :site_id, Integer

聽聽#belongs_to :Site
end
DataMapper.finalize

time=Time.now.strftime("%Y-%m-%d %H:%M")

File.open("Stats.txt","a+") do |vStats|
聽聽Site.all.each do |site|
聽聽聽聽agent = Mechanize.new
聽聽聽聽page = agent.get(site.url)
聽聽聽聽hits = agent.page.search("span.num_hits").map(&:text).map(&:strip)
聽聽聽聽int=hits[0].to_i
聽聽聽聽vStats.puts "#{time} #{site.id} #{int}\n"

聽聽聽聽Statistic.create(
聽聽聽聽:date => time,
聽聽聽聽:number => int,
聽聽聽聽:site_id => site.id
聽聽聽聽)
聽聽end
end

--
Posted via http://www.ruby-forum.com/.

2

Hello All!
This little (crappy :slight_smile: ) code worked fine for a while and stopped
working. I hope somebody out there can give me a hint of what could be
wrong.

The Site table in the db has url in format "http://www.ruby-forum.com"
(without quotes). When I try to puts the url in irb I see the correct
url and the code still writes values to my txt file but doesnt insert
new values to the db.

The error that I get is this:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:
607:in `resolve': absolute URL needed (not "") (ArgumentError)

This line means that the uri you have passed to "get" is the empty string.
You can test this by printing the Site object before calling the url:

require 'rubygems'
require 'mechanize'
require 'data_mapper'

DataMapper.setup(:default,'sqlite3:\Dropbox\Ruby\Stats\StatsDB.sqlite')

class Site
  include DataMapper::Resource
  property :id, Serial
  property :description, String
  property :url, String

  #has n, :Statistics
end

class Statistic
  include DataMapper::Resource
  property :id, Serial
  property :date, DateTime
  property :number, Integer
  property :site_id, Integer

  #belongs_to :Site
end
DataMapper.finalize

time=Time.now.strftime("%Y-%m-%d %H:%M")

File.open("Stats.txt","a+") do |vStats|
  Site.all.each do |site|

       p site # <-------------------------------------------------- add this

    agent = Mechanize.new
    page = agent.get(site.url)
    hits = agent.page.search("span.num_hits").map(&:text).map(&:strip)
    int=hits[0].to_i
    vStats.puts "#{time} #{site.id} #{int}\n"

    Statistic.create(
    :date => time,
    :number => int,
    :site_id => site.id
    )
  end
end

Jesus.

路路路

On Wed, Feb 26, 2014 at 4:30 PM, cristian cristian <lists@ruby-forum.com> wrote:

3

Thank you Jesus, but thats strange because I use the same value from the
web site and write it to the text file and this works fine. Further down
in the code I do this:

vStats.puts "#{time} #{site.id} #{int}\n"

Where I use the variable "int". Also, when I test this in irb I get
this:

irb(main):025:0> Site.all.each do |site|
irb(main):026:1* agent = Mechanize.new
irb(main):027:1> puts site.url
irb(main):028:1> end
http://www.pintamono.com
http://www.pintamono.com
=> [#<Site @id=1 @description="My web" @url="http://www.pintamono.com">,
#<Site @id=2 @description="My web2" @url="http://www.pintamono.com">]

路路路

--
Posted via http://www.ruby-forum.com/.

4

Thank you Jesus, but thats strange because I use the same value from the
web site and write it to the text file and this works fine. Further down
in the code I do this:

vStats.puts "#{time} #{site.id} #{int}\n"

Where I use the variable "int".

Yes, but int is not the URL, and also you are not reaching that line,
because you get an exception first.
Can you try running with the statement I added in the real environment
and check what you get?

Also, when I test this in irb I get
this:

irb(main):025:0> Site.all.each do |site|
irb(main):026:1* agent = Mechanize.new
irb(main):027:1> puts site.url
irb(main):028:1> end
http://www.pintamono.com
http://www.pintamono.com
=> [#<Site @id=1 @description="My web" @url="http://www.pintamono.com">,
#<Site @id=2 @description="My web2" @url="http://www.pintamono.com">]

Yes, looks good, it's strange that it works in irb and not in the
script, but as a sanity check, try running printing the site and the
url before calling agent.get to be sure about what's going on.

Jesus.

路路路

On Thu, Feb 27, 2014 at 9:29 AM, cristian cristian <lists@ruby-forum.com> wrote: