Using Hpricot with HTML's

ruby-talk
1

I have been working on this program for class all day and am not getting
anywhere, so I figured I would try to ask you guys on the forum. Here is
my code so far:

#!/usr/bin/ruby
# Lab 19

require 'rubygems'
require 'fileutils'
require 'hpricot'
require 'net/http'

tamper_args = {
    'my_countries' => ',"Germany", "Japan", "United Kingdom", "United
States"',
    'my_vars' => ',+pop',
    'my_yrs' => ',1950,1960,1970,1980,1990,2000',
    'selected_vars' => 'POP',
    'format_selected' => 'csv'
}

host_name =
"http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php"

puts tamper_screen = Net::HTTP.post_form(URI.parse(host_name),
tamper_args)
puts page_parse = Hpricot(tamper_screen.to_s)

puts page_parse.search("pre")

···

***

It is designed to go through the host_name link and give the populations
for the 4 countries listed, with years 1950, 1960.... through 2000, and
I have to get the data in csv format. I am not sure why I am not able to
do my search the pre tag, which is where the data is stored in the html
doc.

I think it has to do with the .to_s for the page_parse variable, but my
program gives me an error when tamper_screen is not a string.

Any help would be greatly appreciated.
--
Posted via http://www.ruby-forum.com/.

2

Hi Rick,

I have been working on this program for class all day and am not getting
anywhere, so I figured I would try to ask you guys on the forum. Here is
my code so far:

#!/usr/bin/ruby
# Lab 19

require 'rubygems'
require 'fileutils'
require 'hpricot'
require 'net/http'

tamper_args = {
    'my_countries' => ',"Germany", "Japan", "United Kingdom", "United
States"',
    'my_vars' => ',+pop',
    'my_yrs' => ',1950,1960,1970,1980,1990,2000',
    'selected_vars' => 'POP',
    'format_selected' => 'csv'
}

host_name =
"http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php"

puts tamper_screen = Net::HTTP.post_form(URI.parse(host_name),
tamper_args)
puts page_parse = Hpricot(tamper_screen.to_s)

puts page_parse.search("pre")

***

It is designed to go through the host_name link and give the populations
for the 4 countries listed, with years 1950, 1960.... through 2000, and
I have to get the data in csv format. I am not sure why I am not able to
do my search the pre tag, which is where the data is stored in the html
doc.

I think it has to do with the .to_s for the page_parse variable, but my
program gives me an error when tamper_screen is not a string.

Try using the body method. post_form returns a response object, and you
need to grab the body of the response:

  require 'rubygems'
  require 'fileutils'
  require 'hpricot'
  require 'net/http'
  
  tamper_args = {
      'my_countries' => ',"Germany", "Japan", "United Kingdom", "United
  States"',
      'my_vars' => ',+pop',
      'my_yrs' => ',1950,1960,1970,1980,1990,2000',
      'selected_vars' => 'POP',
      'format_selected' => 'csv'
  }
  
  host_name = "http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php"
  
  tamper_screen = Net::HTTP.post_form(URI.parse(host_name), tamper_args)
  
  page_parse = Hpricot(tamper_screen.body)
  puts page_parse.search("pre").inner_text

···

On Sun, Nov 23, 2008 at 04:43:20AM +0900, Rick Rick wrote:

--
Aaron Patterson
http://tenderlovemaking.com/

3

Thank you so much!!! I can finally get past that part.

···

--
Posted via http://www.ruby-forum.com/.

4

Just to continue along with this same program, I am now trying to change
my parameters for the tamper_args to allow the user to key in a country
name, and the program automatically changes the country in
'my_countries' to whatever counties the user wants to examine... Is
there a way to allow for this?

···

--
Posted via http://www.ruby-forum.com/.