HTTP, client cookie and client form handling

(Aleksi) #1

Yannick Turgeon wrote:

Hello,

I've posted this message on february 24th. No reply. I try another time...

I don't know why it wasn't answered, as it's all polite and so...

What my script is doing:
1- It logs in Yahoo website using my user and password
2- It tooks some webpages there and do some process to extract information.
3- It updates a database with this information.

While looking at Ruby doc, I feel at ease to jump in with almost all the
work to be done except the Yahoo login and session management. Perl has a
great module to do this kind of work (LWP::UserAgent) but I suppose Ruby too
can acheive this. How? Is there any module more specific than Net::HTTP or
Net::HTTPS? If not, how am I suppose to use it? I mean, I get Yahoo login
page, extract all hidden parameter and "post" it with my username and
password, then... I don't understand the big picture to how should I manage
sessions in this case. Are cookies used?

Session handling is probably no different from your Perl way of doing it. If that used cookies, by all means use those with Ruby as well.

Net::HTTP is quite versatile, handles proxies for you and so on.

Of these libraries, http://ruby.yi.org/raa/en/library.html#WWW, you might probably want to check

   http://www.acc.umu.se/~r2d2/programming/ruby/webfetcher/

for it's very nice API, although it can't do everything.

   http://sourceforge.net/project/showfiles.php?group_id=65749

as it works with Yahoo to some extent.

Another one which might be of interest is Michael's Mechanize with another nice API.

   http://www.ntecs.de/blog/Blog/WWW-Mechanize.rdoc

  - Aleksi