Performance comparison between screen scrapers

ruby-talk
1

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

···

--
Posted via http://www.ruby-forum.com/.

2

There was a comparision done on this list some time ago. Search for lib names.

···

On 1/11/07, Conrad Chu <conradchu@conradchu.com> wrote:

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

3

I don't know about ScrAPI or HTree, but I recently blogged an informal benchmark run between Rubyful Soup, Hpricot, and the (still developmental) libxml2 HTML parser binding in Libxml-ruby. It's at:

  http://cloverhead.blogspot.com/2006/12/bit-of-benchmarking.html

···

On Thu, 11 Jan 2007 08:36:44 -0000, Conrad Chu <conradchu@conradchu.com> wrote:

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

--
Ross Bamford - rosco@roscopeco.remove.co.uk

4

Conrad Chu wrote:

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

--
Posted via http://www.ruby-forum.com/\.

I haven't used them all but Hpricot is fast (the parser is written in C
with Ragel), error tolerant and perfect for this task. Take a look at
its website for a guide on how to use it.