Faster_csv vs File+split, why it is not faster?

ruby-talk
1

Hi folks,

Why I'm getting this result? is It due just to this specif problem?

the file has 293858 record, here is some record samples:

"MARCOS, LUIS","547 N LAKE ST","","MUNDELEIN","IL","000000000"
"BALDWIN, T & S","4732 NE 203RD ST","","LAKE FOREST PARK","WA","000000000"
"RYBOLT, C","401 CEDAR DR","","CLINTON","IL","000000000"
"WELDT, KRISTINA","1945 N ORLEANS ST","","MCHENRY","IL","000000000"
.....

CODE

require 'benchmark'

Benchmark.bm do |x|
  x.report do
    FasterCSV.foreach("data_test/match.csv") do |row|
    end
  end
end

Benchmark.bm do |x|
  x.report do
    File.new("data_test/match.csv",'r').each{|line|
       row = line.split("\",\"",-1)
       row[0].gsub!('"','')
       row[a.length-1].gsub!('"','')
    }
  end
end

RESULTS

      user system total real
16.180000 0.740000 16.920000 ( *17.246190*)
      user system total real
  5.830000 0.120000 5.950000 ( *6.028469*)

is this true?

···

--
Pablo Q.

2

Is it true that File.split() is faster than FasterCSV? Yeah, I bet it is. Likely reasons are:

* It's written in C
* It doesn't handle all types of CSV data, so it has less work to do

To give some examples, you split code doesn't parse this valid CSV data:

   no,quotes

Or this:

   "embedded
newlines"

Hope that explains things a bit.

James Edward Gray II

···

On Nov 21, 2008, at 11:55 AM, Pablo Q. wrote:

RESULTS

     user system total real
16.180000 0.740000 16.920000 ( *17.246190*)
     user system total real
5.830000 0.120000 5.950000 ( *6.028469*)

is this true?

3

I thought so...

I'm just comparing a single case of FasterCSV to all the implementation of
the library.

Thank you for your time!

···

2008/11/21 James Gray <james@grayproductions.net>

On Nov 21, 2008, at 11:55 AM, Pablo Q. wrote:

RESULTS

    user system total real
16.180000 0.740000 16.920000 ( *17.246190*)
    user system total real
5.830000 0.120000 5.950000 ( *6.028469*)

is this true?

Is it true that File.split() is faster than FasterCSV? Yeah, I bet it is.
Likely reasons are:

* It's written in C
* It doesn't handle all types of CSV data, so it has less work to do

To give some examples, you split code doesn't parse this valid CSV data:

no,quotes

Or this:

"embedded
newlines"

Hope that explains things a bit.

James Edward Gray II

--
Pablo Q.