I get the following warning in a script that uses FasterCSV:
/usr/lib/ruby/gems/1.8/gems/fastercsv-1.2.1/lib/faster_csv.rb:830: warning: already initialized constant CSV
How do I remove the warning?
I would have to see the code to say for sure. My initial thoughts are:
* CSV is loaded in the script when FCSV tries to build its compatibility interface
* FCSV won't build a compatibility interface unless you ask it to
* That's the only time FCSV ever touches the CSV namespace
Hope that gives you some new ideas.
James Edward Gray II
···
On Oct 4, 2007, at 12:25 PM, "none <@ruby-lang.org, venkat\""@>(none) wrote:
I get the following warning in a script that uses FasterCSV:
/usr/lib/ruby/gems/1.8/gems/fastercsv-1.2.1/lib/faster_csv.rb:830: warning: already initialized constant CSV
How do I remove the warning?
James Edward Gray II wrote:
/usr/lib/ruby/gems/1.8/gems/fastercsv-1.2.1/lib/faster_csv.rb:830: warning: already initialized constant CSV
I would have to see the code to say for sure. My initial thoughts are:
* CSV is loaded in the script when FCSV tries to build its compatibility interface
* FCSV won't build a compatibility interface unless you ask it to
* That's the only time FCSV ever touches the CSV namespace
I wrote a class called "CSVImport" using the default CSV class. Today, I modified it to work with FasterCSV class by changing the requires as below:
begin
require 'rubygems'
require 'faster_csv'
FasterCSV.build_csv_interface
rescue
require 'csv'
end
# Rest of the code
The above code in a simple test script doesn't generate any warnings. My script works even with FasterCSV.build_csv_interface commented out in the above code fragment.
I have included the statements that call CSV (or FasterCSV) methods below:
@reader = CSV.open(@csv_file, "r", @col_sep)
if has_header
@header = @reader.shift
else
@header =
end
......
@reader.each do |row|
all_count += 1
next if from_row > all_count
# accumulate rows
count += 1
values = values + row
if count == @batch_size || (to_row > 0 && (to_row == all_count))
yield(values, count)
count, values = 0,
end
break if to_row > 0 and all_count == to_row
end
......
There are no references to any CSV class in my code. Do you think any of these is the culprit?
James Edward Gray II wrote:
/usr/lib/ruby/gems/1.8/gems/fastercsv-1.2.1/lib/faster_csv.rb:830: warning: already initialized constant CSV
I would have to see the code to say for sure. My initial thoughts are:
* CSV is loaded in the script when FCSV tries to build its compatibility interface
* FCSV won't build a compatibility interface unless you ask it to
* That's the only time FCSV ever touches the CSV namespaceI wrote a class called "CSVImport" using the default CSV class. Today, I modified it to work with FasterCSV class by changing the requires as below:
begin
require 'rubygems'
require 'faster_csv'
FasterCSV.build_csv_interface
rescue
require 'csv'
end
# Rest of the code
You have a bug in that. A bare rescue will not catch the LoadError we are interested in this case. Therefore, using just the above code, I don't believe CSV would ever be loaded. But we know it is loaded because…
The above code in a simple test script doesn't generate any warnings. My script works even with FasterCSV.build_csv_interface commented out in the above code fragment.
Ah, that's a sign of a problem. Without that line FCSV would not touch CSV and thus it wouldn't exist for you to call methods on.
My leading guess is that your code is loading CSV with some code you didn't show me.
James Edward Gray II
···
On Oct 4, 2007, at 2:55 PM, venkat wrote:
James Edward Gray II wrote:
I wrote a class called "CSVImport" using the default CSV class. Today, I modified it to work with FasterCSV class by changing the requires as below:
begin
require 'rubygems'
require 'faster_csv'
FasterCSV.build_csv_interface
rescue
require 'csv'
end
# Rest of the codeYou have a bug in that. A bare rescue will not catch the LoadError we are interested in this case. Therefore, using just the above code, I don't believe CSV would ever be loaded. But we know it is loaded because…
I changed that to
rescue LoadError
My leading guess is that your code is loading CSV with some code you didn't show me.
I think I found it.
I am using a module called CSVDBUtils:
# start of csvutils.rb
module CSVDBUtils
···
#
end
require 'csvexp'
require 'csvimp'
CSVImport = CSVDBUtils::CSVImport
CSVExport = CSVDBUtils::CSVExport
# end of csvutils.rb
The csvdbutils.rb requires 'csvimp' and 'csvexp' and declares, for ease of use, CSVImport and CSVExport. If I don't do this, users of CSVImport and CSVExport classes have to instatiate with
CSVDBUtils::CSVImport.new
CSVDBUtils::CSVExport.new
# any other classes added later.
I think the reason the warning was generated was that csvexp.rb requires 'csv'.
I think *FasterCSV#build_csv_interface* should include a check/test if it has been done already and not do it if the interface is built. I am not sure, how it can be done though.
In an application with different ruby source files using CSV and / FasterCSV, it would be difficult to make sure build_csv_interface is called only once or that CSV and FasterCSV aren't both used.
I changed the csvutils.rb to following and it doesn't generate any warnings.
# csvutils.rb
begin
require 'rubygems'
require 'faster_csv'
FasterCSV.build_csv_interface
rescue LoadError
require 'csv'
end
require 'csvexp'
require 'csvimp'
....
# rest of the code
Ofcourse, I had to removed requires from both csvimp and csvexp.
Thanks for quick response and I appreciate your help. I am able to reduce the time of import of 3000 rows from 90 seconds to less than 9 seconds using batch import (by submitting in bulk instead of row by row) and FasterCSV.
Inorder to reduce the time further, I think I need to be able to collect the rows in a single call instead of iterating through rows. Is there a fill_array or each_n or similar?
-Venkat
OK, I'm considering adding more implicit compatibility with the hope of making it a true drop-in replacement for CSV. I'll keep that in mind as I do. Thanks for the feedback.
James Edward Gray II
···
On Oct 4, 2007, at 4:25 PM, venkat wrote:
I think *FasterCSV#build_csv_interface* should include a check/test if it has been done already and not do it if the interface is built. I am not sure, how it can be done though.