FastCSV fcsv and how to keep imported field format (or how to convert it)

ruby-talk
1

Dear Community,

I am struggling with a small problem with FasterCSV. This is a sample
line from a CSV:
14;"0810065750";"2500";"Baiki";;1/21/2011;Notime;08:30;17:30;08:17;20:35;1;1;;;;3:05:41
AM;09:00;;Y;Y;1;;;12:18;3.09;;

Now, this is the code I use to read the file:

csv_data = FCSV.table(input_file, :col_sep => ';', :skip_blanks => true)

employees_id = csv_data[:emp_no].uniq

employees_id.each do |id|
  employees_records_tmp = csv_data.select { |row| row[:emp_no] == id
}.map { |row| row.to_hash }
  employees_records = employees_records_tmp.sort_by { |row|
Date.strptime(row[:date], "%m/%d/%Y") }

puts employees_records[0][:acno]
puts employees_records[0][:acno].class

And the output:
810065750.0
Float

How can I instruct FasterCSV (FCSV) no to convert this particular
column? Because output should be equal input, no changes. Or should I
try to convert all?

Best regards,
Baiki

···

--
Posted via http://www.ruby-forum.com/.

2

I think you should just shut off FasterCSV's conversion, since it isn't doing what you want. table() is just a simple shortcut:

···

On Jan 23, 2011, at 5:47 AM, Dot Baiki wrote:

I am struggling with a small problem with FasterCSV.

Now, this is the code I use to read the file:

csv_data = FCSV.table(input_file, :col_sep => ';', :skip_blanks => true)

How can I instruct FasterCSV (FCSV) no to convert this particular
column? Because output should be equal input, no changes. Or should I
try to convert all?

    #
    # A shortcut for:
    #
    # FasterCSV.read( path, { :headers => true,
    # :converters => :numeric,
    # :header_converters => :symbol }.merge(options) )
    #
    def self.table(path, options = Hash.new)
      read( path, { :headers => true,
                    :converters => :numeric,
                    :header_converters => :symbol }.merge(options) )
    end

So just call read() yourself and leave out the :converters => :numeric option.

Hope that helps.

James Edward Gray II

3

Hello again,

This way seems to be right, but now somehow the :header_converters =>
:symbol is not working anymore. This is the header line from the CSV
file:

Emp No.;AC-No.;No.;Name;Auto-Assign;Date;Timetable;On duty;Off
duty;Clock In;Clock Out;Normal;Real time;Late;Early;Absent;OT Time;Work
Time;Exception;Must C/In;Must
C/Out;Department;NDays;WeekEnd;Holiday;ATT_Time;NDays_OT;WeekEnd_OT;Holiday_OT

But since I use read() (even if I use header_converters => :symbol) the
header line is not converted. I really don't understand it anymore. This
is now my header line:

["Emp No.", "AC-No.", "No.", "Name", "Auto-Assign", "Date", "Timetable",
"On duty", "Off duty", "Clock In", "Clock Out", "Normal", "Real time",
"Late", "Early", "Absent", "OT Time", "Work Time", "Exception", "Must
C/In", "Must C/Out", "Department", "NDays", "WeekEnd", "Holiday",
"ATT_Time", "NDays_OT", "WeekEnd_OT", "Holiday_OT"]

Is there no way to say during import/conversion that column "Emp No."
shall be untouched (expect the header conversion)?

This is what I use now:
csv_data = FCSV.read(input_file, :col_sep => ';', :skip_blanks => true,
:header_converters => :symbol)

Thanks for your help JEG2

Regards,
Baiki

···

--
Posted via http://www.ruby-forum.com/.

4

Hello again,

Yes! Sweet. And BIG THANKS! And shame on me :slight_smile: Of course it works like
a charm now.

Best regards,
Baiki

···

--
Posted via http://www.ruby-forum.com/.

5

since we're just merging the options, couldn't we just add :converters=>nil ?

like,

  csv_data = FCSV.table(input_file, :col_sep => ';', :skip_blanks =>
true, :converters=>nil)

kind regards -botp

···

On Sun, Jan 23, 2011 at 10:10 PM, James Edward Gray II <james@graysoftinc.com> wrote:

def self.table(path, options = Hash.new)
read( path, { :headers => true,
:converters => :numeric,
:header_converters => :symbol }.merge(options) )
end

So just call read() yourself and leave out the :converters => :numeric option.

6

The bug is that you dropped :headers => true. Add that back and you are all set.

James Edward Gray II

···

On Jan 23, 2011, at 9:21 AM, Dot Baiki wrote:

This way seems to be right, but now somehow the :header_converters =>
:symbol is not working anymore. This is the header line from the CSV
file:

Emp No.;AC-No.;No.;Name;Auto-Assign;Date;Timetable;On duty;Off
duty;Clock In;Clock Out;Normal;Real time;Late;Early;Absent;OT Time;Work
Time;Exception;Must C/In;Must
C/Out;Department;NDays;WeekEnd;Holiday;ATT_Time;NDays_OT;WeekEnd_OT;Holiday_OT

But since I use read() (even if I use header_converters => :symbol) the
header line is not converted. I really don't understand it anymore. This
is now my header line:

["Emp No.", "AC-No.", "No.", "Name", "Auto-Assign", "Date", "Timetable",
"On duty", "Off duty", "Clock In", "Clock Out", "Normal", "Real time",
"Late", "Early", "Absent", "OT Time", "Work Time", "Exception", "Must
C/In", "Must C/Out", "Department", "NDays", "WeekEnd", "Holiday",
"ATT_Time", "NDays_OT", "WeekEnd_OT", "Holiday_OT"]

Is there no way to say during import/conversion that column "Emp No."
shall be untouched (expect the header conversion)?

This is what I use now:
csv_data = FCSV.read(input_file, :col_sep => ';', :skip_blanks => true,
:header_converters => :symbol)

7

True!

James Edward Gray II

···

On Jan 23, 2011, at 8:50 AM, botp wrote:

On Sun, Jan 23, 2011 at 10:10 PM, James Edward Gray II > <james@graysoftinc.com> wrote:

   def self.table(path, options = Hash.new)
     read( path, { :headers => true,
                   :converters => :numeric,
                   :header_converters => :symbol }.merge(options) )
   end

So just call read() yourself and leave out the :converters => :numeric option.

since we're just merging the options, couldn't we just add :converters=>nil ?

like,

csv_data = FCSV.table(input_file, :col_sep => ';', :skip_blanks =>
true, :converters=>nil)