I am not able to understand how CSV#convert work?

7stud2 · 16 February 2014 20:50

require 'csv'

str = <<_
name,age,eligible?
arup,27
deep,14
Debu,26,y
Ram,30
Sagar,14

···

_
csv = CSV.new(str,:headers => true)
CSV::Converters[:eligibe] = lambda { |name| name = "Y" unless
name.empty? }
csv.convert('eligible?',&CSV::Converters[:eligibe]) # => [nil]
csv.each { |row| p row }

# ~> /home/kirti/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/csv.rb:2167:in
`block (2 levels) in convert_fields': undefined method `arity' for
nil:NilClass (NoMethodError)

Could any one help me to fix this? But I would expect a fix on this
method only, as I am trying to learn about this method. Basically how
does it work?

Thanks,
Arup

--
Posted via http://www.ruby-forum.com/.

Ammar_Ali · 16 February 2014 22:22

The docs are accurate, even if not very clear. The method can be called with a name *or* a block, but not both.

If a name is given then it should be one of the built-in converters (like :numeric, :date, etc).

With a block you can accept one or two arguments. To convert based on the header name, or index, you want to specify a block with two arguments to get the field info:

csv.convert {|field, info|
  case info[:header]
  when 'eligible?’
    # return converted eligible? value
  when ‘another’
    # return converted field value
  else
    field
  end
}

Regards,
Ammar

···

On Feb 16, 2014, at 10:50 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Could any one help me to fix this? But I would expect a fix on this
method only, as I am trying to learn about this method. Basically how
does it work?

7stud2 · 17 February 2014 20:36

Can any one tell me what wrong I am doing here ?

···

============================================
require 'csv'

str = <<_
name,age,eligible?
arup,27,
deep,14
Debu,26,y
Ram,30
Sagar,14
_

csv = CSV.new(str,:headers => true,:return_headers => true)

CSV.foreach('test.csv') do |row|
p row
end

==========================================

In the output, still no "Y" value present.

# >> ["name", "age", "eligible?"]
# >> ["arup", "27", nil]
# >> ["deep", "14", nil]
# >> ["Debu", "26", "y"]
# >> ["Ram", "30", nil]
# >> ["Sagar", "14", nil]

****************************************

Also I found that, `csv_file.convert do |field, info|..` line of codes
are not getting executed. Why so ?

--
Posted via http://www.ruby-forum.com/.

7stud2 · 18 February 2014 08:39

require 'csv'

str = <<_
name,age,eligible?
arup,27
deep,14
Debu,26,y
Ram,30
Sagar,14

···

_

csv = CSV.new(str,:headers => true,:return_headers => true)
csv.each do |row|
row['eligible?'] = "y" if row['eligible?'].nil?
end

csv.rewind
csv.each { |row| p row.fields }

**output**

["name", "age", "eligible?"]
["arup", "27", nil]
["deep", "14", nil]
["Debu", "26", "y"]
["Ram", "30", nil]
["Sagar", "14", nil]

===============
Again **nil** .... I am getting..very frustrating

--
Posted via http://www.ruby-forum.com/.

7stud2 · 16 February 2014 22:46

Ammar Ali wrote in post #1136869:

If a name is given then it should be one of the built-in converters
(like :numeric, :date, etc).

With a block you can accept one or two arguments. To convert based on
the header name, or index, you want to specify a block with two
arguments to get the field info:

Thanks for the idea. But my code gives all nil, except for 'y' for the
field `eligible?`, for all rows.

require 'csv'

str = <<_
name,age,eligible?
arup,27
deep,14
Debu,26
Ram,30
Sagar,14

···

On Feb 16, 2014, at 10:50 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

_
csv = CSV.new(str,:headers => true,:col_sep => ",")

csv.convert do |field, info|
  #p info
  case info[:header]
  when 'eligible?'
    "Y"
  else
    field
  end
end

csv.each { |row| p row.fields }
# >> ["arup", "27", nil]
# >> ["deep", "14", nil]
# >> ["Debu", "26", nil]
# >> ["Ram", "30", nil]
# >> ["Sagar", "14", nil]

--
Posted via http://www.ruby-forum.com/\.

Ammar_Ali · 17 February 2014 00:05

There are only two columns of data in you sample. If there was at least a final separator, the converter would be called with a nil value, which you can interpret as a false, if that’s what you want.

require 'csv'

str = <<_
name,age,eligible?
arup,27,
deep,14
Debu,26,y
Ram,30
Sagar,14

···

On Feb 17, 2014, at 12:46 AM, Arup Rakshit <lists@ruby-forum.com> wrote:

Thanks for the idea. But my code gives all nil, except for 'y' for the
field `eligible?`, for all rows.

require 'csv'

str = <<_
name,age,eligible?
arup,27
deep,14
Debu,26
Ram,30
Sagar,14
_

_

csv = CSV.new(str,:headers => true)

csv.convert {|field, info|
puts "CALLED"

  case info[:header]
  when 'eligible?'
    field.nil? ? false : (field.downcase[0] == 'y' ? true : false)
  else
    field
  end
}

csv.each { |row| p row.fields }

Good luck,
Ammar

Mike_Stok1 · 17 February 2014 21:38

As you haven't said what you expected it to output this is a guess:

I think that you need to put the converter on the csv object you use to read from the string, so that as each record is read it will be processed by the converter.

For example:

#!/usr/bin/env ruby
require 'csv'

str = <<_
name,age,eligible?
arup,27,
deep,14
Debu,26,y
Ram,30
Sagar,14

···

On Feb 17, 2014, at 3:36 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Can any one tell me what wrong I am doing here ?

============================================
require 'csv'

str = <<_
name,age,eligible?
arup,27,
deep,14
Debu,26,y
Ram,30
Sagar,14
_

csv = CSV.new(str,:headers => true,:return_headers => true)

CSV.open('test.csv','w') do |csv_file|
csv.each do |row|
   csv_file << row
end
csv_file.rewind
csv_file.convert do |field, info|
   case info[:header]
   when 'eligible?'
     "Y"
   else
     field
   end
end
end

CSV.foreach('test.csv') do |row|
p row
end

==========================================

In the output, still no "Y" value present.

# >> ["name", "age", "eligible?"]
# >> ["arup", "27", nil]
# >> ["deep", "14", nil]
# >> ["Debu", "26", "y"]
# >> ["Ram", "30", nil]
# >> ["Sagar", "14", nil]

****************************************

Also I found that, `csv_file.convert do |field, info|..` line of codes
are not getting executed. Why so ?

_

csv = CSV.new(str,
              :headers => true,
              :return_headers => true
             )

csv.convert do |field, info|
  if info.header == 'eligible?'
    'Y'
  else
    field
  end
end

CSV.open('test.csv','w') do |csv_file|
  csv.each do |row|
    csv_file << row
  end
end

CSV.foreach('test.csv') do |row|
p row
end

produces:

~ ∙ ruby ~/tmp/try.rb
["name", "age", "eligible?"]
["arup", "27", "Y"]
["deep", "14", nil]
["Debu", "26", "Y"]
["Ram", "30", nil]
["Sagar", "14", nil]

Both Debu and Arup records had a eligible? field, and when the converter is run for it the value is replaced with a "Y".

Hope this helps,

Mike

--

Mike Stok <mike@stok.ca>
http://www.stok.ca/~mike/

The "`Stok' disclaimers" apply.

7stud2 · 18 February 2014 06:26

Mike Stok wrote in post #1136966:

As you haven't said what you expected it to output this is a guess:

I think that you need to put the converter on the csv object you use to
read from the string, so that as each record is read it will be
processed by the converter.

Ok... Let me try to make it more clear. This is the one, giving me more
trouble. Hope now it would be clear to you guys.

**code**

require 'csv'

str = <<_
name,age,eligible?
arup,27
deep,14
Debu,26,y
Ram,30
Sagar,14

···

On Feb 17, 2014, at 3:36 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

_

csv = CSV.new(str,:headers => true,:return_headers => true)

CSV.foreach('test.csv') do |row|
p row
end
# >> ["name", "age", "eligible?"]
# >> ["arup", "27", " "]
# >> ["deep", "14", " "]
# >> ["Debu", "26", "y"]
# >> ["Ram", "30", " "]
# >> ["Sagar", "14", " "]

But my expected output is :

# >> ["name", "age", "eligible?"]
# >> ["arup", "27","y"]
# >> ["deep", "14", "y"]
# >> ["Debu", "26", "y"]
# >> ["Ram", "30", "y"]
# >> ["Sagar", "14", "y"]

I have tested the below code is not getting executed :

   csv_file.convert do |field, info|
      case info[:header]
      when 'eligible?'
        "Y"
      else
        field
      end
    end

Why so ?

--
Posted via http://www.ruby-forum.com/\.

Ammar_Ali · 18 February 2014 09:31

No, it is not nil. Try this:

csv = CSV.new(str,:headers => true,:return_headers => true)
csv.each do |row|
row['eligible?'] = "y" if row['eligible?'].nil?
p row.fields
end

If you want the changes to persist you need to save them somehow. If you don’t save the changes and call rewind, you need to make the changes again.

Ruby is very advanced, but it does not read minds

Good luck,
Ammar

···

On Feb 18, 2014, at 10:39 AM, Arup Rakshit <lists@ruby-forum.com> wrote:

Again **nil** .... I am getting..very frustrating

7stud2 · 4 March 2014 19:54

Ammar Ali wrote in post #1136869:

With a block you can accept one or two arguments. To convert based on
the header name, or index, you want to specify a block with two
arguments to get the field info:

Why CSV#convert not working here ?

require 'csv'

str = CSV.generate('Hello', :converters => :nonewline, :headers => true
) do |csv|
csv.convert {|s| s.downcase }
end
str # => "Hello"

I expect the output as "hello"

···

On Feb 16, 2014, at 10:50 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

--
Posted via http://www.ruby-forum.com/\.

7stud2 · 17 February 2014 10:03

Ammar Ali wrote in post #1136880:

···

On Feb 17, 2014, at 12:46 AM, Arup Rakshit <lists@ruby-forum.com> wrote:

There are only two columns of data in you sample. If there was at least
a final separator, the converter would be called with a nil value, which
you can interpret as a false, if that's what you want.

Got your point and the below output is also telling the same. In the
output I can see `#<struct CSV::FieldInfo index=2, line=2,
header="eligible?">` only for the first row and third row. This is
because there is a comma separator(,) exist. It means I can apply the
`CSV#convert` method only when there is a data, otherwise or not.

require 'csv'

str = <<_
name,age,eligible?
arup,27,
deep,14
Debu,26,y
Ram,30
Sagar,14
_

csv = CSV.new(str,:headers => true,:return_headers => true)

csv.convert { |field, info|
puts info

  case info[:header]
  when 'eligible?'
    field.nil? ? false : (field.downcase[0] == 'y' ? true : false)
  when 'name','age'
    field
  else
    "y"
  end
}

csv.each { |row| row.fields }

output :

#<struct CSV::FieldInfo index=0, line=2, header="name">
#<struct CSV::FieldInfo index=1, line=2, header="age">
#<struct CSV::FieldInfo index=2, line=2, header="eligible?">
#<struct CSV::FieldInfo index=0, line=3, header="name">
#<struct CSV::FieldInfo index=1, line=3, header="age">
#<struct CSV::FieldInfo index=0, line=4, header="name">
#<struct CSV::FieldInfo index=1, line=4, header="age">
#<struct CSV::FieldInfo index=2, line=4, header="eligible?">
#<struct CSV::FieldInfo index=0, line=5, header="name">
#<struct CSV::FieldInfo index=1, line=5, header="age">
#<struct CSV::FieldInfo index=0, line=6, header="name">
#<struct CSV::FieldInfo index=1, line=6, header="age">

So my actual requirement is something else. But before that I need to
fix this part. As I actually have the data as below :

=====================
name,age,eligible?
arup,27
deep,14
Debu,26,y
Ram,30
Sagar,14

So, on parse time, can I insert the third column value, if it is not
present. It means I am looking for the below :

=====================
name,age,eligible?
arup,27,
deep,14,
Debu,26,y
Ram,30,
Sagar,14,

--
Posted via http://www.ruby-forum.com/\.

Ammar_Ali · 18 February 2014 08:00

Because convert is called during a read not a write. Even if it did, you add the row to csv_file before adding the converter. Don’t bother switching them, it will not be called on write.

It is not clear what you are trying to do. Why read then write then read again? Are you trying to fix bad data? Can that be done at the source instead?

Anyway, you can fix the value while reading, from a string or file, without using convert. For example:

CSV.new(str).each do |row|
# check/change field if needed
row[2] = 'y' unless row[2]

p row
end

Regards,
Ammar

···

On Feb 18, 2014, at 8:26 AM, Arup Rakshit <lists@ruby-forum.com> wrote:

I have tested the below code is not getting executed :

  csv_file.convert do |field, info|
     case info[:header]
     when 'eligible?'
       "Y"
     else
       field
     end
   end

Why so ?

7stud2 · 18 February 2014 14:01

Ammar Ali wrote in post #1137014:

···

On Feb 18, 2014, at 10:39 AM, Arup Rakshit <lists@ruby-forum.com> wrote:

Ruby is very advanced, but it does not read minds

Good luck,
Ammar

@Ammar. Thanks for continuous reply. Actually I am learning and trying
the Ruby's CSV module here in my laptop. Some methods are very easy and
some are very tricky like this one.

For sometime, I am sending this post on hold. Let me try others, hope
there we will meet again..

Thanks again.

--
Posted via http://www.ruby-forum.com/\.

Topic		Replies	Views
Trouble with CSV custom Converter ruby-talk	2	154	16 February 2014
Obtain data from .csv ruby-talk	18	136	26 January 2011
Read thru Csv file and store it in variables ruby-talk	8	124	7 October 2011
Why csv file processing is so slow? ruby-talk	25	1381	4 February 2005
How to find the specific row of CSV file ruby-talk	17	154	3 March 2009

I am not able to understand how CSV#convert work?

===================== name,age,eligible? arup,27 deep,14 Debu,26,y Ram,30 Sagar,14

===================== name,age,eligible? arup,27, deep,14, Debu,26,y Ram,30, Sagar,14,

Related topics

=====================
name,age,eligible?
arup,27
deep,14
Debu,26,y
Ram,30
Sagar,14

=====================
name,age,eligible?
arup,27,
deep,14,
Debu,26,y
Ram,30,
Sagar,14,