Couldn't understand how to use the option :skip_lines

7stud2 · 23 February 2014 15:30

Here is a CSV data in one of my file has :

"DE","Klasse","Deutsch", "x"
"EN","Class","Carpenter",
"DE","Klasse","Mathe",

···

,

Code I wrote :

require 'csv'

input_file_path = File.expand_path('input.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

option = { :skip_blanks => true,
:quote_char => "\'",
:converters => CSV::Converters[:remove_quotes] = lambda do

field>

             field.to_s.tr('"','')
           end
         }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, option) do |in_row|
    row_to_add = in_row.reject(&:empty?)
    out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]
  end
end

**output** ( also expected )

"EN","Class","Carpenter"
"DE","Klasse","Mathe"

But my question is -

How can I skip lines like **,** using option **:skip_lines** (
Class: CSV (Ruby 2.0.0))

Thanks

--
Posted via http://www.ruby-forum.com/\.

Jesus_Gabriel_y_Gala · 24 February 2014 06:40

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,/

(untested)

Jesus.

···

On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Here is a CSV data in one of my file has :

"DE","Klasse","Deutsch", "x"
"EN","Class","Carpenter",
"DE","Klasse","Mathe",
,

Code I wrote :

require 'csv'

input_file_path = File.expand_path('input.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

option = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => CSV::Converters[:remove_quotes] = lambda do
>field>
             field.to_s.tr('"','')
           end
         }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, option) do |in_row|
    row_to_add = in_row.reject(&:empty?)
    out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]
  end
end

**output** ( also expected )

"EN","Class","Carpenter"
"DE","Klasse","Mathe"

But my question is -

How can I skip lines like **,** using option **:skip_lines** (
Class: CSV (Ruby 2.0.0))

abinoam · 4 March 2014 23:08

Hi Arup,

Jesús has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','') }

options = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => :remove_quotes,
           :skip_lines => /,/
         }

Your approach only works because the assignment

CSV::Converters[:remove_quotes] = lambda do |field|
field.to_s.tr('"','')
end

returns the lambda itself.

And, as stated at the documentation,
"A single converter doesn't have to be in an Array."

So, the fragment ...
           :converters => CSV::Converters[:remove_quotes] = lambda do |field|
             field.to_s.tr('"','')
           end

... acts exactly the same as
           :converters => lambda do |field|
             field.to_s.tr('"','')
           end

Abinoam Jr.

···

On Mon, Feb 24, 2014 at 3:40 AM, Jesús Gabriel y Galán <jgabrielygalan@gmail.com> wrote:

On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Here is a CSV data in one of my file has :

"DE","Klasse","Deutsch", "x"
"EN","Class","Carpenter",
"DE","Klasse","Mathe",
,

Code I wrote :

require 'csv'

input_file_path = File.expand_path('input.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

option = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => CSV::Converters[:remove_quotes] = lambda do
>field>
             field.to_s.tr('"','')
           end
         }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, option) do |in_row|
    row_to_add = in_row.reject(&:empty?)
    out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]
  end
end

**output** ( also expected )

"EN","Class","Carpenter"
"DE","Klasse","Mathe"

But my question is -

How can I skip lines like **,** using option **:skip_lines** (
Class: CSV (Ruby 2.0.0))

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,/

(untested)

Jesus.

abinoam · 4 March 2014 23:36

... more

With :skip_line working you can now strip off the "unless
row_to_add.empty?" because all (/,/) rows will be skipped.

Abinoam Jr.

···

On Tue, Mar 4, 2014 at 8:08 PM, Abinoam Jr. <abinoam@gmail.com> wrote:

Hi Arup,

Jesús has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','') }

options = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => :remove_quotes,
           :skip_lines => /,/
         }

Your approach only works because the assignment

CSV::Converters[:remove_quotes] = lambda do |field|
             field.to_s.tr('"','')
           end

returns the lambda itself.

And, as stated at the documentation,
"A single converter doesn't have to be in an Array."

So, the fragment ...
           :converters => CSV::Converters[:remove_quotes] = lambda do |field|
             field.to_s.tr('"','')
           end

... acts exactly the same as
           :converters => lambda do |field|
             field.to_s.tr('"','')
           end

Abinoam Jr.

On Mon, Feb 24, 2014 at 3:40 AM, Jesús Gabriel y Galán > <jgabrielygalan@gmail.com> wrote:

On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Here is a CSV data in one of my file has :

"DE","Klasse","Deutsch", "x"
"EN","Class","Carpenter",
"DE","Klasse","Mathe",
,

Code I wrote :

require 'csv'

input_file_path = File.expand_path('input.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

option = { :skip_blanks => true,
           :quote_char => "\'",
           :converters => CSV::Converters[:remove_quotes] = lambda do
>field>
             field.to_s.tr('"','')
           end
         }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, option) do |in_row|
    row_to_add = in_row.reject(&:empty?)
    out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]
  end
end

**output** ( also expected )

"EN","Class","Carpenter"
"DE","Klasse","Mathe"

But my question is -

How can I skip lines like **,** using option **:skip_lines** (
http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new\)

The :skip_lines option receives an object that responds to match (for
example a Regex). The lines for which this object returns false are
skipped. For example:

:skip_lines => /,/

(untested)

Jesus.

7stud2 · 4 March 2014 23:34

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
}

Thanks, I got your point. One doubt, Are you suggesting me to use
`{..}` instead of `do..end` ? I don't think so. But if yes, then why so
?

···

--
Posted via http://www.ruby-forum.com/\.

abinoam · 4 March 2014 23:39

Hi Arup,

Although there's some stylistic common pratice, you can use what you prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] __inside__ the options Hash.

I'm suggesting you to do the "right" way.
Set it __outside__ and reference it with its index (Symbol :remove_quotes).

Best regards,
Abinoam Jr.

···

On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
}

Thanks, I got your point. One doubt, Are you suggesting me to use
`{..}` instead of `do..end` ? I don't think so. But if yes, then why so
?

--
Posted via http://www.ruby-forum.com/\.

abinoam · 4 March 2014 23:56

... another thing...

It seems all the Converters harassment you are dealing with it's
because of a malformed CSV file.

You have a line with an extra space
"DE","Klasse","Deutsch", "x" # But it should be
"DE","Klasse","Deutsch","x" # without the space before "x"

If you fix this before parsing as csv, the code get much simpler.

# I called the "fixed" version of the file "input_fixed.csv"

input_file_path = File.expand_path('input_fixed.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

options = { :skip_blanks => true, :skip_lines => /,/ }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, options) do |in_row|
    row_to_add = in_row.compact
    row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
    out_row << row_to_add
  end
end

The last column of the some lines will be yielded as nil, not as an
empty string "" as before.

["EN", "Class", "Carpenter", nil]

So you can simply call Array#compact! on it to have

["EN", "Class", "Carpenter"]

And I improved the regexp and the logic to identify the "x" column a
little bit (to fit my taste).

Best regards,
Abinoam Jr.

···

On Tue, Mar 4, 2014 at 8:39 PM, Abinoam Jr. <abinoam@gmail.com> wrote:

Hi Arup,

Although there's some stylistic common pratice, you can use what you prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] __inside__ the options Hash.

I'm suggesting you to do the "right" way.
Set it __outside__ and reference it with its index (Symbol :remove_quotes).

Best regards,
Abinoam Jr.

On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
}

Thanks, I got your point. One doubt, Are you suggesting me to use
`{..}` instead of `do..end` ? I don't think so. But if yes, then why so
?

--
Posted via http://www.ruby-forum.com/\.

abinoam · 5 March 2014 00:06

Dear Arup,

Well... did you edited your original post at the forum after sending it?

The code at Couldn't understand how to use the option :skip_lines - Ruby - Ruby-Forum and the code at
my mail inbox are differents.

    unless row_to_add.empty?
      row_to_add.last[/\s+x/i] ? out_row.puts(row_to_add[0..-2]) :
out_row.puts(row_to_add)
    end

versus

out_row.puts row_to_add unless row_to_add.empty? or
row_to_add.last[/^\s+x$/]

They do different things.

For this "second option" you could do

out_row << row_to_add unless row_to_add.last =~ /\A\s*x\s*\Z/i

instead of

row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
out_row << row_to_add

Keep Rubying,
Abinoam Jr.

···

On Tue, Mar 4, 2014 at 8:56 PM, Abinoam Jr. <abinoam@gmail.com> wrote:

... another thing...

It seems all the Converters harassment you are dealing with it's
because of a malformed CSV file.

You have a line with an extra space
"DE","Klasse","Deutsch", "x" # But it should be
"DE","Klasse","Deutsch","x" # without the space before "x"

If you fix this before parsing as csv, the code get much simpler.

# I called the "fixed" version of the file "input_fixed.csv"

input_file_path = File.expand_path('input_fixed.csv',File.dirname(__FILE__))
output_file_path = File.expand_path('output.csv',File.dirname(__FILE__))

options = { :skip_blanks => true, :skip_lines => /,/ }

CSV.open(output_file_path,'w',:force_quotes => true) do |out_row|
  CSV.foreach(input_file_path, options) do |in_row|
    row_to_add = in_row.compact
    row_to_add.pop if row_to_add.last =~ /\A\s*x\s*\Z/i
    out_row << row_to_add
  end
end

The last column of the some lines will be yielded as nil, not as an
empty string "" as before.

["EN", "Class", "Carpenter", nil]

So you can simply call Array#compact! on it to have

["EN", "Class", "Carpenter"]

And I improved the regexp and the logic to identify the "x" column a
little bit (to fit my taste).

Best regards,
Abinoam Jr.

On Tue, Mar 4, 2014 at 8:39 PM, Abinoam Jr. <abinoam@gmail.com> wrote:

Hi Arup,

Although there's some stylistic common pratice, you can use what you prefer.

My suggestion is not about that.

You are setting the CSV::Converters[:remove_
quotes] __inside__ the options Hash.

I'm suggesting you to do the "right" way.
Set it __outside__ and reference it with its index (Symbol :remove_quotes).

Best regards,
Abinoam Jr.

On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit <lists@ruby-forum.com> wrote:

Abinoam Jr. wrote in post #1138813:

Hi Arup,

Jess has guided you to the right way!
(I've tested and it worked).

But I would give you some suggestions (not directly related to your
question).

I think the "right" way to deal with the converters is:

CSV::Converters[:remove_quotes] = lambda { |field| field.to_s.tr('"','')
}

Thanks, I got your point. One doubt, Are you suggesting me to use
`{..}` instead of `do..end` ? I don't think so. But if yes, then why so
?

--
Posted via http://www.ruby-forum.com/\.

Topic		Replies	Views
How can skip the lines while using CSV by the option (:skip_lines) ruby-talk	2	181	15 February 2014
How to find the specific row of CSV file ruby-talk	17	154	3 March 2009
Newbie question concerning ruby and mysql ruby-talk	7	98	6 June 2008
I am not able to understand how CSV#convert work? ruby-talk	12	159	4 March 2014
If column header contain regexp, delete column ruby-talk	9	135	22 June 2009

Couldn't understand how to use the option :skip_lines

Related topics