# Couldn't understand how to use the option :skip\_lines **URL:** \ **Category:** ruby-talk\ **Created:** [23 February 2014 15:30 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209 "2014-02-23T15:30:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![7stud2](https://avatars.discourse-cdn.com/v4/letter/7/9de053/32.png) [@7stud2](https://rubytalk.org/u/7stud2)\ **Post date:** [23 February 2014 15:30 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/1 "2014-02-23T15:30:14Z")
Here is a CSV data in one of my file has : "DE","Klasse","Deutsch", "x" "EN","Class","Carpenter", "DE","Klasse","Mathe", > **···** > > , > > Code I wrote : > > require 'csv' > > input\_file\_path = File.expand\_path('input.csv',File.dirname(\_\_FILE\_\_)) > output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) > > option = { :skip\_blanks =\> true, >            :quote\_char =\> "\'", >            :converters =\> CSV::Converters[:remove\_quotes] = lambda do > > > field\> > >              field.to\_s.tr('"','') >            end >          } > > CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row| >   CSV.foreach(input\_file\_path, option) do |in\_row| >     row\_to\_add = in\_row.reject(&:empty?) >     out\_row.puts row\_to\_add unless row\_to\_add.empty? or > row\_to\_add.last[/^\s+x$/] >   end > end > > \*\*output\*\* ( also expected ) > > "EN","Class","Carpenter" > "DE","Klasse","Mathe" > > But my question is - > > How can I skip lines like \*\*,\*\* using option \*\*:skip\_lines\*\* ( > [Class: CSV (Ruby 2.0.0)](http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new%5C)) > > Thanks > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Jesus\_Gabriel\_y\_Gala](https://avatars.discourse-cdn.com/v4/letter/j/96bed5/32.png) [@Jesus\_Gabriel\_y\_Gala](https://rubytalk.org/u/Jesus_Gabriel_y_Gala)\ **Post date:** [24 February 2014 06:40 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/2 "2014-02-24T06:40:24Z")
The :skip\_lines option receives an object that responds to match (for example a Regex). The lines for which this object returns false are skipped. For example: :skip\_lines =\> /,/ (untested) Jesus. > **···** > > On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit \ wrote: > > > Here is a CSV data in one of my file has : > > > > "DE","Klasse","Deutsch", "x" > > "EN","Class","Carpenter", > > "DE","Klasse","Mathe", > > , > > > > Code I wrote : > > > > require 'csv' > > > > input\_file\_path = File.expand\_path('input.csv',File.dirname(\_\_FILE\_\_)) > > output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) > > > > option = { :skip\_blanks =\> true, > >            :quote\_char =\> "\'", > >            :converters =\> CSV::Converters[:remove\_quotes] = lambda do > > \>field\> > >              field.to\_s.tr('"','') > >            end > >          } > > > > CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row| > >   CSV.foreach(input\_file\_path, option) do |in\_row| > >     row\_to\_add = in\_row.reject(&:empty?) > >     out\_row.puts row\_to\_add unless row\_to\_add.empty? or > > row\_to\_add.last[/^\s+x$/] > >   end > > end > > > > \*\*output\*\* ( also expected ) > > > > "EN","Class","Carpenter" > > "DE","Klasse","Mathe" > > > > But my question is - > > > > How can I skip lines like \*\*,\*\* using option \*\*:skip\_lines\*\* ( > > [Class: CSV (Ruby 2.0.0)](http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new%5C)) ---
**Author:** ![abinoam](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/abinoam/32/1887_2.png) [@abinoam](https://rubytalk.org/u/abinoam)\ **Post date:** [4 March 2014 23:08 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/3 "2014-03-04T23:08:23Z")
Hi Arup, Jesús has guided you to the right way! 😉 (I've tested and it worked). But I would give you some suggestions (not directly related to your question). I think the "right" way to deal with the converters is: CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') } options = { :skip\_blanks =\> true,            :quote\_char =\> "\'",            :converters =\> :remove\_quotes,            :skip\_lines =\> /,/          } Your approach only works because the assignment CSV::Converters[:remove\_quotes] = lambda do |field|              field.to\_s.tr('"','')            end returns the lambda itself. And, as stated at the documentation, "A single converter doesn't have to be in an Array." So, the fragment ...            :converters =\> CSV::Converters[:remove\_quotes] = lambda do |field|              field.to\_s.tr('"','')            end ... acts exactly the same as            :converters =\> lambda do |field|              field.to\_s.tr('"','')            end Abinoam Jr. > **···** > > On Mon, Feb 24, 2014 at 3:40 AM, Jesús Gabriel y Galán \ wrote: > > > On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit \ wrote: > > > > > Here is a CSV data in one of my file has : > > > > > > "DE","Klasse","Deutsch", "x" > > > "EN","Class","Carpenter", > > > "DE","Klasse","Mathe", > > > , > > > > > > Code I wrote : > > > > > > require 'csv' > > > > > > input\_file\_path = File.expand\_path('input.csv',File.dirname(\_\_FILE\_\_)) > > > output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) > > > > > > option = { :skip\_blanks =\> true, > > >            :quote\_char =\> "\'", > > >            :converters =\> CSV::Converters[:remove\_quotes] = lambda do > > > \>field\> > > >              field.to\_s.tr('"','') > > >            end > > >          } > > > > > > CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row| > > >   CSV.foreach(input\_file\_path, option) do |in\_row| > > >     row\_to\_add = in\_row.reject(&:empty?) > > >     out\_row.puts row\_to\_add unless row\_to\_add.empty? or > > > row\_to\_add.last[/^\s+x$/] > > >   end > > > end > > > > > > \*\*output\*\* ( also expected ) > > > > > > "EN","Class","Carpenter" > > > "DE","Klasse","Mathe" > > > > > > But my question is - > > > > > > How can I skip lines like \*\*,\*\* using option \*\*:skip\_lines\*\* ( > > > [Class: CSV (Ruby 2.0.0)](http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new%5C)) > > > > The :skip\_lines option receives an object that responds to match (for > > example a Regex). The lines for which this object returns false are > > skipped. For example: > > > > :skip\_lines =\> /,/ > > > > (untested) > > > > Jesus. ---
**Author:** ![abinoam](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/abinoam/32/1887_2.png) [@abinoam](https://rubytalk.org/u/abinoam)\ **Post date:** [4 March 2014 23:36 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/4 "2014-03-04T23:36:54Z")
... more With :skip\_line working you can now strip off the "unless row\_to\_add.empty?" because all (/,/) rows will be skipped. Abinoam Jr. > **···** > > On Tue, Mar 4, 2014 at 8:08 PM, Abinoam Jr. \ wrote: > > > Hi Arup, > > > > Jesús has guided you to the right way! 😉 > > (I've tested and it worked). > > > > But I would give you some suggestions (not directly related to your question). > > > > I think the "right" way to deal with the converters is: > > > > CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') } > > > > options = { :skip\_blanks =\> true, > >            :quote\_char =\> "\'", > >            :converters =\> :remove\_quotes, > >            :skip\_lines =\> /,/ > >          } > > > > Your approach only works because the assignment > > > > CSV::Converters[:remove\_quotes] = lambda do |field| > >              field.to\_s.tr('"','') > >            end > > > > returns the lambda itself. > > > > And, as stated at the documentation, > > "A single converter doesn't have to be in an Array." > > > > So, the fragment ... > >            :converters =\> CSV::Converters[:remove\_quotes] = lambda do |field| > >              field.to\_s.tr('"','') > >            end > > > > ... acts exactly the same as > >            :converters =\> lambda do |field| > >              field.to\_s.tr('"','') > >            end > > > > Abinoam Jr. > > > > On Mon, Feb 24, 2014 at 3:40 AM, Jesús Gabriel y Galán \> \ wrote: > > > > > On Sun, Feb 23, 2014 at 4:30 PM, Arup Rakshit \ wrote: > > > > > > > Here is a CSV data in one of my file has : > > > > > > > > "DE","Klasse","Deutsch", "x" > > > > "EN","Class","Carpenter", > > > > "DE","Klasse","Mathe", > > > > , > > > > > > > > Code I wrote : > > > > > > > > require 'csv' > > > > > > > > input\_file\_path = File.expand\_path('input.csv',File.dirname(\_\_FILE\_\_)) > > > > output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) > > > > > > > > option = { :skip\_blanks =\> true, > > > >            :quote\_char =\> "\'", > > > >            :converters =\> CSV::Converters[:remove\_quotes] = lambda do > > > > \>field\> > > > >              field.to\_s.tr('"','') > > > >            end > > > >          } > > > > > > > > CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row| > > > >   CSV.foreach(input\_file\_path, option) do |in\_row| > > > >     row\_to\_add = in\_row.reject(&:empty?) > > > >     out\_row.puts row\_to\_add unless row\_to\_add.empty? or > > > > row\_to\_add.last[/^\s+x$/] > > > >   end > > > > end > > > > > > > > \*\*output\*\* ( also expected ) > > > > > > > > "EN","Class","Carpenter" > > > > "DE","Klasse","Mathe" > > > > > > > > But my question is - > > > > > > > > How can I skip lines like \*\*,\*\* using option \*\*:skip\_lines\*\* ( > > > > [http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new\](http://www.ruby-doc.org/stdlib-2.0/libdoc/csv/rdoc/CSV.html#method-c-new%5C)) > > > > > > The :skip\_lines option receives an object that responds to match (for > > > example a Regex). The lines for which this object returns false are > > > skipped. For example: > > > > > > :skip\_lines =\> /,/ > > > > > > (untested) > > > > > > Jesus. ---
**Author:** ![7stud2](https://avatars.discourse-cdn.com/v4/letter/7/9de053/32.png) [@7stud2](https://rubytalk.org/u/7stud2)\ **Post date:** [4 March 2014 23:34 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/5 "2014-03-04T23:34:48Z")
Abinoam Jr. wrote in post #1138813: > Hi Arup, > > Jess has guided you to the right way! 😉 > (I've tested and it worked). > > But I would give you some suggestions (not directly related to your > question). > > I think the "right" way to deal with the converters is: > > CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') > } Thanks, I got your point. One doubt, Are you suggesting me to use `{..}` instead of `do..end` ? I don't think so. But if yes, then why so ? > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![abinoam](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/abinoam/32/1887_2.png) [@abinoam](https://rubytalk.org/u/abinoam)\ **Post date:** [4 March 2014 23:39 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/6 "2014-03-04T23:39:48Z")
Hi Arup, Although there's some stylistic common pratice, you can use what you prefer. My suggestion is not about that. You are setting the CSV::Converters[:remove\_ quotes] \_\_inside\_\_ the options Hash. I'm suggesting you to do the "right" way. Set it \_\_outside\_\_ and reference it with its index (Symbol :remove\_quotes). Best regards, Abinoam Jr. > **···** > > On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit \ wrote: > > > Abinoam Jr. wrote in post #1138813: > > > > > Hi Arup, > > > > > > Jess has guided you to the right way! 😉 > > > (I've tested and it worked). > > > > > > But I would give you some suggestions (not directly related to your > > > question). > > > > > > I think the "right" way to deal with the converters is: > > > > > > CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') > > > } > > > > Thanks, I got your point. One doubt, Are you suggesting me to use > > `{..}` instead of `do..end` ? I don't think so. But if yes, then why so > > ? > > > > -- > > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![abinoam](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/abinoam/32/1887_2.png) [@abinoam](https://rubytalk.org/u/abinoam)\ **Post date:** [4 March 2014 23:56 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/7 "2014-03-04T23:56:24Z")
... another thing... It seems all the Converters harassment you are dealing with it's because of a malformed CSV file. You have a line with an extra space "DE","Klasse","Deutsch", "x" # But it should be "DE","Klasse","Deutsch","x" # without the space before "x" If you fix this before parsing as csv, the code get much simpler. # I called the "fixed" version of the file "input\_fixed.csv" input\_file\_path = File.expand\_path('input\_fixed.csv',File.dirname(\_\_FILE\_\_)) output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) options = { :skip\_blanks =\> true, :skip\_lines =\> /,/ } CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row|   CSV.foreach(input\_file\_path, options) do |in\_row|     row\_to\_add = in\_row.compact     row\_to\_add.pop if row\_to\_add.last =~ /\A\s\*x\s\*\Z/i     out\_row \<\< row\_to\_add   end end The last column of the some lines will be yielded as nil, not as an empty string "" as before. ["EN", "Class", "Carpenter", nil] So you can simply call Array#compact! on it to have ["EN", "Class", "Carpenter"] And I improved the regexp and the logic to identify the "x" column a little bit (to fit my taste). Best regards, Abinoam Jr. > **···** > > On Tue, Mar 4, 2014 at 8:39 PM, Abinoam Jr. \ wrote: > > > Hi Arup, > > > > Although there's some stylistic common pratice, you can use what you prefer. > > > > My suggestion is not about that. > > > > You are setting the CSV::Converters[:remove\_ > > quotes] \_\_inside\_\_ the options Hash. > > > > I'm suggesting you to do the "right" way. > > Set it \_\_outside\_\_ and reference it with its index (Symbol :remove\_quotes). > > > > Best regards, > > Abinoam Jr. > > > > On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit \ wrote: > > > > > Abinoam Jr. wrote in post #1138813: > > > > > > > Hi Arup, > > > > > > > > Jess has guided you to the right way! 😉 > > > > (I've tested and it worked). > > > > > > > > But I would give you some suggestions (not directly related to your > > > > question). > > > > > > > > I think the "right" way to deal with the converters is: > > > > > > > > CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') > > > > } > > > > > > Thanks, I got your point. One doubt, Are you suggesting me to use > > > `{..}` instead of `do..end` ? I don't think so. But if yes, then why so > > > ? > > > > > > -- > > > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![abinoam](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/abinoam/32/1887_2.png) [@abinoam](https://rubytalk.org/u/abinoam)\ **Post date:** [5 March 2014 00:06 UTC](https://rubytalk.org/t/couldnt-understand-how-to-use-the-option-skip-lines/69209/8 "2014-03-05T00:06:25Z")
Dear Arup, Well... did you edited your original post at the forum after sending it? The code at [Couldn't understand how to use the option :skip\_lines - Ruby - Ruby-Forum](https://www.ruby-forum.com/topic/4423224) and the code at my mail inbox are differents.     unless row\_to\_add.empty?       row\_to\_add.last[/\s+x/i] ? out\_row.puts(row\_to\_add[0..-2]) : out\_row.puts(row\_to\_add)     end versus     out\_row.puts row\_to\_add unless row\_to\_add.empty? or row\_to\_add.last[/^\s+x$/] They do different things. For this "second option" you could do     out\_row \<\< row\_to\_add unless row\_to\_add.last =~ /\A\s\*x\s\*\Z/i instead of     row\_to\_add.pop if row\_to\_add.last =~ /\A\s\*x\s\*\Z/i     out\_row \<\< row\_to\_add Keep Rubying, Abinoam Jr. > **···** > > On Tue, Mar 4, 2014 at 8:56 PM, Abinoam Jr. \ wrote: > > > ... another thing... > > > > It seems all the Converters harassment you are dealing with it's > > because of a malformed CSV file. > > > > You have a line with an extra space > > "DE","Klasse","Deutsch", "x" # But it should be > > "DE","Klasse","Deutsch","x" # without the space before "x" > > > > If you fix this before parsing as csv, the code get much simpler. > > > > # I called the "fixed" version of the file "input\_fixed.csv" > > > > input\_file\_path = File.expand\_path('input\_fixed.csv',File.dirname(\_\_FILE\_\_)) > > output\_file\_path = File.expand\_path('output.csv',File.dirname(\_\_FILE\_\_)) > > > > options = { :skip\_blanks =\> true, :skip\_lines =\> /,/ } > > > > CSV.open(output\_file\_path,'w',:force\_quotes =\> true) do |out\_row| > >   CSV.foreach(input\_file\_path, options) do |in\_row| > >     row\_to\_add = in\_row.compact > >     row\_to\_add.pop if row\_to\_add.last =~ /\A\s\*x\s\*\Z/i > >     out\_row \<\< row\_to\_add > >   end > > end > > > > The last column of the some lines will be yielded as nil, not as an > > empty string "" as before. > > > > ["EN", "Class", "Carpenter", nil] > > > > So you can simply call Array#compact! on it to have > > > > ["EN", "Class", "Carpenter"] > > > > And I improved the regexp and the logic to identify the "x" column a > > little bit (to fit my taste). > > > > Best regards, > > Abinoam Jr. > > > > On Tue, Mar 4, 2014 at 8:39 PM, Abinoam Jr. \ wrote: > > > > > Hi Arup, > > > > > > Although there's some stylistic common pratice, you can use what you prefer. > > > > > > My suggestion is not about that. > > > > > > You are setting the CSV::Converters[:remove\_ > > > quotes] \_\_inside\_\_ the options Hash. > > > > > > I'm suggesting you to do the "right" way. > > > Set it \_\_outside\_\_ and reference it with its index (Symbol :remove\_quotes). > > > > > > Best regards, > > > Abinoam Jr. > > > > > > On Tue, Mar 4, 2014 at 8:34 PM, Arup Rakshit \ wrote: > > > > > > > Abinoam Jr. wrote in post #1138813: > > > > > > > > > Hi Arup, > > > > > > > > > > Jess has guided you to the right way! 😉 > > > > > (I've tested and it worked). > > > > > > > > > > But I would give you some suggestions (not directly related to your > > > > > question). > > > > > > > > > > I think the "right" way to deal with the converters is: > > > > > > > > > > CSV::Converters[:remove\_quotes] = lambda { |field| field.to\_s.tr('"','') > > > > > } > > > > > > > > Thanks, I got your point. One doubt, Are you suggesting me to use > > > > `{..}` instead of `do..end` ? I don't think so. But if yes, then why so > > > > ? > > > > > > > > -- > > > > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).