Ruby gsub! problem

ruby-talk
1

Hi all,

I need to escape / in a gsub!

eg

line = line.gsub!(///, '')

Unfortunately I don't know how to escape the / (forward slash) - the data I'm munging has this in it (annoyingly)

Kev

2

Hi all,

I need to escape / in a gsub!

eg

line = line.gsub!(///, '')

line.gsub!(/\//, '')

···

On 30-Sep-05, at 2:50 AM, Kev Jackson wrote:

Unfortunately I don't know how to escape the / (forward slash) - the data I'm munging has this in it (annoyingly)

Kev

!DSPAM:433ce04c757271272720660!

--
Jeremy Tregunna
jtregunna@blurgle.ca

"If debugging is the process of removing bugs, then programming must be the process of putting them in." --Dykstra

3

Kev Jackson wrote:

Hi all,

I need to escape / in a gsub!

eg

line = line.gsub!(///, '')

Unfortunately I don't know how to escape the / (forward slash) - the
data I'm munging has this in it (annoyingly)

Kev

If you use this method, you don't need to escape the slashes:

"09/30/05".sub( %r{(\d\d)/(\d\d)/(\d\d)}, '\3-\1-\2')

"09/30/05".sub( %r!(\d\d)/(\d\d)/(\d\d)!, '\3-\1-\2')

"09/30/05".sub( %r](\d\d)/(\d\d)/(\d\d)], '\3-\1-\2')

4

Nicest way to escape slashes is using a different regexp marker. I.e.

line.gsub!(%r{/}, '')

also be carefull to write either

line.gsub!(/a/, 'b')

or

line = line.gsub(/a/, 'b')

because gsub! does return nil if no substitution occured. E.g.

"123".gsub!(/a/, 'b') # => nil

hope to help,

Brian

···

On 30/09/05, Kev Jackson <kevin.jackson@it.fts-vn.com> wrote:

Hi all,

I need to escape / in a gsub!

eg

line = line.gsub!(///, '')

Unfortunately I don't know how to escape the / (forward slash) - the
data I'm munging has this in it (annoyingly)

Kev

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/

5

Jeremy Tregunna wrote:

Hi all,

I need to escape / in a gsub!

eg

line = line.gsub!(///, '')

line.gsub!(/\//, '')

Unfortunately I don't know how to escape the / (forward slash) - the data I'm munging has this in it (annoyingly)

Ta very much, I knew it was something like that but I couldn't get it quite right

Much appreciated

Kev

···

On 30-Sep-05, at 2:50 AM, Kev Jackson wrote:

6

Another gsub/regex problem

I've got a load of data with newlines (\n) scattered throughout, I want to remove all the newlines from this file, before

filename.each do |line| ... end

I'm playing with various combinations for gsub! and unfortunately I can't strip all of them out

filename.to_s.gsub!(/\n/, '')

removes some of them, the problem is

...ahh

input.to_s.gsub!(/(.*)\n(.*)/,'\1\2')

nope still doesn't work

Any ideas - (i'm looking through the pickaxe1 book but can't find anything) - off to google now

Kev

7

The typical Ruby idiom for this is:

File.foreach(filename) do |line|
     line.chomp!

     # ... processing code goes here ...
end

Is there some reason this doesn't work for you?

James Edward Gray II

···

On Sep 30, 2005, at 2:49 AM, Kev Jackson wrote:

Another gsub/regex problem

I've got a load of data with newlines (\n) scattered throughout, I want to remove all the newlines from this file, before

filename.each do |line| ... end

8

The typical Ruby idiom for this is:

File.foreach(filename) do |line|
    line.chomp!

    # ... processing code goes here ...
end

I didn't ry it to be honest, the real problem was that the data (when exported by another tool), has (\n) line endings at arbitrary places (ie 3 in a typical 'line' of data). So I really wanted to strip them out before resplitting on the real line delimiter ';'. I actually worked it out, I was trying filename.to_s.gsub!(/\n/, ''), but although ruby won't complain about this, it won't operate on it, if instead I spli on my delimiter ';' and then do a line.gsub!(/\n/,'') everything works fine

Thanks for the help

Is the idiom above more for processing multiple files (for each file, for each line do some processing close) kind of thing?

Kev

9

Hi,

I'm guessing the reason it worked after splitting and not before is due to you not using a multiline regexp.
I believe in 1.8 regexp was changed to automatically be multi-line...if I recall correctly.
So it would have to be like gsub(/\n/m,"")

-Justin

···

-----Original Message-----
From: Kev Jackson [mailto:kevin.jackson@it.fts-vn.com]
Sent: Sun 10/2/2005 8:30 PM
To: ruby-talk ML
Subject: Re: ruby gsub! problem

The typical Ruby idiom for this is:

File.foreach(filename) do |line|
    line.chomp!

    # ... processing code goes here ...
end

I didn't ry it to be honest, the real problem was that the data (when
exported by another tool), has (\n) line endings at arbitrary places (ie
3 in a typical 'line' of data). So I really wanted to strip them out
before resplitting on the real line delimiter ';'. I actually worked it
out, I was trying filename.to_s.gsub!(/\n/, ''), but although ruby won't
complain about this, it won't operate on it, if instead I spli on my
delimiter ';' and then do a line.gsub!(/\n/,'') everything works fine

Thanks for the help

Is the idiom above more for processing multiple files (for each file,
for each line do some processing close) kind of thing?

Kev

10

filename.to_s.gsub!(/\n/, '') would replace all newlines in the
filename. That is certainly not what you want. You could do

string = File.read(filename).gsub(/\n/, '')

to get the content of the file without newlines

and

lines = File.read(filename).gsub(/\n/, '').split(";", -1)

to get what you want. The snippet from James is for processing each
line of a file after each other and may be a less memory intensive way
to achieve your goal.

hope to help,

Brian

···

On 03/10/05, Kev Jackson <kevin.jackson@it.fts-vn.com> wrote:

>
> The typical Ruby idiom for this is:
>
> File.foreach(filename) do |line|
> line.chomp!
>
> # ... processing code goes here ...
> end
>
I didn't ry it to be honest, the real problem was that the data (when
exported by another tool), has (\n) line endings at arbitrary places (ie
3 in a typical 'line' of data). So I really wanted to strip them out
before resplitting on the real line delimiter ';'. I actually worked it
out, I was trying filename.to_s.gsub!(/\n/, ''), but although ruby won't
complain about this, it won't operate on it, if instead I spli on my
delimiter ';' and then do a line.gsub!(/\n/,'') everything works fine

Thanks for the help

Is the idiom above more for processing multiple files (for each file,
for each line do some processing close) kind of thing?

Kev

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/

11

I don't think so. The /m option more or less only turns on that .
matches newline, so /\n/ and /\n/m would work alike.

regards,

Brian

···

On 03/10/05, Collins, Justin <collinsj@seattleu.edu> wrote:

Hi,

I'm guessing the reason it worked after splitting and not before is due to you not using a multiline regexp.
I believe in 1.8 regexp was changed to automatically be multi-line...if I recall correctly.
So it would have to be like gsub(/\n/m,"")

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/

12

filename.to_s.gsub!(/\n/, '') would replace all newlines in the
filename. That is certainly not what you want. You could do

string = File.read(filename).gsub(/\n/, '')

to get the content of the file without newlines

and

lines = File.read(filename).gsub(/\n/, '').split(";", -1)

to get what you want. The snippet from James is for processing each
line of a file after each other and may be a less memory intensive way
to achieve your goal.

hope to help,

Here's what I ended up with - love blocks, without them there'd be a lot more resource handling...

File.open("sections.sql", "w") do |output|
File.open("TBLSECTION.dat","r") do |input|
  lines=0
# here I was trying to remove the \n
  input.to_s.gsub!(/\n/, '')
# didn't work so moved into munge_fields
  input.each(";") do |line|
    munge_fields line
    munge_data(line, lines+1)
    p line
    output << line << "\n"
    lines+=1
  end
  p "Total Lines in file : " + lines.to_s
end

both munge_fields and munge_data do the sub! thing to bring the data back to the format I want, and although I considered splitting to get an array of strings then creating a builder to rebuild my output, for now it works fine with simple regexps

Kev

13

> [snip]

Here's what I ended up with - love blocks, without them there'd be a lot
more resource handling...

File.open("sections.sql", "w") do |output|
File.open("TBLSECTION.dat","r") do |input|
  lines=0
# here I was trying to remove the \n
  input.to_s.gsub!(/\n/, '')

Here your create a string from the input and substitute the newlines
against the empty string. You then forget about the string and it gets
garbage collected. Try

content = input.read
content.gsub!(/\n/, '')

and replace the below against

content.gsub!(/;/, ";\n")

and then write the content to a file

output << content

# didn't work so moved into munge_fields
  input.each(";") do |line|
    munge_fields line
    munge_data(line, lines+1)
    p line
    output << line << "\n"
    lines+=1
  end
  p "Total Lines in file : " + lines.to_s
end
end

both munge_fields and munge_data do the sub! thing to bring the data
back to the format I want, and although I considered splitting to get an
array of strings then creating a builder to rebuild my output, for now
it works fine with simple regexps

Kev

Or you do it in a more concise way:

File.open("sections.sql", "w") do |output|
  output << File.read("TBLSECTION.dat").gsub(/\n/, '').gsub(/;/, ";\n")
end

hope to help,

Brian

···

On 03/10/05, Kev Jackson <kevin.jackson@it.fts-vn.com> wrote:

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/