Using external text file to do search and replace?

ruby-talk
1

I recently started to take an interest in Ruby programming and with
the help of some people and by reading Learn to program by Chris Pine
I made myself a little program that has come to a halt because I don't
know if or how I can use an external file where I store the strings I
want to search and replace in text type files.

This is what I have so far:

txt_files = Dir.glob('**/*.txt').each do |path|
puts path
txts = path.to_s
  file = File.open(txts).readlines.each { |line|
    if line.match(/PROMOS:/)
      then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    end }
  file2=File.open(txts, "w")
  file2.write( file )
end

What I want is to be able to use an external file where I store the
values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
Za-z0-9]+/, 'PROMOn=some_other_text'".
I will need to use RubyScript2Exe because I'm not sure that on some
other machines I will have ruby installed and it's easier to just put
the strings I want to search and replace in an external file that is
located in the same folder as the script or on a predefined path.

Any ideas, hints, improvements and critiques are highly welcome.

2

Alle mercoledì 22 agosto 2007, Dan George ha scritto:

I recently started to take an interest in Ruby programming and with
the help of some people and by reading Learn to program by Chris Pine
I made myself a little program that has come to a halt because I don't
know if or how I can use an external file where I store the strings I
want to search and replace in text type files.

This is what I have so far:

txt_files = Dir.glob('**/*.txt').each do |path|
puts path
txts = path.to_s
  file = File.open(txts).readlines.each { |line|
    if line.match(/PROMOS:/)
      then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    end }
  file2=File.open(txts, "w")
  file2.write( file )
end

What I want is to be able to use an external file where I store the
values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
Za-z0-9]+/, 'PROMOn=some_other_text'".
I will need to use RubyScript2Exe because I'm not sure that on some
other machines I will have ruby installed and it's easier to just put
the strings I want to search and replace in an external file that is
located in the same folder as the script or on a predefined path.

Any ideas, hints, improvements and critiques are highly welcome.

First some comments about your code:
* path is already a string, so calling to_s on it does nothing.
* You can replace the File.open(txts).readlines part with
File.readlines(txts), which is (in my opinion) clearer and doesn't force you
to remember to close the file (which by the way, you don't do).
* When you write to file2, you can use the block form of File.open, which
takes care of closing the file for you.

Here's how what I'd have written:

Dir.glob('**/*.txt').each do |path|
  lines = File.readlines(path)
  if line.match(/PROMOS:/)
    lines.map!{|l| l.gsub(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
  end
  File.open(path, 'w'){|f| f.write lines}
end

As you can see, I've also replaced the each/gsub! combination with map!/gsub,
which, in my opinion makes clearer what you're doing (changing the contents
of the array).

As for storing the search/replacement pairs on a file, I'd use YAML. It's
included in the standard library, so there shouldn't be problems with
RubyScript2Exe. You can get information on yaml for ruby at
http://yaml4r.sourceforge.net/ (look in particular at the cookbook and doc
sections). A simple example could be this:

'PROMO1=[A-za-z0-9]+': 'PROMO1=some_text'
'PROMO2=[A-za-z0-9]+': 'PROMO1=some_other_text'
...

When read into ruby using YAML.load, this would return the following hash:

{
'PROMO1=[A-za-z0-9]+' => 'PROMO1=some_text',
'PROMO2=[A-za-z0-9]+' => 'PROMO1=some_other_text'
}

You could then create regexps using Regexp.new. (Actually, you can also store
the regexps directly in the yaml file, prefixing them with the
string !ruby/regexp, but I think the file is easier to read/write this way).

I hope this helps.

Stefano

3

# First line of file is regular expression; second is
# substitution.
temp = IO.readlines( "substitute" ).map{|s| s.chomp }
reg_exp = Regexp.new( temp[0] )
new_text = temp[1]

Dir[ "**/*.txt" ].each{|path|
  p path
  changed = false
  lines = IO.readlines( path ).map{|s|
    if s =~ /PROMOS:/
      old_s = s.dup
      s.gsub!( reg_exp, new_text )
      changed = true if s != old_s
    end
    s
  }
  # Don't write to file unless there was a change.
  if changed
    File.open( path, "w" ){|f| f.puts lines }
    puts " ---> changed"
  end
}

···

On Aug 22, 6:55 am, Dan George <end...@gmail.com> wrote:

I recently started to take an interest in Ruby programming and with
the help of some people and by reading Learn to program by Chris Pine
I made myself a little program that has come to a halt because I don't
know if or how I can use an external file where I store the strings I
want to search and replace in text type files.

This is what I have so far:

txt_files = Dir.glob('**/*.txt').each do |path|
puts path
txts = path.to_s
  file = File.open(txts).readlines.each { |line|
    if line.match(/PROMOS:/)
      then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    end }
  file2=File.open(txts, "w")
  file2.write( file )
end

What I want is to be able to use an external file where I store the
values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
Za-z0-9]+/, 'PROMOn=some_other_text'".
I will need to use RubyScript2Exe because I'm not sure that on some
other machines I will have ruby installed and it's easier to just put
the strings I want to search and replace in an external file that is
located in the same folder as the script or on a predefined path.

Any ideas, hints, improvements and critiques are highly welcome.

4

Thanks everyone for your reply.

I'm visiting some friends at the moment so I can't try anything yet
but as soon as I get home I'll try your suggestions, even if this will
have to wait a few days. Maybe I can read the cookbook and docs for
YAML till now.

Thank you again!

5

I tried what you said and it shows it works and I get an Exit code: 0
but nothing is modified.

Here's what I have now:

require 'yaml'
promo = File.open('promo.yaml')
yp = YAML::load_documents(promo) do |item|
txt_files = Dir.glob('**/*.txt').each do |path|
puts path
  file = File.open(path).readlines.each { |line|
    if line.match(/PROMOS/)
      then line.gsub!(item['search'], item['sub'])
    end }
  File.open(path, 'w'){|f| f.write file}
end
end

And my YAML file looks like this:

···

On Aug 22, 5:43 pm, Stefano Crocco <stefano.cro...@alice.it> wrote:

Alle mercoledì 22 agosto 2007, Dan George ha scritto:

> I recently started to take an interest in Ruby programming and with
> the help of some people and by reading Learn to program by Chris Pine
> I made myself a little program that has come to a halt because I don't
> know if or how I can use anexternalfile where I store the strings I
> want to search and replace in text type files.

> This is what I have so far:

> txt_files = Dir.glob('**/*.txt').each do |path|
> puts path
> txts = path.to_s
> file = File.open(txts).readlines.each { |line|
> if line.match(/PROMOS:/)
> then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
> end }
> file2=File.open(txts, "w")
> file2.write( file )
> end

> What I want is to be able to use anexternalfile where I store the
> values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
> Za-z0-9]+/, 'PROMOn=some_other_text'".
> I will need to use RubyScript2Exe because I'm not sure that on some
> other machines I will have ruby installed and it's easier to just put
> the strings I want to search and replace in anexternalfile that is
> located in the same folder as the script or on a predefined path.

> Any ideas, hints, improvements and critiques are highly welcome.

First some comments about your code:
* path is already a string, so calling to_s on it does nothing.
* You can replace the File.open(txts).readlines part with
File.readlines(txts), which is (in my opinion) clearer and doesn't force you
to remember to close the file (which by the way, you don't do).
* When you write to file2, you can use the block form of File.open, which
takes care of closing the file for you.

Here's how what I'd have written:

Dir.glob('**/*.txt').each do |path|
  lines = File.readlines(path)
  if line.match(/PROMOS:/)
    lines.map!{|l| l.gsub(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
  end
  File.open(path, 'w'){|f| f.write lines}
end

As you can see, I've also replaced the each/gsub! combination with map!/gsub,
which, in my opinion makes clearer what you're doing (changing the contents
of the array).

As for storing the search/replacement pairs on a file, I'd use YAML. It's
included in the standard library, so there shouldn't be problems with
RubyScript2Exe. You can get information on yaml for ruby athttp://yaml4r.sourceforge.net/(look in particular at the cookbook and doc
sections). A simple example could be this:

'PROMO1=[A-za-z0-9]+': 'PROMO1=some_text'
'PROMO2=[A-za-z0-9]+': 'PROMO1=some_other_text'
...

When read into rubyusingYAML.load, this would return the following hash:

{
'PROMO1=[A-za-z0-9]+' => 'PROMO1=some_text',
'PROMO2=[A-za-z0-9]+' => 'PROMO1=some_other_text'

}

You could then create regexpsusingRegexp.new. (Actually, you can also store
the regexps directly in the yaml file, prefixing them with the
string !ruby/regexp, but I think the file is easier to read/write this way).

I hope this helps.

Stefano

---
search: /PROMO1=[A-Za-z0-9]+/
sub: PROMO1=some_text

Can anyone please tell me what's wrong with it?

6

Alle giovedì 6 settembre 2007, Dan George ha scritto:

I tried what you said and it shows it works and I get an Exit code: 0
but nothing is modified.

Here's what I have now:

require 'yaml'
promo = File.open('promo.yaml')
yp = YAML::load_documents(promo) do |item|
txt_files = Dir.glob('**/*.txt').each do |path|
puts path
file = File.open(path).readlines.each { |line|
if line.match(/PROMOS/)
then line.gsub!(item['search'], item['sub'])
end }
File.open(path, 'w'){|f| f.write file}
end
end

And my YAML file looks like this:

---
search: /PROMO1=[A-Za-z0-9]+/
sub: PROMO1=some_text

Can anyone please tell me what's wrong with it?

The reason your code doesn't work is that the search string is stored in the
YAML file as a string, not as a regexp. To use it as a regexp, you need to
create a regexp from it: remove the delimiting / in the regexp in the YAML
file and replace the call to gsub! with:

  gsub(Regexp.new(item['search']), item['sub'])

and it should work.

Without creating the regexp, gsub! would look for a
literal '/PROMO1=[A-Za-z0-9]+/' in the argument, not for a Regexp. (As I
wrote in my previous post, you can store directly a in the YAML file:

  search: !ruby/regexp /PROMO1=[A-Za-z0-9]+/
).

By the way, you don't need to store the return value of YAML::load_documents
in a variable.

I hope this helps

Stefano

7

Thanks a lot for your help Stefano! It works great!

Cheers :slight_smile:

···

On Sep 6, 8:06 pm, Stefano Crocco <stefano.cro...@alice.it> wrote:

Alle giovedì 6 settembre 2007, Dan George ha scritto:

> I tried what you said and it shows it works and I get an Exit code: 0
> but nothing is modified.

> Here's what I have now:

> require 'yaml'
> promo = File.open('promo.yaml')
> yp = YAML::load_documents(promo) do |item|
> txt_files = Dir.glob('**/*.txt').each do |path|
> puts path
> file = File.open(path).readlines.each { |line|
> if line.match(/PROMOS/)
> then line.gsub!(item['search'], item['sub'])
> end }
> File.open(path, 'w'){|f| f.write file}
> end
> end

> And my YAML file looks like this:

> ---
> search: /PROMO1=[A-Za-z0-9]+/
> sub: PROMO1=some_text

> Can anyone please tell me what's wrong with it?

The reason your code doesn't work is that the search string is stored in the
YAML file as a string, not as a regexp. To use it as a regexp, you need to
create a regexp from it: remove the delimiting / in the regexp in the YAML
file and replace the call to gsub! with:

  gsub(Regexp.new(item['search']), item['sub'])

and it should work.

Without creating the regexp, gsub! would look for a
literal '/PROMO1=[A-Za-z0-9]+/' in the argument, not for a Regexp. (As I
wrote in my previous post, you can store directly a in the YAML file:

  search: !ruby/regexp /PROMO1=[A-Za-z0-9]+/
).

By the way, you don't need to store the return value of YAML::load_documents
in a variable.

I hope this helps

Stefano