Read from several .txt files by a pattern and write the matched lines to a new file

ruby-talk
1

Just to be said. I am new to ruby.

This should iterate over an array of files, read them and match the
lines by a given pattern. And print to console, But how to proceed if i
want all of the matched lines to be written to a new file?

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
         f = File.open(element, 'r')
              lines = f.readlines
              lines.each do |line|
                  if line.match(pattern)
                      puts line
                        end
                     end
                     f.close
                end

I have tried different solutions as
File.open('newfile.txt', 'w') do |wirte|
write.puts(line)

but it did not work

Any suggestions? Thanks

···

--
Posted via http://www.ruby-forum.com/.

2

Just to be said. I am new to ruby.

This should iterate over an array of files, read them and match the
lines by a given pattern. And print to console, But how to proceed if i
want all of the matched lines to be written to a new file?

You could invoke the script with an output redirection. That's just
one option, of course.

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
f = File.open(element, 'r')
lines = f.readlines
lines.each do |line|
if line.match(pattern)
puts line
end
end
f.close
end

I have tried different solutions as
File.open('newfile.txt', 'w') do |wirte|
write.puts(line)

but it did not work

What was the issue? Can you share an error message?

Any suggestions? Thanks

Watch your spelling.

Other than that: your task can be solved with other tools pretty easily

$ find . -type f -name Msg.log -exec fgrep failed {} + >newfile.txt

Kind regards

robert

···

On Mon, Jun 4, 2012 at 2:13 PM, Robert Hansen <lists@ruby-forum.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

3

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
         f = File.open(element, 'r')
              lines = f.readlines
              lines.each do |line|
                  if line.match(pattern)
             File.open('newfile.txt', 'w') do |write| <-- Problem
                     write.puts(line)
                        end
                     end
                   end
                     f.close
                end

My problem is where the "if statement" begins. I am sorry to say that my
compiler does not give me any error message, but the "newfile.txt" will
not be created and therefore not written to. This makes no sense to me
since I am a new beginner at Programming/scripting

As for your code,

"$ find . -type f -name Msg.log -exec fgrep failed {} + >newfile.txt"

looks pretty and neat, but I am afraid it is way over my head.

Kind regard, Robert Hansen

···

--
Posted via http://www.ruby-forum.com/.

4

Robert Hansen wrote in post #1063024:

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
         f = File.open(element, 'r')
              lines = f.readlines
              lines.each do |line|
                  if line.match(pattern)
             File.open('newfile.txt', 'w') do |write| <-- Problem
                     write.puts(line)
                        end
                     end
                   end
                     f.close
                end

My problem is where the "if statement" begins. I am sorry to say that my
compiler does not give me any error message, but the "newfile.txt" will
not be created and therefore not written to. This makes no sense to me
since I am a new beginner at Programming/scripting

As for your code,

"$ find . -type f -name Msg.log -exec fgrep failed {} + >newfile.txt"

looks pretty and neat, but I am afraid it is way over my head.

Kind regard, Robert Hansen

Somewhere in the script i changed the working directory, so it was my
mistake, but feel free to suggest better code. Everything will be
appreciated as I am still learning.

Kind regard Robert Hansen

···

--
Posted via http://www.ruby-forum.com/\.

5

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
f = File.open(element, 'r')
lines = f.readlines
lines.each do |line|
if line.match(pattern)
File.open('newfile.txt', 'w') do |write| <-- Problem
write.puts(line)
end
end
end
f.close
end

My problem is where the "if statement" begins. I am sorry to say that my
compiler does not give me any error message, but the "newfile.txt" will
not be created and therefore not written to. This makes no sense to me
since I am a new beginner at Programming/scripting

You open and close the file all the time with 'w' which means 'open
for writing, truncate'. This will lead to an empty file as you
notice. You should open it only once because that is more efficient.
You could also replace 'w' with 'a' (append) but as I said, that is
far less efficient.

As for your code,

"$ find . -type f -name Msg.log -exec fgrep failed {} + >newfile.txt"

looks pretty and neat, but I am afraid it is way over my head.

:slight_smile: Standard shell tools.

"find ." - finds in the directory hierarchy rooted at the current dir
"-type f" selects files
"name Msg.log" selects all items with name "Msg.log"
"-exec ... {} +" executed the given command with multiple file names
"fgrep failed" selects lines which contain the literal string "failed"
">newfile.txt" redirects output to the given file.

Kind regards

robert

···

On Mon, Jun 4, 2012 at 3:32 PM, Robert Hansen <lists@ruby-forum.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

6

It's always better to use the block form to open files. It ensures
that they are closed even if there are exceptions.
To iterate over the lines of a file one at a time I like to use File#foreach:

pattern = /(failed)/m
File.open('newfile.txt', 'w') do |out|
  Dir.glob("**/Msg.log").entries.each do |file|
    next unless File.file?(file)
    File.foreach(file) do |line|
      out.puts(line) if line.match(pattern)
    end
  end
end

Hope this helps,

Jesus.

···

On Mon, Jun 4, 2012 at 4:01 PM, Robert Hansen <lists@ruby-forum.com> wrote:

Robert Hansen wrote in post #1063024:

$files = Array.new

Dir.glob("**/Msg.log").entries.each { |n| $files.push(n) if
File.file?(n) }
puts "\n#{$files.size} files found\n"

pattern = /(failed)/m

$files.each do |element|
f = File.open(element, 'r')
lines = f.readlines
lines.each do |line|
if line.match(pattern)
File.open('newfile.txt', 'w') do |write| <-- Problem
write.puts(line)
end
end
end
f.close
end

My problem is where the "if statement" begins. I am sorry to say that my
compiler does not give me any error message, but the "newfile.txt" will
not be created and therefore not written to. This makes no sense to me
since I am a new beginner at Programming/scripting

As for your code,

"$ find . -type f -name Msg.log -exec fgrep failed {} + >newfile.txt"

looks pretty and neat, but I am afraid it is way over my head.

Kind regard, Robert Hansen

Somewhere in the script i changed the working directory, so it was my
mistake, but feel free to suggest better code. Everything will be
appreciated as I am still learning.