Read and modify logs

ruby-talk
1

I'm trying to read the entries in a logfile so I can enter its contents
in a database, the lines are results I want to record, but in order to
be sure that I don't have duplicates I'd like to be able to remove
entries that I have read. Is there an easy way to do this in Ruby so I
can remove read lines from a file that is open once they've been
captured?

I'm not as worried about the conn_to_db.print line, that is something I
have working ok, so if it looks funny please ignore it.

What I have so far, and its untested as of yet as I am trying to get the
form right, until I am sure its working .

Example:
while running
  sleep 10
  # read file into an array
  f = file.open("mylogfile.log")
  saved = File.open("archive.log", "a")
  results = f.readlines
  # Is there a way to remove everything that's been read here?
  f.close
  results.each_line do |sending|
    conn_to_db.print("data #{sending}")
    # Make sure we keep a backup
    saved.puts("#{sending}") s.flush
  end
  # Close the archive file
  saved.close
end

Thanks for any help!

···

--
Posted via http://www.ruby-forum.com/.

2

There's at least a couple of ways to remove duplicate lines.

1) results.uniq!
2) results.each { |l| b<<l } # b.keys gives you the set of unique lines

Hope that helps,

Jayanth

···

On Tue, Mar 31, 2009 at 9:11 PM, Michael Furmaniuk <mfurmaniuk@yahoo.com>wrote:

I'm trying to read the entries in a logfile so I can enter its contents
in a database, the lines are results I want to record, but in order to
be sure that I don't have duplicates I'd like to be able to remove
entries that I have read. Is there an easy way to do this in Ruby so I
can remove read lines from a file that is open once they've been
captured?

I'm not as worried about the conn_to_db.print line, that is something I
have working ok, so if it looks funny please ignore it.

What I have so far, and its untested as of yet as I am trying to get the
form right, until I am sure its working .

Example:
while running
sleep 10
# read file into an array
f = file.open("mylogfile.log")
saved = File.open("archive.log", "a")
results = f.readlines
# Is there a way to remove everything that's been read here?
f.close
results.each_line do |sending|
   conn_to_db.print("data #{sending}")
   # Make sure we keep a backup
   saved.puts("#{sending}") s.flush
end
# Close the archive file
saved.close
end

Thanks for any help!
--
Posted via http://www.ruby-forum.com/\.

3

Sorry, I misread your question.

One silly way to achieve what you want is to reopen the file for reading.
That would pretty much erase the file. Not sure if that's what you want...

f.close
f = File.open("mylogfile.log","w")
f.close

Though, I don't see why you can't maintain a hash of all the lines instead
of having to remove lines from the log file. Its never a good idea to muck
about with log files.

Jayanth

···

On Thu, Apr 2, 2009 at 10:29 AM, Srijayanth Sridhar <srijayanth@gmail.com>wrote:

There's at least a couple of ways to remove duplicate lines.

1) results.uniq!
2) results.each { |l| b<<l } # b.keys gives you the set of unique lines

Hope that helps,

Jayanth

On Tue, Mar 31, 2009 at 9:11 PM, Michael Furmaniuk <mfurmaniuk@yahoo.com > >wrote:

> I'm trying to read the entries in a logfile so I can enter its contents
> in a database, the lines are results I want to record, but in order to
> be sure that I don't have duplicates I'd like to be able to remove
> entries that I have read. Is there an easy way to do this in Ruby so I
> can remove read lines from a file that is open once they've been
> captured?
>
> I'm not as worried about the conn_to_db.print line, that is something I
> have working ok, so if it looks funny please ignore it.
>
> What I have so far, and its untested as of yet as I am trying to get the
> form right, until I am sure its working .
>
> Example:
> while running
> sleep 10
> # read file into an array
> f = file.open("mylogfile.log")
> saved = File.open("archive.log", "a")
> results = f.readlines
> # Is there a way to remove everything that's been read here?
> f.close
> results.each_line do |sending|
> conn_to_db.print("data #{sending}")
> # Make sure we keep a backup
> saved.puts("#{sending}") s.flush
> end
> # Close the archive file
> saved.close
> end
>
> Thanks for any help!
> --
> Posted via http://www.ruby-forum.com/\.
>
>

4

Srijayanth Sridhar wrote:

One silly way to achieve what you want is to reopen the file for
reading.

I wanted to avoid that as I am using this log file to maintain results
from other scripts and if the timing goes wrong I might close the file,
another opens, then I open it again to erase and lose the latest entry.
Mostly I was wondering if there was a way to do this in one operation.

Though, I don't see why you can't maintain a hash of all the lines
instead
of having to remove lines from the log file. Its never a good idea to
muck
about with log files.

hmmm...I suppose I could read the file and write out ones I recorded
with a notation so I skip those in the future. I'll have to think about
that.

Thanks

···

--
Posted via http://www.ruby-forum.com/\.