Question about file management [read]

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
  def self.ip(log)
    file = File.open(log)
    file.each_line do |line|
      if line.include?("Ban")
         a = line.split(" ")
         date = a[0]
         ip = a[6]
         puts "date: #{date} IP: #{ip}"
      end
    end
    file.close
  end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

Panagiotis (atmosx) Atmatzidis

email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4

···

--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
   def self.ip(log)
     file = File.open(log)

Ha! There you are not using block form of File.open and you do not have the #close in ensure! :slight_smile:

     file.each_line do |line|
       if line.include?("Ban")
          a = line.split(" ")
          date = a[0]
          ip = a[6]
          puts "date: #{date} IP: #{ip}"
       end
     end
     file.close
   end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
   loop do
     io.each do |line|
       # whatever...
     end

     # v2: pos = io.tell
     sleep 10 # whatever interval is appropriate
     # v1: io.seek(0, IO::SEEK_SET)
     # v2: io.seek(pos, IO::SEEK_CUR)
   end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute position

Kind regards

  robert

···

On 12.12.2009 21:05, Panagiotis Atmatzidis wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

Hello,

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
  def self.ip(log)
    file = File.open(log)

Ha! There you are not using block form of File.open and you do not have the #close in ensure! :slight_smile:

I don't understand what do you mean by 'ensure'.

    file.each_line do |line|
      if line.include?("Ban")
         a = line.split(" ")
         date = a[0]
         ip = a[6]
         puts "date: #{date} IP: #{ip}"
      end
    end
    file.close
  end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
loop do
   io.each do |line|
     # whatever...
   end

   # v2: pos = io.tell
   sleep 10 # whatever interval is appropriate
   # v1: io.seek(0, IO::SEEK_SET)
   # v2: io.seek(pos, IO::SEEK_CUR)
end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute position

Kind regards

  robert

Oh, this solutions, might work but I think that it's a bit overcomplicated for me. I did not used IO and "seek" yet...

I found online [1] a few examples but I did not had the time to test them.

Maybe something like:

file = File.open(path, File::WRONLY|File::APPEND)

Will do.

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[1] File Access

Panagiotis (atmosx) Atmatzidis

email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4
- --
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."

···

On 13 Δεκ 2009, at 1:20 μ.μ., Robert Klemme wrote:

On 12.12.2009 21:05, Panagiotis Atmatzidis wrote:

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
  def self.ip(log)
    file = File.open(log)

Ha! There you are not using block form of File.open and you do not have the #close in ensure! :slight_smile:

I don't understand what do you mean by 'ensure'.

There full syntax you can have with begin end blocks is

begin
rescue
else
ensure
end

See also
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html
http://blog.rubybestpractices.com/posts/rklemme/001-Using_blocks_for_Robustness.html

    file.each_line do |line|
      if line.include?("Ban")
         a = line.split(" ")
         date = a[0]
         ip = a[6]
         puts "date: #{date} IP: #{ip}"
      end
    end
    file.close
  end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
loop do
   io.each do |line|
     # whatever...
   end

   # v2: pos = io.tell
   sleep 10 # whatever interval is appropriate
   # v1: io.seek(0, IO::SEEK_SET)
   # v2: io.seek(pos, IO::SEEK_CUR)
end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute position

Oh, this solutions, might work but I think that it's a bit overcomplicated for me. I did not used IO and "seek" yet...

#seek is not used in the simplest case. What about reading up on #seek and #tell or trying out the code? I don't see where this is complicated. If that's too complicated for you then I either my explanation made it look more complicated that it actually is or you should abandon software development. You'll encounter far more complicated things in the course of it.

I found online [1] a few examples but I did not had the time to test them.

Maybe something like:

file = File.open(path, File::WRONLY|File::APPEND)

Will do.

Will do what?

[1] File Access

I am not sure whether pleac is a proper source to enter the world of ruby. AFAIK their aim is to compare languages - not to present an introductory read of particular languages. There are quite a few tutorials and books out there for Ruby.

Kind regards

  robert

···

On 13.12.2009 14:37, Panagiotis Atmatzidis wrote:

On 13 2009, at 1:20, Robert Klemme wrote:

On 12.12.2009 21:05, Panagiotis Atmatzidis wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

hi,

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
def self.ip(log)
   file = File.open(log)

Ha! There you are not using block form of File.open and you do not have the #close in ensure! :slight_smile:

I don't understand what do you mean by 'ensure'.

There full syntax you can have with begin end blocks is

begin
rescue
else
ensure
end

See also
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html
http://blog.rubybestpractices.com/posts/rklemme/001-Using_blocks_for_Robustness.html

Okay

   file.each_line do |line|
     if line.include?("Ban")
        a = line.split(" ")
        date = a[0]
        ip = a[6]
        puts "date: #{date} IP: #{ip}"
     end
   end
   file.close
end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
loop do
  io.each do |line|
    # whatever...
  end

  # v2: pos = io.tell
  sleep 10 # whatever interval is appropriate
  # v1: io.seek(0, IO::SEEK_SET)
  # v2: io.seek(pos, IO::SEEK_CUR)
end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute position

Oh, this solutions, might work but I think that it's a bit overcomplicated for me. I did not used IO and "seek" yet...

#seek is not used in the simplest case. What about reading up on #seek and #tell or trying out the code? I don't see where this is complicated. If that's too complicated for you then I either my explanation made it look more complicated that it actually is or you should abandon software development. You'll encounter far more complicated things in the course of it.

Given the fact that I'm messing with Ruby less than 8 days, I'll pass that note.

By complicated I meant that, ruby might have a build-in function or gem that already deals with this situation of monitoring etc.

I found online [1] a few examples but I did not had the time to test them.
Maybe something like: file = File.open(path, File::WRONLY|File::APPEND)
Will do.

Will do what?

[1] File Access

I am not sure whether pleac is a proper source to enter the world of ruby. AFAIK their aim is to compare languages - not to present an introductory read of particular languages. There are quite a few tutorials and books out there for Ruby.

Okay, thanks

Kind regards

  robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

Panagiotis (atmosx) Atmatzidis

email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4
- --
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."

···

On 13 Δεκ 2009, at 4:35 μ.μ., Robert Klemme wrote:

On 13.12.2009 14:37, Panagiotis Atmatzidis wrote:

On 13 2009, at 1:20, Robert Klemme wrote:

On 12.12.2009 21:05, Panagiotis Atmatzidis wrote: