Drb server using file/tail to read file, and update client?

(Christopher Aldridge) #1

Hi!

I'm probably doing something completely stupid which is why I'd like
your suggestions please.

[scenerio]
I'm using the file/tail module on a drb server to tail an httpd access
log(for example).

When a client connects, it tells the server to read the log and then
(the client)reports to stdout each new line that the server finds.
[/scenerio]

The only way I could get this to work, was to listen on a drb object
on the client within a new thread.. Then send the file/tail request to
the drb server, (along with the client connection details)..

After which, the server reads each new line of the logfile and calls
the client object which simply does:
puts line

This works, but I feel like there is (or should be) some way of making
the client persistently accept each new line all within the <client>
initiated transaction.

Any suggestions or am I doing it the only way it can be done?

Thanks all!

(Ryan Leavengood) #2

Christopher Aldridge said:

Any suggestions or am I doing it the only way it can be done?

If I understand your problem correctly, the issue is that you now have to
constantly poll the server to see if there are more lines, when really
what you want is to have the server tell the clients of the new lines.

So your options are either to code the clients so they also have a DRb
server, which the server connects to when outputting lines from the log
file, or possible to use Rinda and a Tuplespace.

Here is an example of the former:

Server:

require 'drb/drb'

class TailServer
  def initialize
    @clients = []
  end

  def register(uri)
    puts "Client registered at: #{uri}"
    @clients << DRbObject.new(nil, uri)
  end

  def sendline(line)
    puts "Sending line: #{line}"
    @clients.each do |c|
      c.addline(line)
    end
  end
end

ts = TailServer.new
DRb.start_service('druby://localhost:3456', ts)

# Obviously the code below needs to be replaced with code
# that actually tails the log file
i = 0
loop do
  sleep(5)
  ts.sendline("This is line #{i}.")
  i+=1
end
DRb.thread.join

Client:

require 'drb/drb'

class TailClient
  def addline(line)
    puts line
  end
end

# nil in the first parameter opens a random port
DRb.start_service(nil, TailClient.new)
server = DRbObject.new(nil, 'druby://localhost:3456')
server.register(DRb.uri)
DRb.thread.join
__END__

Ryan

(Joel VanderWerf) #3

Ryan Leavengood wrote:

Christopher Aldridge said:

Any suggestions or am I doing it the only way it can be done?

If I understand your problem correctly, the issue is that you now have to
constantly poll the server to see if there are more lines, when really
what you want is to have the server tell the clients of the new lines.

So your options are either to code the clients so they also have a DRb
server, which the server connects to when outputting lines from the log
file, or possible to use Rinda and a Tuplespace.

IIRC, the client can also call a method with a block. The block is
undumped, so it remains on the client side, with a proxy on the server
side. When the server's implementation of the method yields to the
block, it calls the code on the client side. DRb is sooo elegant!

An example of that idea is embedded in a little distributed chat server
example that comes with my foxtails project, using distributed observers
wired up to fox gui widgets. But it's probably easier to code this up
from scratch, once you know that DRb supports distributed block yields.

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

(Christopher Aldridge) #4

Thank you for your response..

This was exactly what I did before asking and I feel like it's hacky..
Do you have an example of the latter instead of the former? Perhaps
something with tuplespace/rinda?

Here's the code I've been using in case anyone can use it:

<client defs>
  def Execute(global, system)
    Thread.new do
      begin
        DRb.start_service global['clientConnectionString'], EbdnxClient.new
        DRb.thread.join
      rescue DRb::DRbBadURI
        puts "rescued"
        Vars.new.session(global)
        begin
          DRb.start_service global['clientConnectionString'], EbdnxClient.new
          DRb.thread.join
        rescue DRb::DRbBadURI
          puts "Couldnt bind for monitoring. Exiting."
          exit 0
        end
      end
    end
    sleep 2
    begin
      DRbObject.new(nil, system['connectionString']).Execute(global, system)
      puts "client> server received job. pending status.."
    rescue
      puts "Could not connect to #{system['connectionString']}\ntrying
again in 2 seconds..."
      sleep2
      begin
        DRbObject.new(nil, system['connectionString']).Execute(global, system)
      rescue
        puts "Unable to connect to server #{system['connectionString']}"
        exit 0
      end
    end
    if system['authFailure'] == "true"
      puts "Authentication Failure"
      exit 1
    end
    DRbObject.new(nil, system['connectionString']).Monitor(global,
system) if global['output'] != "quiet"
  end

</client defs>
<server defs>
def Monitor(global, system)
    begin
      ebdnxLogStructure = "whatever.txt"
      if File.exists?("B:/LOGS/" + ebdnxLogStructure) == false
        sleep 7
        if File.exists?("B:/LOGS/" + ebdnxLogStructure) == false
          puts "log file still not found. sleeping 5 and forcing continue."
          sleep 5
        else
          puts "log file found"
        end
      end
      File::Tail::Logfile.open("B:/LOGS/" + ebdnxLogStructure, :rewind =>
5) do |log|
          log.tail { |line|
          if /<\/ebdnx>/.match(line)
            begin
              StatusToClient.new.Update(line, global)
            rescue DRb::DRbConnError
              puts "server> completed."
              break
            end
          else
            StatusToClient.new.Update(line, global)
          end
        }
      end
    rescue
      puts "server> unable to tail log: B:/LOGS/" + ebdnxLogStructure
    end
end
</server defs>

Obviously these are implemented in classes but Id like to see a way of
updating the client without having start a new drb service(on the
client).

Is this possible? Thank you!!

···

On 8/9/05, Ryan Leavengood <mrcode@netrox.net> wrote:

Christopher Aldridge said:
>
> Any suggestions or am I doing it the only way it can be done?

If I understand your problem correctly, the issue is that you now have to
constantly poll the server to see if there are more lines, when really
what you want is to have the server tell the clients of the new lines.

So your options are either to code the clients so they also have a DRb
server, which the server connects to when outputting lines from the log
file, or possible to use Rinda and a Tuplespace.

Here is an example of the former:

Server:

require 'drb/drb'

class TailServer
  def initialize
    @clients = []
  end

  def register(uri)
    puts "Client registered at: #{uri}"
    @clients << DRbObject.new(nil, uri)
  end

  def sendline(line)
    puts "Sending line: #{line}"
    @clients.each do |c|
      c.addline(line)
    end
  end
end

ts = TailServer.new
DRb.start_service('druby://localhost:3456', ts)

# Obviously the code below needs to be replaced with code
# that actually tails the log file
i = 0
loop do
  sleep(5)
  ts.sendline("This is line #{i}.")
  i+=1
end
DRb.thread.join

Client:

require 'drb/drb'

class TailClient
  def addline(line)
    puts line
  end
end

# nil in the first parameter opens a random port
DRb.start_service(nil, TailClient.new)
server = DRbObject.new(nil, 'druby://localhost:3456')
server.register(DRb.uri)
DRb.thread.join
__END__

Ryan

(Joel VanderWerf) #5

Joel VanderWerf wrote:

Ryan Leavengood wrote:

Christopher Aldridge said:

Any suggestions or am I doing it the only way it can be done?

If I understand your problem correctly, the issue is that you now have to
constantly poll the server to see if there are more lines, when really
what you want is to have the server tell the clients of the new lines.

So your options are either to code the clients so they also have a DRb
server, which the server connects to when outputting lines from the log
file, or possible to use Rinda and a Tuplespace.

IIRC, the client can also call a method with a block. The block is
undumped, so it remains on the client side, with a proxy on the server
side. When the server's implementation of the method yields to the
block, it calls the code on the client side. DRb is sooo elegant!

An example of that idea is embedded in a little distributed chat server
example that comes with my foxtails project, using distributed observers
wired up to fox gui widgets. But it's probably easier to code this up
from scratch, once you know that DRb supports distributed block yields.

Here is an example of using callbacks by yielding from the server:

==== svr.rb ====
require 'drb'

$callback = nil

class C
  def foo
    3.times do |i|
      yield i
    end
  end

  def bar(&pr)
    $callback = pr
  end
end

obj = C.new

DRb.start_service('druby://:22337', obj)

begin
  loop do
    $callback.call if $callback
    sleep 1
  end
rescue DRb::DRbConnError
  puts "client is gone"
end

···

=========

==== clnt.rb ====
require 'drb'

DRb.start_service()
obj = DRbObject.new_with_uri('druby://:22337')

obj.foo do |i|
  p i
end

obj.bar do
  puts "Got callback!"
end

puts "press return to quit"
gets

output:

0
1
2
press return to quit
Got callback!
Got callback!
Got callback!
Got callback!

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

(Christopher Aldridge) #6

Got it Joel.

I'll give it a try tomorrow morning.

Thanks again! :slight_smile:

···

On 8/9/05, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

Joel VanderWerf wrote:
> Ryan Leavengood wrote:
>
>>Christopher Aldridge said:
>>
>>
>>>Any suggestions or am I doing it the only way it can be done?
>>
>>
>>If I understand your problem correctly, the issue is that you now have to
>>constantly poll the server to see if there are more lines, when really
>>what you want is to have the server tell the clients of the new lines.
>>
>>So your options are either to code the clients so they also have a DRb
>>server, which the server connects to when outputting lines from the log
>>file, or possible to use Rinda and a Tuplespace.
>
>
> IIRC, the client can also call a method with a block. The block is
> undumped, so it remains on the client side, with a proxy on the server
> side. When the server's implementation of the method yields to the
> block, it calls the code on the client side. DRb is sooo elegant!
>
> An example of that idea is embedded in a little distributed chat server
> example that comes with my foxtails project, using distributed observers
> wired up to fox gui widgets. But it's probably easier to code this up
> from scratch, once you know that DRb supports distributed block yields.
>

Here is an example of using callbacks by yielding from the server:

==== svr.rb ====
require 'drb'

$callback = nil

class C
  def foo
    3.times do |i|
      yield i
    end
  end

  def bar(&pr)
    $callback = pr
  end
end

obj = C.new

DRb.start_service('druby://:22337', obj)

begin
  loop do
    $callback.call if $callback
    sleep 1
  end
rescue DRb::DRbConnError
  puts "client is gone"
end

==== clnt.rb ====
require 'drb'

DRb.start_service()
obj = DRbObject.new_with_uri('druby://:22337')

obj.foo do |i|
  p i
end

obj.bar do
  puts "Got callback!"
end

puts "press return to quit"
gets

output:

0
1
2
press return to quit
Got callback!
Got callback!
Got callback!
Got callback!

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

(Christopher Aldridge) #7

Joel, FYI -- This worked GREAT! :slight_smile: Thank you!!!!!!!!!!

···

On 8/10/05, Christopher Aldridge <caldridge@gmail.com> wrote:

Got it Joel.

I'll give it a try tomorrow morning.

Thanks again! :slight_smile:

On 8/9/05, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:
> Joel VanderWerf wrote:
> > Ryan Leavengood wrote:
> >
> >>Christopher Aldridge said:
> >>
> >>
> >>>Any suggestions or am I doing it the only way it can be done?
> >>
> >>
> >>If I understand your problem correctly, the issue is that you now have to
> >>constantly poll the server to see if there are more lines, when really
> >>what you want is to have the server tell the clients of the new lines.
> >>
> >>So your options are either to code the clients so they also have a DRb
> >>server, which the server connects to when outputting lines from the log
> >>file, or possible to use Rinda and a Tuplespace.
> >
> >
> > IIRC, the client can also call a method with a block. The block is
> > undumped, so it remains on the client side, with a proxy on the server
> > side. When the server's implementation of the method yields to the
> > block, it calls the code on the client side. DRb is sooo elegant!
> >
> > An example of that idea is embedded in a little distributed chat server
> > example that comes with my foxtails project, using distributed observers
> > wired up to fox gui widgets. But it's probably easier to code this up
> > from scratch, once you know that DRb supports distributed block yields.
> >
>
> Here is an example of using callbacks by yielding from the server:
>
> ==== svr.rb ====
> require 'drb'
>
> $callback = nil
>
> class C
> def foo
> 3.times do |i|
> yield i
> end
> end
>
> def bar(&pr)
> $callback = pr
> end
> end
>
> obj = C.new
>
> DRb.start_service('druby://:22337', obj)
>
> begin
> loop do
> $callback.call if $callback
> sleep 1
> end
> rescue DRb::DRbConnError
> puts "client is gone"
> end
> =========
>
> ==== clnt.rb ====
> require 'drb'
>
> DRb.start_service()
> obj = DRbObject.new_with_uri('druby://:22337')
>
> obj.foo do |i|
> p i
> end
>
> obj.bar do
> puts "Got callback!"
> end
>
> puts "press return to quit"
> gets
> ========
>
> output:
>
> 0
> 1
> 2
> press return to quit
> Got callback!
> Got callback!
> Got callback!
> Got callback!
>
> --
> vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
>
>