Strange DRb and fork issues

ruby-talk
1

Hi all,

I'm busy experimenting with DRb and I've hit some strange behaviour
which I can't get seem to find the cause of. I think it might be
something to do with DRb and forks but I'm unsure. The process I'm
trying to achieve is to have a Server process running as a DRb
service. It can spawn client processes (the create_client method)
which also run using a DRb service.

Upon spawning the clients create DRb proxy objects to the server and
register using the register_client method passing through their client
process pid and url. The server then creates DRb proxy objects with
the information it receives and stores it in a hash with the client
pid as the key. We can retrieve an array of client pid using the
client_pids method on the server.

I have encountered the following problem. When I start the server
process and then start the client process independent from the server
process everything seems to work fine. I can retrieve from the server
the array of client pids and everything exits gracefully.

However when I create the client from within the server process using
the create_client method (like it hopefully would normally do) the
server registers the client pid and stores it in the hash but on
subsequent calls to get the array of pids it comes back empty(!) and a
ruby process hangs around afterwards. I've attached some files showing
this behaviour. Both tests in theory should be doing the same thing;
registering a client with a server and then retrieving the array of
client pids.

farrel@nicodemus ~/Desktop $ ruby server.rb -n test_register_client
Loaded suite server
Started
Registered client druby://localhost:41362, 1 clients registered
.
Finished in 0.310428 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
farrel@nicodemus ~/Desktop ps   PID TTY TIME CMD 5404 pts/4 00:00:00 bash 5920 pts/4 00:00:00 ps farrel@nicodemus \~/Desktop ruby server.rb -n test_create_client
Loaded suite server
Started
Registered client druby://localhost:2276, 1 clients registered
F
Finished in 0.315975 seconds.

  1) Failure:
test_create_client(TestServer) [server.rb:68]:
<[5923]> expected but was
<[]>.

1 tests, 1 assertions, 1 failures, 0 errors
farrel@nicodemus ~/Desktop $ ps
  PID TTY TIME CMD
5404 pts/4 00:00:00 bash
5923 pts/4 00:00:00 ruby
5924 pts/4 00:00:00 ps

As you can see the pid array comes back empty despite the server
printing out the message that it has registered the client and for
some reason there's a ruby process still hanging around.

Any DRb experts out there who can help?

Thanks
Farrel

base.rb (377 Bytes)

client.rb (240 Bytes)

server.rb (1.58 KB)

2

Hey Farrel-

  I'll save you a lot of hassle. Go grab the slave gem off rubyforge. It does exactly what you are trying to do. It allows you to fork child objects and it sets up a drb connection and a heartneat between the child and parent so if the parent dies, the children will also die.

http://codeforpeople.com/lib/ruby/slave/slave-1.2.1/README

Cheers-
-- Ezra Zygmuntowicz-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

···

On Apr 29, 2007, at 4:17 AM, Farrel Lifson wrote:

Hi all,

I'm busy experimenting with DRb and I've hit some strange behaviour
which I can't get seem to find the cause of. I think it might be
something to do with DRb and forks but I'm unsure. The process I'm
trying to achieve is to have a Server process running as a DRb
service. It can spawn client processes (the create_client method)
which also run using a DRb service.

<snip>

Any DRb experts out there who can help?

Thanks
Farrel
<base.rb>
<client.rb>
<server.rb>

3

Thanks, I'll check it out, although I would really like to work out
what the problem is with my code. I'll look through the source of
Slave and see if I can find some hints.

Farrel

···

On 29/04/07, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:

I'll save you a lot of hassle. Go grab the slave gem off rubyforge.
It does exactly what you are trying to do. It allows you to fork
child objects and it sets up a drb connection and a heartneat between
the child and parent so if the parent dies, the children will also die.

4

i reccomend trying to using slave.rb if you can - of course, as the author,
i'm biased. it's really tricky connecting parent/child drb processes: the key
is that you cannot carry the drb objects across the fork, you must setup each
new drb object in the child and arrange for the parent to find it. this, as
ezra mentions, exactly what slave does. if you have examle code we can
suggest how to plug slave into it.

regards.

-a

···

On Mon, 30 Apr 2007, Farrel Lifson wrote:

On 29/04/07, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:

I'll save you a lot of hassle. Go grab the slave gem off rubyforge.
It does exactly what you are trying to do. It allows you to fork
child objects and it sets up a drb connection and a heartneat between
the child and parent so if the parent dies, the children will also die.

Thanks, I'll check it out, although I would really like to work out what the
problem is with my code. I'll look through the source of Slave and see if I
can find some hints.

--
be kind whenever possible... it is always possible.
- the dalai lama

5

Hi Ara,

In my code I used fork with a block (taken from server.rb):
def create_client(hostname=nil,port=nil)
    pid = fork do
      @klass.start(hostname,port,self.url)
    end
    Process.detach(pid)
    pid
  end
end

Could that be causing the strange issue I'm having? I assumed that was
a safe way of using fork as the child only runs what's in the block.
My main puzzle is why the server loses it's reference to the client
process after it has registered.

I'm trying to learn some of the low level details of DRb, but
unfortunately the documentation out there is not very detailed.

Farrel

···

On 30/04/07, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

i reccomend trying to using slave.rb if you can - of course, as the author,
i'm biased. it's really tricky connecting parent/child drb processes: the key
is that you cannot carry the drb objects across the fork, you must setup each
new drb object in the child and arrange for the parent to find it. this, as
ezra mentions, exactly what slave does. if you have examle code we can
suggest how to plug slave into it.

6

Hi Ara,

In my code I used fork with a block (taken from server.rb):
def create_client(hostname=nil,port=nil)
  pid = fork do
    @klass.start(hostname,port,self.url)
  end
  Process.detach(pid)
  pid
end

that looks reasonable.

Could that be causing the strange issue I'm having? I assumed that was a
safe way of using fork as the child only runs what's in the block. My main
puzzle is why the server loses it's reference to the client process after it
has registered.

hard to say without seeing the code :wink:

I'm trying to learn some of the low level details of DRb, but unfortunately
the documentation out there is not very detailed.

the samples directory covers nearly everything - do you have that?

cheers.

-a

···

On Mon, 30 Apr 2007, Farrel Lifson wrote:
--
be kind whenever possible... it is always possible.
- the dalai lama

7

Hi Ara,

Thanks for the help, I hope you don't mind me badgering you but from
my time on the list I know you are one of the resident DRb experts so
I'm grabbing the opportunity to get some expert advice here.

I attached the code I'm using in my first email, comprising three
files (base.rb, client.rb and server.rb) with the unit tests
illustrating my problems in server.rb . If it got stripped out let me
know and I can try and resend it. In the meantime I will check out the
DRb samples directorty and see if that helps.

Thanks again,
Farrel

···

On 30/04/07, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Mon, 30 Apr 2007, Farrel Lifson wrote:

>
> Hi Ara,
>
> In my code I used fork with a block (taken from server.rb):
> def create_client(hostname=nil,port=nil)
> pid = fork do
> @klass.start(hostname,port,self.url)
> end
> Process.detach(pid)
> pid
> end

that looks reasonable.

> Could that be causing the strange issue I'm having? I assumed that was a
> safe way of using fork as the child only runs what's in the block. My main
> puzzle is why the server loses it's reference to the client process after it
> has registered.

hard to say without seeing the code :wink:

8

i'll try to look at it soon...

cheers.

-a

···

On Mon, 30 Apr 2007, Farrel Lifson wrote:

Hi Ara,

Thanks for the help, I hope you don't mind me badgering you but from
my time on the list I know you are one of the resident DRb experts so
I'm grabbing the opportunity to get some expert advice here.

I attached the code I'm using in my first email, comprising three
files (base.rb, client.rb and server.rb) with the unit tests
illustrating my problems in server.rb . If it got stripped out let me
know and I can try and resend it. In the meantime I will check out the
DRb samples directorty and see if that helps.

Thanks again,
Farrel

--
be kind whenever possible... it is always possible.
- the dalai lama