DRb Basics

ruby-talk
1

I'm finally getting around to playing with DRb and I have some general questions.

Let's say I would like to share a Hash with a whole slew of computers. Clients should be able to load objects into the Hash and examine the data the Hash contains.

Here's the trivial implementation. First the server:

#!/usr/local/bin/ruby -w

# hash_server.rb

require "drb"

data = {:server => "data"}

DRb.start_service("druby://localhost:61676", data)
DRb.thread.join

__END__

Then the client:

#!/usr/local/bin/ruby -w

require "drb"

DRb.start_service
data = DRbObject.new(nil, "druby://localhost:61676")

### Whatever operations go here... ###
data[:client] = "more data"
data.each { |(from, data)| puts "#{from} => #{data}"}

__END__

Now I'll list some some general questions and hope some guru feels like enlightening me:

1. This is mostly a curiosity, but how does this work, really?

I guess I half expected that each() call not to work because of the block, which can't be marshaled, obviously. Was the Hash marshaled down to the client and each() called there? How often does this transference happen, every method call? I'm trying to decide how up to date my data will be, at any given time, mainly.

Any details you can provide would be helpful. As you can see, I'm still trying to get my head around all of this.

2. How does the use of DRbUndumped change the answer(s) to #1.

3. I *assume* the above is not very (thread) safe. Should I wrap the Hash in class with modify and read methods, then synchronize their work? Again, I want 40 computers to be able to trade data in and out of here all at once.

4. How does Rinda::TupleSpace relate to what I'm trying to do here?

5. I've poked around in the documents on ruby-doc.org, read the Chad Fowler article, and read the short sections in the Pickaxe2. Any good DRb documentation I'm missing?

Thanks much!

James Edward Gray II

2

I know your question is/was about DRb but Roxy will do the block.

http://rubyforge.org/projects/roxy ... Full source available ... only
185 lines of code ...

Undumped functionality doesn't exist yet.
And I haven't done anything like Rinda on top of it yet.

j.

···

On 10/19/05, James Edward Gray II <james@grayproductions.net> wrote:

I'm finally getting around to playing with DRb and I have some
general questions.

Let's say I would like to share a Hash with a whole slew of
computers. Clients should be able to load objects into the Hash and
examine the data the Hash contains.

Here's the trivial implementation. First the server:

#!/usr/local/bin/ruby -w

# hash_server.rb

require "drb"

data = {:server => "data"}

DRb.start_service("druby://localhost:61676", data)
DRb.thread.join

__END__

Then the client:

#!/usr/local/bin/ruby -w

require "drb"

DRb.start_service
data = DRbObject.new(nil, "druby://localhost:61676")

### Whatever operations go here... ###
data[:client] = "more data"
data.each { |(from, data)| puts "#{from} => #{data}"}

__END__

Now I'll list some some general questions and hope some guru feels
like enlightening me:

1. This is mostly a curiosity, but how does this work, really?

I guess I half expected that each() call not to work because of the
block, which can't be marshaled, obviously. Was the Hash marshaled
down to the client and each() called there? How often does this
transference happen, every method call? I'm trying to decide how up
to date my data will be, at any given time, mainly.

Any details you can provide would be helpful. As you can see, I'm
still trying to get my head around all of this.

2. How does the use of DRbUndumped change the answer(s) to #1.

3. I *assume* the above is not very (thread) safe. Should I wrap
the Hash in class with modify and read methods, then synchronize
their work? Again, I want 40 computers to be able to trade data in
and out of here all at once.

4. How does Rinda::TupleSpace relate to what I'm trying to do here?

5. I've poked around in the documents on ruby-doc.org, read the Chad
Fowler article, and read the short sections in the Pickaxe2. Any
good DRb documentation I'm missing?

Thanks much!

James Edward Gray II

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

3

I recently gave a presentation on DrB and Rinda at the St. Louis Ruby
User Group. You can find it at
Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos. It's the
  DistributedRuby.pdf link. Hopefully this will answer some of your
questions.

···

On 10/19/05, James Edward Gray II <james@grayproductions.net> wrote:

I'm finally getting around to playing with DRb and I have some
general questions.

Let's say I would like to share a Hash with a whole slew of
computers. Clients should be able to load objects into the Hash and
examine the data the Hash contains.

Here's the trivial implementation. First the server:

#!/usr/local/bin/ruby -w

# hash_server.rb

require "drb"

data = {:server => "data"}

DRb.start_service("druby://localhost:61676", data)
DRb.thread.join

__END__

Then the client:

#!/usr/local/bin/ruby -w

require "drb"

DRb.start_service
data = DRbObject.new(nil, "druby://localhost:61676")

### Whatever operations go here... ###
data[:client] = "more data"
data.each { |(from, data)| puts "#{from} => #{data}"}

__END__

Now I'll list some some general questions and hope some guru feels
like enlightening me:

1. This is mostly a curiosity, but how does this work, really?

I guess I half expected that each() call not to work because of the
block, which can't be marshaled, obviously. Was the Hash marshaled
down to the client and each() called there? How often does this
transference happen, every method call? I'm trying to decide how up
to date my data will be, at any given time, mainly.

Any details you can provide would be helpful. As you can see, I'm
still trying to get my head around all of this.

2. How does the use of DRbUndumped change the answer(s) to #1.

3. I *assume* the above is not very (thread) safe. Should I wrap
the Hash in class with modify and read methods, then synchronize
their work? Again, I want 40 computers to be able to trade data in
and out of here all at once.

4. How does Rinda::TupleSpace relate to what I'm trying to do here?

5. I've poked around in the documents on ruby-doc.org, read the Chad
Fowler article, and read the short sections in the Pickaxe2. Any
good DRb documentation I'm missing?

Thanks much!

James Edward Gray II

--
R. Mark Volkmann
Partner, Object Computing, Inc.

4

I have managed to answer this question for myself, thanks to:

Answer: Yes, it's not thread safe. Use a Mutex.

James Edward Gray II

···

On Oct 19, 2005, at 4:59 PM, James Edward Gray II wrote:

3. I *assume* the above is not very (thread) safe. Should I wrap the Hash in class with modify and read methods, then synchronize their work? Again, I want 40 computers to be able to trade data in and out of here all at once.

5

1. This is mostly a curiosity, but how does this work, really?

I guess I half expected that each() call not to work because of the block, which can't be marshaled, obviously. Was the Hash marshaled down to the client and each() called there? How often does this transference happen, every method call? I'm trying to decide how up to date my data will be, at any given time, mainly.

I have learned a little now and can sort of answer this, I think. Under normal operation, DRb marshals an Array of method name and arguments over the wire to the target object, then marshals back some results.

Some things change this though, like the use of blocks or DRbUndumped...

2. How does the use of DRbUndumped change the answer(s) to #1.

This causes DRb to send a proxy object who's every call will be forwarded back to the local copy over the wire. This is what makes blocks possible.

5. I've poked around in the documents on ruby-doc.org, read the Chad Fowler article, and read the short sections in the Pickaxe2. Any good DRb documentation I'm missing?

This link is one of the more helpful explanations:

James Edward Gray II

···

On Oct 19, 2005, at 4:59 PM, James Edward Gray II wrote:

6

This is a scary cool system for sharing data. I've been playing with some toy implementations and it blows my mind...

Anybody using this that wouldn't mind sharing an overview of what you're using it for?

James Edward Gray II

···

On Oct 19, 2005, at 4:59 PM, James Edward Gray II wrote:

4. How does Rinda::TupleSpace relate to what I'm trying to do here?

7

Mark Volkmann offered:

I recently gave a presentation on DrB and Rinda at the St. Louis Ruby
User Group. You can find it at
Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos. It's the
DistributedRuby.pdf link. Hopefully this will answer some of your
questions.

"You are not a member of the group stlruby.
If you believe you are a member, Find your membership"

The stlruby group's files aren't publicly accessible.

Cheers,
Dave

8

Just to be clear, DRb *does* do the block. I'm just trying to figure out how... :wink:

James Edward Gray II

···

On Oct 19, 2005, at 6:32 PM, Jeff Wood wrote:

I know your question is/was about DRb but Roxy will do the block.

9

This was tremendously helpful to my studies! Thank you so much.

Are you using DRb for something? Can I ask what for?

James Edward Gray II

···

On Oct 19, 2005, at 8:32 PM, Mark Volkmann wrote:

I recently gave a presentation on DrB and Rinda at the St. Louis Ruby
User Group. You can find it at
Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos. It's the
    DistributedRuby.pdf link. Hopefully this will answer some of your
questions.

10

I use TupleSpace and RingServer in multi-system monitoring application with self-configuring client daemons. It will be released once I get it robust enough, maybe in a week.

The server starts up and creates a TupleSpace and announces it via a RingServer.

Monitoring daemons look for a RingServer and attach to the TupleSpace. They register themselves with the server and wait on a Stream (implemented on top of the TupleSpace) for requests. When they get a request, they run the indicated test and drop the result back on an output Stream.

The server reads the result Stream and puts it in the database.

This makes the monitoring daemons incredibly small and flexible. They have no configuration and no state. All the hard work is done on the server.

···

On Oct 21, 2005, at 4:45 PM, James Edward Gray II wrote:

On Oct 19, 2005, at 4:59 PM, James Edward Gray II wrote:

4. How does Rinda::TupleSpace relate to what I'm trying to do here?

This is a scary cool system for sharing data. I've been playing with some toy implementations and it blows my mind...

Anybody using this that wouldn't mind sharing an overview of what you're using it for?

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

11

You can click Home, join the group, and then get the file.

James Edward Gray II

···

On Oct 20, 2005, at 1:11 AM, Dave Burt wrote:

"You are not a member of the group stlruby.
If you believe you are a member, Find your membership"

The stlruby group's files aren't publicly accessible.

12

My bad ... I could have sworn I'd read in the docs that blocks were
verboten.

j.

···

On 10/20/05, James Edward Gray II <james@grayproductions.net> wrote:

On Oct 19, 2005, at 6:32 PM, Jeff Wood wrote:

> I know your question is/was about DRb but Roxy will do the block.

Just to be clear, DRb *does* do the block. I'm just trying to figure
out how... :wink:

James Edward Gray II

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

13

James Edward Gray II <james@grayproductions.net> writes:

···

On Oct 19, 2005, at 6:32 PM, Jeff Wood wrote:

I know your question is/was about DRb but Roxy will do the block.

Just to be clear, DRb *does* do the block. I'm just trying to figure
out how... :wink:

James Edward Gray II

The block stays with the the caller and the method on the remote
object does a RPC call for each block invocation.

remote_array.sort{|x| x}

would cause at least #{remote_array.size} RPC calls from the remote
process to the process where the block is declared.

YS.

14

This is an awesome real world example. Thank you! Can't wait to see it...

James Edward Gray II

···

On Oct 21, 2005, at 9:47 PM, Eric Hodel wrote:

I use TupleSpace and RingServer in multi-system monitoring application with self-configuring client daemons. It will be released once I get it robust enough, maybe in a week.

15

[cool stuff snipped]

This makes the monitoring daemons incredibly small and flexible.
They have no configuration and no state. All the hard work is done
on the server.

This sounds very neat. I look forward to the release so I can check
out the source code.

Ryan

···

On 10/21/05, Eric Hodel <drbrain@segment7.net> wrote:

16

> I recently gave a presentation on DrB and Rinda at the St. Louis Ruby
> User Group. You can find it at
> Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos. It's the
> DistributedRuby.pdf link. Hopefully this will answer some of your
> questions.

This was tremendously helpful to my studies! Thank you so much.

You're welcomed. Any feedback on this is welcomed. If there are errors
in it, I'll fix them.

Are you using DRb for something? Can I ask what for?

I'm not. I'm in the middle of creating a course on Ruby and that
presentation will likely be one of the sections in the course. I was
just learning as I experimented with drb and Rinda. People on this
mailing list answered some of my questions along the way so I was
happy to contribute it back.

···

On 10/21/05, James Edward Gray II <james@grayproductions.net> wrote:

On Oct 19, 2005, at 8:32 PM, Mark Volkmann wrote:

--
R. Mark Volkmann
Partner, Object Computing, Inc.

17

James Edward Gray II wrote:

···

On Oct 20, 2005, at 1:11 AM, Dave Burt wrote:

"You are not a member of the group stlruby.
If you believe you are a member, Find your membership"

The stlruby group's files aren't publicly accessible.

You can click Home, join the group, and then get the file.

But what if I don't want to join the St. Louis Ruby Users Group? I don't
even know where St. Louis is!

Cheers,
Dave

18

Objects can be dumped or not dumped.

Blocks and Procs are objects.

Blocks and Procs are objects that cannot be dumped.

Any object that cannot be dumped is wrapped in a DRbObject and that DRbObject is sent to the remote end.

···

On Oct 20, 2005, at 12:58 PM, Yohanes Santoso wrote:

James Edward Gray II <james@grayproductions.net> writes:

On Oct 19, 2005, at 6:32 PM, Jeff Wood wrote:

I know your question is/was about DRb but Roxy will do the block.

Just to be clear, DRb *does* do the block. I'm just trying to figure
out how... :wink:

James Edward Gray II

The block stays with the the caller and the method on the remote
object does a RPC call for each block invocation.

remote_array.sort{|x| x}

would cause at least #{remote_array.size} RPC calls from the remote
process to the process where the block is declared.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

19

I tried to attach the presentation to an email, but it was rejected
because the message was too large. Curt Hibbs helped me post it to the
"WhyRuby" site at http://rubyforge.org/docman/?group_id=251\. Look for
"Distributed Ruby (drb and Rinda)".

BTW, St. Louis is in Missouri which is near the center of the United
States. It's the place with the big arch.

···

On 10/20/05, Dave Burt <dave@burt.id.au> wrote:

James Edward Gray II wrote:
> On Oct 20, 2005, at 1:11 AM, Dave Burt wrote:
>
>> "You are not a member of the group stlruby.
>> If you believe you are a member, Find your membership"
>>
>> The stlruby group's files aren't publicly accessible.
>
> You can click Home, join the group, and then get the file.

But what if I don't want to join the St. Louis Ruby Users Group? I don't
even know where St. Louis is!

--
R. Mark Volkmann
Partner, Object Computing, Inc.

20

Mark Volkmann wrote:

I tried to attach the presentation to an email, but it was rejected
because the message was too large. Curt Hibbs helped me post it to the
"WhyRuby" site at http://rubyforge.org/docman/?group_id=251\. Look for
"Distributed Ruby (drb and Rinda)".

Thanks, it looks great.

BTW, St. Louis is in Missouri which is near the center of the United
States. It's the place with the big arch.

Oh, I thought the place with the big arch was Paris, France.

(But Google Images tells me your arch is way cooler than their old blocky
thing!)

Cheers,
Dave