Drb and PStore behaviour

ruby-talk
1

I'm doing a remote database object so i can do:

result = database.find do |record|
    record[:name] == "john"
end

in the server object i have something like:

class Foo
     include DRb::DRbUndumped

     def find(&block)
             result = nil
             @store.transaction do
                     result = @store[:records].find_all do |record|
                                        yield(record)
                     end
             end
             return result
     end
end

well, the problem is yielding that block. All the database is "cloned/copied" (?) over the network when doing the find. I don't want to copy all that data, just return the found records.

I tried adding DRb::DRbUndumped to class PStore, but the same result.
The database size is about 3 MB.

Do you know how to solve this problem?

2

The problem is that you can't pass a block from the client to the server because block's aren't serializable.

Jamey

Javier Valencia wrote:

···

I'm doing a remote database object so i can do:

result = database.find do |record|
   record[:name] == "john"
end

in the server object i have something like:

class Foo
    include DRb::DRbUndumped

    def find(&block)
            result = nil
            @store.transaction do
                    result = @store[:records].find_all do |record| yield(record)
                    end
            end
            return result
    end
end

well, the problem is yielding that block. All the database is "cloned/copied" (?) over the network when doing the find. I don't want to copy all that data, just return the found records.

I tried adding DRb::DRbUndumped to class PStore, but the same result.
The database size is about 3 MB.

Do you know how to solve this problem?

3

Javier Valencia wrote:

I'm doing a remote database object so i can do:

result = database.find do |record|
   record[:name] == "john"
end

in the server object i have something like:

class Foo
    include DRb::DRbUndumped

    def find(&block)

               ^^^^^^
You don't need this, since you are using yield.

            result = nil
            @store.transaction do
                    result = @store[:records].find_all do |record|
                                      yield(record)
                    end
            end
            return result
    end
end

well, the problem is yielding that block. All the database is
"cloned/copied" (?) over the network when doing the find. I don't want
to copy all that data, just return the found records.

Yielding to a DRb client is not itself a problem. The block stays on the
client side. In some cases that's ok, but in this case, it generates too
much traffic because every record iterated over by find_all is
serialized to the client, so that the client can call the block on it.

I tried adding DRb::DRbUndumped to class PStore, but the same result.
The database size is about 3 MB.

I doubt the database object as a whole is being serialized, just each
record individually.

Maybe you want do define query objects and pass them as an argument to find?

result = database.find(QueryEqual.new(:name, "john"))

Then only the query object and the results need to get serialized.

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

4

Oh, so the answer to my problem will be to rewrite using sockets and marshaling for that.

(the same happened with xmlrpc).
thanks for the quick response.

Jamey Cribbs wrote:

···

The problem is that you can't pass a block from the client to the server because block's aren't serializable.

Jamey

Javier Valencia wrote:

I'm doing a remote database object so i can do:

result = database.find do |record|
   record[:name] == "john"
end

in the server object i have something like:

class Foo
    include DRb::DRbUndumped

    def find(&block)
            result = nil
            @store.transaction do
                    result = @store[:records].find_all do |record| yield(record)
                    end
            end
            return result
    end
end

well, the problem is yielding that block. All the database is "cloned/copied" (?) over the network when doing the find. I don't want to copy all that data, just return the found records.

I tried adding DRb::DRbUndumped to class PStore, but the same result.
The database size is about 3 MB.

Do you know how to solve this problem?

5

Joel VanderWerf wrote:

Javier Valencia wrote:

I'm doing a remote database object so i can do:

result = database.find do |record|
  record[:name] == "john"
end

in the server object i have something like:

class Foo
   include DRb::DRbUndumped

   def find(&block)
   

              ^^^^^^
You don't need this, since you are using yield.

           result = nil
           @store.transaction do
                   result = @store[:records].find_all do |record|
                                     yield(record)
                   end
           end
           return result
   end
end

well, the problem is yielding that block. All the database is
"cloned/copied" (?) over the network when doing the find. I don't want
to copy all that data, just return the found records.
   
Yielding to a DRb client is not itself a problem. The block stays on the
client side. In some cases that's ok, but in this case, it generates too
much traffic because every record iterated over by find_all is
serialized to the client, so that the client can call the block on it.

I tried adding DRb::DRbUndumped to class PStore, but the same result.
The database size is about 3 MB.
   
I doubt the database object as a whole is being serialized, just each
record individually.

Maybe you want do define query objects and pass them as an argument to find?

result = database.find(QueryEqual.new(:name, "john"))

Then only the query object and the results need to get serialized.

Interesting point, i'll try it.

And thanks for the correction.