Concerning Marshalling

ruby-talk
1

Hello,

I was trying to marshal some objects, however some of these objects
contain references to singletons which of course can't be marshalled.
That's why I tried something as follows:

class Container
end

class SpecialContainer < Container
  include Singleton
end

class Test
  attr_accessor :container
  def _dump(depth)
    old = @container
    if old == SpecialContainer.instance
      @container = nil
    end
    result = super
    @container = old
    return result
  end
  def self._load(str)
    super
    if @container == nil
      @container = SpecialContainer.instance
    end
    self
  end
end

Sadly, however, it seems there is no default implementation for _dump,
which means that in this case the super call results in a
NoMethodError. Is there any other way to resolve this problem without
custom-coding a full _dump method?

Regards,
Christophe

2

That's why I tried something as follows:

Perhaps best to use marshal_dump/marshal_load

Guy Decoux

3

for some reason the mailing list seemed to have eaten my post about this so
here it is again - sorry if it's a dup:

harp:~ > cat dumpable_singleton.rb

   module DumpableSingleton
     module InstanceMethods
       attr_accessor 'ikey'
       def _dump *a, &b
         Marshal::dump ikey, *a, &b
       end
       def == other
         self.ikey == other.ikey
       end
       alias_method 'eql?', '=='
     end
     module ClassMethods
       def new *a, &b
         ikey =
         ikey.push *a
         ikey.push b if b
         if instances.has_key? ikey
           instances[ ikey ]
         else
           begin
             ikey.each{|k| Marshal::dump k}
           rescue TypeError, Exception
             raise ArgumentError, "all arguments must be dumpable!"
           end
           instances[ (obj = super(*a, &b)).ikey = ikey ] = obj
         end
       end
       def _load buf, *a, &b
         new(*(ikey = Marshal::load(buf, *a, &b)))
       end
       attr_accessor 'instances'
     end
     def self::included other
       other.module_eval{ include InstanceMethods }
       other.extend ClassMethods
       other.instances = {}
       other
     end
   end

   class C
     include DumpableSingleton
     def initialize(*a, &b); end
   end

   x = C::new 42
   y = C::new 42
   z = C::new 43

   puts '-' * 42
   C::instances.each {|k,c| p k => c}
   p x == y
   p x == z
   p y == z

   xd, yd, zd = [x, y, z].map{|c| Marshal::load(Marshal::dump(c))}

   puts '-' * 42
   C::instances.each {|k,c| p k => c}
   p xd == yd
   p xd == zd
   p yd == zd

   puts '-' * 42
   C::instances.each {|k,c| p k => c}
   p x == xd
   p y == yd
   p z == zd

harp:~ > ruby dumpable_singleton.rb

···

On Sun, 25 Sep 2005, christophe.poucet@gmail.com wrote:

Hello,

I was trying to marshal some objects, however some of these objects
contain references to singletons which of course can't be marshalled.
That's why I tried something as follows:

class Container
end

class SpecialContainer < Container
include Singleton
end

class Test
attr_accessor :container
def _dump(depth)
   old = @container
   if old == SpecialContainer.instance
     @container = nil
   end
   result = super
   @container = old
   return result
end
def self._load(str)
   super
   if @container == nil
     @container = SpecialContainer.instance
   end
   self
end
end

Sadly, however, it seems there is no default implementation for _dump,
which means that in this case the super call results in a
NoMethodError. Is there any other way to resolve this problem without
custom-coding a full _dump method?

Regards,
Christophe

   ------------------------------------------
   {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
   {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
   true
   false
   ------------------------------------------
   {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
   {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
   true
   false
   ------------------------------------------
   {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
   {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
   true

i've used something like this in in a few small projects.

hth.

-a
--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death
Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================

4

ts wrote:

"c" == christophe poucet@gmail com <christophe.poucet@gmail.com> writes:

> That's why I tried something as follows:

Perhaps best to use marshal_dump/marshal_load

Guy Decoux

That's a good suggestion. The marshal_dump/marshal_load pair can be very
useful, because it integrates well with the whole Marshal.dump or
Marshal.load process. An inherent problem with _dump and _load is that
you have to use strings, so you are responsible for somehow preserving
and restoring references to other objects that are also being dumped.
This is critical when the original dump call was on an object that
referred to your object, not directly to your object.

With marshal_dump/marshal_load, you produce/consume an arbitrary
Marshallable object, possibly containing references, and all the code in
marshal.c takes over the hard task of preserving and restoring
references. Probably an example will say it better:

class Node
  attr_accessor :prev_node, :next_node

  attr_accessor :temp_data
    # don't want this to persist (maybe it's a cache, or simply
    # undumpable, like a proc, or a file, or a singleton).

  def marshal_dump
    dumped_obj = [prev_node, next_node]
    dumped_obj
  end

  def marshal_load(dumped_obj)
    @prev_node, @next_node = dumped_obj
  end
end

node1 = Node.new
node2 = Node.new
node1.next_node = node2
node2.prev_node = node1
node1.temp_data = STDOUT
node2.temp_data = proc {"foo"}

p node1

p Marshal.load(Marshal.dump(node1))

__END__

output:

#<Node:0xb7dff098 @next_node=#<Node:0xb7dfef08
@prev_node=#<Node:0xb7dff098 ...>, @temp_data=#<Proc:0xb7dff37c@-:23>>,
@temp_data=#<IO:0xb7e4c078>>
#<Node:0xb7dfed64 @prev_node=nil, @next_node=#<Node:0xb7dfed28
@prev_node=#<Node:0xb7dfed64 ...>, @next_node=nil>>

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

5

Hello,

Guy and Joel: Thank you for your helpful comments, this was exactly
what I was looking for and this was sadly not mentioned in the RDoc of
Marshal :/. For the moment I had been using YAML, but this will not be
a sustainable solution once the number of objects to dump grows in
size. Eventually I want to be able to store the objects into a
database based upon a key that is unique per object (all my classes
derive from Managed that ensures a unique identifier) As for the
references being kept, great!

Ara: Your solution is definitely an interesting one, however I fear
that it is not a sustainable solution for larger projects and the
problem was in more than just singletons, singletons being a symptom of
the problem.

Cheers,
Christophe / vincenz

6

Concerning tracking references.

I want to store each object as a separate value. I can assure that
each object has a unique id (all the objects that need to be stored
separately are instance_of? Managed), however if I marshal the objects
one by one, then the references they might hold internally to each
other will of course no long work. How would I solve this problem?
Obviously I need to replace the references by some sort value that
refers to it, or otherwise anything that is referred will be stored
multiple times. However I'm not quite sure how to put the hooks in
place for this.

Christophe.

7

christophe (dot) poucet (at) gmail (dot) com wrote:

Concerning tracking references.

I want to store each object as a separate value. I can assure that
each object has a unique id (all the objects that need to be stored
separately are instance_of? Managed), however if I marshal the objects
one by one, then the references they might hold internally to each
other will of course no long work. How would I solve this problem?
Obviously I need to replace the references by some sort value that
refers to it, or otherwise anything that is referred will be stored
multiple times. However I'm not quite sure how to put the hooks in
place for this.

Christophe.

Here's one way:

class Node
  @@nodes = Hash.new {|h,k| raise ArgumentError, "Missing node
#{k.inspect}"}

  # The id of a node can be anything but +nil+.
  attr_reader :id

  def initialize id
    @id = id
    @@nodes[id] = self
  end

  attr_accessor :prev_node, :next_node

  attr_accessor :temp_data
    # don't want this to persist (maybe it's a cache, or simply
    # undumpable, like a proc, or a file, or a singleton).

  def marshal_dump
    [prev_node, next_node].map {|node| node && node.id}
  end

  def marshal_load(dumped_obj)
    @prev_node, @next_node = dumped_obj.map do |node_id|
      node_id.nil? ? nil : @@nodes[node_id]
    end
  end

  def self.delete id
    @@nodes.delete id
  end
end

node1 = Node.new "One"
node2 = Node.new 2

node1.next_node = node2
node2.prev_node = node1

node1.temp_data = STDOUT
node2.temp_data = proc {"foo"}

p node1
node1_dumped = Marshal.dump(node1)
p Marshal.load(Marshal.dump(node1))

Node.delete 2
p Marshal.load(node1_dumped)
  # This should raise an exception, to show that the referred node is not
  # stored in node1_dumped, and not in the @@nodes hash.

__END__

#<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
@next_node=#<Node:0xb7dfe24c
@temp_data=#<Proc:0xb7dfe9e0@marshal-refs2.rb:40>, @id=2,
@prev_node=#<Node:0xb7dfe260 ...>>>
#<Node:0xb7dfdeb4 @next_node=#<Node:0xb7dfe24c
@temp_data=#<Proc:0xb7dfe9e0@marshal-refs2.rb:40>, @id=2,
@prev_node=#<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
@next_node=#<Node:0xb7dfe24c ...>>>, @prev_node=nil>
marshal-refs2.rb:2: Missing node 2 (ArgumentError)
        from marshal-refs2.rb:2:in `call'
        from marshal-refs2.rb:24:in `default'
        from marshal-refs2.rb:24:in `'
        from marshal-refs2.rb:24:in `marshal_load'
        from marshal-refs2.rb:23:in `map'
        from marshal-refs2.rb:23:in `marshal_load'
        from marshal-refs2.rb:47:in `load'
        from marshal-refs2.rb:47

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407