Custom serialization problem 2 (marshal_dump, marshal_load)

ruby-talk
1

Hi,

I have a problem with the marshal_dump and marshal_load...
I built two classes XXX, YYY and implemented both methods like this:

class YYY
   @data = "string"
   @version = 1

   def marshal_dump()
       return [@version,@data]
   end

   def marshal_load(var)
       @version = var[0]
       case @version
            when 1
                @data = var[1]
            else
                #do something else
            end
   end
end

class XXX
   @data = Array of YYY
   @version = 1

   def marshal_dump()
       return [@version,@data]
   end

   def marshal_load(var)
       @version = var[0]
       case @version
            when 1
                @data = var[1]
            else
                #do something else
            end
   end
end

The serialization works correctly when the number of objects YYY is
small, but when it got more than 5000, the serialization doesn't work.
(the serializaed file has more and less 1 meg)

When I try to not to use marshal_dump and marshal_load. the
serialization works again???

Can someone tell me, where is the difference between customized
serialization and no-customized one???

Thanks you very much

Sayoyo

···

--
Posted via http://www.ruby-forum.com/.

2

sayoyo Sayoyo wrote:

Hi,

I have a problem with the marshal_dump and marshal_load...
I built two classes XXX, YYY and implemented both methods like this:

class YYY
   @data = "string"
   @version = 1

   def marshal_dump()
       return [@version,@data]
   end

   def marshal_load(var)
       @version = var[0]
       case @version
            when 1
                @data = var[1]
            else
                #do something else
            end
   end
end

class XXX
   @data = Array of YYY
   @version = 1

   def marshal_dump()
       return [@version,@data]
   end

   def marshal_load(var)
       @version = var[0]
       case @version
            when 1
                @data = var[1]
            else
                #do something else
            end
   end
end

The serialization works correctly when the number of objects YYY is
small, but when it got more than 5000, the serialization doesn't work.
(the serializaed file has more and less 1 meg)

When I try to not to use marshal_dump and marshal_load. the
serialization works again???

Can someone tell me, where is the difference between customized
serialization and no-customized one???

Thanks you very much

Sayoyo

What version of ruby? What platform?

Same problem when you serialize to memory instead of file?

What exactly is the observed problem? Crash, exception, data error?

The following version of your code seems to work:

class YYY
   attr_reader :data, :version

   def initialize
     @data = "string"
     @version = 1
   end

   def marshal_dump()
      return [@version,@data]
   end

   def marshal_load(var)
      @version = var[0]
      case @version
           when 1
               @data = var[1]
           else
               #do something else
           end
   end
end

class XXX
   attr_reader :data, :version

   def initialize
     @data = (0..1000).map { YYY.new }
     @version = 1
   end

   def marshal_dump()
       return [@version,@data]
   end

   def marshal_load(var)
       @version = var[0]
       case @version
            when 1
                @data = var[1]
            else
                #do something else
            end
   end
end

xxx = XXX.new

x2 = Marshal.load(Marshal.dump(xxx))

p x2.data[777]

File.open('/tmp/foo', "wb") do |f|
   Marshal.dump(xxx, f)
end

x3 = File.open('/tmp/foo', "rb") do |f|
   Marshal.load(f)
end

p x3.data[888]

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

3

Joel VanderWerf wrote:

    @data = (0..1000).map { YYY.new }

I tried it for 1_000_000 with no problem, and that generated a 1.4Mb file.

Ruby 1.8.6-p114, linux.

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

4

Hi,

I'm using ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux] and
Rails 1.2.5

The problem I have, is the class XXX should have an array of YYY as
data, but it turn out I got only NIL when I deserialized, then I got an
exception for unable to use [] for a NIL. It happens only when I have a
lot of data.

The example I posted here is an simplification of my data, actually I
have 5 classes, the structure:
A--
  >-Array of B
  >- C

B--
  >-Array of D

C--
  >-Array of E

and each of them have more and less 10 attributes... When I have 5000 of
B, the serialized file has almost 1 meg. And I also realized each time I
serialized the same data, the file got a different size each time...

How can I verify the serailization?

···

--
Posted via http://www.ruby-forum.com/.

5

sayoyo Sayoyo wrote:

Hi,

I'm using ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux] and Rails 1.2.5

The problem I have, is the class XXX should have an array of YYY as data, but it turn out I got only NIL when I deserialized, then I got an exception for unable to use for a NIL. It happens only when I have a lot of data.

The example I posted here is an simplification of my data, actually I have 5 classes, the structure:
A--
  >-Array of B
  >- C

B--
  >-Array of D

C--
  >-Array of E

and each of them have more and less 10 attributes... When I have 5000 of B, the serialized file has almost 1 meg. And I also realized each time I serialized the same data, the file got a different size each time...

How can I verify the serailization?

I would try to come up with a minimal characterization of the problem--minimize the classes, attrs, objects so that if you take away anything else the problem goes away. That should help point to the problem.

Also, you could try running this on other systems and ruby versions....

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407