Marshaling NArray objects

ruby-talk
1

I'm trying to marshal narray objects by first converting them to
ordinary arrays but I can't seem to get it right. I understand that I
have to implement marshal_dump and marshal_load for NArray so I've
tried this:

class NArray
  def marshal_dump
    self.to_a
  end

def marshal_load(array)
   NArray.to_narray(array)
end
end

And lots of other stuff. I won't bore you with the error messages,
suffice to say that I can't seem to get it right.

alex

···

--
Alex Polite
http://flosspick.org - finding the right open source

2

I'm trying to marshal narray objects by first converting them to
ordinary arrays but I can't seem to get it right. I understand that I
have to implement marshal_dump and marshal_load for NArray so I've
tried this:

class NArray
def marshal_dump
   self.to_a
end

It's this part.

def marshal_load(array)
  NArray.to_narray(array)
end

You need something like
replace(NArray.to_narray(array))

Unfortunately, replace isn't a method for NArray.

···

On May 24, 2006, at 11:35 PM, Alex Polite wrote:

And lots of other stuff. I won't bore you with the error messages,
suffice to say that I can't seem to get it right.

alex

--
Alex Polite
http://flosspick.org - finding the right open source

3

Alex Polite wrote:

I'm trying to marshal narray objects by first converting them to
ordinary arrays but I can't seem to get it right. I understand that I
have to implement marshal_dump and marshal_load for NArray so I've
tried this:

class NArray
def marshal_dump
   self.to_a
end

def marshal_load(array)
  NArray.to_narray(array)
end
end

And lots of other stuff. I won't bore you with the error messages,
suffice to say that I can't seem to get it right.

One problem is that #marshal_load needs to populate self with the data,
rather than just instantiating a new object. In other words, don't
bother returning a new NArray from marshal_load--it won't be used.

Example:

class C
  attr_accessor :x, :y

  def marshal_dump
    [x, y]
  end

  def marshal_load(ary)
    p ary
    @x, @y = ary # <-- populate self with persisted data
  end
end

c = C.new
c.x = "foo"
c.y = "bar"

s = Marshal.dump(c)
p Marshal.load(s)

__END__

Output:

["foo", "bar"]
#<C:0xb7dbc220 @y="bar", @x="foo">

Is there a NArray method like Array#replace ? That would probably do the
trick, if you called it in marshal_load with the narray as its argument.

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

4

Joel VanderWerf wrote:

Is there a NArray method like Array#replace ? That would probably do the
trick, if you called it in marshal_load with the narray as its argument.

This seems to work as a replace method:

a = b

Ye cats, NArray is nice. Why isn't it in the stdlib, again?

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

5

There doesn't seem to be anything like replace in narray. Maybe I
should resort to using a wrapper class around my narrays?

class NArrayWrapper
  def initialize(narray)
     @narray = narray
  end

   def marshal_dump
      @narray.to_a
   end

   def marshal_load(array)
     @narray = NArray.to_narray(array)
   end
end

But then I'd have to write a whole bunch of accessor methods and stuff.
I bet there's some C/ruby/narray wizardry that will replicate replace.

alex

···

On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

  def marshal_load(ary)
    p ary
    @x, @y = ary # <-- populate self with persisted data
  end

Is there a NArray method like Array#replace ? That would probably do the
trick, if you called it in marshal_load with the narray as its argument.

--
Alex Polite
http://flosspick.org - finding the right open source

6

This seems to work as a replace method:

a = b

···

On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

     1 require 'narray'
     2 class NArray
     3 def marshal_dump
     4 self.to_a
     5 end
     6
     7 def marshal_load(array)
     8 newnarray = NArray.to_narray(array)
     9 self = newnarray
    10 end
    11 end
    12
    13 na = NArray.float(10)
    14
    15 Marshal.load(Marshal.dump(na))

Results in

marshaling_narrays.rb:9:in `=': wrong argument type NArray (expected
Data) (TypeError)
        from marshaling_narrays.rb:9:in `marshal_load'
        from marshaling_narrays.rb:15

--
Alex Polite
http://flosspick.org - finding the right open source

7

are you sure need it? you can write/read narrays from file quite easily:

     harp:~ > cat a.rb
     require 'narray'

     na = NArray.int 2, 3

     na[0, true] = 42

     p na

     open('dat', 'w'){|f| f.write na.to_s}

     buf = IO.read('dat')

     na = NArray.to_na buf, NArray::INT, 2, 3

     p na

     harp:~ > ruby a.rb
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]

so, assuming that you do, it's easy enough:

     harp:~ > cat a.rb
     require 'narray'

     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data => to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

     na = NArray.int 2, 3

     na[0, true] = 42

     dumped = Marshal.dump na

     na = Marshal.load dumped

     p na

     harp:~ > ruby a.rb
     NArray.int(2,3):
     [ [ 42, 0 ],
       [ 42, 0 ],
       [ 42, 0 ] ]

regards.

-a

···

On Thu, 25 May 2006, Alex Polite wrote:

On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

  def marshal_load(ary)
    p ary
    @x, @y = ary # <-- populate self with persisted data
  end

Is there a NArray method like Array#replace ? That would probably do the
trick, if you called it in marshal_load with the narray as its argument.

There doesn't seem to be anything like replace in narray. Maybe I
should resort to using a wrapper class around my narrays?

class NArrayWrapper
def initialize(narray)
   @narray = narray
end

def marshal_dump
    @narray.to_a
end

def marshal_load(array)
   @narray = NArray.to_narray(array)
end
end

But then I'd have to write a whole bunch of accessor methods and stuff.
I bet there's some C/ruby/narray wizardry that will replicate replace.

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

8

Alex Polite wrote:

This seems to work as a replace method:

a = b

===============
    1 require 'narray'
    2 class NArray
    3 def marshal_dump
    4 self.to_a
    5 end
    6
    7 def marshal_load(array)
    8 newnarray = NArray.to_narray(array)
    9 self = newnarray
   10 end
   11 end
   12
   13 na = NArray.float(10)
   14
   15 Marshal.load(Marshal.dump(na))

Results in

marshaling_narrays.rb:9:in `=': wrong argument type NArray (expected
Data) (TypeError)
       from marshaling_narrays.rb:9:in `marshal_load'
       from marshaling_narrays.rb:15

That's a bug in NArray. It should define an allocator function by
calling rb_define_alloc_func(), otherwise the T_OBJECT allocator is used
instead of an allocator that returns a T_DATA.

In this case, #marshal_load is being called on an object that is already
corrupt, before we even try to replace its contents: it is marked as
T_OBJECT, and so NArray's methods will fail on it, expecting T_DATA.

In any case, with the bug fixed or not, this won't work as written. The
typecode and shape would have to be added to the persisted data, as
Ara's solution points out. But more that that, there would be no way for
NArray to correctly allocate storage without knowing the typecode, so
the load method has to be a class method (_load), not an instance method
(marshal_load).

···

On 5/25/06, Joel VanderWerf <vjoel@path.berkeley.edu> wrote:

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

9

are you sure need it?

yeah. I'm marshaling objects that have narrays as a component among others.

     harp:~ > cat a.rb
     require 'narray'

     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data => to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

Thanks a bunch. That did the trick. Just one more thing. When I load a
marshalled object I can access the attributes on that object but I can
call it's methods. This is from irb. I've googled for stuff like
"marshaling methods" to no avail.

Alex

···

On 5/25/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

--
Alex Polite
http://flosspick.org - finding the right open source

10

you must have something else going on, you needn't do anything special to
marshal methods:

     harp:~ > cat a.rb
     require 'narray'
     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data => to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

     na = Marshal.load(Marshal.dump(NArray.int(4,2)))

     p na
     na[true, 0] = 42
     p na
     p na.size
     p na.shape

     harp:~ > ruby a.rb
     NArray.int(4,2):
     [ [ 0, 0, 0, 0 ],
       [ 0, 0, 0, 0 ] ]
     NArray.int(4,2):
     [ [ 42, 42, 42, 42 ],
       [ 0, 0, 0, 0 ] ]
     8
     [4, 2]

you'll have to show us your problem - probably you are returning the wrong kind
of object from the class _load method.

regards.

-a

···

On Sat, 27 May 2006, Alex Polite wrote:

On 5/25/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

are you sure need it?

yeah. I'm marshaling objects that have narrays as a component among others.

     harp:~ > cat a.rb
     require 'narray'

     class NArray
       def _dump *ignored
         Marshal.dump :typecode => typecode, :shape => shape, :data => to_s
       end
       def self._load buf
         h = Marshal.load buf
         typecode = h[:typecode]
         shape = h[:shape]
         data = h[:data]
         to_na data, typecode, *shape
       end
     end

Thanks a bunch. That did the trick. Just one more thing. When I load a
marshalled object I can access the attributes on that object but I can
call it's methods. This is from irb. I've googled for stuff like
"marshaling methods" to no avail.

Alex

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama