Saving array

ruby-talk
1

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

2

Phidippus wrote:

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

There are numerous ways. You might use Marshal, for instance.

YAML is another possibility:

require “yaml”
#…
File.open(“myarray”,“w”) {|f| f.puts array.to_yaml }

Later on…

array = nil
File.open(“myarray”) {|f| array = YAML.load(f) }

That’s just one way.

HTH,
Hal

3

Phidippus wrote:

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

You can use marshalling:

array = …
File.open(“array”, “wb”) do |f|
Marshal.dump(array, f)
end

To reload:

array = nil
File.open(“array”, “r”) do |f|
array = Marshal.load(f)
end

4

$irb
irb(main):001:0> Marshal.dump([1,2,3],open(‘ary.dat’,‘w+’))
=> #<File:ary.dat>
irb(main):002:0> exit

$ls -l ary.dat
-rw-r–r-- 1 gabriele Administ 10 Mar 10 20:55 ary.dat

$irb
irb(main):001:0> Marshal.load(open(‘ary.dat’))
=> [1, 2, 3]

···

il 10 Mar 2004 12:36:16 -0800, mopthisandthat@hotmail.com (Phidippus) ha scritto::

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

5

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

Thank you so much.
T.O.

mopthisandthat@hotmail.com (Phidippus) wrote in message news:a2208ca9.0403101236.37762216@posting.google.com…

···

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

6

“Phidippus” mopthisandthat@hotmail.com wrote in message
news:a2208ca9.0403101236.37762216@posting.google.com…

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

Hi,

An instance of PStore, maybe?

http://www.mattriffle.com/mirrors/ruby_book/html/lib_standard.html

(do a browser search on “PStore”).

  • dan
7

Joel VanderWerf wrote:

To reload:

array = nil
File.open(“array”, “r”) do |f|
^^^ should be “rb” (it matters on windows)

···

array = Marshal.load(f)
end

8

“gabriele renzi” surrender_it@remove.yahoo.it schrieb im Newsbeitrag
news:c30v401l703lrie4ogcjbsa5rvmir30v3v@4ax.com…

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

$irb
irb(main):001:0> Marshal.dump([1,2,3],open(‘ary.dat’,‘w+’))
=> #<File:ary.dat>
irb(main):002:0> exit

You should rather do

Marshal.dump([1,2,3],open(‘ary.dat’,‘wb’)).close()

in order to properly close the array. But the block form is even better
as it ensures closing of the file under all conditions (others posted this
form already):

File.open(‘ary.dat’,‘wb’) {|f| Marshal.dump([1,2,3],f) }

Similar for reading.

robert
···

il 10 Mar 2004 12:36:16 -0800, mopthisandthat@hotmail.com (Phidippus) > ha scritto::

9

Hmm is narray.rb part of ruby-1.8.x? or a separate package? I’m guessing
that the yaml package implements the #to_yaml method for all the containers
(array, hash etc) which are part of ruby “main”. (all thanks to open class
definitions in ruby =D ). So if narray.rb is not part of the stdlib, they
wouldn’t have implemented the #to_yaml for you.

Solutions? In your program you can do something like

class NArray
def to_yaml ()
… #Implementation (sorry I’m not familiar with NArray)
end
end

Archit

P.S. On further scrutiny it revealed that narray.rb is indeed NOT a part
of the stdlib

irb(main):002:0> require ‘narray’
LoadError : No such file to load – narray
from (irb):2:in require' from (irb):2 irb(main):003:0> require 'narray.rb' LoadError : No such file to load -- narray.rb from (irb):3:in require’
from (irb):3
irb(main):004:0> x = NArray.new
NameError : uninitialized constant NArray
from (irb):4
irb(main):005:0>

mopthisandthat@hotmail.com (Phidippus) writes:

···

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

Thank you so much.
T.O.

mopthisandthat@hotmail.com (Phidippus) wrote in message news:a2208ca9.0403101236.37762216@posting.google.com…

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

10

Er, how about you please provide US with what you are doing, what
errors your getting.

That’s a far easier way to help you than having people guess what might
be wrong with your method.

···

On Mar 11, 2004, at 3:04 PM, Phidippus wrote:

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

11

Hi,

From: “Phidippus” mopthisandthat@hotmail.com
Newsgroups: comp.lang.ruby
Sent: Friday, March 12, 2004 7:04 AM

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

When the object not a sparse array,

Marshal.load(Marshal.dump(narray.to_a))
require “soap/marshal”; SOAPMarshal.load(SOAPMarshal.dump(narray.to_a))
require “yaml”; YAML.load(narray.to_a.to_yaml)

could be the answer.

SOAP marshaller supports sparse array, but cannot handle a NArray
object for now.

Regards,
// NaHi

12

I tried, but it did not seem to work either…

irb(main):001:0> require “pstore”
true
irb(main):002:0> require “narray”
true
irb(main):003:0> array = NArray.int(2,2,2)
NArray.int(2,2,2):
[ [ [ 0, 0 ],
[ 0, 0 ] ],
[ [ 0, 0 ],
[ 0, 0 ] ] ]
irb(main):004:0> db = PStore.new(“myarray”)
#<PStore:0x4020ab00 @transaction=false, @abort=false, @filename=“myarray”>
irb(main):005:0> db.transaction do
irb(main):006:1* db[“data”] = array
irb(main):007:1> end
TypeError: can’t dump NArray
from /usr/local/lib/ruby/1.6/pstore.rb:111:in dump' from /usr/local/lib/ruby/1.6/pstore.rb:111:in transaction’
from (irb):5
irb(main):008:0>

“dhtapp” dhtapp@cox.net wrote in message news:A264c.52975$aZ3.14689@fed1read04…

···

“Phidippus” mopthisandthat@hotmail.com wrote in message
news:a2208ca9.0403101236.37762216@posting.google.com…

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

Hi,

An instance of PStore, maybe?

http://www.mattriffle.com/mirrors/ruby_book/html/lib_standard.html

(do a browser search on “PStore”).

  • dan
13

Gavin Kistner wrote:

···

On Mar 11, 2004, at 3:04 PM, Phidippus wrote:

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

Er, how about you please provide US with what you are doing, what errors
your getting.

That’s a far easier way to help you than having people guess what might
be wrong with your method.

The problem is that ary.to_yaml dumps empty data:

— !ruby/object:NArray {}

IIRC there is a similar problem with rbtree.

–
Joel VanderWerf California PATH, UC Berkeley
mailto:vjoel@path.berkeley.edu Ph. (510) 231-9446
http://www.path.berkeley.edu FAX (510) 231-9565

14

Sorry. NArray is not a standard package. I got it from

http://www.ir.isas.ac.jp/~masa/ruby/index-e.html

Thanks.

Archit Baweja bighead@users.sourceforge.net wrote in message news:m2oer3yrou.fsf@dhcp52.moberg…

···

Hmm is narray.rb part of ruby-1.8.x? or a separate package? I’m guessing
that the yaml package implements the #to_yaml method for all the containers
(array, hash etc) which are part of ruby “main”. (all thanks to open class
definitions in ruby =D ). So if narray.rb is not part of the stdlib, they
wouldn’t have implemented the #to_yaml for you.

Solutions? In your program you can do something like

class NArray
def to_yaml ()
… #Implementation (sorry I’m not familiar with NArray)
end
end

Archit

P.S. On further scrutiny it revealed that narray.rb is indeed NOT a part
of the stdlib

irb(main):002:0> require ‘narray’
LoadError : No such file to load – narray
from (irb):2:in require' from (irb):2 irb(main):003:0> require 'narray.rb' LoadError : No such file to load -- narray.rb from (irb):3:in require’
from (irb):3
irb(main):004:0> x = NArray.new
NameError : uninitialized constant NArray
from (irb):4
irb(main):005:0>

mopthisandthat@hotmail.com (Phidippus) writes:

Thank you for the responses. But I forgot to mention one thing. My
arrays are multidimensional arrays of class NArray (using narray.rb).
I tried using YAML and marshalling, but they do not seem to work
(although I may be doint totally wrong). If you can, please give me
further advices.

Thank you so much.
T.O.

mopthisandthat@hotmail.com (Phidippus) wrote in message news:a2208ca9.0403101236.37762216@posting.google.com…

After I do bunch of computations and create a huge array with bunch of
numbers, I like to use the already-computed array in another program
rather than re-computing it each time I run the program. How can I
save the array object in one program and read it from another program
in ruby?

15

Phidippus wrote:

TypeError: can’t dump NArray
from /usr/local/lib/ruby/1.6/pstore.rb:111:in dump' from /usr/local/lib/ruby/1.6/pstore.rb:111:in transaction’
from (irb):5

Here’s one way to do it in YAML. Hopefully we can make this simpler in
the future. Not too bad for now.

class NArray
    def is_complex_yaml?
        true
    end
    def to_yaml_type
        "!ruby/narray"
    end
    def to_yaml( opts = {} )
        opts[:DocType] = self.class if Hash === opts
        YAML::quick_emit( self.object_id, opts ) { |out|
            out.seq( to_yaml_type ) { |seq|
                seq.concat( self.to_a )
            }
        }
    end
end

YAML.add_ruby_type( 'narray' ) { |type, val| NArray[ *val ] ) }

_why

16

why the lucky stiff wrote:

Here’s one way to do it in YAML. Hopefully we can make this simpler in
the future. Not too bad for now.

Thanks for the tip, _why. Now, my CGenerator lib generates ruby
extensions which are automatically YAML friendly. (Separate announcement
upcoming.)

One question: CGenerator uses types of the form

!ruby/cshadow:SubclassName

for the shadow classes generated from client code. It doesn’t seem
likely to have been used, but can I check that somewhere? Is there a
registry where I could declare this type family?

TIA.

17

Joel VanderWerf wrote:

One question: CGenerator uses types of the form

!ruby/cshadow:SubclassName

for the shadow classes generated from client code. It doesn’t seem
likely to have been used, but can I check that somewhere? Is there a
registry where I could declare this type family?

you’ve seen the yaml.org type repository? [1] well, we should have a
type repository for ruby types. at one time this was destined for
ruby.yaml.org. we could start one up on wiki. [2]

let me check with yaml-core. in the meantime, you might follow omap’s
type spec. [3] send me a document with URI, shorthand, kind, definition
and any examples you’d like. also: a link to the cshadow home.

_why

[1] http://yaml.org/tags.html
[2] http://yaml.kwiki.org/
[3] http://yaml.org/tags/omap.html

18

I haven’t gotten around to setting up a specification page for my
ruby/cshadow type, like you suggested, because I’m not certain what the
URI should be. Something based on the site where I distribute the
software? Let me know if I should be RTM somewhere, I didn’t try yet.
Anyway, is this URI necessary or is the shortcut (which would be
!ruby/cshadow, I guess) good enough?

Also, I’m wondering about the logistics of YAML.add_ruby_type. I can do
it when my class is loaded, but only if YAML itself has been loaded
(which I don’t need to do). If not, the code that’s using my lib and
YAML needs to do it. It’s just a method call, but kind of a pain to
remember to do it. Has anyone come up with clever solution to this?