[ANN] arrayfields-3.0.0

ruby-talk
1

URLS:

   http://raa.ruby-lang.org/project/arrayfields/
   http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

   allow keyword access to arrays:

     require 'arrayfields'

     fields = 'name', 'age'
     row = [ 'bob', 30 ]

     row.fields = fields

     row[ 'name' ] #=> 'bob'
     row.indices 'name', 'age' #=> [ 'bob', 30 ]

   assigning to un-named fields appends:

     stack =
     stack.fields = %w(zero one)
     stack['zero'] = 'zero'
     stack['one'] = 'one'
     stack #=> [ 'zero', 'one' ]

   *very* useful for database work

     relation = pgconn.query sql
     relation.size #=> 65536

     # yikes! do we really want to re-construct a hash for for each tuple when
     # we already have Arrays?

     fields = %w(ssn name position)
     table.each{|tuple| tuple.fields = fields}

     table[34578]['ssn'] #=> 574865032

LIST OF OVERRIDDEN METHODS

   Array#
   Array#=
   Array#at
   Array#delete_at
   Array#fill
   Array#values_at
   Array#indices
   Array#indexes
   Array#slice
   Array#slice!

LIST OF NEW Array METHODS

   Array#fields=
   Array#each_with_field

DOCS/USAGE/SAMPLE

   lib/arrayfields.rb
   test/arrayfields.rb

AUTHOR

   ara.t.howard@noaa.gov

-a

···

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

2

Hi,

Cool, I'm gonna give it a whirl.

Thanks a lot,
Joao

···

--- "Ara.T.Howard" <Ara.T.Howard@noaa.gov> wrote:

URLS:

   http://raa.ruby-lang.org/project/arrayfields/
  
http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

   allow keyword access to arrays:

     require 'arrayfields'

     fields = 'name', 'age'
     row = [ 'bob', 30 ]

     row.fields = fields

     row[ 'name' ] #=> 'bob'
     row.indices 'name', 'age' #=> [ 'bob', 30 ]

__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

3

Ara.T.Howard wrote:

URLS:

  http://raa.ruby-lang.org/project/arrayfields/
  http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

  allow keyword access to arrays:

    require 'arrayfields'

    fields = 'name', 'age'
    row = [ 'bob', 30 ]

    row.fields = fields

    row[ 'name' ] #=> 'bob'
    row.indices 'name', 'age' #=> [ 'bob', 30 ]

Hm, sorta like something I've been using:

···

---
module AccessibleIndex
   def index_accessor(h)
     h.each do |sym, idx|
       define_method sym do
         self[idx]
       end
       define_method "#{sym}=" do |val|
         self[idx] = val
       end
     end
   end
end

a = [0,1,2,3,4,5]
class << a
   extend AccessibleIndex
   index_accessor :x => 3
end

a.x = "three"
p a # ==> [0, 1, 2, "three", 4, 5]
---

But this approach doesn't help with values_at and the like.

4

Hi,

At Wed, 30 Jun 2004 05:02:58 +0900,
Ara.T.Howard wrote in [ruby-talk:104864]:

   http://raa.ruby-lang.org/project/arrayfields/
   http://www.codeforpeople.com/lib/ruby/arrayfields/

$ ruby-1.8 -r arrayfields.rb -e '[:x]'
-e:1:in `': Symbol as array index (TypeError)
  from -e:1
$ grep VERSION arrayfields.rb
    VERSION = '3.1.0'

`and' and `or' operators have same precedence.

BTW, you couldn't use Struct?

  tuples =
  fields = Struct.new(nil, res.fields)

  tuples = res.collect {|tuple| fields.new(*tuple)}

···

--
Nobu Nakada

5

Ara, any chance of getting arrayfields to not modify the Array methods but
instead simply add an accessor to Array which returns an object that
encapsulates this new functionality?

So, instead of saying:

row = ['bob', 30]
row.fields = ['name', 'age']
row['name'] #=> 'bob'

you would say something like:

row = ['bob', 30]
row.arrayfields.fields = ['name', 'age']
row.arrayfields['name'] #=> 'bob'

My thinking is that if arrayfields functionality were contained in its own
object that linked back to a parent array, you could create arrayfield
objects and attach them to other "array-like" objects, such as database rows
and so on.

But also, it might prevent problems that arise later from taking over Array's
methods where people want to use both arrayfields, and regular Array
functionality.

Just an idea.

  Sean O'Dell

6

cool. your way is perhaps safer. :wink:

the reason i wrote this in the first place was that i was doing alot of
postgresql work and it was returning these huge result sets that i would then
write code that looked like:

   if tuple[17] =~ pat or tuple[9].to_i < 42
     ...
   end

and i was reading my own code thinking - huh? so then i started doing:

   res = pgconn.exec sql

   tuples = res.result
   fields = res.fields

   tuples =
     tuples.collect do |tuple|
       h = {}
       fields.each do |field|
         h[field] = tuple.shift
       end
       h
     end

this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
module. it still requires work:

   res = pgconn.exec sql

   tuples = res.result
   fields = res.fields

   tuples.each{|t| t.fields = fields}

but now all 42000 tuples SHARE a copy of fields for doing their lookups - i
still have to iterate over all of em - but i don't need to create any new
objects and my code new reads like:

   if tuple['name'] =~ pat or tuple['age'].to_i < 42
     ...
   end

which is a lot nicer.

in any case it's just one of those things that i ended up using alot. i'd
like to re-write it in C but don't have the time right now...

cheers.

-a

···

On Wed, 30 Jun 2004, Joel VanderWerf wrote:

---
module AccessibleIndex
  def index_accessor(h)
    h.each do |sym, idx|
      define_method sym do
        self[idx]
      end
      define_method "#{sym}=" do |val|
        self[idx] = val
      end
    end
  end
end

a = [0,1,2,3,4,5]
class << a
  extend AccessibleIndex
  index_accessor :x => 3
end

a.x = "three"
p a # ==> [0, 1, 2, "three", 4, 5]
---

But this approach doesn't help with values_at and the like.

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

7

Hi,

At Wed, 30 Jun 2004 10:52:28 +0900,
nobu.nokada@softhome.net wrote in [ruby-talk:104888]:

$ ruby-1.8 -r arrayfields.rb -e '[:x]'
-e:1:in `': Symbol as array index (TypeError)
  from -e:1

Sorry, this is not a problem, I was a stupid.

···

--
Nobu Nakada

8

Hi,

At Wed, 30 Jun 2004 05:02:58 +0900,
Ara.T.Howard wrote in [ruby-talk:104864]:

   http://raa.ruby-lang.org/project/arrayfields/
   http://www.codeforpeople.com/lib/ruby/arrayfields/

$ ruby-1.8 -r arrayfields.rb -e '[:x]'
-e:1:in `': Symbol as array index (TypeError)
  from -e:1
$ grep VERSION arrayfields.rb
   VERSION = '3.1.0'

`and' and `or' operators have same precedence.

the instance only is extended, and only then after instance.fields= has been
called. eg.

   ~ > ruby -r arrayfields-3.0.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
   42

   ~ > ruby -r arrayfields-3.1.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
   42

so, in the normal case only the method Array#fields= is added when you

   require 'arrayfields'

i didn't want to polute the namespace any more than needed.

BTW, you couldn't use Struct?

tuples =
fields = Struct.new(nil, res.fields)

tuples = res.collect {|tuple| fields.new(*tuple)}

i suppose i could... but there the would have a lot of object create on large
result sets no?

regards.

-a

···

On Wed, 30 Jun 2004 nobu.nokada@softhome.net wrote:
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

9

well - theoretically you lose zero Array functionality since ArrayFields
methods are used if, and only if, the idx paramter is a String or Symbol which
would never be done with a normal Array.

that being said:

   jib:~/shared > cat a.rb
   require 'arrayfields'

   class FieldedArray
     attr :fieldset
     def initialize fields, array
       @a = array
       self.fields = fields
     end
     def method_missing(meth, *args, &block)
       @a.send(meth, *args, &block)
     end
     def fields= fields
       extend ArrayFields unless defined? @fieldset

       @fieldset =
         if ArrayFields::FieldSet === fields
           fields
         else
           ArrayFields::FieldSet.new fields
         end
     end
     attr_reader :fieldset
     def fields
       @fieldset and @fieldset.fields
     end
   end

   fa = FieldedArray.new %w(zero one two), [0, 1, 2]
   fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}

   jib:~/shared > ruby a.rb
   zero : 0
   one : 1
   two : 2

i can't bloody believe that worked - go matz!

what'ya think of adding the above to 3.3.0 (along with gem spec)?

-a

···

On Thu, 1 Jul 2004, Sean O'Dell wrote:

Ara, any chance of getting arrayfields to not modify the Array methods but
instead simply add an accessor to Array which returns an object that
encapsulates this new functionality?

So, instead of saying:

row = ['bob', 30]
row.fields = ['name', 'age']
row['name'] #=> 'bob'

you would say something like:

row = ['bob', 30]
row.arrayfields.fields = ['name', 'age']
row.arrayfields['name'] #=> 'bob'

My thinking is that if arrayfields functionality were contained in its own
object that linked back to a parent array, you could create arrayfield
objects and attach them to other "array-like" objects, such as database rows
and so on.

But also, it might prevent problems that arise later from taking over Array's
methods where people want to use both arrayfields, and regular Array
functionality.

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

10

Ara.T.Howard wrote:

  if tuple[17] =~ pat or tuple[9].to_i < 42
    ...
  end

...

  if tuple['name'] =~ pat or tuple['age'].to_i < 42
    ...
  end

Is there any reason you didn't just use either constants or variables like:

if tuple[NAME] =~ pat or tuple[AGE].to_i < 42
   ...
end

or if your indices weren't constant, set them from a DB query and then do:

if tuple[name] =~ pat or tuple[age].to_i < 42
   ...
end

arrayfields looks great, and makes things much easier, but it just seems odd to me that you'd approach it the way you did.

Ben

11

Okay, admittedly, sometimes I'm too dense for my own good, but I'm
confused. It looks to me like, internally, each array is using a hash to
keep track of the fields to indices mapping, and that hash is not shared
amongst arrays with the same set of fields. So you end up using more memory
using arrayfields for something like database result sets than if you just
used hashes.

I put together a few simple little test programs to try it out, using
arrayfields and using hashes to store similar sets of data, and in practice
it looks like arrayfields uses almost 2x the amount of RAM for a given data
set, as well. Am I missing something, here?

Thanks,

Kirk Haines

···

On Wed, 30 Jun 2004 06:37:56 +0900, Ara.T.Howard wrote

and i was reading my own code thinking - huh? so then i started doing:

   res = pgconn.exec sql

   tuples = res.result
   fields = res.fields

   tuples =
     tuples.collect do |tuple|
       h = {}
       fields.each do |field|
         h[field] = tuple.shift
       end
       h
     end

this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
module. it still requires work:

   res = pgconn.exec sql

   tuples = res.result
   fields = res.fields

   tuples.each{|t| t.fields = fields}

but now all 42000 tuples SHARE a copy of fields for doing their
lookups - i still have to iterate over all of em - but i don't need
to create any new objects and my code new reads like:

   if tuple['name'] =~ pat or tuple['age'].to_i < 42
     ...
   end

which is a lot nicer.

12

Hi,

At Wed, 30 Jun 2004 11:22:54 +0900,
Ara.T.Howard wrote in [ruby-talk:104892]:

> BTW, you couldn't use Struct?
>
> tuples =
> fields = Struct.new(nil, res.fields)
>
> tuples = res.collect {|tuple| fields.new(*tuple)}

i suppose i could... but there the would have a lot of object create on large
result sets no?

I don't like to hold such huge dataset in one time, so I'd use
PGresult#each instead as possible.

···

--
Nobu Nakada

13

Ara.T.Howard said:

the reason i wrote this in the first place was that i was doing alot of
postgresql work and it was returning these huge result sets that i would
then
write code that looked like:

   if tuple[17] =~ pat or tuple[9].to_i < 42
     ...
   end

FYI, for those using the DBI libraries, this functionality is already
available.

db = DBI.connect "DBI:yada yada ..."

=> #<DBI::DatabaseHandle:0x40e6b0 ... yada...>

row = db.select_one("SELECT z_route_num, z_step_num FROM s_route_step")

=> [710, 1]

row['z_route_num']

=> 710

row[0]

=> 710

row['z_step_num']

=> 1

row[1]

=> 1

If you wish to use DBI::Row independently, here is a demo ...

values = [0,1,2]

=> [0, 1, 2]

names = %w(zero one two)

=> ["zero", "one", "two"]

r = DBI::Row.new(names, values)

=> [0, 1, 2]

r['one']

=> 1

···

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

14

I like your solution above where none of the arrayfields code goes directly
into the Array class. Benefits: no overhead for Arrays that don't use the
arrayfields feature, and the ability to work with other objects that aren't
Array's but which have array-like functionality.

  Sean O'Dell

···

On Wednesday 30 June 2004 12:32, Ara.T.Howard wrote:

On Thu, 1 Jul 2004, Sean O'Dell wrote:
> Ara, any chance of getting arrayfields to not modify the Array methods
> but instead simply add an accessor to Array which returns an object that
> encapsulates this new functionality?
>
> So, instead of saying:
>
> row = ['bob', 30]
> row.fields = ['name', 'age']
> row['name'] #=> 'bob'
>
> you would say something like:
>
> row = ['bob', 30]
> row.arrayfields.fields = ['name', 'age']
> row.arrayfields['name'] #=> 'bob'
>
> My thinking is that if arrayfields functionality were contained in its
> own object that linked back to a parent array, you could create
> arrayfield objects and attach them to other "array-like" objects, such as
> database rows and so on.
>
> But also, it might prevent problems that arise later from taking over
> Array's methods where people want to use both arrayfields, and regular
> Array functionality.

well - theoretically you lose zero Array functionality since ArrayFields
methods are used if, and only if, the idx paramter is a String or Symbol
which would never be done with a normal Array.

that being said:

   jib:~/shared > cat a.rb
   require 'arrayfields'

   class FieldedArray
     attr :fieldset
     def initialize fields, array
       @a = array
       self.fields = fields
     end
     def method_missing(meth, *args, &block)
       @a.send(meth, *args, &block)
     end
     def fields= fields
       extend ArrayFields unless defined? @fieldset

       @fieldset =
         if ArrayFields::FieldSet === fields
           fields
         else
           ArrayFields::FieldSet.new fields
         end
     end
     attr_reader :fieldset
     def fields
       @fieldset and @fieldset.fields
     end
   end

   fa = FieldedArray.new %w(zero one two), [0, 1, 2]
   fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}

   jib:~/shared > ruby a.rb
   zero : 0
   one : 1
   two : 2

i can't bloody believe that worked - go matz!

what'ya think of adding the above to 3.3.0 (along with gem spec)?

15

yes.

   constants:

     can't be reused - obviously - so

       tuple_a[NAME]

       tuple_b[NAME]

     won't work of NAME is different for both

     this also doesn't work when the relation is modified. in general there is
     no guaruntee on the order of fields returned to this is never a good
     option.

   variables:

     i like to do this

       name, age, ssn = tuple.values_at 'name', 'age', 'ssn'

     so i already used up those var names - not that i couldn't use 'name_idx',
     etc. but this starts getting messy fast if you are dealing for four or
     five tables with 10-20 fields each.

lastly there is one nice feature of using the arrayfields method: say you are
generating a report from a relation using something like you are suggesting

   res = pgconn.exec sql
   tuples, fields = res.tuples, res.fields

   name, age, ssn = %w(name age ssn).map{|f| fields.index f}

   YAML::dump(
     tuples.map do |t|
       Hash[
         'name' => tuple[name],
         'age' => tuple[age],
         'ssn' => tuple[ssn],
       ]
     end
   )

and later the relation is modified to now have a new column you'll need to
modify the code. whereas if you use the fields this code will work even when
the table is modified

   res = pgconn.exec sql
   tuples, fields = res.tuples, res.fields

   YAML::dump(
     tuples.map do |t|
       t.fields = fields
       fields.inject({}){|h,f| h[f] = t[f]; h}
     end
   )

of course you could write the code to have this feature either way - my point
is just that when you extract the indexes into named vars you limit the way
your code will work as the database add/drops fields... i was originally also
doing alot of this sort of thing:

   def where_clause tuple, op = ' or '
     clauses =
       tuple.fields.map do |field|
         "#{ field } = '#{ tuple[field] }'"
       end

     clauses.join op
   end

which just started to get painful when the tuple didn't know it's own field
names... you can imagine doing alot of this stuff if you have to map fields
to var names and pass this mapping into each sql generation method...

cheers.

-a

···

On Wed, 30 Jun 2004, Ben Giddings wrote:

Is there any reason you didn't just use either constants or variables like:

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

16

not as dense as i am :wink: you are correct about the ram. i 'fixed' myself
into this when i was making some updates for testing - let me have a look at
it and post a fix....

-a

···

On Wed, 30 Jun 2004, Kirk Haines wrote:

Okay, admittedly, sometimes I'm too dense for my own good, but I'm
confused. It looks to me like, internally, each array is using a hash to
keep track of the fields to indices mapping, and that hash is not shared
amongst arrays with the same set of fields. So you end up using more memory
using arrayfields for something like database result sets than if you just
used hashes.

I put together a few simple little test programs to try it out, using
arrayfields and using hashes to store similar sets of data, and in practice
it looks like arrayfields uses almost 2x the amount of RAM for a given data
set, as well. Am I missing something, here?

Thanks,

Kirk Haines

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================