[ANN] Article: An Exercise in Metaprogramming with Ruby

I failed to post this link before, so here it is now:

聽聽http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it "didn't do that well" in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we'll see what effect it has.

So there you have it. No bots or artificial inflation, please. :wink:

This article, by the way, was adapted from a talk given in January
to the Austin on Rails group.

Cheers,
Hal

Interesting article... just about the level I needed. A decent
example, not uselessly trivial, but not terribly complex either, so I
can follow enough of the metaprogramming to truly understand what's
going on.

Thanks...

路路路

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it "didn't do that well" in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we'll see what effect it has.

So there you have it. No bots or artificial inflation, please. :wink:

This article, by the way, was adapted from a talk given in January
to the Austin on Rails group.

Cheers,
Hal

Hal wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

Wow. Very nice. I'm doing a fair amount of meta programing myself these days and having resources like this for reference and information is great.

I like the way you dynamically create the class using Object#const_set. Metaprogramming sure beats generating code the traditional way.

路路路

--
John Long
http://wiseheartdesign.com

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it "didn't do that well" in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we'll see what effect it has.

Nice article Hal. It's a shame ZD requires registration to comment
there though.

So there you have it. No bots or artificial inflation, please. :wink:

I'll be sharing it around at work, just a bit of natural inflation :slight_smile:

路路路

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:

This article, by the way, was adapted from a talk given in January
to the Austin on Rails group.

Cheers,
Hal

--
thanks,
-pate
-------------------------

That's so darn cool, I think I'm just going to have to add it to FasterCSV... :wink:

Excellent article Hal!

James Edward Gray II

路路路

On Mar 24, 2006, at 1:38 PM, rubyhacker@gmail.com wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

Nice article. As others have said, a clear survey of some intermediate
techniques. A couple of suggestions:

1) More sample output or interactive session information. You build up the
People class kind of piecemeal without giving intermediate output along the
way. I think that might be hard for someone new to metaprogramming to
follow.
2) You use some advanced syntax that might confuse others (example: splat
operator for attributes). There is a lot packed into that article - I would
reduce tricks like that as much as possible.

Good luck with page views!

路路路

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

My editor said it "didn't do that well" in terms of page views. And I
said,
well, I should have posted it to ruby-talk. And she said: Do that now,
and we'll see what effect it has.

So there you have it. No bots or artificial inflation, please. :wink:

This article, by the way, was adapted from a talk given in January
to the Austin on Rails group.

Cheers,
Hal

I'm a Ruby newbie. So far I am loving everything I learn about Ruby. I'm trying to find a real app to create with it. I have need for a client program that talks to an LDAP server and that makes calls to an ONC/RPC server that we wrote here at my job in C++. Do these exist for Ruby?

Thanks!

Ernest

Of course, un-pedagogically, it could be compressed a tad:

聽聽class DataRecord

聽聽聽聽def self.make(file_name)
聽聽聽聽聽聽header = File.open(file_name){|f|f.gets}.split(/,/)
聽聽聽聽聽聽struct = File.basename(file_name,'.txt').capitalize
聽聽聽聽聽聽record = Struct.new(struct, *header)

聽聽聽聽聽聽class<<record;self;end.send :define_method, :read do
聽聽聽聽聽聽聽聽File.open(file_name) do |f|
聽聽聽聽聽聽聽聽聽聽f.gets
聽聽聽聽聽聽聽聽聽聽f.inject([]){|a,l| a << record.new(*eval("[#{l}]")) }
聽聽聽聽聽聽聽聽end
聽聽聽聽聽聽end

聽聽聽聽聽聽record
聽聽聽聽end

聽聽end

聽聽data = DataRecord.make('people.txt')
聽聽list = data.read # or: Struct::People.read
聽聽person = list[2]
聽聽puts person.name
聽聽if person.age < 18
聽聽聽聽puts "under 18"
聽聽else
聽聽聽聽puts "over 18"
聽聽end
聽聽puts "Weight is: %.2f kg" % kg = person.weight / 2.2

cheers,
andrew

路路路

--
Andrew L. Johnson http://www.siaris.net/
聽聽聽聽聽聽What have you done to the cat? It looks half-dead.
聽聽聽聽聽聽聽聽聽聽-- Schroedinger's wife

Thanks very much, you've inspired me to clean up some crusty code I've been ignoring for a while!

路路路

On 24 Mar 2006, at 19:38, rubyhacker@gmail.com wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

[...]

i like the article!

yeah, and even if code using the fields is coupled tightly to the
created classes, the solution is highly reusable.
-- henon

路路路

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

Interesting article... just about the level I needed. A decent
example, not uselessly trivial, but not terribly complex either, so I
can follow enough of the metaprogramming to truly understand what's
going on.

+1

Great stuff Hal, thanks much.

路路路

On 3/24/06, Matthew Moss <matthew.moss.coder@gmail.com> wrote:

Thanks...

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:
> I failed to post this link before, so here it is now:
>
> http://www.devsource.com/article2/0,1895,1928561,00.asp
>
> My editor said it "didn't do that well" in terms of page views. And I
> said,
> well, I should have posted it to ruby-talk. And she said: Do that now,
> and we'll see what effect it has.
>
> So there you have it. No bots or artificial inflation, please. :wink:
>
> This article, by the way, was adapted from a talk given in January
> to the Austin on Rails group.
>
>
> Cheers,
> Hal
>
>
>

--
Bill Guindon (aka aGorilla)
The best answer to most questions is "it depends".

Question along these lines, suppose you add an attribute to the
'People' class after the initial creation (say by adding another
column to the people.txt file), do the 'old' people classes get the
new attribute as well? If so, what's the initial value? I suspect it
would be nil.

thanks,

路路路

On 3/24/06, John W. Long <ng@johnwlong.com> wrote:

Hal wrote:
> I failed to post this link before, so here it is now:
>
> http://www.devsource.com/article2/0,1895,1928561,00.asp
>

Wow. Very nice. I'm doing a fair amount of meta programing myself these
days and having resources like this for reference and information is great.

I like the way you dynamically create the class using Object#const_set.
Metaprogramming sure beats generating code the traditional way.

--
John Long
http://wiseheartdesign.com

--
Keith Sader
ksader@gmail.com
http://www.saderfamily.org/roller/page/ksader

Meinrad Recheis wrote:

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

[...]

i like the article!

yeah, and even if code using the fields is coupled tightly to the
created classes, the solution is highly reusable.
-- henon

The point about coupling (mentioned in the second-to-last paragraph of
the article) is important, and I feel it is dismissed to easily in the
article. There are some tradeoffs to consider, though perhaps they are
out of the scope of the article, which is intended as an exercise, not
as a complete guide:

1. Suppose your code needs to _discover_ what fields are in the file?
You can use #instance_methods(false), but that is not perfect: you have
to filter out "to_s" and "inspect", which were added by #make. And what
if you add a new method in addition to the ones generated by #make? The
field names could be stored in a list kept in class instance variable...

2. Once you have discovered the field names, you have to use
send(fieldname) and send("#{fieldname}=") to access them. That's more
awkward and (at least in the second case) less efficient than Hash#
and #=. Who's the "second class citizen" in this case?

3. If you really know the field names "in advance" (that is, you have
enough information to hard code them into your program), rather than "by
discovery", then maybe it is better to use a different metaprogramming
style in which the fields are declared using class methods:

  class Person
    field :name, :favorite_ice_cream, ...
  end

In this way, some rudimentary error checking can be performed when
reading the file, rather than failing later when trying to serve ice
cream to the person. (I just hate it when my ruby-scripted robo-fridge
serves me passion fruit and rabbit dropping ice cream.) This is not
always the best way to go (what if, as the article points out, fields
get added to the file?), but one more thing to keep in mind.

4. Is it always a good idea to couple the class name with the file name?
Maybe the class's identity should be associated with the set of fields
defined in the header? Why not _reuse_ the anonymous class if the header
is the same as those in some other file you imported earlier? (This
could be done using a hash mapping sorted lists of column names to
classes.) That would make it possible to use == to compare objects read
from different files. Further, it would let you use x.class == y.class
to determine if x and y came from files with compatible formats (same
fields, but maybe in a different order).

5. Maybe Struct would serve just as well, since it takes care of
everything in the class_eval block. For example:

  klass = Struct.new(*names.map{|s|s.intern})

None of these are necessarily problems, depending on what you are trying
to do, but alternate solutions (for example, using hashes) are worth
considering. Metaprogramming is not always the best solution, though it
is good to have it in your pocket.

Some minor quibbles:

1. In DataRecord.make, if the file happens to be empty, data.gets.chomp
will raise an exception and the file will not be closed. Similarly in
the #read method of the generated class. Why not use a block with File.open?

2. The second way of referring to the class:

    require 'my-csv'
    DataRecord.make("people.txt") # Ignore the return value and
    list = People.read # refer to the class by name.

should raise the hackles on a ruby programmer's neck. It's a violation
of DRY: you have typed the string "people" in two places, and your
program breaks if (a) the filename changes or (b) the way "people.txt"
is transformed into "People" changes. Maybe you _want_ that breakage
(maybe you want the program to fail if someone tries to run it on
"other-people.txt" or on "places.txt"). Or maybe not: it's another
tradeoff. (To be fair, the article doesn't claim that the version with
People hard-coded can read places.txt.)

3. Is it really a good idea to encourage people to eval("[#{line}]") ???

路路路

On 3/24/06, rubyhacker@gmail.com <rubyhacker@gmail.com> wrote:

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

I failed to post this link before, so here it is now:

  http://www.devsource.com/article2/0,1895,1928561,00.asp

That's so darn cool, I think I'm just going to have to add it to FasterCSV... :wink:

Developer at play:

$ irb -r lib/faster_csv.rb
>> class FullName < Struct.new(:first, :last)
>> def initialize( first, last, other = Hash.new )
>> super(first, last)
>> @middle, @suffix = other.values_at(:middle, :suffix)
>> end
>> attr_accessor :middle, :suffix
>> end
=> nil
>> names = [ FullName.new("Santa", "Clause"),
?> FullName.new("James", "Gray", :middle => "Edward", :suffix => "II"),
?> FullName.new("Easter", "Bunny") ]
=> [#<struct FullName first="Santa", last="Clause">, #<struct FullName first="James", last="Gray">, #<struct FullName first="Easter", last="Bunny">]
>> csv = FasterCSV.dump(names)
=> "class,FullName\n@middle,@suffix,first=,last=\n,Santa,Clause\nEdward,II,James,Gray\n,Easter,Bunny\n"
>> puts csv
class,FullName
@middle,@suffix,first=,last=
,Santa,Clause
Edward,II,James,Gray
,Easter,Bunny
=> nil
>> reloaded = FasterCSV.load(csv)
=> [#<struct FullName first="Santa", last="Clause">, #<struct FullName first="James", last="Gray">, #<struct FullName first="Easter", last="Bunny">]
>> reloaded.find { |name| name.first == "James" }.middle
=> "Edward"

That's using the development version of FasterCSV. Thanks for the idea Hal! :wink:

James Edward Gray II

路路路

On Mar 24, 2006, at 2:28 PM, James Edward Gray II wrote:

On Mar 24, 2006, at 1:38 PM, rubyhacker@gmail.com wrote:

i'm totally with you on this joel. still, i think one can have a bit of both:

     harp:~ > cat a.rb
     require "arrayfields"
     require "csv"

     csv = <<-csv
     latitude,longitude,description
     47.23,59.34,Omaha
     32.17,39.24,New York City
     73.11,48.91,Carlsbad Caverns
     csv

     class CSVTable < ::Array
       attr "fields"
       def initialize arg
         CSV::parse(arg) do |row|
           row.map!{|c| c.to_s}
           if @fields
             self << row
           else
             @row_class = Class::new(::Array) do
               define_method("initialize") do |a|
                 self.fields = row
                 replace a
               end
             end
             @fields = row
           end
         end
         @fields.each{|field| column_attr field}
       end
       def << row
         super @row_class::new(row)
       end
       def column_attr(ca)
         singleton_class = class << self; self; end
         singleton_class.module_eval{ define_method(ca){ map{|r| r[ca]}} }
       end
       def (*a, &b)
         m = a.first
         return(send(m)) if [String, Symbol].map{|c| c === m}.any? && respond_to?(m)
         super
       end
     end

     table = CSVTable::new csv

     p table
     puts

     p table.fields
     puts

     table.fields.each{|f| puts "#{ f }: #{ table[f].join(', ') }"}
     puts

     table.each{|row| puts row.fields.map{|f| "#{ f }: #{ row[f] }"}.join(', ') }
     puts

     harp:~ > ruby a.rb
     [["47.23", "59.34", "Omaha"], ["32.17", "39.24", "New York City"], ["73.11", "48.91", "Carlsbad Caverns"]]

     ["latitude", "longitude", "description"]

     latitude: 47.23, 32.17, 73.11
     longitude: 59.34, 39.24, "48.91
     description: Omaha, New York City, Carlsbad Caverns

     latitude: 47.23, longitude: 59.34, description: Omaha
     latitude: 32.17, longitude: 39.24, description: New York City
     latitude: 73.11, longitude: 48.91, description: Carlsbad Caverns

regards.

-a

路路路

On Sun, 26 Mar 2006, Joel VanderWerf wrote:

The point about coupling (mentioned in the second-to-last paragraph of the
article) is important, and I feel it is dismissed to easily in the article.
There are some tradeoffs to consider, though perhaps they are out of the
scope of the article, which is intended as an exercise, not as a complete
guide:

1. Suppose your code needs to _discover_ what fields are in the file? You
can use #instance_methods(false), but that is not perfect: you have to
filter out "to_s" and "inspect", which were added by #make. And what if you
add a new method in addition to the ones generated by #make? The field names
could be stored in a list kept in class instance variable...

2. Once you have discovered the field names, you have to use send(fieldname)
and send("#{fieldname}=") to access them. That's more awkward and (at least
in the second case) less efficient than Hash# and #=. Who's the "second
class citizen" in this case?

3. If you really know the field names "in advance" (that is, you have enough
information to hard code them into your program), rather than "by
discovery", then maybe it is better to use a different metaprogramming style
in which the fields are declared using class methods:

class Person
   field :name, :favorite_ice_cream, ...
end

In this way, some rudimentary error checking can be performed when reading
the file, rather than failing later when trying to serve ice cream to the
person. (I just hate it when my ruby-scripted robo-fridge serves me passion
fruit and rabbit dropping ice cream.) This is not always the best way to go
(what if, as the article points out, fields get added to the file?), but one
more thing to keep in mind.

--
share your knowledge. it's a way to achieve immortality.
- h.h. the 14th dali lama

The point about coupling (mentioned in the second-to-last paragraph of
the
article) is important, and I feel it is dismissed to easily in the
article.
There are some tradeoffs to consider, though perhaps they are out of the
scope of the article, which is intended as an exercise, not as a complete
guide:

1. Suppose your code needs to _discover_ what fields are in the file?
You
can use #instance_methods(false), but that is not perfect: you have to
filter out "to_s" and "inspect", which were added by #make. And what
if you
add a new method in addition to the ones generated by #make? The field
names
could be stored in a list kept in class instance variable...

2. Once you have discovered the field names, you have to use
send(fieldname)
and send("#{fieldname}=") to access them. That's more awkward and (at
least
in the second case) less efficient than Hash# and #=. Who's the
"second
class citizen" in this case?

3. If you really know the field names "in advance" (that is, you have
enough
information to hard code them into your program), rather than "by
discovery", then maybe it is better to use a different metaprogramming
style
in which the fields are declared using class methods:

class Person
   field :name, :favorite_ice_cream, ...
end

In this way, some rudimentary error checking can be performed when
reading
the file, rather than failing later when trying to serve ice cream to the
person. (I just hate it when my ruby-scripted robo-fridge serves me
passion
fruit and rabbit dropping ice cream.) This is not always the best way
to go
(what if, as the article points out, fields get added to the file?),
but one
more thing to keep in mind.

i'm totally with you on this joel. still, i think one can have a bit of
both:

...

    class CSVTable < ::Array
      attr "fields"

Sure, that's more or less what I meant by storing the fields, but I was
thinking of using a class instance variable to keep that information at
the class level (assuming you might want to reuse one table class and
one row class for several files).

Using arrayfields is nice, since you have the symbolic # and #=
interfaces, as with hashes, as well as the array interface. But you have
neither a declared list of what fields should be in the file (what I was
suggesting for error checking purposes), nor the ability to refer to
fields directly with a "first class citizen" method (what Hal's article
was advocating):

p table[1].latitude

Nothing wrong with any of these approaches, it's just good to be aware
of all of them.

Btw, you can use a block with #any? :

        return(send(m)) if [String, Symbol].map{|c| c === m}.any? &&
respond_to?(m)

    return(send(m)) if [String, Symbol].any? {|c| c === m} && respond_to?(m)

路路路

ara.t.howard@noaa.gov wrote:

On Sun, 26 Mar 2006, Joel VanderWerf wrote:

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Joel VanderWerf wrote:

[snip snip snip]

Joel,

Thanks for your comments. I have read them with interest
(and everyone else's) but am too busy to reply at length.

In short, yes, there are flaws in the approach I took, and
there is more than one way to do it. For the most part, it's
just an exercise.

Thanks,
Hal