Adding a dynamic method handler? (long post)

Hi,

I've been using method_missing overly much in my code lately, and it's
prompted me to think a lot about it's limitations. I've been wishing
for a version of method_missing that allows the dynamic methods to act
more like they are real methods on the object. I think this could be
done by implementing a new method hook especially for dynamic methods:
dynamic_method.

dynamic_method would be called before method_missing if a method
lookup fails. If dynamic_method fails to handle the message,
method_missing will recieve the message for handling.

A couple of the immediate benefits afforded by a good implementation
of a dynamic method handler:
- the object will respond_to? the method
- you can call method(:foo) to get a copy of the dynamic method.

dynamic_method would be used something like this:

  class Foo
    def dynamic_method(name)
      if name.to_s =~ /^foo/
        # create the method (as a proc)
        return lambda do |*args|
          args.map{|arg| name.to_s.sub(/^foo/, arg.to_s) }
        end
      end
    end
  end

  f = Foo.new
    ==>#<Foo:0x589a58>
  f.respond_to? :foobar
    ==>true
  f.respond_to? :barfoo
    ==>false
  f.foobar(*%w[one two three])
    ==>["onebar", "twobar", "threebar"]
  f.barfoo(*%w[one two three])
  NoMethodError: undefined method `barfoo' for #<Foo:0x589a58>
  
  f.method(:foobaz).call(*%w[one two three])
    ==>["onebaz", "twobaz", "threebaz"]

As you can see, a user-defined dynamic_method returns either a
callable object (Proc, Method, etc) or nil. If dynamic_method(message)
returns nil, it is assumed that the object does not
respond_to?(message), and method(message) should raise a
NoMethodError.

And here's a lightweight example implementation:

  module Kernel
    alias_method :old_method_missing, :method_missing
    def method_missing(name, *args, &block)
      m = dynamic_method(name)
      if m
        m.call(*args, &block)
      else
        m = Kernel.instance_method(:old_method_missing)
        m.bind(self).call(name, *args, &block)
      end
    end
    
    alias_method :old_respond_to?, :respond_to?
    def respond_to?(msg)
      (old_respond_to?(msg) || dynamic_method(msg)) ? true : false
    end
    
    alias_method :old_method, :method
    def method(name)
      old_method(name)
    rescue NameError => e
      m = dynamic_method(name)
      raise e unless m
      m
    end
    
    def dynamic_method(name)
      nil
    end
  end

Here's a lightweight version of OpenStruct written using dynamic_method:

  class OpenStruct
    def initialize(hash = {})
      @table = {}
      hash.each do |key, value|
        @table[key.to_sym] = value
      end
    end
    
    def dynamic_method(name)
      if name.to_s =~ /\=\z/
        lambda{|val| @table[name.to_s.chop.intern] = val }
      elsif @table.keys.include?(name)
        lambda{ @table[name] }
      end
    end
  end
  
And using it:

  os = OpenStruct.new
    ==>#<OpenStruct:0x511454 @table={}>
  os.red = 23
    ==>23
  os.blue = 42
    ==>42
  os.green = 56
    ==>56
  os.respond_to? :blue
    ==>true
  os.respond_to? :periwinkle
    ==>false
  os_blue = os.method(:blue)
    ==>#<Proc:0x0038743c@(eval):13>
  os.blue = 1024
    ==>1024
  os_blue.call
    ==>1024

This is not a complete idea (let alone implementation) at this time...
I just wanted to see if anyone had an opinion on whether the idea was
worth anything, or could make suggestions to improve the interface.

Now here's me second-guessing myself: The implementation is pretty
complicated; adding another dynamic message handler may not be worth
the confusion. It would be one more thing to explain to people, and
while method_missing is an elegant addition to a language, I'm not
sure this would be. Especially considering the need to add more
complexity to method lookups.

Still, I think that even if this idea here isn't worthy, putting it
out there might help someone else come up with a more elegant
solution.

So, any thoughts?

cheers,
Mark

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca0502152303763354f7@mail.gmail.com...

Hi,

I've been using method_missing overly much in my code lately, and it's
prompted me to think a lot about it's limitations.

<snip/>

So, any thoughts?

I'm wondering in which situation you need this. Although I understand the
benefits of your approach I don't see the use case for this.

Kind regards

    robert

Mark Hubbart wrote:

I've been wishing
for a version of method_missing that allows the dynamic methods to act
more like they are real methods on the object.

I am not sure what you mean by dynamic methods. But in case only Structs are
concerned wouldn't it make more sence to redefine method_missing for
*Structs(Open-Super, ...).?

benny

Basically this is for any time that you want the code re-use and ease
of implementation afforded by method_missing, but the benefits of
still having the methods behave mostly as if they were actually
defined, rather than handled dynamically. This is useful for quickly
defining wrapper objects, or objects that delegate to multiple other
objects.

The idea is that this would be a way of dynamically creating methods
for an object, without resorting to relatively permanent methods like
"class << self; define_method(:foo){...}; end".

cheers,
Mark

···

On Wed, 16 Feb 2005 18:19:50 +0900, Robert Klemme <bob.news@gmx.net> wrote:

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca0502152303763354f7@mail.gmail.com...
> Hi,
>
> I've been using method_missing overly much in my code lately, and it's
> prompted me to think a lot about it's limitations.

<snip/>

> So, any thoughts?

I'm wondering in which situation you need this. Although I understand the
benefits of your approach I don't see the use case for this.

Wrote Robert Klemme <bob.news@gmx.net>, on Wed, Feb 16, 2005 at 06:19:50PM +0900:

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca0502152303763354f7@mail.gmail.com...
> Hi,
>
> I've been using method_missing overly much in my code lately, and it's
> prompted me to think a lot about it's limitations.

<snip/>

> So, any thoughts?

I'm wondering in which situation you need this. Although I understand the
benefits of your approach I don't see the use case for this.

I think I see what Mark was getting at. As I understand it, if I defined
a proxy object that used method_missing to forward all method calls to
an underlying object, I could call

  proxy.to_ary

and if the underlying object was an Array, this would work.

However, if I passed that into a library that was using duck-typing, and
that lib did

  proxy.responds_to? :to_ary

the answer would be false. So, my Array proxy doesn't look as much like
an Array as it needs to.

Do I understand correctly?

Cheers,
Sam

···

--
Sam Roberts <sroberts@certicom.com>

benny wrote:

Mark Hubbart wrote:

I've been wishing
for a version of method_missing that allows the dynamic methods to act
more like they are real methods on the object.

I am not sure what you mean by dynamic methods. But in case only Structs
are concerned wouldn't it make more sence to redefine method_missing for
*Structs(Open-Super, ...).?

benny

Oh, I am sorry: this was kind of stupid, since method_missing belongs to
Kernel :frowning:

benny

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca050216125273686a45@mail.gmail.com...

>
> "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> news:de63abca0502152303763354f7@mail.gmail.com...
> > Hi,
> >
> > I've been using method_missing overly much in my code lately, and

it's

> > prompted me to think a lot about it's limitations.
>
> <snip/>
>
> > So, any thoughts?
>
> I'm wondering in which situation you need this. Although I understand

the

> benefits of your approach I don't see the use case for this.

Basically this is for any time that you want the code re-use and ease
of implementation afforded by method_missing, but the benefits of
still having the methods behave mostly as if they were actually
defined, rather than handled dynamically. This is useful for quickly
defining wrapper objects, or objects that delegate to multiple other
objects.

The idea is that this would be a way of dynamically creating methods
for an object, without resorting to relatively permanent methods like
"class << self; define_method(:foo){...}; end".

I'm sorry if I am being stubborn (or dump), but this is still pretty much
abstract. I'd like to know which concrete use case made this behavior
necessary.

Kind regards

    robert

···

On Wed, 16 Feb 2005 18:19:50 +0900, Robert Klemme <bob.news@gmx.net> wrote:

Yes, that's the general rationale I had for this. Getting more of the
reflection methods to work for dynamically defined methods.

cheers,
Mark

···

On Thu, 17 Feb 2005 08:56:34 +0900, Sam Roberts <sroberts@certicom.com> wrote:

Wrote Robert Klemme <bob.news@gmx.net>, on Wed, Feb 16, 2005 at 06:19:50PM +0900:
>
> "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> news:de63abca0502152303763354f7@mail.gmail.com...
> > Hi,
> >
> > I've been using method_missing overly much in my code lately, and it's
> > prompted me to think a lot about it's limitations.
>
> <snip/>
>
> > So, any thoughts?
>
> I'm wondering in which situation you need this. Although I understand the
> benefits of your approach I don't see the use case for this.

I think I see what Mark was getting at. As I understand it, if I defined
a proxy object that used method_missing to forward all method calls to
an underlying object, I could call

  proxy.to_ary

and if the underlying object was an Array, this would work.

However, if I passed that into a library that was using duck-typing, and
that lib did

  proxy.responds_to? :to_ary

the answer would be false. So, my Array proxy doesn't look as much like
an Array as it needs to.

Do I understand correctly?

benny wrote:

benny wrote:

Mark Hubbart wrote:

I've been wishing
for a version of method_missing that allows the dynamic methods to act
more like they are real methods on the object.

I am not sure what you mean by dynamic methods. But in case only Structs
are concerned wouldn't it make more sence to redefine method_missing for
*Structs(Open-Super, ...).?

benny

Oh, I am sorry: this was kind of stupid, since method_missing belongs to
Kernel :frowning:

benny

I had expected it to be part of Object (would be way more flexible IMHO)

benny

Maybe stubborn, but that's not always a bad thing. There are many
things I'm very glad Matz is stubborn about :slight_smile:

I don't think I ever implied that this behavior was *necessary*. You
can implement similar functionality with what is currently available.
It's just a pain in the neck to do it; You either have to break down
and def a bunch of methods, or override respond_to? and method in your
class. And sometimes neither of those is the *best* solution.

I did give a specific use case where it would be very useful, though.
When wrapping a class, or doing runtime refactoring (like what
pathname does, which is, for the most part, a refactored wrapper for
File and Dir), this could be very handy. Like method_missing, it would
allow you to handle large amounts of similar methods at once, letting
you condense code; while still getting almost all the benefits of
actually defining each individual method.

  class DirectoryItem
    def initialize(path)
      @path = path
    end
    [...]
    def dynamic_method(name)
      @@file_methods = (File.methods - Object.methods).map{|s|s.to_sym}
      @@dir_methods = (Dir.methods - Object.methods).map{|s|s.to_sym}
      if @@file_methods.include? name
        if File.method(name).arity == 1
          lambda{ File.send(name, @path) }
        elsif name == :truncate
          lambda{|len| File.send(name, @path, len) }
        elsif File.method(name).arity == 2
          lambda{|path| File.send(name, @path, path) }
        end
      elsif @@dir_methods.include? name
        # handle Dir methods here
      end
    end
  end

The equivalent portion of code using 'def' would be much, much longer,
and very repetitive. The equivalent code using method_missing would be
about the same length, but if anyone tried to check it's capabilities,
it would seem to almost be an empty object.

cheers,
Mark

···

On Thu, 17 Feb 2005 17:19:54 +0900, Robert Klemme <bob.news@gmx.net> wrote:

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca050216125273686a45@mail.gmail.com...
> On Wed, 16 Feb 2005 18:19:50 +0900, Robert Klemme <bob.news@gmx.net> > wrote:
> >
> > "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> > news:de63abca0502152303763354f7@mail.gmail.com...
> > > Hi,
> > >
> > > I've been using method_missing overly much in my code lately, and
it's
> > > prompted me to think a lot about it's limitations.
> >
> > <snip/>
> >
> > > So, any thoughts?
> >
> > I'm wondering in which situation you need this. Although I understand
the
> > benefits of your approach I don't see the use case for this.
>
> Basically this is for any time that you want the code re-use and ease
> of implementation afforded by method_missing, but the benefits of
> still having the methods behave mostly as if they were actually
> defined, rather than handled dynamically. This is useful for quickly
> defining wrapper objects, or objects that delegate to multiple other
> objects.
>
> The idea is that this would be a way of dynamically creating methods
> for an object, without resorting to relatively permanent methods like
> "class << self; define_method(:foo){...}; end".

I'm sorry if I am being stubborn (or dump), but this is still pretty much
abstract. I'd like to know which concrete use case made this behavior
necessary.

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca050217095240cde47c@mail.gmail.com...

>
> "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> news:de63abca050216125273686a45@mail.gmail.com...
> > >
> > > "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> > > news:de63abca0502152303763354f7@mail.gmail.com...
> > > > Hi,
> > > >
> > > > I've been using method_missing overly much in my code lately,

and

> it's
> > > > prompted me to think a lot about it's limitations.
> > >
> > > <snip/>
> > >
> > > > So, any thoughts?
> > >
> > > I'm wondering in which situation you need this. Although I

understand

> the
> > > benefits of your approach I don't see the use case for this.
> >
> > Basically this is for any time that you want the code re-use and

ease

> > of implementation afforded by method_missing, but the benefits of
> > still having the methods behave mostly as if they were actually
> > defined, rather than handled dynamically. This is useful for quickly
> > defining wrapper objects, or objects that delegate to multiple other
> > objects.
> >
> > The idea is that this would be a way of dynamically creating methods
> > for an object, without resorting to relatively permanent methods

like

> > "class << self; define_method(:foo){...}; end".
>
> I'm sorry if I am being stubborn (or dump), but this is still pretty

much

> abstract. I'd like to know which concrete use case made this behavior
> necessary.

Maybe stubborn, but that's not always a bad thing. There are many
things I'm very glad Matz is stubborn about :slight_smile:

I don't think I ever implied that this behavior was *necessary*. You
can implement similar functionality with what is currently available.
It's just a pain in the neck to do it; You either have to break down
and def a bunch of methods, or override respond_to? and method in your
class. And sometimes neither of those is the *best* solution.

I did give a specific use case where it would be very useful, though.
When wrapping a class, or doing runtime refactoring (like what
pathname does, which is, for the most part, a refactored wrapper for
File and Dir), this could be very handy. Like method_missing, it would
allow you to handle large amounts of similar methods at once, letting
you condense code; while still getting almost all the benefits of
actually defining each individual method.

  class DirectoryItem
    def initialize(path)
      @path = path
    end
    [...]
    def dynamic_method(name)
      @@file_methods = (File.methods - Object.methods).map{|s|s.to_sym}
      @@dir_methods = (Dir.methods - Object.methods).map{|s|s.to_sym}
      if @@file_methods.include? name
        if File.method(name).arity == 1
          lambda{ File.send(name, @path) }
        elsif name == :truncate
          lambda{|len| File.send(name, @path, len) }
        elsif File.method(name).arity == 2
          lambda{|path| File.send(name, @path, path) }
        end
      elsif @@dir_methods.include? name
        # handle Dir methods here
      end
    end
  end

The equivalent portion of code using 'def' would be much, much longer,
and very repetitive. The equivalent code using method_missing would be
about the same length, but if anyone tried to check it's capabilities,
it would seem to almost be an empty object.

Although I agree to that - wouldn't it be more efficient to define all
forwarding methods in DirectoryItem class once and for all? That way you
get these benefits:

- faster as methods can be invoked directly and no lambdas have to be
created
- reduced mem usage as not every instance has its own copy of the lambdas

Drawback is of course that methods added to File and Dir later don't get
invoked - but it's unlikely in this case I'd say.

Also, a suggestion for improvement: wrap dynamic_method with a method
similar to this, which will reduce the number of created lambdas:

# untested
def dyn_create(name)
  m = dynamic_method(name)
  class<<self;self;end.class_eval do
    define_method(name.to_sym,*a,&m)
  end
end

Then invoke this method via respond_to?, method_missing and method. Then
the method is defined on first access. What do you think?

Kind regards

    robert

···

On Thu, 17 Feb 2005 17:19:54 +0900, Robert Klemme <bob.news@gmx.net> wrote:
> > On Wed, 16 Feb 2005 18:19:50 +0900, Robert Klemme <bob.news@gmx.net> > > wrote:

That's an interesting way of doing it. I suppose if at a later time
the method needs to be modified, you could always undef it and let the
dyn_create method catch it the next time through.

On the other hand, the more I think about this, the more complicated
it seems to get. I'm not really as gung ho on the idea as when I first
thought of it. Maybe if it was worked into the syntax somehow, it
would be worthwhile; but I wouldn't know where to start. Perhaps a
better way to go would be to make it easier to wrap respond_to?() and
methods(). Which is coming in 2.0 anyway :slight_smile:

cheers,
Mark

···

On Fri, 18 Feb 2005 17:39:47 +0900, Robert Klemme <bob.news@gmx.net> wrote:

"Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
news:de63abca050217095240cde47c@mail.gmail.com...
> On Thu, 17 Feb 2005 17:19:54 +0900, Robert Klemme <bob.news@gmx.net> > wrote:
> >
> > "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> > news:de63abca050216125273686a45@mail.gmail.com...
> > > On Wed, 16 Feb 2005 18:19:50 +0900, Robert Klemme <bob.news@gmx.net> > > > wrote:
> > > >
> > > > "Mark Hubbart" <discordantus@gmail.com> schrieb im Newsbeitrag
> > > > news:de63abca0502152303763354f7@mail.gmail.com...
> > > > > Hi,
> > > > >
> > > > > I've been using method_missing overly much in my code lately,
and
> > it's
> > > > > prompted me to think a lot about it's limitations.
> > > >
> > > > <snip/>
> > > >
> > > > > So, any thoughts?
> > > >
> > > > I'm wondering in which situation you need this. Although I
understand
> > the
> > > > benefits of your approach I don't see the use case for this.
> > >
> > > Basically this is for any time that you want the code re-use and
ease
> > > of implementation afforded by method_missing, but the benefits of
> > > still having the methods behave mostly as if they were actually
> > > defined, rather than handled dynamically. This is useful for quickly
> > > defining wrapper objects, or objects that delegate to multiple other
> > > objects.
> > >
> > > The idea is that this would be a way of dynamically creating methods
> > > for an object, without resorting to relatively permanent methods
like
> > > "class << self; define_method(:foo){...}; end".
> >
> > I'm sorry if I am being stubborn (or dump), but this is still pretty
much
> > abstract. I'd like to know which concrete use case made this behavior
> > necessary.
>
> Maybe stubborn, but that's not always a bad thing. There are many
> things I'm very glad Matz is stubborn about :slight_smile:
>
> I don't think I ever implied that this behavior was *necessary*. You
> can implement similar functionality with what is currently available.
> It's just a pain in the neck to do it; You either have to break down
> and def a bunch of methods, or override respond_to? and method in your
> class. And sometimes neither of those is the *best* solution.
>
> I did give a specific use case where it would be very useful, though.
> When wrapping a class, or doing runtime refactoring (like what
> pathname does, which is, for the most part, a refactored wrapper for
> File and Dir), this could be very handy. Like method_missing, it would
> allow you to handle large amounts of similar methods at once, letting
> you condense code; while still getting almost all the benefits of
> actually defining each individual method.
>
> class DirectoryItem
> def initialize(path)
> @path = path
> end
> [...]
> def dynamic_method(name)
> @@file_methods = (File.methods - Object.methods).map{|s|s.to_sym}
> @@dir_methods = (Dir.methods - Object.methods).map{|s|s.to_sym}
> if @@file_methods.include? name
> if File.method(name).arity == 1
> lambda{ File.send(name, @path) }
> elsif name == :truncate
> lambda{|len| File.send(name, @path, len) }
> elsif File.method(name).arity == 2
> lambda{|path| File.send(name, @path, path) }
> end
> elsif @@dir_methods.include? name
> # handle Dir methods here
> end
> end
> end
>
> The equivalent portion of code using 'def' would be much, much longer,
> and very repetitive. The equivalent code using method_missing would be
> about the same length, but if anyone tried to check it's capabilities,
> it would seem to almost be an empty object.

Although I agree to that - wouldn't it be more efficient to define all
forwarding methods in DirectoryItem class once and for all? That way you
get these benefits:

- faster as methods can be invoked directly and no lambdas have to be
created
- reduced mem usage as not every instance has its own copy of the lambdas

Drawback is of course that methods added to File and Dir later don't get
invoked - but it's unlikely in this case I'd say.

Also, a suggestion for improvement: wrap dynamic_method with a method
similar to this, which will reduce the number of created lambdas:

# untested
def dyn_create(name)
  m = dynamic_method(name)
  class<<self;self;end.class_eval do
    define_method(name.to_sym,*a,&m)
  end
end

Then invoke this method via respond_to?, method_missing and method. Then
the method is defined on first access. What do you think?