Python-style decorators

I've been using ruby for about 8 months now and I've come to appreciate
its powerful features. But I miss python's decorators, so I made
something similar.

The code is attached; here are some examples. It's really easy to use a
decorator:

class C
  memoized
  def tak(x, y, z)
    return z if x <= y
    tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
  end
end

And really easy to write one:

class Module
  decorator
  def memoized(name, f)
    lambda do |*args|
      ((@__memo_cache ||= {})[[name, args]] ||=
[f.bind(self).call(*args)])[0]
    end
  end
end

You can use this to do all of the usual decorator things, like
memoization, method call tracing, synchronization, currying, type
checking, basic profiling, print warnings for deprecated methods, or
anything else you can think of.

One important limitation of this implementation is that you cannot
"stack" decorators like you can in python. That's fixable, and hopefully
I'll have time to make another version of this library with stackable
decorators. But until then, maybe someone will find this useful.

I'd love to hear feedback about this library! If you have any comments
or questions, please let me know.

kr

Attachments:
http://www.ruby-forum.com/attachment/52/deco.rb

···

--
Posted via http://www.ruby-forum.com/.

This is quite interesting.

I'm not sure how I feel about the use of declarative style. I'm not a
big fan of public, private, protected to begin with b/c of this. It
also complicates the code dealing with method_added and wrapping
methods... I wonder how robust it is. (This is another good example of
where some built in AOP functionality could improve things.)

Though it's a bit less convenient, it might be better to just name the
method:

  class C
    def tak(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
    memoized :tak
  end

Unfortunately, not as nice, but the underlying code would certainly
get simplified.

Dreaming a little. I wonder, if there were a callback for when a class/
module closes, then maybe you do do it lazily? Also, I wonder if this
corresponds to Matz' idea of ":"-notation he used for pre and post.
So,

  class C
    def tak:memoized(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
  end

Of course we could always do:

  class C
    def_memoized :tak do |x, y, z|
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
  end

Though the lack of block support and closed scope make that not quite
the same.

Oh, one last thing. Could you give some other examples? Memoization
perhaps isn't the best, since most times it is as easy as:

  class C
    def initialize
      @_tak = {}
    end

    def tak(x,y,z)
      @_tak[x,y,z]] ||= (
        return z if x <= y
        tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
      )
    end
  end

T.

···

On Aug 3, 4:53 pm, Keith Rarick <k...@essembly.com> wrote:

I've been using ruby for about 8 months now and I've come to appreciate
its powerful features. But I misspython'sdecorators, so I made
something similar.

The code is attached; here are some examples. It's really easy to use a
decorator:

class C
  memoized
  def tak(x, y, z)
    return z if x <= y
    tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
  end
end

And really easy to write one:

class Module
  decorator
  def memoized(name, f)
    lambda do |*args|
      ((@__memo_cache ||= {})[[name, args]] ||=
[f.bind(self).call(*args)])[0]
    end
  end
end

You can use this to do all of the usual decorator things, like
memoization, method call tracing, synchronization, currying, type
checking, basic profiling, print warnings for deprecated methods, or
anything else you can think of.

One important limitation of this implementation is that you cannot
"stack" decorators like you can inpython. That's fixable, and hopefully
I'll have time to make another version of this library with stackable
decorators. But until then, maybe someone will find this useful.

I'd love to hear feedback about this library! If you have any comments
or questions, please let me know.

kr

Attachments:http://www.ruby-forum.com/attachment/52/deco.rb

Keith Rarick wrote:

I'd love to hear feedback about this library! If you have any comments
or questions, please let me know.

Depending on how you've implemented it, the declarative style is not thread-safe. You would need to ensure that the decorators are only aggregated in the current thread, rather than in the class/metaclass for example, since they would step on each other in the latter case. This is because aggregating decorators and eventually applying them to some syntactic construct is not atomic; things can happen between each aggregation and the eventual consumption.

The imperative version would be safer, since it could largely be made atomic.

I like the idea in general though. I proposed (and roughly implemented) something similar for JRuby to be able to compile to static Java signatures, while still providing fully dynamic bodies:

class Foo
   signature [String, Fixnum]
   def bar(a, b)
     ...
   end
end

This would compile to foo(RubyString a, RubyFixnum b) with appropriate type checks and conversions, allowing us to provide a real Java method signature. In JRuby, this would be implemented on a per-thread-basis, or we'd use the imperative version:

signature :bar, [String, Fixnum]

- Charlie

This is quite interesting.

Thanks. I'd like it to be more than interesting; I want it to be
useful! Hopefully I can make some improvements.

I'm not sure how I feel about the use of declarative style.
I'm not a big fan of public, private, protected to begin
with b/c of this.

I can understand if you have reservations about the style. Personally,
I'm used to it and I find it easy to read.

One notable difference between the public, private, protected notation
and these decorators is that these must appear immediately before each
method they should apply to. A decorator's effects don't stick around
beyond the very next method, so there's no danger of having it go
unnoticed further down the file.

It also complicates the code dealing
with method_added and wrapping methods... I wonder how
robust it is. (This is another good example of where some
built in AOP functionality could improve things.)

I worry about the interaction with existing method_added() or
singleton_method_added() hooks. I tested some straightforward examples
of those and found no problems. Also, this code is working with no
trouble in a rather large rails app in the company I work for.

The code would be simpler and safer if ruby treated metaclasses and
classes consistently by calling metaclass.method_added() instead of
(or in addition to) singleton_method_added(). (See my earlier mail
with subject "method_added hook and class methods" for more.)

Perhaps the following notation would be better:

class C
  decorate :memoized
  def tak(x, y, z)
    return z if x <= y
    tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
  end
end

It would obviate the need to redefine the decorator method itself and
thus simplify the implementation. Also, this notation is more explicit
about the mechanism.

Though it's a bit less convenient, it might be better to
just name the method:

  class C
    def tak(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
    memoized :tak
  end

Unfortunately, not as nice, but the underlying code would
certainly get simplified.

Yes, the implementation would be pretty easy. (There's even a similar
example, called "once", in the pickaxe book.) However, putting the
decorator at the bottom makes it easy to miss, especially if the
method body is long.

Dreaming a little. I wonder, if there were a callback for when
a class/module closes, then maybe you do do it lazily?

Yeah, when I started thinking about how to do this I looked for such a
callback but didn't find one.

Also, I
wonder if this corresponds to Matz' idea of ":"-notation he
used for pre and post. So,

  class C
    def tak:memoized(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
  end

That's very interesting! I didn't know about that notation before. I
just read about pre, post, and wrap methods, which seem similar but
less useful. They are invoked at the method call, rather than the
method definition, so they have less chance to affect the method's
interface.

Now, if you could define arbitrary methods to be used with the
":"-notation, like def tak:memoized(x, y, z) in your example above,
that would be really useful.

Oh, one last thing. Could you give some other examples?

Sure. The one I find most useful is tracing:

class Module
  TRACE_LEVEL = [0]

  decorator
  def traced(name, meth)
    lambda do |*args|
      s = '. ' * TRACE_LEVEL[0]
      puts s + "calling #{name}(#{args.map{|a|a.inspect}.join(',')})"
      TRACE_LEVEL[0] += 1
      r = begin
        begin
          meth.bind(self).call(*args)
        ensure
          TRACE_LEVEL[0] -= 1
        end
      rescue => ex
        puts(s + "! #{ex.class}: " + ex)
        raise ex
      end
      puts(s + '=> ' + r.inspect)
      return r
    end
  end
end

Then you can turn tracing on (or off) for any function easily:

class Calc
  class << self
    traced
    def fact(n)
      return 1 if n < 2
      return n * fact(n - 1)
    end

    traced
    def bomb(n)
      raise 'boo' if n < 2
      return n * bomb(n - 1) if n < 5
      begin
        return n * bomb(n - 1)
      rescue
        return n * fact(n - 1)
      end
    end
  end
end

Calc.fact(5)
Calc.bomb(5)

I've also used decorators to do database object lookups automatically.
For example, assume that a (hypothetical) web framework will call
Profile.show("37") when the user makes a request.

class Module
  decorator
  def lookup(name, f)
    lambda do |id|
      f.bind(self).call(self.class.find(id.to_i))
    end
  end
end

class Profile
  lookup
  def show(profile)
    return profile.name + ' is a nice person.'
  end

  lookup
  def edit(profile)
  end
end

Type checking (if you like that sort of thing):

(This one requires a small change that I will post shortly.)

class Module
  decorator
  def checked(name, f, types)
    lambda do |*args|
      [args, types].transpose.each do |a, t|
        raise TypeError if !a.is_a?(t)
      end
      f.bind(self).call(*args)
    end
  end
end

class C
  checked Integer, String
  def warn(level, message)
    STDERR.puts '!'*level + message
  end
end

You can also do general pre- and postconditions.

Deprecation warnings:

(Adapted from http://wiki.python.org/moin/PythonDecoratorLibrary\.\)

class Module
  decorator
  def deprecated(name, f)
    lambda do |*args|
      STDERR.puts "Warning: function #{name} is deprecated."
      f.bind(self).call(*args)
    end
  end
end

kr

···

On 8/6/07, Trans <transfire@gmail.com> wrote:

I'm not sure how I feel about the use of declarative style. I'm not a
big fan of public, private, protected to begin with b/c of

...

Though it's a bit less convenient, it might be better to just name the
method:

  class C
    def tak(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
    memoized :tak
  end

Very much for reasons like this do I keepcauggesting that def be a
statement that returns the defined method instance. Then you could do:

class C
  memoized def tak( x, y, z )
    #...
  end
end

Further (not that I know python) I imagine this would enable the
"stacking" that the OP wanted:

class C
  memoized awesomificated def tak( ... )
    #...
  end
end

...as long as the decorating methods in play kept on returning the
method being affected.

···

On Aug 6, 10:23 am, Trans <transf...@gmail.com> wrote:

You're right; it is not. But that doesn't bother me too much.
Shared-state concurrency is usually a bad idea. Then again, ideally
this library would be correct under all circumstances including
multiple threads.

The real problem is that ruby provides no way to intercept method
definitions as they occur. The best you can do is notice that they
happen after the fact.

The best interface would be an add_method method that one could
override. The interpreter has an add_method function in C, but it's
not exposed to ruby code.

kr

···

On 9/4/07, Charles Oliver Nutter <charles.nutter@sun.com> wrote:

Depending on how you've implemented it, the declarative style is not
thread-safe.

Here's the change.

kr

deco.rb (1.63 KB)

···

On 8/7/07, Keith Rarick <kr@essembly.com> wrote:

Type checking (if you like that sort of thing):

(This one requires a small change that I will post shortly.)

Trans wrote:

Perhaps the following notation would be better:

class C
  decorate :memoized
  def tak(x, y, z)
    return z if x <= y
    tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
  end
end

It would obviate the need to redefine thedecoratormethod itself and
thus simplify the implementation. Also, this notation is more explicit
about the mechanism.

That's not a bad idea really. It would make it clear when a decorator
is being used too.

what about:
   class C
     decorate :memoized do
       def ...
     end
   end

That makes the implementation much more simple and solid than relying on method_added which may be incorrectly overridden somewhere.

Daniel

···

On Aug 7, 1:45 pm, "Keith Rarick" <k...@essembly.com> wrote:

Hmmmm, I'm just catching up after being too busy for a few weeks to
drink from the firehose that is ruby-talk.

I just wrote this yesterday:
http://talklikeaduck.denhaven2.com/articles/2007/09/04/block-your-privates

After noticing that some folks like ot artifically indent methods
definitions after private and its friends, it occurred to me that it
might be nice if you could write

class Foo
  private do
     def method1
     end

    def method2
    end
  end
...
end

I ran into similar issues in trying to metahack an implementation of
this idea, although I did find a way to accomplish the same effect
albeit with somewhat less pretty code, see the article for the
details.

···

On 8/7/07, Keith Rarick <kr@essembly.com> wrote:

One notable difference between the public, private, protected notation
and these decorators is that these must appear immediately before each
method they should apply to. A decorator's effects don't stick around
beyond the very next method, so there's no danger of having it go
unnoticed further down the file.

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

[....]

The real problem is that ruby provides no way to intercept method
definitions as they occur. The best you can do is notice that they
happen after the fact.

what about http://blog.ntecs.de/articles/2004/08/16/implementing-python-decorators-in-ruby
? This seem a very useful feature to have in ruby 2.0

···

On Sep 5, 12:15 am, "Keith Rarick" <k...@essembly.com> wrote:

I like that. I think the gain in robustness more than offsets the
slight degradation (IMO) in readability. What would it look like for
class methods? Something like this? I don't know how I would implement
example 2 below.

class C

  # Example 1 (instance method)
  decorate :memoized do
    def tak(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
  end

  # Example 2 (class method)
  decorate :memoized do
    def self.tbk(x, y, z)
      return z if x <= y
      tbk(tbk(x - 1, y, z), tbk(y - 1, z, x), tbk(z - 1, x, y))
    end
  end

  # Example 3 (class method)
  class << self
    decorate :memoized do
      def tck(x, y, z)
        return z if x <= y
        tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
      end
    end
  end

end

kr

···

On 8/29/07, Daniel DeLorme <dan-ml@dan42.com> wrote:

what about:
   class C
     decorate :memoized do
       def ...
     end
   end

That makes the implementation much more simple and solid than relying on
method_added which may be incorrectly overridden somewhere.

Interesting. As long as we're willing to modify the ruby interpreter,
why not just implement real decorators? There are some problems with
my interface (and the one described on ntecs.de) that I was willing to
live with in a pure ruby implementation. First, it's not very DRY as
each decorator needs logic to lookup and replace the method and return
its name. It's also a little fragile. Most importantly, though,
replacing the method *after* it's been defined isn't as clean as
wrapping the method object right before it gets bound to a name. Any
method_added hooks will, unfortunately, be called twice.

I'm not personally going to try getting a ruby patch accepted.

kr

···

On 9/16/07, Matteo Gottardi <gomatteo@gmail.com> wrote:

what about http://blog.ntecs.de/articles/2004/08/16/implementing-python-decorators-in-ruby
? This seem a very useful feature to have in ruby 2.0

Just a thought...

There's this trick I always want to use to define class-level module
methods that would be "inherited" through the class hierarchy.
Unfortunately it's doesn't work because the method still winds up in a
class rather than a module. However, it can be used for other things.
And this might be a good case. Using your examples:

  # Example 1 (instance method)

  def memoized.tak(x, y, z)
    return z if x <= y
    tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
  end

  # Example 3 (class method)

  class << self
    def memoized.tck(x, y, z)
      return z if x <= y
      tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
    end
  end

Interestingly, the only place I've used this trick so far is for
#meta, which allows me to define class methods by-passing access
privacy. For instance I can use it to do:

  class X
    meta.attr_writer :foo
  end

instead of

  class X
    class << self
      attr_writer :foo
    end
  end

It should even be possible to combine this with example #3 to do:

  # Example 3 (class method)

  def meta.memoized.tck(x, y, z)
    return z if x <= y
    tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
  end

T.

···

On Aug 29, 4:40 pm, "Keith Rarick" <k...@essembly.com> wrote:

On 8/29/07, Daniel DeLorme <dan...@dan42.com> wrote:

> what about:
> class C
> decorate :memoized do
> def ...
> end
> end

> That makes the implementation much more simple and solid than relying on
> method_added which may be incorrectly overridden somewhere.

I like that. I think the gain in robustness more than offsets the
slight degradation (IMO) in readability. What would it look like for
class methods? Something like this? I don't know how I would implement
example 2 below.

class C

  # Example 1 (instance method)
  decorate :memoized do
    def tak(x, y, z)
      return z if x <= y
      tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
    end
  end

  # Example 2 (class method)
  decorate :memoized do
    def self.tbk(x, y, z)
      return z if x <= y
      tbk(tbk(x - 1, y, z), tbk(y - 1, z, x), tbk(z - 1, x, y))
    end
  end

  # Example 3 (class method)
  class << self
    decorate :memoized do
      def tck(x, y, z)
        return z if x <= y
        tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
      end
    end
  end

end

Just a thought...
[snip]

That notation looks really nice. It's much better than my original
proposal. I'm working on implementing it right now and I'll post it as
soon as it's ready.

It should even be possible to combine this with example #3 to do:

  # Example 3 (class method)

  def meta.memoized.tck(x, y, z)
    return z if x <= y
    tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
  end

Unfortunately, that produces a syntax error. :frowning: Ruby doesn't seem to
let you give more than one dot in the name.

kr

···

On 8/29/07, Trans <transfire@gmail.com> wrote:

> Just a thought...
> [snip]

That notation looks really nice. It's much better than my original
proposal. I'm working on implementing it right now and I'll post it as
soon as it's ready.

Cool. It will be interesting to see how it turns out.

> It should even be possible to combine this with example #3 to do:

> # Example 3 (class method)

> def meta.memoized.tck(x, y, z)
> return z if x <= y
> tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
> end

Unfortunately, that produces a syntax error. :frowning: Ruby doesn't seem to
let you give more than one dot in the name.

Figures. Well, chalk up another one to Ruby curmudgeon.

T.

···

On Aug 30, 12:30 pm, "Keith Rarick" <k...@essembly.com> wrote:

On 8/29/07, Trans <transf...@gmail.com> wrote:

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.method(:bar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Put these together and it means that once bar has been defined in this
way it can never become an instance method.

If only there were some way to get the body of an unbound method as a
string I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

kr

···

On 8/30/07, Trans <transfire@gmail.com> wrote:

Cool. It will be interesting to see how it turns out.

> Cool. It will be interesting to see how it turns out.

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.method(:bar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Hmm that is why I put my defs into modules, even one module per
method, than I just use the module instead of the method, but honestly
your original code is just great if you adapt to the
decorator do
end
idiom you could potentially create a new Module from the block and
work with it, but I am not sure what that would be good for, are you
not looking for over-perfection here ;)?

If only there were some way to get the body of an unbound method as a
string

But there is, oops that is Squeak, not Ruby :frowning:

I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

kr

Cheers
Robert

···

On 8/30/07, Keith Rarick <kr@essembly.com> wrote:

On 8/30/07, Trans <transfire@gmail.com> wrote:

--
I'm an atheist and that's it. I believe there's nothing we can know
except that we should be kind to each other and do what we can for
other people.
-- Katharine Hepburn

So you've hit the same wall I did with module inheritance. That's
unfortunate. Though in some ways I'm glad you have. Maybe you'll give
some consideration to a long standing opinion of mine, that either the
distinction between class and module should just be torn down, or that
singleton classes should be "singleton modules" instead. After all, we
can't subclass singleton classes, so why not allow us to include them
instead? That would solve both issues in one stroke, and I dare say,
improve Ruby's meta-programming by leaps and bounds (pun intended :wink:

However, even though I think such a change is a good thing to pursue
(I'm even tempted to revive Suby just to offer this one distinct
feature), I offer this:

  class X

    class Foo

      def initialize(klass)
        @klass = klass
      end

      def singleton_method_added(meth)
        return if meth == :singleton_method_added
        m = method(meth)
        @klass.send(:define_method, meth, m.to_proc)
      end
    end

    def self.fooized
      @foo ||= Foo.new(self)
    end

    def fooized.try
      10 + 3
    end

  end

  p X.new.try

Honestly, I half-expect I'm delusional, b/c I'm not sure how this
manges to work, but it seems to do so. Maybe you can use it as a
jumping board.

Yours,
T.

···

On Aug 30, 4:18 pm, "Keith Rarick" <k...@essembly.com> wrote:

On 8/30/07, Trans <transf...@gmail.com> wrote:

> Cool. It will be interesting to see how it turns out.

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.method(:bar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Put these together and it means that once bar has been defined in this
way it can never become an instance method.

If only there were some way to get the body of an unbound method as a
string I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

Nodewrap lets you do this, but it's not safe:

irb(main):001:0> class Foo; def foo; puts "HERE"; end; end
=> nil
irb(main):002:0> class Bar; end
=> nil
irb(main):005:0> Bar.add_method(:bar, Foo.instance_method(:foo).body, 0)
=> nil
irb(main):006:0> Bar.new.bar
HERE
=> nil

Paul

···

On Fri, Aug 31, 2007 at 08:18:38AM +0900, Keith Rarick wrote:

2. An unbound method (such as what you get by calling
foo.method(:bar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Hopefully I am not too egocentric about my own approach, but if so you
could maybe explain ;)?
I fail to grasp where exactly you hit that wall, I have this
impression that it is the fact that you bind your methods too early,
if you would just refrain from using methods that are defined inside
the class, but would always do some include/extend trick, would the
wall not just come trembling down?
Sorry if I am missing the obvious here.

Cheers
Robert

···

On 8/31/07, Trans <transfire@gmail.com> wrote:

So you've hit the same wall I did with module inheritance. That's
unfortunate.

--
I'm an atheist and that's it. I believe there's nothing we can know
except that we should be kind to each other and do what we can for
other people.
-- Katharine Hepburn