A plugin system using extend

ruby-talk
1

Hi.

I tried to design a simple plugin system using :extend called upon
instance (http://ruby-doc.org/core/classes/Object.html#M000335 usual
documented behavior), so as to inject some methods redefinitions while
keeping the ability to call super in order to fall back on the default
behavior if needed.

It looks like this: http://gist.github.com/339025
It works as expected, as one can see by reading the commented output at
the end of the file: the inheritance chain is altered, with the plugin
"subclassing" the original class of the plugin receiver.

Yet, I tried to introduce some modularity in the system, so I went for:


It fails but I don't understand why. The inheritance chain is not
altered.
I tried to use :include instead of :extend, which gives merely the same
behavior (overwriting instance methods), but this time the plugin module
is added as the parent of the original class within the inheritance
chain, so it is useless for the purpose :wink:

I'll be glad if someone could give me a hint on this :slight_smile:
Thank you!

···

--
Posted via http://www.ruby-forum.com/.

2

I'm not sure if this helps, but I did something like this before:

http://www.oriontransfer.co.nz/blog/2009-10/loading-anonymous-ruby-classes/index

Kind regards,
Samuel

···

On 21/03/2010, at 3:46 PM, Jean-denis Vauguet wrote:

Hi.

I tried to design a simple plugin system using :extend called upon
instance (class Object - RDoc Documentation usual
documented behavior), so as to inject some methods redefinitions while
keeping the ability to call super in order to fall back on the default
behavior if needed.

It looks like this: plugins system: standalone · GitHub
It works as expected, as one can see by reading the commented output at
the end of the file: the inheritance chain is altered, with the plugin
"subclassing" the original class of the plugin receiver.

Yet, I tried to introduce some modularity in the system, so I went for:
plugins system: nested · GitHub
It fails but I don't understand why. The inheritance chain is not
altered.
I tried to use :include instead of :extend, which gives merely the same
behavior (overwriting instance methods), but this time the plugin module
is added as the parent of the original class within the inheritance
chain, so it is useless for the purpose :wink:

I'll be glad if someone could give me a hint on this :slight_smile:
Thank you!
--
Posted via http://www.ruby-forum.com/\.

3

Hi, I got it to do what I think you are looking for by having plugins pass
the object to extend. gist:339101 · GitHub
To test it, I had the redefined object say things in reverse. I also added
another module to show that you can activate different behaviours in
different objects.

Then I played around with it a little bit more, trying to make it more
modular, and behave similar to ActiveRecord's scopes

I wanted to try to make the plugin agnostic to the method it was activating
so that you could, for example, make a plugin that could then be applied to
any method of any object. But I just can never seem to get define_method to
work right -.- every time I try to do that, I seem to struggle a lot with
it, and usually end up using eval with a string, because dynamically adding
a method is so difficult.

Oh well, I'm kind of happy with it.

Guess I should pick up PragProg's Metaprogramming Ruby book... or try out
Lisp >:D

···

On Sat, Mar 20, 2010 at 9:46 PM, Jean-denis Vauguet <jd@vauguet.fr> wrote:

Hi.

I tried to design a simple plugin system using :extend called upon
instance (class Object - RDoc Documentation usual
documented behavior), so as to inject some methods redefinitions while
keeping the ability to call super in order to fall back on the default
behavior if needed.

It looks like this: plugins system: standalone · GitHub
It works as expected, as one can see by reading the commented output at
the end of the file: the inheritance chain is altered, with the plugin
"subclassing" the original class of the plugin receiver.

Yet, I tried to introduce some modularity in the system, so I went for:
plugins system: nested · GitHub
It fails but I don't understand why. The inheritance chain is not
altered.
I tried to use :include instead of :extend, which gives merely the same
behavior (overwriting instance methods), but this time the plugin module
is added as the parent of the original class within the inheritance
chain, so it is useless for the purpose :wink:

I'll be glad if someone could give me a hint on this :slight_smile:
Thank you!
--
Posted via http://www.ruby-forum.com/\.

4

Thank you Josh. Actually I've already tested what you wrote and that's
just fine as long as what you're redefining belongs to the Base::Server
instance (that's the purpose of :extend).

I posted a "mockup" of what I'd like in a single file:

Basically, it's the "same" thing, but the purpose here is to redefine
instance methods of *other classes* than the Base::Server which has the
"include Plugins"; plus, to do so not for a particular instance of these
classes, but for any of their instances.

Problem is: in this situation, one cannot access a particular instance
of the class to be altered within a given plugin. In my example, this
means the Backward plugin should alter any instance of the Base::Speaker
class once loaded. So you've got to work at the class level somehow
(Base::Speaker). Using :extend at this class level seems useless to me
here (it makes plugins redefinitions available as class methods for
Base::Speaker, not instance's); and using :include does not override the
class' instance methods, for the plugin module is added *before* the
class in it's inheritance chain (say: [Base::Speaker,
Base::Plugins::Backward::SpeakerRedef, Object, Kernel] once the Backward
plugin is loaded).

A workaround should be to undef (or alias) the original instance method
when the plugin's module is :included, so that a call to the "original"
method force the object to go finding the method in the plugin's module,
but it feels clumsy to me. Maybe that's the only way to achieve this?
After all, that was the point of alias_method_chain, wasn't it?

Enlight me :slight_smile:

···

--
Posted via http://www.ruby-forum.com/.

5

Another idea I had is the following:
- any loaded lugin registers as a "callback" for the classes it wants to
alter (instances of the classes, actually!);
- through the Plugins module, which is mix-ined within the Base module
using :extend, have any class nested within the base module to be able
(forced?), everytime they're initialized, to have their instances
:extend the plugins which have registered as callback for the class.

Not really sure about it, but...
Theoretically it would allow for per-instance :extend, thus overriding
behavior of any instance without the need for aliasing. Same behavior as
my standalone and Josh codes, but same flexibility as my previous gist
attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

···

--
Posted via http://www.ruby-forum.com/.

6

If I understand what you are saying correctly, I've used this exact strategy in the past.

I allowed plugins to register themselves on the class object (you can also do that automatically as they are defined with an inherited hook), and then I just extended all instances with the registered plugins as they were created.

It worked great and felt very natural to me when using it.

James Edward Gray II

···

On Mar 21, 2010, at 2:13 AM, Jean-denis Vauguet wrote:

Another idea I had is the following:
- any loaded lugin registers as a "callback" for the classes it wants to
alter (instances of the classes, actually!);
- through the Plugins module, which is mix-ined within the Base module
using :extend, have any class nested within the base module to be able
(forced?), everytime they're initialized, to have their instances
:extend the plugins which have registered as callback for the class.

Not really sure about it, but...
Theoretically it would allow for per-instance :extend, thus overriding
behavior of any instance without the need for aliasing. Same behavior as
my standalone and Josh codes, but same flexibility as my previous gist
attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

7

James Edward Gray II wrote:

behavior of any instance without the need for aliasing. Same behavior as
my standalone and Josh codes, but same flexibility as my previous gist
attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

If I understand what you are saying correctly, I've used this exact
strategy in the past.

I allowed plugins to register themselves on the class object (you can
also do that automatically as they are defined with an inherited hook),
and then I just extended all instances with the registered plugins as
they were created.

It worked great and felt very natural to me when using it.

James Edward Gray II

Thank you for this feedback. You understood well I guess. This is good
pattern IMO too: you both get rid of dirty aliasing, and enable clean
inheritance with super. I did not have the time to try it out yet, but I
guess it should just work :slight_smile: If it performs zell, I'll try to make it
more generic (a kind of "Pluginable" gem?).

By the way, would you have any code snippet to illustrate the pattern?
That could come in handy when refactoring my code, for I'm sure you're
definitely more skilled at writing this kind of stuff than I currently
am :wink:

···

On Mar 21, 2010, at 2:13 AM, Jean-denis Vauguet wrote:

--
Posted via http://www.ruby-forum.com/\.

8

James, what's the advantage of this over simply using "include"? If all instances get to use the plugin module then you can as well include it in the class.

I see it like this: if all instances of a class should be extended with plugin behavior, then simply use "include". If only some instances should, then use "extend".

If plugins also need to be removed at runtime, then a more complicated solution is necessary (e.g. cooking your own version of "extend" and using method_missing to delegate to plugged in code - access to instance variables can then be tricky but you can cope by explicitly passing self to each method).

Kind regards

  robert

···

On 03/21/2010 06:14 PM, James Edward Gray II wrote:

On Mar 21, 2010, at 2:13 AM, Jean-denis Vauguet wrote:

Another idea I had is the following:
- any loaded lugin registers as a "callback" for the classes it wants to alter (instances of the classes, actually!);
- through the Plugins module, which is mix-ined within the Base module using :extend, have any class nested within the base module to be able (forced?), everytime they're initialized, to have their instances :extend the plugins which have registered as callback for the class.

Not really sure about it, but...
Theoretically it would allow for per-instance :extend, thus overriding behavior of any instance without the need for aliasing. Same behavior as my standalone and Josh codes, but same flexibility as my previous gist attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

If I understand what you are saying correctly, I've used this exact strategy in the past.

I allowed plugins to register themselves on the class object (you can also do that automatically as they are defined with an inherited hook), and then I just extended all instances with the registered plugins as they were created.

It worked great and felt very natural to me when using it.

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

9

Sure. I added some code like that to Prawn a while back (though I don't think they kept it):

I hope that helps.

James Edward Gray II

···

On Mar 21, 2010, at 12:44 PM, Jean-denis Vauguet wrote:

James Edward Gray II wrote:

On Mar 21, 2010, at 2:13 AM, Jean-denis Vauguet wrote:

behavior of any instance without the need for aliasing. Same behavior as
my standalone and Josh codes, but same flexibility as my previous gist
attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

If I understand what you are saying correctly, I've used this exact
strategy in the past.

I allowed plugins to register themselves on the class object (you can
also do that automatically as they are defined with an inherited hook),
and then I just extended all instances with the registered plugins as
they were created.

It worked great and felt very natural to me when using it.

James Edward Gray II

Thank you for this feedback. You understood well I guess. This is good
pattern IMO too: you both get rid of dirty aliasing, and enable clean
inheritance with super. I did not have the time to try it out yet, but I
guess it should just work :slight_smile: If it performs zell, I'll try to make it
more generic (a kind of "Pluginable" gem?).

By the way, would you have any code snippet to illustrate the pattern?
That could come in handy when refactoring my code, for I'm sure you're
definitely more skilled at writing this kind of stuff than I currently
am :wink:

10

There's one massive advantage: the inheritance order. include puts the plugin behind the base class, which is much less useful than extend's behavior of putting it in front of the base class (behind the singleton class, technically). To me, this is the very reason this system is so natural for plugins.

James Edward Gray II

···

On Mar 21, 2010, at 1:35 PM, Robert Klemme wrote:

On 03/21/2010 06:14 PM, James Edward Gray II wrote:

On Mar 21, 2010, at 2:13 AM, Jean-denis Vauguet wrote:

Another idea I had is the following:
- any loaded lugin registers as a "callback" for the classes it wants to alter (instances of the classes, actually!);
- through the Plugins module, which is mix-ined within the Base module using :extend, have any class nested within the base module to be able (forced?), everytime they're initialized, to have their instances :extend the plugins which have registered as callback for the class.

Not really sure about it, but...
Theoretically it would allow for per-instance :extend, thus overriding behavior of any instance without the need for aliasing. Same behavior as my standalone and Josh codes, but same flexibility as my previous gist attempted to achieve.

I'll give it a try when I get the time to, unless I'm told this is BS :wink:

If I understand what you are saying correctly, I've used this exact strategy in the past.
I allowed plugins to register themselves on the class object (you can also do that automatically as they are defined with an inherited hook), and then I just extended all instances with the registered plugins as they were created.
It worked great and felt very natural to me when using it.

James, what's the advantage of this over simply using "include"? If all instances get to use the plugin module then you can as well include it in the class.

11

Robert Klemme wrote:

Not really sure about it, but...

It worked great and felt very natural to me when using it.

James, what's the advantage of this over simply using "include"? If all
instances get to use the plugin module then you can as well include it
in the class.

As JEGII stated, the main reason to do so is related to the inheritance
chain order. If you want to be able to redefine an instance method while
keeping access to the original implementation, extend is necessary.
include will make the redefinition available but as it's mixed-in, the
original definition trumps the redef. With extend, the order is reversed
(the original class is kind of subclassed by the module extend receives
as an argument).

This is terrific behavior for callbacks and plugins. It's a easy as pie
to do with classes, and a little bit more tortuous to achieve for class
instances.

I see it like this: if all instances of a class should be extended with
plugin behavior, then simply use "include". If only some instances
should, then use "extend".

I think there's some confusion about what extend and include really are.
We often read extend is to make module methods available as class
methods while include makes them instance methods, but that's not true.

extend is a method of the Object class, and it can handle any object as
a receiver, be it a class, an instance of a class, or a module, a
singleton, etc. If you extend a class, then you'll get class methods. An
instance, instance methods. And what's so great about extend is, once
again, the way it alters the inheritance chain: the class calling extend
for a module gets subclassed by the module-now-a-class. Which means at
least two things: if you redefine the module passed to extend, then the
changes are not propagated, it's made available only for new extending
objects; if you had a method on the object which is redefined into the
extended module, then the module-now-a-class version is the first match,
and you can call super to reach the original (class) definition.

Quite different is include, a keyword not a method, which has only one
behavior: quoting the Pickaxe, "it makes a reference from the class to
the included module. If multiple classes include that module, they'll
all point to the same thing". So you get shared, instance methods. The
mixed-in module is appended right next to the class including it within
the inheritance chain, so that if you call an instance method of the
class, even if it's been redefined by the mixed-in module, the first
match's still the class'.

So, the real difference between extend and include, aside from their
nature, is not really about whether they're talking to classes or
instances, it's more about their behavior: extend is useful for
redefining things (once), include (mixin) is useful for adding (shared)
things. I mean, following Yehuda Katz
(http://yehudakatz.com/2010/02/25/rubys-implementation-does-not-define-its-semantics/\),
you can either stick to the implementation, or, you can consider the
semantic/purpose, the latter being more accurate and useful at the same
time IMO :slight_smile:

···

On 03/21/2010 06:14 PM, James Edward Gray II wrote:

--
Posted via http://www.ruby-forum.com/\.

12

This isn't totally accurate. extend() is a stupid simple shortcut that really is just an include. This code:

  obj.extend(Whatever)

is identical to:

  class << obj
    include Whatever
  end

So it really does all work the same.

The reason it moves the methods in front of the main class though is that the singleton class is in front of the main class. Thus including the module behind that class still has them in front of the main class.

I talked about this quite a bit in my presentation at LSRC last year:

  Video: http://lsrc2009.confreaks.com/module-magic-james-edward-gray-ii-28-aug-2009.html
  Slides: http://grayproductions.net/ruby/module_magic.tar.gz

The spirit of what you said is all right on though.

James Edward Gray II

···

On Mar 21, 2010, at 2:51 PM, Jean-denis Vauguet wrote:

extend is a method of the Object class, and it can handle any object as
a receiver, be it a class, an instance of a class, or a module, a
singleton, etc. If you extend a class, then you'll get class methods. An
instance, instance methods. And what's so great about extend is, once
again, the way it alters the inheritance chain: the class calling extend
for a module gets subclassed by the module-now-a-class.

Quite different is include, a keyword not a method, which has only one
behavior: quoting the Pickaxe, "it makes a reference from the class to
the included module. If multiple classes include that module, they'll
all point to the same thing". So you get shared, instance methods.

So, the real difference between extend and include, aside from their
nature, is not really about whether they're talking to classes or
instances, it's more about their behavior: extend is useful for
redefining things (once), include (mixin) is useful for adding (shared)
things.

13

James Edward Gray II wrote:

Adding the simplest possible extension system to prawn. · JEG2/prawn@7e25baf · GitHub

I hope that helps.

Thank you, this is art. I'm happy to see I did it roughly the same way,
minus the (very smart) tests :slight_smile:

···

--
Posted via http://www.ruby-forum.com/\.

14

James Edward Gray II wrote:

stuff.

This isn't totally accurate. extend() is a stupid simple shortcut that
really is just an include. This code:

  obj.extend(Whatever)

is identical to:

  class << obj
    include Whatever
  end

So it really does all work the same.

That's even better than expected! I did not think about the singleton
class nor did I look at the real implementations (which demonstrates it
*is* important to master along the semantics ;)).

Thanks for the correction.

···

On Mar 21, 2010, at 2:51 PM, Jean-denis Vauguet wrote:

The reason it moves the methods in front of the main class though is
that the singleton class is in front of the main class. Thus including
the module behind that class still has them in front of the main class.

I talked about this quite a bit in my presentation at LSRC last year:

  Video:
http://lsrc2009.confreaks.com/module-magic-james-edward-gray-ii-28-aug-2009.html
  Slides: Gray Soft / Not Found

The spirit of what you said is all right on though.

--
Posted via http://www.ruby-forum.com/\.

15

I just finished writing my plugins system. It's working like a charm,
thank you for your feedbacks!

I may extract a standalone gem which would allow any module (or class,
but typically it's to be called on a project module) to enable plugins
and project internals redefinition on the fly, without the need for any
aliasing or explicit callback definition. I need to streamline my
current code, though.

···

--
Posted via http://www.ruby-forum.com/.

16

Hi.

I encounter a little issue while trying to make it really dynamic.

http://gist.github.com/344898 sums it up.

···

--
Posted via http://www.ruby-forum.com/.

17

Jean-denis Vauguet wrote:

Hi.

I encounter a little issue while trying to make it really dynamic.

344898’s gists · GitHub sums it up.

Fixed, although I don't really understand *why* at the moment :slight_smile:

···

--
Posted via http://www.ruby-forum.com/\.