Code organisation in a plugin system

Martin_DeMello · 24 January 2022 04:57

Looking for some good ideas for this...

I have a desktop app that uses a plugin system to organise various file
converters. You can see the code here:

martindemello/pangrid/blob/master/lib/pangrid/plugin.rb

require_relative 'xw'
require_relative 'utils'

module Pangrid

class PluginDependencyError < StandardError
  attr_accessor :name, :gems

  def initialize(name, gems)
    @name, @gems = name, gems
  end
end

# Load all the gem dependencies of a plugin
def self.require_for_plugin(name, gems)
  missing = []
  gems.each do |gem|
    begin
      require gem
    rescue LoadError => e

This file has been truncated. show original

Plugins are defined by subclassing Plugin, which calls an inherited hook to
add them to the plugin registry, and the top level app calls
Plugin.load_all, which basically does a glob of all files under plugins/
and calls `require #{file}` on each in turn.

I'm now trying to use some of the individual plugins as a library in
another app, and I ran into the annoying issue that the plugin code
subclasses Plugin, it wants plugin.rb to be required first. This is clearly
an issue with my code, since I have been relying on the plugins only loaded
via Plugin.load_all, and I should technically add require_relative
"../plugin" to each of them. However this made me realise that the whole
thing feels like a bit of a code smell, since other than that subclass hook
the plugins themselves are independent of the base class and should be
usable in isolation.

Is there a nicer way to accomplish both goals:
1. The main app should autodiscover and load everything in the plugins/
directory, and have the relevant classes register themselves
2. I should be able to just say `require pangrid/plugin/foo` to use one
plugin by itself without going through the plugin system

martin

Ismail_Arilik · 24 January 2022 07:47

Hi Martin.

You can gemify your Plugin class and say that plugin writers should use
this gem. I don't see any annoying point about it since other way would be
too much implicit. Although we use a dynamic language, some explicity would
be better I think. Also object oriented paradigm requires some.

şunu yazdı:

···

24 Oca 2022 Pzt 07:57 tarihinde Martin DeMello <martindemello@gmail.com>

Looking for some good ideas for this...

I have a desktop app that uses a plugin system to organise various file
converters. You can see the code here:
https://github.com/martindemello/pangrid/blob/master/lib/pangrid/plugin.rb

Plugins are defined by subclassing Plugin, which calls an inherited hook
to add them to the plugin registry, and the top level app calls
Plugin.load_all, which basically does a glob of all files under plugins/
and calls `require #{file}` on each in turn.

I'm now trying to use some of the individual plugins as a library in
another app, and I ran into the annoying issue that the plugin code
subclasses Plugin, it wants plugin.rb to be required first. This is clearly
an issue with my code, since I have been relying on the plugins only loaded
via Plugin.load_all, and I should technically add require_relative
"../plugin" to each of them. However this made me realise that the whole
thing feels like a bit of a code smell, since other than that subclass hook
the plugins themselves are independent of the base class and should be
usable in isolation.

Is there a nicer way to accomplish both goals:
1. The main app should autodiscover and load everything in the plugins/
directory, and have the relevant classes register themselves
2. I should be able to just say `require pangrid/plugin/foo` to use one
plugin by itself without going through the plugin system

martin

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Mohit_Sindhwani1 · 24 January 2022 09:07

Hi Martin,

I have a desktop app that uses a plugin system to organise various file converters. You can see the code here: https://github.com/martindemello/pangrid/blob/master/lib/pangrid/plugin.rb

I love this idea - we did something similar for a C++ application that auto-detected DLLs in a plugins directory exactly for importing data formats!

Plugins are defined by subclassing Plugin, which calls an inherited hook to add them to the plugin registry, and the top level app calls Plugin.load_all, which basically does a glob of all files under plugins/ and calls `require #{file}` on each in turn.

This sounds like something that a PluginsManager should do in your desktop application. Define which directories to scan and require or load plugins, run the initialise code of the plugin, if required, etc. It could also handle other things like offering an info panel that shows the plugins available and versions, etc.

I'm now trying to use some of the individual plugins as a library in another app, and I ran into the annoying issue that the plugin code subclasses Plugin, it wants plugin.rb to be required first. This is clearly an issue with my code, since I have been relying on the plugins only loaded via Plugin.load_all, and I should technically add require_relative "../plugin" to each of them. However this made me realise that the whole thing feels like a bit of a code smell, since other than that subclass hook the plugins themselves are independent of the base class and should be usable in isolation.

Yes, I agree. They should inherit from Plugin if there is common behaviour that you would like them to have. If not, there is no real reason for them to use the Plugin.

Is there a nicer way to accomplish both goals:
1. The main app should autodiscover and load everything in the plugins/ directory, and have the relevant classes register themselves

Yes, PluginManager - part of the main app.

2. I should be able to just say `require pangrid/plugin/foo` to use one plugin by itself without going through the plugin system

Yes, since they exist - any code should be able to do that if they don't have a reliance on Plugin anyway.

Two more thoughts:
a) Consider gems - it might make some of this better
b) You might need to play a bit with LOAD_PATH in case plugins is not on the LOAD_PATH (i.e., under lib, etc.)

Hope this helps.

Best Regards,
Mohit.

···

On 2022-1-24 12:57 pm, Martin DeMello wrote:

Martin_DeMello · 24 January 2022 20:53

Plugin is a slight misnomer in that I'm using it for internal code
structure, not as a way to let users of the gem extend the functionality. I
would rather not split this into multiple gems. I do see your point about
not trying to be *too* implicit, but the Plugin class saves me from having
to write code like

require 'foo'
require 'bar'
...
PLUGINS['foo-reader'] = FooReader
PLUGINS['bar-reader'] = BarReader
PLUGINS['bar-writer'] = BarWriter
...

and keep changing it every time I add a new plugin.

martin

···

On Sun, Jan 23, 2022 at 11:48 PM İsmail Arılık <arilik.ismail@gmail.com> wrote:

Hi Martin.

You can gemify your Plugin class and say that plugin writers should use
this gem. I don't see any annoying point about it since other way would be
too much implicit. Although we use a dynamic language, some explicity would
be better I think. Also object oriented paradigm requires some.

24 Oca 2022 Pzt 07:57 tarihinde Martin DeMello <martindemello@gmail.com>
şunu yazdı:

Looking for some good ideas for this...

I have a desktop app that uses a plugin system to organise various file
converters. You can see the code here:
https://github.com/martindemello/pangrid/blob/master/lib/pangrid/plugin.rb

Plugins are defined by subclassing Plugin, which calls an inherited hook
to add them to the plugin registry, and the top level app calls
Plugin.load_all, which basically does a glob of all files under plugins/
and calls `require #{file}` on each in turn.

I'm now trying to use some of the individual plugins as a library in
another app, and I ran into the annoying issue that the plugin code
subclasses Plugin, it wants plugin.rb to be required first. This is clearly
an issue with my code, since I have been relying on the plugins only loaded
via Plugin.load_all, and I should technically add require_relative
"../plugin" to each of them. However this made me realise that the whole
thing feels like a bit of a code smell, since other than that subclass hook
the plugins themselves are independent of the base class and should be
usable in isolation.

Is there a nicer way to accomplish both goals:
1. The main app should autodiscover and load everything in the plugins/
directory, and have the relevant classes register themselves
2. I should be able to just say `require pangrid/plugin/foo` to use one
plugin by itself without going through the plugin system

martin

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Martin_DeMello · 24 January 2022 21:02

The class I'm calling Plugin is pretty much a PluginsManager. The problem
I'm having a hard time with is that I can scan all the files in the
plugins/ directory and `require` them all, but then I have to see what
classes within those files are actually meant to be plugins and only
register those classes. Defining Plugin.inherited is a convenient way to do
that. The plugins do technically conform to a common interface (they need
to define one or both of `read()` and `write()`), but I'm already using
duck typing rather than `is_a? Plugin` to check if they satisfy the
interface, so the inheritance is purely to let the classes explicitly mark
themselves as plugins so that they can be found by the plugin manager.

martin

···

On Mon, Jan 24, 2022 at 1:07 AM Mohit Sindhwani <mo_mail@onghu.com> wrote:

> Plugins are defined by subclassing Plugin, which calls an inherited
> hook to add them to the plugin registry, and the top level app calls
> Plugin.load_all, which basically does a glob of all files under
> plugins/ and calls `require #{file}` on each in turn.

This sounds like something that a PluginsManager should do in your
desktop application. Define which directories to scan and require or
load plugins, run the initialise code of the plugin, if required, etc.
It could also handle other things like offering an info panel that shows
the plugins available and versions, etc.

Mohit_Sindhwani1 · 25 January 2022 03:49

Hi Martin,

The class I'm calling Plugin is pretty much a PluginsManager. The problem I'm having a hard time with is that I can scan all the files in the plugins/ directory and `require` them all, but then I have to see what classes within those files are actually meant to be plugins and only register those classes. Defining Plugin.inherited is a convenient way to do that. The plugins do technically conform to a common interface (they need to define one or both of `read()` and `write()`), but I'm already using duck typing rather than `is_a? Plugin` to check if they satisfy the interface, so the inheritance is purely to let the classes explicitly mark themselves as plugins so that they can be found by the plugin manager.

What we did in our system (C++ DLL) was that it must have a method that we could call (e.g. plugin_initialize) and if calling that failed, it meant that it was not a plugin.

[1] Perhaps, you could consider something like:
* a real plugin must include a 'plugin_interface' a separate class to which you can delegate the plugin initialisation functions that you want
* still have a plugins manager that enumerates these and checks if they are plugins before adding them to your main app

[2] But that would mean that they would need to be able to include 'plugin.rb' or similar - where you started

Best Regards,
Mohit.
2022-1-25 | 11:48 am.

Topic		Replies	Views
Seeking design ideas for a lightweight plugin system ruby-talk	5	160	24 March 2014
[ANN] GemPlugin 0.1 -- A Mongrel Spinoff ruby-talk	8	118	8 March 2006
Loading "plugins" ruby-talk	7	66	26 March 2005
Loading all plugins in a directory ruby-talk	6	239	27 August 2015
Plugin type discovery/registration - How do I discover what classes were loaded when I dynamically require in ruby files from a directory? ruby-talk	4	82	27 February 2006

Code organisation in a plugin system

Related topics