Superclass of eigenclass

Gary Wright wrote:

Confusion comes from:
-- the super keyword uses the method lookup chain, not the superclass
chain
-- the ancestor method reports on the method lookup chain, not the
inheritance chain

Aaaah! The penny finally drops! Thanks a million for this, Gary. I think
I'd have been banging my head against a brick wall for ages without your
explanation.

I'd probably still stand by my earlier assertion that it's a conceptual
mess, however. It's just that now that I know what the semantics really
are, I can work with it. Bad naming trips up even very experienced
people. For example, in RPL, no lesser luminaries than David Flanagan
and Matz write on page 238 that super "invokes a method with the same
name as the current one, in the superclass of the current class". Given
your first point above, it's clear that this is not just misleading,
it's also Just Plain Wrong (tm).

That said, for my own purposes, I now have a pretty good picture of the
semantics (naming snafus and all) and can definitely work with that
picture. So thanks again.

Regards,

Danny

···

--
Posted via http://www.ruby-forum.com/\.

Right. So then the question: is there a compelling reason that they should be allowed to be subclassed? (I mean, other than the obvious: "Because that's not the way Ruby was designed" answer.) I cannot come up with any compelling technical reason why the artificial limitation of being unable to subclass "virtual classes"[1] should exist. I also don't see what, besides this artificial limitation, is keeping Ruby from having, at its heart, a fully prototype based inheritance system.

- Josh

[1] That's what the exception calls them, so I'd throw that into your pot of valid names in the running.

···

On May 14, 2009, at 8:25 PM, Jörg W Mittag wrote:

Joshua Ballanco wrote:

As it turns out, they can be very
handy for implementing instance-specific methods,

Exactly. It is my understanding that they were only added as a place
to "stick" per-object methods. MRI already had classes, so why not use
them? If you stick per-object methods in an eigenclass you need
exactly 0 new constructs in the interpreter, 0 changes to method
lookup and just one if-condition in superclass lookup. Plus, you can
also implement mixins the same way. That's a *lot* of leverage for
essentially one line of code.

AFAIK, the fact that they were exposed to the programmer is more of an
accident than a well-designed thought-out language feature.

Gary Wright wrote:
> Confusion comes from:
> -- the super keyword uses the method lookup chain, not thesuperclass
> chain
> -- the ancestor method reports on the method lookup chain, not the
> inheritance chain

I have to disagree with Gary's nomenclature. Inheritance and method
lookup are basically the same thing, so trying to distinguish "method
lookup chain" from "inheritance chain" is hard to do. Maybe we can
say "class hierarchy" when we want to exclude modules as the
superclass method does and "method inheritance chain" (or something)
when we want to include modules.

Aaaah! The penny finally drops! Thanks a million for this, Gary. I think
I'd have been banging my head against a brick wall for ages without your
explanation.

I'd probably still stand by my earlier assertion that it's a conceptual
mess, however. It's just that now that I know what the semantics really
are, I can work with it. Bad naming trips up even very experienced
people. For example, in RPL, no lesser luminaries than David Flanagan
and Matz write on page 238 that super "invokes a method with the same
name as the current one, in thesuperclassof the current class".

You're right. I should have said something like "in the nearest
ancestor of the current class" or something to that effect, so that
modules are included. I may also want to update this paragraph to
point out that a singleton method of an object can chain to a regular
instance method of the object.

Thanks for pointing out this error!

···

On May 12, 9:29 am, Danny O cuiv <danny.oc...@gmail.com> wrote:

That said, for my own purposes, I now have a pretty good picture of the
semantics (naming snafus and all) and can definitely work with that
picture. So thanks again.

Regards,

Danny

--
Posted viahttp://www.ruby-forum.com/.

Joshua Ballanco wrote:

Joshua Ballanco wrote:

As it turns out, they can be very
handy for implementing instance-specific methods,

Exactly. It is my understanding that they were only added as a place
to "stick" per-object methods. MRI already had classes, [...]

AFAIK, the fact that they were exposed to the programmer is more of an
accident than a well-designed thought-out language feature.

Right. So then the question: is there a compelling reason that they
should be allowed to be subclassed? (I mean, other than the obvious:
"Because that's not the way Ruby was designed" answer.) I cannot come
up with any compelling technical reason why the artificial limitation
of being unable to subclass "virtual classes"[1] should exist. I also
don't see what, besides this artificial limitation, is keeping Ruby
from having, at its heart, a fully prototype based inheritance system.

There are lots of areas in Ruby, where things aren't as simple,
orthogonal or consistent as they should be. Unfortunately[1],
simplicity, orthogonality or consistency are not in any way design
goals. I really like your idea, but you need to find a more compelling
argument than "it makes eigenclasses consistent with classes and
simplifies the object system". But a full-fledged prototype system
would be pretty cool.

jwm

[1] Or fortunately? After all, Brainfuck is certainly simple,
    orthogonal and consistent ...

···

On May 14, 2009, at 8:25 PM, Jörg W Mittag wrote:

The purpose of an eigenclass is to hold per object methods. As such, there is always always a one-to-one relationship between an eigenclass and its "eigeninstance" (sorry, but I can't think of a clearer term at the moment).

The idea of subclassing an eigenclass contradicts that one-to-one relationship. If you could subclass an eigenclass, there would be no way to cause ruby to search the new subclass for instance methods since it wouldn't have a matching eigeninstance (sorry, again). It would not be in the method lookup chain of any object.

Just to be complete, I'll point out that there is a way for eigenclasses to have a subclass and that is when the eigeninstances are themselves classes. This is most clear in 1.9. If B is a subclass of A, then B.eigenclass will be a subclass of A.eigenclass. Even in this case A and B have to exist before their eigenclasses can be accessed and not the other way around.

Gary Wright

···

On May 15, 2009, at 7:46 AM, Joshua Ballanco wrote:

Right. So then the question: is there a compelling reason that they should be allowed to be subclassed? (I mean, other than the obvious: "Because that's not the way Ruby was designed" answer.) I cannot come up with any compelling technical reason why the artificial limitation of being unable to subclass "virtual classes"[1] should exist.

Gary Wright wrote:

Confusion comes from:
-- the super keyword uses the method lookup chain, not thesuperclass
chain
-- the ancestor method reports on the method lookup chain, not the
inheritance chain

I have to disagree with Gary's nomenclature.

No problem. I just made them up for that post.

Inheritance and method
lookup are basically the same thing, so trying to distinguish "method
lookup chain" from "inheritance chain" is hard to do.

But since they are only 'basically the same' and not exactly the same it is important to distinguish between them. This is more important in 1.8.X than in 1.9.X since in 1.9.X 'superclass' tracks the method lookup chain more closely. In 1.8.X 'superclass' is pretty hard to explain when you try to include singleton classes.

Maybe we can say "class hierarchy" when we want to exclude modules as the
superclass method does and "method inheritance chain" (or something)
when we want to include modules.

I could quibble with those definitions, but my main point was that 'ancestors' doesn't match the 'superclass chain' (because of modules) nor does it always match the 'method lookup chain' (because of
eigenclasses).

Gary Wright

···

On May 13, 2009, at 6:20 PM, David Flanagan wrote:

On May 12, 9:29 am, Danny O cuiv <danny.oc...@gmail.com> wrote:

But a full-fledged prototype system

would be pretty cool.

But there are! There is Ara's prototype, which needs a little fix if I
am not mistaken or your humble servant's one:

Sorry for the blog link, but I have not yet repackaged PPP into Lab419 :(.
HTH
Robert

···

2009/5/16 Jörg W Mittag <JoergWMittag+Usenet@googlemail.com>:

And there, IMHO, is where things break down. If the definition of an
eigenclass is a 'class' which holds per object methods, then the class
of a class which has subclasses is not an eigenclass since it provides
methods not only to it's sole instance but also to it's instance's
subclass(es).

That's why I prefer to think of the objects which serve as the class
of a class as metaclasses, which is in line with the terminology used
in several other languages.

This whole area of Ruby reminds me a bit of Plato's cave, we see
images on the wall cast by "smoke" and "mirrors" in the
implementation, and try to name them. And the smoke has been shifting
a bit and the mirrors have been moving slightly. When you step outside
of the cave and look at the implementation you find:

There's a flag called FL_SINGLETON which when set in the header of a
class object means:

   * It will get cloned if you clone its instance
   * You can't copy it.
    * You can't make a subclass of it, unless you are ruby itself
creating a subclass in which case
     subclasses of the "singleton" metaclass are created under the covers.
   * It will be bypassed by the Object#class method
   * It will be filtered out of the results Module#ancestors method
   * It will be skipped in the search for methods by
Module#instance_methods and it's related methods
  * Its methods are considered singleton methods
  * It has a hidden 'class' instance variable __attached__ which
points to it's sole instance
  * Its skipped over by ObjectSpace.each_object, so
ObjectSpace.each_object(Class) won't yield it.
  * It will prevent dumping by Marshall.dump

So what exactly does the singleton in FL_SINGLETON mean. Well, it
does mean that a class marked with FL_SINGLETON has only one direct
instance. It doesn't mean in the case of a singleton class which
happens to be a metaclass that it doesn't have subclasses, just that
you as a Ruby programmer can only create or add to its subclasses by
subclassing its attached class.

And while you might think that singleton method was a synonym for
instance specific method, once again this is only true when it's a
singleton method within an instance-specific singleton class.

class Foo
    def self.blat
        :blat
    end
end

class Bar < Foo
end

Foo.singleton_methods => [:blat]
Bar.singleton_methods => [:blat]

The 2nd ed of the Pickaxe used the term "virtual class" and seems to
call the FL_SINGLETON bit the 'V' bit. Looking at Ruby 1.8.6, the
term virtual class only appears in errors raised when you try to
   1. Try to make a subclass of a class marked with FL_SINGLETON and
you'll get the error
"can't make subclass of virtual class"
   2. Try to add a singleton method to an instance of an immediate
class (e.g. a Fixnum), since there's no klass ptr so nowhere to insert
the needed instance specific class in the lookup chain, and you'll get
the error "no virtual class for %s"
   3. Try to instantiate an instance of a class marked with
FL_SINGLETON and you'll get the error can't create instance of virtual
class"

And in Ruby 1.9, these error descriptions have all be changed to
remove the usage of "virtual class" and replace it with "singleton
class".

I kind of wish that FL_SINGLETON had actually been named FL_VIRTUAL,
or FL_HIDDEN, since what the flag does mostly is to hide such classes
in the standard library provided API. (ignoring the leak from class <
a;self;end

Back in the mists of Ruby history, it became popular to call any class
marked with FL_SINGLETON as a metaclass, probably because that's what
_why named the leak method in the Poignant Guide.

That added another layer of confusion, because to many of us metaclass
means the class of a class, and an instance-specific class ain't a
metaclass, it's a class whether it's hidden or not.

And I think that that's where the name eigenclass got coined, as an
alternative to singleton class, for better or worse. And with the
publication of "The Ruby Programming Language" it's gotten a bit of a
blessing from Matz, although the Ruby 1.9 implementation uses
singleton class in those errors where virtual class was used before.

I personally think that the distinction between metaclasses (in the
generally accepted sense of the class of a class) and
instance-specific classes is more important than the accidental
similarities between singleton classes used as instance-specific
classes and singleton-classes used as metaclasses.

And I guess eigenclass is not a bad term to use for a class which
provides behavior to a single instance, i.e. an instance-specific
class, but I don't like it as a term for a metaclass.

But that's just me, after all it's all just images caused by smoke and
mirrors <G>

···

On Sat, May 16, 2009 at 1:24 AM, Gary Wright <gwtmp01@mac.com> wrote:

On May 15, 2009, at 7:46 AM, Joshua Ballanco wrote:

Right. So then the question: is there a compelling reason that they
should be allowed to be subclassed? (I mean, other than the obvious:
"Because that's not the way Ruby was designed" answer.) I cannot come up
with any compelling technical reason why the artificial limitation of being
unable to subclass "virtual classes"[1] should exist.

The purpose of an eigenclass is to hold per object methods. As such, there
is always always a one-to-one relationship between an eigenclass and its
"eigeninstance" (sorry, but I can't think of a clearer term at the moment).

The idea of subclassing an eigenclass contradicts that one-to-one
relationship. If you could subclass an eigenclass, there would be no way to
cause ruby to search the new subclass for instance methods since it wouldn't
have a matching eigeninstance (sorry, again). It would not be in the method
lookup chain of any object.

Just to be complete, I'll point out that there is a way for eigenclasses to
have a subclass and that is when the eigeninstances are themselves classes.
This is most clear in 1.9. If B is a subclass of A, then B.eigenclass will
be a subclass of A.eigenclass. Even in this case A and B have to exist
before their eigenclasses can be accessed and not the other way around.

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

Rick Denatale wrote:

But that's just me, after all it's all just images caused by smoke and mirrors <G>

:slight_smile: Perhaps. But there's also the more mundane possibility that it's
just inconsistent semantics and bad terminology.

There seems to be two forces contributing to this. Firstly, the
profusion of terms (many of which are clearly misnomers) is getting out
of hand, and is unhelpful to newcomers. Secondly, many people still
resort to referring to MRI runtime structures when discussing semantics
even though Ruby is now being implemented on the JVM, on the CLR, on the
Objective-C runtime and presumably elsewhere. There is now a clear need
to be able to discuss semantics at a sufficient level of abstraction,
i.e. purely in terms of what is revealed by the reflection API. If there
are any inconsistencies or anomalies revealed by the API, then this
should be addressed directly.

The central point that I think should be made here is that per-instance
behaviour has nothing to do with inheritance, per se. One obvious
language rule will be that per-instance behaviour should take precedence
over instance-common behaviour and one convenient way of implementing
that is to snap an eigenclass into the chain between the instance and
its class. But that implementation-specific runtime structure won't
serve as a definition of the semantics for implementations that work
differently. Take, for example, a "Ruby in the browser" (perhaps one
already exists?) that translates onto the JavaScript object model. It
would make perfect sense for this to implement per-instance behaviour as
object properties. The class << obj syntax would then just be a way of
manipulating the per-instance behaviour collectively. The only
assumption being that method resolution is correctly (and hopefully
efficiently) implemented. And the eigenclass would be purely a
conceptual notion, existing only as several properties of an object,
taken together. No problem whatsoever.

It seems that the snapping of eigenclasses into the structure was
intended to be a private implementation detail. Certain reflective
methods were written in such a way as to skip over the eigenclasses, as
though they were not to be regarded as part of the public (semantic)
inheritance hierarchy. However, there were one or two leaks. For
example, eigenclass.superclass reports the original class of the
instance (in 1.9 at least). And, in a similar vein, the keyword super
chains from an instance-specific method to a regular instance method.
(As it happens, it transpires that this is convenient way of enabling
interception.) Ideally, one would like to stem the leaks as far as
possible without causing too much breakage of existing code.

Here's a few conclusions I've come to. I'd be interested to hear what
others think.

(i) The correct way to present method resolution is the way
Flanagan/Matz do on page 258 of their book, i.e. as an algorithm. It's
simple, intuitive and easy to remember. Ruby implementations are free to
implement this algorithm any way that's convenient (and fast) in their
given context. The wrong way to present method resolution is the Pickaxe
approach of using diagrams from MRI's runtime structure. Apart from not
being portable (which will become a bigger and bigger problem), it's
backwards. That runtime structure exists like that in order to support
certain desired semantics. So deriving the semantics from the structure
is neither here nor there. Just state them up front.

(ii) The keyword super should just be defined to work according to the
method resolution algorithm, not the inheritance hierarchy. This is what
happens at the moment, so it's just a formalising of the de facto
situation and doesn't break anything. This would preserves the
convenient chaining from a per-instance method to the regular instance
method and makes it clear that there are potentially lots of other
useful chaining opportunities. As Gary pointed out to David earlier in
this thread, the sequence of classes produced by the method resolution
algorithm is different from the inheritance hierarchy because of module
mixin and eigenclasses. (And in Ruby, module mixin is not inheritance,
regardless of the similarity.)

(iii) The superclass method should be defined to return Object for the
eigenclass of a regular object. In other words, the original statement I
alluded to from page 261 of RPL would now be true. The current situation
(returning the class of the instance) is an implementation leak. I don't
think there would be much breakage resulting from this and it cleans
things up.

(iv) In principle, class method resolution could also be defined purely
by an algorithm, as a language rule, which would then not require
explicit semantic inheritance to work.

Lastly, the terminology situation needs to be addressed by the
community. The term eigenclass is really spot on for a conceptual
container for per-instance behaviour and it's great that RPL adopted it.
The other terms are not acceptable synonyms; they are misnomers.

(a) metaclass
Even _why, in the 2005 piece that led to the widespread use of the term
metaclass, acknowledged that the term was wrong: 'This definition
doesn’t really work with Ruby, though, since "a class which defines a
class" is simply: a Class'. The simplest way to illustrate this is to
compare Smalltalk and Ruby. In Smalltalk, if you evaluate the expression
"Integer class", you get "Integer class" (i.e. the same thing). The
answer being the same as the question is an expression of the anonymous
nature of the metaclass, but is read by a Smalltalk programmer as "the
metaclass of Integer". In Ruby, if you evaluate the expression
"Integer.class", you get "Class". Thus, even if the eigenclasses of
class objects in Ruby play a similar role to metaclasses in Smalltalk,
they are not metaclasses. By definition.

(b) singleton class
As pointed out in the "Ruby in the browser" example above, there is no
need for the eigenclass to actually exist as a regular class, let alone
have a single instance. Clearly then, the term singleton class would
only be appropriate to some implementations and would consequently be a
misnomer.

While I'm at it, the term "singleton method" is also a misnomer. The
term "singleton" derives from set theory. A singleton set is a set with
one element. As such, the GoF pattern is a perfect fit. However, there
is no singleton set anywhere in this situation (unless you get really
silly in your choice of set). You can have many instances, each with
many per-instance methods. The only appearance of the word "one" is on
one side of the cardinality of an association (and not on the method
side). For reasons of symmetry, eigenmethod is a far better term.

Regards,

Danny.

···

--
Posted via http://www.ruby-forum.com/\.

Danny O cuiv wrote:

There seems to be two forces contributing to this. Firstly, the
profusion of terms (many of which are clearly misnomers) is getting out
of hand, and is unhelpful to newcomers. Secondly, many people still
resort to referring to MRI runtime structures when discussing semantics
even though Ruby is now being implemented on the JVM, on the CLR, on the
Objective-C runtime and presumably elsewhere.

Yet more runtimes: the Parrot VM (Cardinal), a modified version of the
64-Bit Gemstone Smalltalk VM (MagLev), the Flash/ActionScript Tamarin
VM (Red Sun) and the SAP NetWeaver/ABAP VM (Blue Ruby). Plus
specialized Ruby VMs (Rubinius, tinyrb).

There is now a clear need
to be able to discuss semantics at a sufficient level of abstraction,
i.e. purely in terms of what is revealed by the reflection API. If there
are any inconsistencies or anomalies revealed by the API, then this
should be addressed directly.

Let's not forget the current ISO standardization work for Ruby.
Anybody remember Open OfficeXML? It was standardized by
reverse-engineering MS Word, exactly the way that you describe (which
is basically the way RubySpec works: reverse-engineering MRI and
YARV). In the end, it was so incredibly hard to implement that even
Microsoft itself shipped a working implementation of ODF (i.e. the
native OpenOffice.Org XML format) *before* OOXML (i.e. the native MS
Office XML format). [In fact, OOXML is widely believed to be
un-implementable, because of logical contradictions in the spec.]

We *need* a specification of Ruby's semantics, independent of MRI
and/or YARV, both for the sake of all the other implementations and
ISO Ruby.

Unfortunately, there is really only one person that could write such a
spec.

Ideally, of course, that specification would be both formal and
abstract enough, so that we can automatically generate an interpreter
from it, or at least a testsuite. Or feed it to an automatic theorem
prover, and have it prove interesting properties like memory safety,
$SAFE-level safety, consistency etc.

And now, I'm going down to the stables to feed my unicorn :slight_smile:

The central point that I think should be made here is that per-instance
behaviour has nothing to do with inheritance, per se. One obvious
language rule will be that per-instance behaviour should take precedence
over instance-common behaviour and one convenient way of implementing
that is to snap an eigenclass into the chain between the instance and
its class. But that implementation-specific runtime structure won't
serve as a definition of the semantics for implementations that work
differently. Take, for example, a "Ruby in the browser" (perhaps one
already exists?) that translates onto the JavaScript object model.

HotRuby was (is?) an implementation of not "Ruby in the browser" but
"YARV in the browser". It was basically a YARV bytecode interpreter
loop written in JavaScript (meaning it requires that the Ruby
sourcecode gets pre-compiled with the actual YARV compiler, the
bytecode dumped to disk and then delivered to the browser as a binary
blob).

YARI (Yet Another Ruby Interpreter) aka IoRuby was an implementation
of Ruby (1.6, I think – it was before my time) on the Io runtime in
Io, which is a prototype-based OO language.

There has been talk about implementing Ruby on top of V8, which is an
ECMAScript engine.

There has been talk about implementing Ruby on top of NekoVM, which
has a unified class/prototype model.

None of these implementations actually have come even close to doing
anything even remotely useful and of course couldn't even dream of
running (let alone *passing*) RubySpec. But, there is at least *some*
interest in running Ruby on top of "prototype-ish" runtimes.

jwm