[ANN] Article: Seeing Metaclasses Clearly

Hi --

Nice article, but I disagree in the point that @@vars are simpler than class instance variables. Class instance variables have odd semantics in the current Ruby which means that they will probably do some detail different than you expect.

But you have to do:

class Foo
  @var1 =
  @var2 = {}
  class << self
    attr_accessor :var1, :var2
  end
end

to get reasonable access to those class instance variables inside an instance method, and even then you have to do self.class.var1

Once you 'get it' it's not TERRIBLY difficult, but I would say that the above is more than enough justification for calling @@foo simpler. Simpler, but also confusing in the inherited-class cases.

Also, they're different. In 2.0, it sounds like they will be more
similar to each other, because class vars will be truly scoped per
class. I'm not in favor of that. Class variables, in my view, are
responsible already for a huge amount of misunderstanding of
*instance* variables (because the @/@@ thing always makes people think
there must be some connection or similarity between the two entities,
and then they get confused when there isn't).

I'm not sure what the best thing would be. I tend to root for the
removal of class variables entirely. I might even be able to tolerate
#class_attr_* methods, which actually make no sense (since there's
really no more reason to create special methods, in that area, for
Class objects than any other object) but might keep things smoother.

David

···

On Mon, 18 Apr 2005, Gavin Kistner wrote:

On Apr 18, 2005, at 6:33 AM, Florian Groß wrote:

--
David A. Black
dblack@wobblini.net

why the lucky stiff wrote:

The `class_def' method is intended to be used to add instance methods (like the `initialize' I'm adding in Dwemthy's Array):

class Dragon; end
Dragon.class_def :initialize do
    @life = 11; @strength = 12
end

Oh, that's already done by Module#define_method then. Perhaps you can just make that method public?

[Regarding the term "metaclass"]

> I only use the term "metaclass" because it is the term predominantly used
> in the PickAxe II. While Dave does interchangibly use "virtual class"
> and "singleton class", the only term he uses to generically refer to the
> construct is "metaclass".

I read the Pickaxe differently, especially the box on p. 382. It
paraphrases Matz as having said: "You can call it _metaclass_ but,
unlike Smalltalk, it's not a class of a class; it's a singleton class
of a class."

I don't have the pickaxe with me (its at work), but let me quote from "Putting
Metaclasses to Work" by Ira R. Forman and Scott H. Danforth. The quote is
longish, but useful in understanding metaclasses. From the Preface:

"If one thinks of objects as cookies, then classes are analogous to cookie
cutters. Cookie cutters are teamplates that are used to make and determine
the properties of cookies; classes make and determine the properties of
objects. But ow are cookie cutters themselvfes made? Let us ssay that they
are pressed from sheet metal using a cookie cutter press (a piece of heavy
machinery). So, if a cookie cutter press is ed to make cookie cutters, what
is used to make classes? The answer is a metaclass."

So, the instances of a metaclass are classes. Ruby happens to implement
metaclasses as singleton classes (or virtual classes, whatever). So the
singleton class of a class is a metaclass, but the singleton class of
something that is not a class, would not be a meta class.

I would modify Why's metaid library thusly:

  class Object
    def singletonclass
      class << self
        self
      end
    end
  end
  class Class
    def metaclass
      singletonclass
    end
  end

Allowing ...

o = Object.new

=> #<Object:0x40253228>

Object.metaclass

=> #<Class:Object>

o.metaclass

NoMethodError: undefined method `metaclass' for #<Object:0x40253228>
  from (irb):4

o.singletonclass

=> #<Class:#<Object:0x40253228>>

Object.metaclass == Object.singletonclass

=> true

(I think I misread the p. 382 thing about "virtual". It does actually
sound like Matz is accepting it as a synonym for "singleton", though I
avoid it because I don't see anything "virtual" about singleton
classes. They're objects.)

I think the "virtual" is a modifier of "class", not "object". A virtual class
is something that is like a class in someways (it holds instance methods),
but unlike a class in other ways (can't create new instances of it).

···

On Mon, 18 Apr 2005, why the lucky stiff wrote:

On Monday 18 April 2005 03:55 pm, David A. Black wrote:

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

Florian Groß a écrit :

why the lucky stiff wrote:

The `class_def' method is intended to be used to add instance methods (like the `initialize' I'm adding in Dwemthy's Array):

class Dragon; end
Dragon.class_def :initialize do
    @life = 11; @strength = 12
end

Oh, that's already done by Module#define_method then. Perhaps you can just make that method public?

  def class_def name, &blk
   class_eval { self.define_method name, &blk }
  end

It was the other solution I thought for.

···

--
Lionel Thiry

Personal website: http://users.skynet.be/lthiry/

+1 - in fact, I think that should go into the FAQ, under "What is the
difference between a singleton class and a metaclass?"

martin

···

Jim Weirich <jim@weirichhouse.org> wrote:

So, the instances of a metaclass are classes. Ruby happens to implement
metaclasses as singleton classes (or virtual classes, whatever). So the
singleton class of a class is a metaclass, but the singleton class of
something that is not a class, would not be a meta class.

I would modify Why's metaid library thusly:

  class Object
    def singletonclass
      class << self
        self
      end
    end
  end
  class Class
    def metaclass
      singletonclass
    end
  end

So, the instances of a metaclass are classes. Ruby happens to implement
metaclasses as singleton classes (or virtual classes, whatever). So the
singleton class of a class is a metaclass, but the singleton class of
something that is not a class, would not be a meta class.

A singleton class never create an instance, in ruby.

Don't make confusion with another language, which use

  "Putting Metaclasses to Work: A New Dimension in Object-Oriented
   Programming"

as references.

Guy Decoux

Hi --

I wrote:

(I think I misread the p. 382 thing about "virtual". It does actually
sound like Matz is accepting it as a synonym for "singleton", though I
avoid it because I don't see anything "virtual" about singleton
classes. They're objects.)

I think the "virtual" is a modifier of "class", not "object". A virtual class
is something that is like a class in someways (it holds instance methods),
but unlike a class in other ways (can't create new instances of it).

It is a modifier of "class", not "object"; I think Matz (via Pickaxe)
is saying that "virtual class" is an acceptable synonym for "singleton
class". Actually I don't like it both for my reason ("virtual", taken
on its own, doesn't evoke something as concrete as a singleton class)
and for your reason (it has a meaning in other languages that is only
a partial fit, as I understand it, for Ruby's singleton classes).

David

···

On Tue, 19 Apr 2005, Jim Weirich wrote:

--
David A. Black
dblack@wobblini.net

Hi --

···

On Tue, 19 Apr 2005, Jim Weirich wrote:

I would modify Why's metaid library thusly:

class Object
   def singletonclass

Aww, don't you like the _ in my RCR? :slight_smile: (singleton_class)

David

--
David A. Black
dblack@wobblini.net

Hi --

I don't have the pickaxe with me (its at work), but let me quote from "Putting
Metaclasses to Work" by Ira R. Forman and Scott H. Danforth. The quote is
longish, but useful in understanding metaclasses. From the Preface:

"If one thinks of objects as cookies, then classes are analogous to cookie
cutters. Cookie cutters are teamplates that are used to make and determine
the properties of cookies; classes make and determine the properties of
objects. But ow are cookie cutters themselvfes made? Let us ssay that they
are pressed from sheet metal using a cookie cutter press (a piece of heavy
machinery). So, if a cookie cutter press is ed to make cookie cutters, what
is used to make classes? The answer is a metaclass."

So, the instances of a metaclass are classes. Ruby happens to implement
metaclasses as singleton classes (or virtual classes, whatever). So the
singleton class of a class is a metaclass, but the singleton class of
something that is not a class, would not be a meta class.

I have some trouble applying the quotation to Ruby, since classes make
but don't determine the properties of objects, and also -- even more
-- because that description of a metaclass sounds more like class
Class than like singleton classes of classes (since class Class is the
only thing in Ruby from which arbitrarily many Class objects can be
"pressed"). So it doesn't really seem like singleton classes of
classes are accounted for in the quotation.

(I will try to consolidate my comments next time and not respond three
times in a row :slight_smile:

David

···

On Tue, 19 Apr 2005, Jim Weirich wrote:

--
David A. Black
dblack@wobblini.net

David A. Black said:

I have some trouble applying the quotation to Ruby, since classes make
but don't determine the properties of objects,

I think you misunderstand "properties". It is true that the class does
not (directly) influence the attributes/instance variables of an object.
But properties also include the methods of the object, which are directly
influenced by the class.

Actually, the objections regarding "making objects" and "instance variable
properties" goes more to saying that the singleton classes of classes are
not /classes/, rather than saying they are not /meta/.

What I was trying to focus on is the tendency for folks to call /any/
singleton class a metaclass. I was just arguing that not all singleton
classes are metaclasses. I guess we can continue to discuss whether some
subset of singleton classes might be metaclasses.

Aww, don't you like the _ in my RCR? :slight_smile: (singleton_class)

I should have refered to the RCR before responding. I added my vote in
favor of it!

···

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

Hi --

David A. Black said:

I have some trouble applying the quotation to Ruby, since classes make
but don't determine the properties of objects,

I think you misunderstand "properties". It is true that the class does
not (directly) influence the attributes/instance variables of an object.
But properties also include the methods of the object, which are directly
influenced by the class.

I was actually thinking of methods, mainly. And you've swapped in a
new wording :slight_smile: I'd go along with "directly influenced", but not
"determined"; that is, an object's "birth class" certainly has a role
to play in the object's method capabilities, but is not a determinant.

Actually, the objections regarding "making objects" and "instance variable
properties" goes more to saying that the singleton classes of classes are
not /classes/, rather than saying they are not /meta/.

I must have garbled something I said somewhere along the line; I
definitely would never (knowingly :slight_smile: suggest that singleton classes
are not classes. See below....

What I was trying to focus on is the tendency for folks to call /any/
singleton class a metaclass. I was just arguing that not all singleton
classes are metaclasses. I guess we can continue to discuss whether some
subset of singleton classes might be metaclasses.

What I got from the quotation, and your gloss on it, was that
"metaclass" is recognized (outside of Ruby) to mean a factory for
classes, and that therefore it could be used to refer to a Class
object's singleton class. I would argue the opposite: singleton
classes, in every case, are non-factories, so to the extent that
"metaclass" might suggest factory-ness (i.e., factory of Class
objects), it could be misleading -- not only as a general term for
"singleton class" (which we both agree it isn't), but even as a
specialized term for "singleton class of Class object".

Correspondingly, if there is a "template for making cookie cutters
themselves" (to paraphrase) in Ruby, it would be the object Class
itself. So Class could be called *the* metaclass :slight_smile:

David

···

On Wed, 20 Apr 2005, Jim Weirich wrote:

--
David A. Black
dblack@wobblini.net

David A. Black said:

I'd go along with "directly influenced", but not
"determined"; that is, an object's "birth class" certainly has a role
to play in the object's method capabilities, but is not a determinant.

As do all the ancestors in the

What I got from the quotation, and your gloss on it, was that
"metaclass" is recognized (outside of Ruby) to mean a factory for
classes, and that therefore it could be used to refer to a Class
object's singleton class.

The analogy in the quote did indeed focus on the creation aspect, and I
did indeed gloss over the fact that singleton classes don't create new
instances. Perhaps the analogy was poorly suited for my point.

What I was trying to communicate with the quote is that Metaclasses are to
classes what classes are to objects. As classes hold the behavior for
objects, metaclasses hold the behavior classes. A Metaclass is a class
whose instances are classes (or in the case of a singleton metaclass, whos
one and only instance is a class).

I must have garbled something I said somewhere along the line; I
definitely would never (knowingly :slight_smile: suggest that singleton classes
are not classes. See below....

And just as the inability to create new instances does not disqualify
singleton classes from being classes, the inability for a singleton
metaclass does not disqualify it from being a metaclass.

Correspondingly, if there is a "template for making cookie cutters
themselves" (to paraphrase) in Ruby, it would be the object Class
itself.

Indeed, Class is the Metaclass from which all the singleton metaclasses
derive.

···

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

Jim Weirich said:

As do all the ancestors in the

Oops ............................^^^^^^^^

Sorry, a half formed thought escaped through my fingers. Just ignore this
part.

···

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

Hi --

David A. Black said:

I'd go along with "directly influenced", but not
"determined"; that is, an object's "birth class" certainly has a role
to play in the object's method capabilities, but is not a determinant.

As do all the ancestors in the

What I got from the quotation, and your gloss on it, was that
"metaclass" is recognized (outside of Ruby) to mean a factory for
classes, and that therefore it could be used to refer to a Class
object's singleton class.

The analogy in the quote did indeed focus on the creation aspect, and I
did indeed gloss over the fact that singleton classes don't create new
instances. Perhaps the analogy was poorly suited for my point.

What I was trying to communicate with the quote is that Metaclasses are to
classes what classes are to objects. As classes hold the behavior for
objects, metaclasses hold the behavior classes. A Metaclass is a class
whose instances are classes (or in the case of a singleton metaclass, whos
one and only instance is a class).

Although... here's something interesting:

irb(main):001:0> a = ""
=> ""
irb(main):002:0> c = (class << a; self; end)
=> #<Class:#<String:0x401f8b6c>>
irb(main):003:0> a.instance_of?(c)
=> false
irb(main):004:0> a.instance_of?(String)
=> true
irb(main):005:0> s = class << String; self; end
=> #<Class:String>
irb(main):006:0> String.instance_of?(s)
=> false
irb(main):007:0> String.instance_of?(Class)
=> true

I guess being an instance_of something is a unique relationship (only
one class-of-which-you-are-an-instance to a customer), and having a
singleton class doesn't mean you're an instance of it. That seems to
reinforce the non-factory aspect of all singleton classes, in the
sense that it disallows even the one "instance of" relationship that
one might have expected them to claim to have. (I expected it -- I
think I once knew, but had forgotten about, the above behavior.)

I must have garbled something I said somewhere along the line; I
definitely would never (knowingly :slight_smile: suggest that singleton classes
are not classes. See below....

And just as the inability to create new instances does not disqualify
singleton classes from being classes, the inability for a singleton
metaclass does not disqualify it from being a metaclass.

OK... but that, again, is where the quotation then becomes confusing,
since its chief point is that a "metaclass" is, specifically, a
factory for classes. If we use the word "metaclass" to mean singleton
class of a class, we're definitely giving up any "cookie-cutter"
analogy, and that seemed to be the whole thrust of that quotation.

Correspondingly, if there is a "template for making cookie cutters
themselves" (to paraphrase) in Ruby, it would be the object Class
itself.

Indeed, Class is the Metaclass from which all the singleton metaclasses
derive.

And all other Class objects too. But then "[Mm]etaclass" becomes
really kind of overloaded -- in the sense that on the one hand it is a
suitable term for Class (which is truly the thing that stamps out the
cookie cutters themselves), while on the other hand it gets used as a
term for something which is unequivocally *not* in the cookie-cutter,
meta-cookie-cutter, or factory family at all.

David

···

On Wed, 20 Apr 2005, Jim Weirich wrote:

--
David A. Black
dblack@wobblini.net

Although... here's something interesting:

[... elided IRB session that illustrates that singleton classes have no
instances ...]

Wow ... I didn't see that coming. That has implications for elsewhere in the
thread discussing the Ruby object model diagram where everyone is drawing
ASCII art with arrows labeled "instance-of". All those diagrams are rendered
invalid in one fell swoop. Wow. (I notice that Matz's diagram in the RI docs
does not explicitly label the horizontal arrows "instance-of" ).

So, what do we call the relationship between a class and its singleton class?
"singleton-instance-of"? Maybe.

Given that the Forman/Danforth formal definition of a metaclass is "A
metaclass is an object whose instances are classes", then the only
Forman/Danforth metaclass in Ruby is the class Class.

Although, after rereading the first few chapters of the book, it is clear that
the Class singleton classes fulfill a very similar role to the classical
Forman/Danforth metaclasses. One could argue that Forman's use of
"instance-of" really maps to "singleton-instance-of" ... but at that point we
are arguing definitions and it not worth pursuing.

I guess being an instance_of something is a unique relationship (only
one class-of-which-you-are-an-instance to a customer), and having a
singleton class doesn't mean you're an instance of it. That seems to
reinforce the non-factory aspect of all singleton classes, in the
sense that it disallows even the one "instance of" relationship that
one might have expected them to claim to have. (I expected it -- I
think I once knew, but had forgotten about, the above behavior.)

It also makes the name "Singleton" class a little odd. I thought it was
because a singleton class could only have a single instance. Now I find out
it can't even have that. Perhaps they should be Zeroton classes :slight_smile:

> Indeed, Class is the Metaclass from which all the singleton metaclasses
> derive.

And all other Class objects too.

Only things (that would be called metaclasses if they could only have
instances) are derived (i.e. inherit) from Class. All other class objects
are merely instances of Class.

But then "[Mm]etaclass" becomes
really kind of overloaded -- in the sense that on the one hand it is a
suitable term for Class (which is truly the thing that stamps out the
cookie cutters themselves), while on the other hand it gets used as a
term for something which is unequivocally *not* in the cookie-cutter,
meta-cookie-cutter, or factory family at all.

The formal definition does not require metaclasses (or classes) to create
instances ... only that they /have/ instances (which is you so clearly
demonstrated singleton classes have no instances). The factory aspect was
entirely part of the analogy and not part of the formal definition.

···

On Tuesday 19 April 2005 09:29 pm, David A. Black wrote:

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

David A. Black schrieb:

Although... here's something interesting:

irb(main):001:0> a = ""
=> ""
irb(main):002:0> c = (class << a; self; end)
=> #<Class:#<String:0x401f8b6c>>
irb(main):003:0> a.instance_of?(c)
=> false
irb(main):004:0> a.instance_of?(String)
=> true
irb(main):005:0> s = class << String; self; end
=> #<Class:String>
irb(main):006:0> String.instance_of?(s)
=> false
irb(main):007:0> String.instance_of?(Class)
=> true

But note that

   irb(main):001:0> a = ""
   => ""
   irb(main):002:0> c = (class << a; self; end)
   => #<Class:#<String:0x2dae6d0>>
   irb(main):003:0> c === a
   => true

Regards,
Pit

Hi --

It also makes the name "Singleton" class a little odd. I thought it was
because a singleton class could only have a single instance. Now I find out
it can't even have that. Perhaps they should be Zeroton classes :slight_smile:

I always thought it was singleton in the sense that, unlike the "birth
class", it was not shared with other objects. There is of course a
long history of discussion about the term "singleton", and the fact
that it's overloaded.... I've suggested "own class", and others have
suggested other terms, though I don't know that even a pronouncement
from Matz will change usage completely.

Indeed, Class is the Metaclass from which all the singleton metaclasses
derive.

And all other Class objects too.

Only things (that would be called metaclasses if they could only have
instances) are derived (i.e. inherit) from Class. All other class objects
are merely instances of Class.

But being a (mere) instance of Class is, I think, by far the closest
thing in Ruby to matching the proto-cookie-cutter thing. (I know
that's just an analogy, but Class actually fits it rather well.) Not
that every instance of Class is a metaclass, but rather that Class is
a metaclass, on account of the fact that it can create new classes
(meta or otherwise).

But then "[Mm]etaclass" becomes
really kind of overloaded -- in the sense that on the one hand it is a
suitable term for Class (which is truly the thing that stamps out the
cookie cutters themselves), while on the other hand it gets used as a
term for something which is unequivocally *not* in the cookie-cutter,
meta-cookie-cutter, or factory family at all.

The formal definition does not require metaclasses (or classes) to create
instances ... only that they /have/ instances (which is you so clearly
demonstrated singleton classes have no instances). The factory aspect was
entirely part of the analogy and not part of the formal definition.

Like "singleton", though, this may be a case where Matz isn't using it
in a way that fits precisely with any previous formal definitions. I
guess that's one of the hazards of writing a language with a cool,
uncommon design :slight_smile:

David

···

On Wed, 20 Apr 2005, Jim Weirich wrote:

--
David A. Black
dblack@wobblini.net

Hi --

···

On Wed, 20 Apr 2005, Pit Capitain wrote:

David A. Black schrieb:

irb(main):001:0> a = ""
=> ""
irb(main):002:0> c = (class << a; self; end)
=> #<Class:#<String:0x401f8b6c>>
irb(main):003:0> a.instance_of?(c)
=> false
irb(main):004:0> a.instance_of?(String)
=> true
irb(main):005:0> s = class << String; self; end
=> #<Class:String>
irb(main):006:0> String.instance_of?(s)
=> false
irb(main):007:0> String.instance_of?(Class)
=> true

But note that

irb(main):001:0> a = ""
=> ""
irb(main):002:0> c = (class << a; self; end)
=> #<Class:#<String:0x2dae6d0>>
irb(main):003:0> c === a
=> true

Yes, a.kind_of(c) is true, but that's never been a one-only
relationship.

David

--
David A. Black
dblack@wobblini.net

[Jim Weirich <jim@weirichhouse.org>, 2005-04-20 06.15 CEST]

> Although... here's something interesting:
>
[... elided IRB session that illustrates that singleton classes have no
instances ...]

Wow ... I didn't see that coming. That has implications for elsewhere in the
thread discussing the Ruby object model diagram where everyone is drawing
ASCII art with arrows labeled "instance-of". All those diagrams are rendered
invalid in one fell swoop. Wow. (I notice that Matz's diagram in the RI docs
does not explicitly label the horizontal arrows "instance-of" ).

Wow, it was only me. Since when I am everyone? Thanks for the compliment,
anyway.

Before your discoveries make you jump into conclusions, you should apply a
little of scientific thought. You have two facts: one, that the diagram
shows that Object --instanceof--> (Object). The other: that

$ ruby -e 'p Object.instance_of?(class<<Object;self;end)'
false

One possible explanation for this discrepancy is that the diagram is wrong.
Another is that instance_of? doesn't take into account singleton classes.

I guess it's the second one :).

Good luck.

···

On Tuesday 19 April 2005 09:29 pm, David A. Black wrote:

It also makes the name "Singleton" class a little odd. I thought it was
because a singleton class could only have a single instance. Now I find out
it can't even have that. Perhaps they should be Zeroton classes :slight_smile:

The instance-of relationship is not about creatability of new objects: it is about which object represents the behaviour of which other object, period. In the case of singleton classes, the instance is created *before* the class is, and it doesn't follow that this peculiar creation order makes that relationship a lesser-instance-of or a non-instance-of.

Where the "singleton" name really breaks is with indirect instance-of relationships: for example, (Object) has many indirect instance-of relationships, with *all* Ruby classes in the system, and you can instantiate the (Object) class as many times as you want to, and you always do whenever you are subclassing any class whatsoever, because every instance of Class is also an instance of (Object).

(note: the general instance-of such as Module#=== is transitive in the sense that it also includes instance-of-subclass-of and so on: every object is an instance-of Object.)

The formal definition does not require metaclasses (or classes) to create
instances ... only that they /have/ instances (which is you so clearly
demonstrated singleton classes have no instances).

I have trouble following that... we seem to both agree and disagree on the same thing at once...!

Maybe it's because of confusing the Object#class method with which is the direct-instance-of relationship of an object. The latter cannot be observed in Ruby: if an object x has no S-class, then it's directly-instance-of the result of x.class, and if an object has a S-class, it's directly-instance-of the result of (class<<x;self;end) and not of x.class, but there's no way to figure out whether an object has an S-class other than to cause that S-class to spring into existence (!).

Does this all make sense to you?

  ,-o---------o---------o---------o-. ,----. |
  > The Diagram is the Program (TM) | | ,-o----------------------------o-.
  `-o-----------------------------o-' | | Mathieu Bouchard (Montréal QC) |

···

On Wed, 20 Apr 2005, Jim Weirich wrote:
    > >---' | http://artengine.ca/matju |
    > > `-o------------------------------'