Difference between a method »living« in a module (as namespace) and in main

Hi,

I’m struggling to get my head around the difference between a method being within main and the same method »living« in a module.

The following Ruby script is a simplified example of where it worked and started for me:

···

—-
  def tst
    puts "Tst: in #{inspect}"
  end
  puts "Main: in #{inspect}"
  tst

#=> Main: in main
#=> Tst: in main
—-

After wrapping this script into a module as a namespace to organize the code it returns a NameError:
—-
  module M
    def tst
      puts "Tst: in #{inspect}"
    end
    puts "Main: in #{inspect}"
    tst
  end

#=> Main: in M
#=> undefined local variable or method `tst’ for M:Module
—-

I thought of main just being one level above modules in the hierarchy and therefore didn’t expect a different behavior. I’ve found already interesting articles esp. re. self [1] and can fix the NameError by defining my tst method within a module on the self object:
—-
  module M
    def self.tst
      puts "Tst: in #{inspect}"
    end
    puts "Main: in #{inspect}"
    tst
  end
#=> Main: in main
#=> Tst: in main
—-

Although the code’s working now, I’m not really sure what’s the reason behind this: why is it that I have to define a method within the same namespace on the self object to get access to it but can access it ´directly´ from within main?

Any feedback or hints for specific further readings about this would be more than welcome!

Cheers,
Michael

_____
[1] http://yehudakatz.com/2009/11/15/metaprogramming-in-ruby-its-all-about-the-self/

Hi Michael,

I'll start with a lemma:

A Module is not a namespace.

Modules are small chunks of functionality that can be mixed into existing
classes or objects (using "include".) They're Ruby's version of multiple
inheritance; part of the OO-model, not the lexical.

Hi,

I’m struggling to get my head around the difference between a method being
within main and the same method »living« in a module.

The following Ruby script is a simplified example of where it worked and
started for me:
​[snip]

After wrapping this script into a module as a namespace to organize the
code it returns a NameError:
​[snip]

A Module is not a namespace; it's more like a lightweight class. If you
replaced the keyword "module" with "class" in your sample above, would you
still expect the same things of it?

I thought of main just being one level above modules in the hierarchy and

therefore didn’t expect a different behavior. I’ve found already
interesting articles esp. re. self [1] and can fix the NameError by
defining my tst method within a module on the self object:
​[snip]

Main is just an object, like any other. It just happens to "inherit" all
the methods that aren't defined inside any module or class.

Although the code’s working now, I’m not really sure what’s the reason

behind this: why is it that I have to define a method within the same
namespace on the self object to get access to it but can access it
´directly´ from within main?

Any feedback or hints for specific further readings about this would be
more than welcome!

Defining a method inside a class or module means that method can be invoked
on an *instance* of the class or module.

···

On 16 May 2016 at 06:56, Michael Schwarze <michael@schwarze-web.de> wrote:

~~~
  class Foo
    def foo
    end
  end
  Foo.new.foo

  module Fu
    def fu
    end
  end
  class Foo
    include Fu
  end
  Foo.new.fu
~~~

To make it be invoked on the class or module itself, you have to define it
in the class or module's eigenclass. (Think of the eigenclass as: a special
class that only this object instantiates.) You can define a method on the
eigenclass either directly:

~~~
  class Foo
    def self.bar
    end
  end
  Foo.bar #=> nil
~~~

or by reopening the eigenclass:

~~~
  class Foo
    class <<self
      def bar
      end
    end
  end
  Foo.bar #=> nil

  # OR:

  class <<Foo
    def bar
    end
  end
  Foo.bar #=> nil
~~~

These examples also work if you replace the outermost "class" with "module".

Any time you "def" a method outside of any class or module, it gets shoved
into Main's eigenclass as a private method.

​Cheers
--
  Matthew Kerwin
  http://matthew.kerwin.net.au/

Hi Matthew,

Thank you so much, that solved the riddle for me:

A Module is not a namespace. […] They’re Ruby’s version of multiple inheritance; part of the OO-model, not the lexical. […] A Module is […] more like a lightweight class. If you replaced the keyword "module" with "class" in your sample above, would you still expect the same things of it?

I took the wrong turn with putting modules into the lexical part of Ruby and that’s why it did not work as expected for me. Seeing them within Ruby’s OO-model and comparing them to classes instead has brought me back on track and the behavior makes sense for me again, esp. after this:

Defining a method inside a class or [*]module[*] means that method can be invoked on an *instance* of the class or [*]module[*].

Regards,
Michael

···

Am 16.05.2016 um 01:19 schrieb Matthew Kerwin <matthew@kerwin.net.au>:

I'll start with a lemma:

A Module is not a namespace.

I disagree: a module is also a namespace. The confusion is probably
related to the "also" and the fact that methods defined in a module
are treated differently than constants. Additionally you can define
singleton methods of the module which then look like namespaced in the
module as well as instance methods which are intended to be mixed into
other classes or instances.

Modules are small chunks of functionality that can be mixed into existing
classes or objects (using "include".) They're Ruby's version of multiple
inheritance; part of the OO-model, not the lexical.

That is one usage of them. Creating namespaces is another.

Defining a method inside a class or module means that method can be invoked
on an *instance* of the class or module.

There are no instances of modules. There are only instance of classes
which can mix in modules.

Any time you "def" a method outside of any class or module, it gets shoved
into Main's eigenclass as a private method.

No. Rather, it's a private method of Object:

$ ruby -e 'def foo;end;m=method(:foo);p
m,m.owner,Object.private_instance_methods.grep(/foo/)'
#<Method: Object#foo>
Object
[:foo]

Compare:

$ ruby -e 'def foo;end;p method(:foo); def
self.bar;end;m=method(:bar);p m,m.owner'
#<Method: Object#foo>
#<Method: main.bar>
#<Class:#<Object:0x000000015f2408>>

You can see that #foo is made an instance method of Object while #bar
is defined in main's eigenclass.

$ ruby -e 'def foo;end;def self.bar;end;class
X;end;X.new.instance_eval{p foo;p bar}'
nil
-e:1:in `block in <main>': undefined local variable or method `bar'
for #<X:0x000000016ebd00> (NameError)
from -e:1:in `instance_eval'
from -e:1:in `<main>'

Kind regards

robert

···

On Mon, May 16, 2016 at 1:19 AM, Matthew Kerwin <matthew@kerwin.net.au> wrote:

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can
- without end}
http://blog.rubybestpractices.com/

Hi Matthew,

Thank you so much, that solved the riddle for me:

A Module is not a namespace. […]

It is:

$ ruby -e 'module A; class B; end;p B;end'

They’re Ruby’s version of multiple inheritance; part of the OO-model, not the lexical. […] A Module is […] more like a lightweight class. If you replaced the keyword "module" with "class" in your sample above, would you still expect the same things of it?

I took the wrong turn with putting modules into the lexical part of Ruby and that’s why it did not work as expected for me. Seeing them within Ruby’s OO-model and comparing them to classes instead has brought me back on track and the behavior makes sense for me again, esp. after this:

Defining a method inside a class or [*]module[*] means that method can be invoked on an *instance* of the class or [*]module[*].

Please see my comments in the other thread.

Kind regards

robert

···

On Mon, May 16, 2016 at 10:12 AM, Michael Schwarze <michael@schwarze-web.de> wrote:

Am 16.05.2016 um 01:19 schrieb Matthew Kerwin <matthew@kerwin.net.au>:

A::B

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can
- without end}
http://blog.rubybestpractices.com/

Hi Robert, Matthew,

Thanks again for your input on this which helped me a lot in digging deeper into this topic.

[…] A Module is not a namespace.

I disagree: a module is also a namespace.

Found some additional (uniting?) insight [1] on this: »A Ruby module is the container part of a class *without the factory. You can’t instantiate a module*, but you can put things inside of a module. Modules can hold methods, constants, classes, and even other modules.«

Learned a lot from both of you and Russ about the OO-part, but

That is one usage of them [modules]. Creating namespaces is another.

how does this relate to my original somewhat contrived but still real and probably more name-spacing (lexical?) related problem?

···

Am 16.05.2016 um 13:03 schrieb Robert Klemme <shortcutter@googlemail.com>:
On Mon, May 16, 2016 at 1:19 AM, Matthew Kerwin <matthew@kerwin.net.au> wrote:

~~~
module M
  def tst
    puts "Tst: in #{inspect}"
  end
  puts "Main: in #{inspect}"
  tst
end
#=> NameError: undefined local variable or method `tst’ for M:Module
~~~

From my initial assumption (modules as namespaces) and supported (?) by Robert’s explanation I would assume my code not failing here. But it does. Matthew’s explanation re. the OO-part and modules being like classes, therefore defining my method with self.tst at the class hence module level, made perfectly sense to me. And worked. But what about the name space part of the story?

What I’m trying to achieve with my name space / module is to have some methods which can be invoked from anywhere as I was just refactoring a quite straight forward Ruby script (no OO); exactly as Robert laid out:

Am 16.05.2016 um 15:15 schrieb Robert Klemme <shortcutter@googlemail.com>:

[…] methods defined at top level look a bit like functions which are available independently from any instance. Making them private instance methods of Object allows that; they can be invoked anywhere without a receiver.

But that’s only working with main / private instance methods of Object…so where does my method tst live in the above example? As the error message states it’s obviously not in my module M. But if it were in main it should be available from anywhere, shouldn’t it? Hence, tst neither seems to be living in main…

Regards,
Michael

_____
[1] Russ Olsen, »Eloquent Ruby«: Chapter 15 »Use Modules as Name Spaces«

Mit freundlichem Gruß
Michael Schwarze

> I'll start with a lemma:
>
>> A Module is not a namespace.

I disagree: a module is also a namespace.

​Well, I did call it a lemma.​

The confusion is probably
related to the "also" and the fact that methods defined in a module
are treated differently than constants. Additionally you can define
singleton methods of the module which then look like namespaced in the
module as well as instance methods which are intended to be mixed into
other classes or instances.

​Indeed, but that can also be done with classes, no?​

> Modules are small chunks of functionality that can be mixed into existing
> classes or objects (using "include".) They're Ruby's version of multiple
> inheritance; part of the OO-model, not the lexical.

That is one usage of them. Creating namespaces is another.

> Defining a method inside a class or module means that method can be
invoked
> on an *instance* of the class or module.

There are no instances of modules. There are only instance of classes
which can mix in modules.

Following the multiple-inheritance model, an instance of a class that
includes a module is also an instance of the module. However accurately or
inaccurately this might reflect the actual MRI implementation, or your
personal model of Ruby's inheritance, it serves as an illustrative
statement.​

> Any time you "def" a method outside of any class or module, it gets
shoved
> into Main's eigenclass as a private method.

No. Rather, it's a private method of Object:

​Oh yeah, I knew that. I even blogged about it once. 'main' is my one blind
spot in Ruby.

Kind regards

robert

​Cheers

···

On 16 May 2016 at 21:03, Robert Klemme <shortcutter@googlemail.com> wrote:

On Mon, May 16, 2016 at 1:19 AM, Matthew Kerwin <matthew@kerwin.net.au> > wrote:

--
  Matthew Kerwin
  http://matthew.kerwin.net.au/

Hi Robert,

Thanks for joining in on this

···

Am 16.05.2016 um 13:06 schrieb Robert Klemme <shortcutter@googlemail.com>:
[…]
Please see my comments in the other thread.

although I have to admit that I’m confused again after reading your comments in the other thread :wink: Matthew’s explanation made perfectly sense to me as it explained the behavior of my code…need to digest your input and play a little bit around with it.

Cheers
Michael

The confusion is probably
related to the "also" and the fact that methods defined in a module
are treated differently than constants. Additionally you can define
singleton methods of the module which then look like namespaced in the
module as well as instance methods which are intended to be mixed into
other classes or instances.

Indeed, but that can also be done with classes, no?

Yes, but why the "but"? Did somebody claim this is not true for classes?

> Modules are small chunks of functionality that can be mixed into
> existing
> classes or objects (using "include".) They're Ruby's version of multiple
> inheritance; part of the OO-model, not the lexical.

That is one usage of them. Creating namespaces is another.

> Defining a method inside a class or module means that method can be
> invoked
> on an *instance* of the class or module.

There are no instances of modules. There are only instance of classes
which can mix in modules.

Following the multiple-inheritance model, an instance of a class that
includes a module is also an instance of the module. However accurately or
inaccurately this might reflect the actual MRI implementation, or your
personal model of Ruby's inheritance, it serves as an illustrative
statement.

Yes, one can look at it that way. I still feel uncomfortable with this
as every object can only have one type and not many. It's the
difference between

x.kind_of? AnyClass
x.class == AnyClass

> Any time you "def" a method outside of any class or module, it gets
> shoved
> into Main's eigenclass as a private method.

No. Rather, it's a private method of Object:

Oh yeah, I knew that. I even blogged about it once. 'main' is my one blind
spot in Ruby.

Yes, it is odd. I never understood the motivation for doing this.
Maybe it's because methods defined at top level look a bit like
functions which are available independently from any instance. Making
them private instance methods of Object allows that; they can be
invoked anywhere without a receiver.

Kind regards

robert

···

On Mon, May 16, 2016 at 2:23 PM, Matthew Kerwin <matthew@kerwin.net.au> wrote:

On 16 May 2016 at 21:03, Robert Klemme <shortcutter@googlemail.com> wrote:

On Mon, May 16, 2016 at 1:19 AM, Matthew Kerwin <matthew@kerwin.net.au> >> wrote:

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can
- without end}
http://blog.rubybestpractices.com/

> Hi Matthew,
>
> Thank you so much, that solved the riddle for me:
>
>>
>> A Module is not a namespace. […]

It is:

$ ruby -e 'module A; class B; end;p B;end'
A::B

Please forgive my imprecision. If my over simplification helped resolve an
issue and give someone understanding then surely it still has some value?​

···

On 16 May 2016 at 21:06, Robert Klemme <shortcutter@googlemail.com> wrote:

On Mon, May 16, 2016 at 10:12 AM, Michael Schwarze > <michael@schwarze-web.de> wrote:
>> Am 16.05.2016 um 01:19 schrieb Matthew Kerwin <matthew@kerwin.net.au>:

> They’re Ruby’s version of multiple inheritance; part of the OO-model,
not the lexical. […] A Module is […] more like a lightweight class. If you
replaced the keyword "module" with "class" in your sample above, would you
still expect the same things of it?
>
> I took the wrong turn with putting modules into the lexical part of Ruby
and that’s why it did not work as expected for me. Seeing them within
Ruby’s OO-model and comparing them to classes instead has brought me back
on track and the behavior makes sense for me again, esp. after this:
>
>> Defining a method inside a class or [*]module[*] means that method can
be invoked on an *instance* of the class or [*]module[*].

Please see my comments in the other thread.

Kind regards

robert

--
  Matthew Kerwin
  http://matthew.kerwin.net.au/

Thanks again for your input on this which helped me a lot in digging deeper into this topic.

You're welcome!

Found some additional (uniting?) insight [1] on this: »A Ruby module is the container part of a class *without the factory. You can’t instantiate a module*, but you can put things inside of a module. Modules can hold methods, constants, classes, and even other modules.«

Learned a lot from both of you and Russ about the OO-part, but

That is one usage of them [modules]. Creating namespaces is another.

how does this relate to my original somewhat contrived but still real and probably more name-spacing (lexical?) related problem?

~~~
module M
  def tst
    puts "Tst: in #{inspect}"
  end
  puts "Main: in #{inspect}"
  tst
end
#=> NameError: undefined local variable or method `tst’ for M:Module
~~~

From my initial assumption (modules as namespaces) and supported (?) by Robert’s explanation I would assume my code not failing here. But it does. Matthew’s explanation re. the OO-part and modules being like classes, therefore defining my method with self.tst at the class hence module level, made perfectly sense to me. And worked. But what about the name space part of the story?

In this case there is no "name space part of the story". Modules have
different purposes but not all of them show up in each use case. The
namespace part comes into play if you define constants OR define
instance methods of the module instance:

Case 1: constants

irb(main):001:0> module Ns1; module Ns2; class X; end; p X; end end
Ns1::Ns2::X
=> Ns1::Ns2::X
irb(main):002:0> class Ns3; module Ns2; class X; end; p X; end end
Ns3::Ns2::X
=> Ns3::Ns2::X
irb(main):003:0> module Ns4; FOO = 123; end
=> 123
irb(main):004:0> Ns4::FOO
=> 123
irb(main):005:0> Ns4.FOO
NoMethodError: undefined method `FOO' for Ns4:Module
from (irb):3
from /usr/bin/irb:11:in `<main>'

Case 2: instance methods of the module (implicitly defined in the
eigenclass as Matthew has shown):

irb(main):001:0> module Ns1; def self.tst; printf "tst in %p\n", self end end
=> :tst
irb(main):002:0> Ns1.tst
tst in Ns1
=> nil
irb(main):003:0> Ns1::tst
tst in Ns1
=> nil

Note the different possible calling syntax: for a constant only "::"
works but for a method "." works as well.

What I’m trying to achieve with my name space / module is to have some methods which can be invoked from anywhere as I was just refactoring a quite straight forward Ruby script (no OO); exactly as Robert laid out:

[…] methods defined at top level look a bit like functions which are available independently from any instance. Making them private instance methods of Object allows that; they can be invoked anywhere without a receiver.

More precisely: they (private methods) must be invoked without a
receiver. If you want to define methods which are available everywhere
why don't you just define them on top level? If you want to put them
into a namespace then you need to define instance methods of the
module as done in the example above and as Matthew already
demonstrated. So you would do

irb(main):001:0> module Ns1; def self.foo; printf "foo in %p\n", self; end end
=> :foo
irb(main):002:0> module Ns2; Ns1::foo; end
foo in Ns1
=> nil
irb(main):003:0> module Ns2; ::Ns1::foo; end
foo in Ns1
=> nil

And, you see, the global namespace is not polluted:

irb(main):004:0> module Ns2; foo; end
NameError: undefined local variable or method `foo' for Ns2:Module
from (irb):4:in `<module:Ns2>'
from (irb):4
from /usr/bin/irb:11:in `<main>'
irb(main):005:0> foo
NameError: undefined local variable or method `foo' for main:Object
from (irb):5
from /usr/bin/irb:11:in `<main>'

But that’s only working with main / private instance methods of Object…so where does my method tst live in the above example? As the error message states it’s obviously not in my module M. But if it were in main it should be available from anywhere, shouldn’t it? Hence, tst neither seems to be living in main…

It is an instance method defined in the module. This means, it can be
executed on instances of a class mixing in that module OR by any
instance extended by that module:

irb(main):001:0> module M; def tst; printf "tst in %p\n", self end end
=> :tst

irb(main):002:0> M.tst
NoMethodError: undefined method `tst' for M:Module
from (irb):2
from /usr/bin/irb:11:in `<main>'
irb(main):003:0> M::tst
NoMethodError: undefined method `tst' for M:Module
from (irb):3
from /usr/bin/irb:11:in `<main>'
irb(main):004:0> class X; include M end
=> X

irb(main):005:0> X.new.tst
tst in #<X:0x0000000177a1e0>
=> nil

irb(main):009:0> o = Object.new
=> #<Object:0x00000001763d00>
irb(main):010:0> o.tst
NoMethodError: undefined method `tst' for #<Object:0x00000001763d00>
from (irb):10
from /usr/bin/irb:11:in `<main>'
irb(main):011:0> o.extend M
=> #<Object:0x00000001763d00>
irb(main):012:0> o.tst
tst in #<Object:0x00000001763d00>
=> nil

[1] Russ Olsen, »Eloquent Ruby«: Chapter 15 »Use Modules as Name Spaces«

Kind regards

robert

···

On Mon, May 16, 2016 at 5:55 PM, Michael Schwarze <michael@schwarze-web.de> wrote:

Am 16.05.2016 um 13:03 schrieb Robert Klemme <shortcutter@googlemail.com>:
On Mon, May 16, 2016 at 1:19 AM, Matthew Kerwin <matthew@kerwin.net.au> wrote:
Am 16.05.2016 um 15:15 schrieb Robert Klemme <shortcutter@googlemail.com>:

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can
- without end}
http://blog.rubybestpractices.com/

Hi Robert,

Thanks for joining in on this

You're welcome - even though I am not sure I made matters better or worse.

[…]
Please see my comments in the other thread.

although I have to admit that I’m confused again after reading your comments in the other thread :wink:

I am sorry for the added confusion.

Matthew’s explanation made perfectly sense to me as it explained the behavior of my code…need to digest your input and play a little bit around with it.

That is usually the best approach in my experience. That way, new
knowledge will stick much better. :slight_smile:

Cheers

robert

···

On Mon, May 16, 2016 at 1:40 PM, Michael Schwarze <michael@schwarze-web.de> wrote:

Am 16.05.2016 um 13:06 schrieb Robert Klemme <shortcutter@googlemail.com>:

--
[guy, jim, charlie].each {|him| remember.him do |as, often| as.you_can
- without end}
http://blog.rubybestpractices.com/

Hi Robert,

Thank you so much for taking your time again and your extensive explanation; that’s really more than expected!

If you want to define methods which are available everywhere why don’t you just define them on top level?

Good idea; that solves it.

If you want to put them into a namespace then you need to define instance methods of the module

Had a somewhat weird understanding of the namespaces in Ruby; I think I missed the point here that everything’s an object in Ruby…

Cheers,
Michael

···

Am 16.05.2016 um 22:37 schrieb Robert Klemme <shortcutter@googlemail.com>: