Namespaces implemented in ruby

Rich and Nathaniel and I were standing around at the conference this
weekend talking about Nathaniel's idea for implementing namespaces in
pure ruby, just so ruby users could play around with them until we get
them in ruby 2.0 (or maybe 1.9.1). Here's my first pass at an
implementation of that idea. It uses a slightly modified version of
Florian Gross's Binding.of_caller method.

The idea is pretty simple; each scope contains two local variables
called __ns (which namespace is currently being modified) and __use
(which namespaces are searched during method invocation). When a method
is added, a proxy method is created that does the method searching. No
method caching is done, but that could be implemented. The global
namespace is known as :__global.

The syntax is something like this:

  namespace :namespace do
    # methods created in this scope will be added to :namespace
  end

  use :namespace do
    # methods called in this scope will be searched in :namespace first,
    # then :__global.
  end

Hopefully someone can take this and extend it to properly handle
singleton methods and inheritance.

There is also one bug that I don't know how to address:

  class Foo; def foo; puts "global namespace"; end; end
  namespace :n { class Foo; def foo; puts "namespace n"; end; end }
  p = proc { Foo.new.foo }
  use :n { p.call }

This should print "global namespace" but currently prints "namespace n".
The problem is that both the outer scope and the inner scope share the
same __use variable, and I don't know any good way to actually make the
variable truly block-local. I'm all ears if anyone knows how to fix
this.

Paul

···

---

require 'thread'

def caller_binding(&block)
  old_critical = Thread.critical
  Thread.critical = true

  count = 0
  result, error = callcc do |cc|
    tracer = lambda do |*args|
      type, context = args[0], args[4]
      if type == "return"
        count += 1
        if count == 2
          set_trace_func(nil)
          cc.call(eval("binding", context), nil)
        end
      elsif type != "line"
        set_trace_func(nil)
        cc.call(nil, lambda { raise(ArgumentError, "unable to get binding" ) })
      end
    end
    set_trace_func(tracer)
    return nil
  end

  Thread.critical = false
  error.call if error
  yield result
end

def method_table
  h = {}
  methods.each do |name|
    h[name] = method(name)
  end
  return h
end

def namespace(ns)
  caller_binding do |b|
    old_ns = eval("defined?(__ns) ? __ns : nil", b)
    eval("__ns = #{ns.inspect}", b)
    eval("__mt ||= []; __mt << method_table", b)
    begin
      use(ns) do
        yield
      end
    ensure
      if old_ns then
        eval("__ns = #{old_ns.inspect}", b)
      else
        eval("__ns = nil", b)
      end

      eval("__mt.pop", b)
    end
  end
end

def use(ns)
  caller_binding do |b|
    eval("__use ||= []; __use << #{ns.inspect}", b)
    begin
      yield
    ensure
      eval("__use.pop", b)
    end
  end
end

class Object
  # TODO: need singleton_method_added too
  def self.method_added(name)
    caller_binding do |b|
      in_ma = eval("defined?(__in_ma) ? true : false", b)
      return if in_ma

      ns = eval("defined?(__ns) ? __ns : nil", b)
      ns ||= :__global
      mt = class << self; @mt ||= {}; end
      mt[ns] = instance_method(name)

      undef_method name
      self.class_eval <<-END
        __in_ma = true
        def #{name}(*args, &block)
          caller_binding do |b|
            use = eval("defined?(__use) ? __use : nil", b)
            use ||= []
            use = use.reverse + [:__global]
            use.each do |ns|
              # TODO: also search up the inheritance hierarchy
              mt = class << self.class; @mt ||= {}; end
              u = mt[ns]
              next if not u
              m = u.bind(self)
              return m.call(*args, &block)
            end
            raise NoMethodError, "undefined method `#{name}' for TODO"
          end
        end
      END
    end
  end
end

class Foo
  namespace :n do
    def foo; 42; end
  end

  namespace :m do
    def foo; 10; end
  end

  def foo; 1; end
end

def bar
  p Foo.new.foo
end

use :n do
  p Foo.new.foo #=> 42 (namespace n)

  use :m do
    p Foo.new.foo #=> 10 (namespace m)
  end

  use :z do
    p Foo.new.foo #=> 42 (namespace n)
  end

  bar #=> 1 (global namespace)
end

use :z do
  p Foo.new.foo #=> 1 (global namespace)
end

p Foo.new.foo #=> 1 (global namespace)

I hate to interrupt a techincal and probably excellent initial post
about namepsaces in ruby with such a noob question, but...

What would namespaces do for ruby that I, as a developer, shoud care
about? My current experience of them in my C++ is that they're magic
lines I put in my headers to allow access to classes of useful
functions, and thus I have them lumped in my brain as much like a
library, but different in a way that I understand less well than I
should.

Thanks for your patience.

The goal behind namespaces is the same as David Alan Black's Ruby
Behaviors (http://www.wobblini.net/ruby/behaviors/\). Unfortunately Ruby
Behaviors is not thread-safe, so we desire a better solution. At the
first RubyConf, David Simmons introduced us to the idea of selector
namespaces, and Matz wants to add them to Ruby 2.0.

Namespaces as I have implemented them (which are not entirely unlike
selector namespaces) differ from C++ namespaces greatly. C++ does not
allow modifying a class once it has been defined; Ruby does. The result
is that in Ruby, two different libraries can both add the same method to
a builtin class. Using namespaces in Ruby, each version of the method
can be added to a different namespace, and user code can select which
version of the method it wants (C++ namespaces would not allow two
functions from the same class to reside in different namespaces; to get
equivalent behavior in C++, one would use function overloading).

An example in Ruby:

  class String
    namespace :GPM {
      def %(*args)
        # namespace GPM modifies String#% to add formatting for GPM
        # integers and rationals
      end
    }
  end

  class String
    namespace :BigDecimal {
      def %(*args)
        # namespace BigDecimal modifies String#% to add formatting for
        # BigDecimals
      end
    }
  end

  bd = BigDecimal.new("10") ** 65536

  puts "%06f" % bd # this outputs the wrong answer

  use :BigDecimal {
    puts "%06f" % bd # this gives the right answer
  }

I don't think this is an ideal syntax for namespaces; I'm only working
with what Ruby gives me. Ideally constants could also be defined in a
namespace, so you could have the global String class or your own String
class in your own namespace.

On a side note, I used to be strongly against modifying classes. I have
since realized that Ruby only provides one mechanism for method
dispatch, and that mechanism dispatches based on the class of the
receiver. The user is then left with two options: to write his own
dispatch mechanism or make use of the builtin dispatch mechanism in
Ruby. The latter is far simpler.

On another side note, you should NEVER put a using declaration inside a
C++ header unless it resides in its own scope. See Herb Sutter's
GOTW#53 (http://www.gotw.ca/gotw/053.htm\) for a better explanation than
I can give.

Paul

···

On Thu, Oct 20, 2005 at 11:11:43AM +0900, NBarnes wrote:

What would namespaces do for ruby that I, as a developer, shoud care
about? My current experience of them in my C++ is that they're magic
lines I put in my headers to allow access to classes of useful
functions, and thus I have them lumped in my brain as much like a
library, but different in a way that I understand less well than I
should.

Paul Brannan wrote:

What would namespaces do for ruby that I, as a developer, shoud care
about? My current experience of them in my C++ is that they're magic
lines I put in my headers to allow access to classes of useful
functions, and thus I have them lumped in my brain as much like a
library, but different in a way that I understand less well than I
should.

The goal behind namespaces is the same as David Alan Black's Ruby
Behaviors (http://www.wobblini.net/ruby/behaviors/\). Unfortunately Ruby
Behaviors is not thread-safe, so we desire a better solution. At the
first RubyConf, David Simmons introduced us to the idea of selector
namespaces, and Matz wants to add them to Ruby 2.0.

Namespaces as I have implemented them (which are not entirely unlike
selector namespaces) differ from C++ namespaces greatly. C++ does not
allow modifying a class once it has been defined; Ruby does. The result
is that in Ruby, two different libraries can both add the same method to
a builtin class. Using namespaces in Ruby, each version of the method
can be added to a different namespace, and user code can select which
version of the method it wants (C++ namespaces would not allow two
functions from the same class to reside in different namespaces; to get
equivalent behavior in C++, one would use function overloading).

An example in Ruby:

  class String
    namespace :GPM {
      def %(*args)
        # namespace GPM modifies String#% to add formatting for GPM
        # integers and rationals
      end
    }
  end

  class String
    namespace :BigDecimal {
      def %(*args)
        # namespace BigDecimal modifies String#% to add formatting for
        # BigDecimals
      end
    }
  end

  bd = BigDecimal.new("10") ** 65536

  puts "%06f" % bd # this outputs the wrong answer

  use :BigDecimal {
    puts "%06f" % bd # this gives the right answer
  }

I don't think this is an ideal syntax for namespaces; I'm only working
with what Ruby gives me. Ideally constants could also be defined in a
namespace, so you could have the global String class or your own String
class in your own namespace.

The granularity used for declarations above would typically not be necessary.

   # orig.rb
   class SomeClass
     def foo()
       puts 'foo'
     end
   end

   # ns.rb
   namespace NS
     class SomeClass
       def foo()
         puts 'bar'
       end
     end
   end

   # test1.rb
   require 'orig'
   require 'ns'

   SomeClass.new.foo # 'foo'
   SomeClass.new.NS:foo # 'bar'

The alternative with more explicit invocation is just:

   with NS do SomeClass.new.foo end

Of course, truly excellent would be just being able to do
this with modules or Traits (some of the new proposals are
definitely overlapping) so that an explicit class would not
be required:

   # test2.rb
   require 'orig'

   module AdditionalBehaviour
     def foo()
       puts 'quux'
     end
   end

   with AdditionalBehaviour do SomeClass.new.foo end # 'quux'
   SomeClass.new.AdditionalBehaviour:foo # 'quux'
   SomeClass.new.foo # 'foo'

This, then, definitely strides into the AOP territory.

···

On Thu, Oct 20, 2005 at 11:11:43AM +0900, NBarnes wrote:

On a side note, I used to be strongly against modifying classes. I have
since realized that Ruby only provides one mechanism for method
dispatch, and that mechanism dispatches based on the class of the
receiver. The user is then left with two options: to write his own
dispatch mechanism or make use of the builtin dispatch mechanism in
Ruby. The latter is far simpler.

On another side note, you should NEVER put a using declaration inside a
C++ header unless it resides in its own scope. See Herb Sutter's
GOTW#53 (http://www.gotw.ca/gotw/053.htm\) for a better explanation than
I can give.

Paul