Autoload with a block?

ruby-talk
1

I've just discovered Ruby's autoload feature. It looks to imply something like
this:

autoload :Foo, 'foo'
Bar.autoload :Baz, 'bar/baz'

This implies that to have namespace'd modules autoloaded properly, I would end
up with a structure like this:

autoload :Foo, 'foo'

(and, in foo.rb)

module Foo
  autoload :Bar, 'foo/bar'
end

What if I want to abstract away such a system? In other words, what if I want
a light, portable version of Rails-style autoloading? In a perfect world, I
would be able to do something like this:

autoload :Foo do
  require 'foo'
  Foo.autoload :Bar ...
end

In that simplest form, it would at least centralize all my loading stuff in
one place. In a more meta form, I could run through some search paths I'm
likely to use (in my own app, for instance), and generate an autoload scheme
based on that.

Does this sound like a good idea? Or should I be hacking it together with
const_missing, instead?

2

No one had anything to say about this... Hmm.

In case you didn't realize: I'm testing all of this in Ruby 1.9.

I thought of a sneakier approach: The rdoc specifies that autoload will call
Kernel::require:

http://ruby-doc.org/core-1.9/classes/Kernel.html#M006100

And it does, indeed, seem to behave this way -- so long as I don't redefine
Kernel::require. A simple test:

$ irb1.9
irb(main):001:0> module Kernel
irb(main):002:1> alias_method :require_without_me, :require
irb(main):003:1> def require(*args, &block)
irb(main):004:2> puts 'Require called:'
irb(main):005:2> p args
irb(main):006:2> p block
irb(main):007:2> require_without_me *args, &block
irb(main):008:2> end
irb(main):009:1> end
=> nil
irb(main):010:0> autoload :Foo, 'foo'
=> nil
irb(main):011:0> Foo
loaded foo.rb
=> Foo
irb(main):012:0> require 'foo/bar'
Require called:
["foo/bar"]
nil
loaded bar.rb
=> true
irb(main):013:0>

For what it's worth, both foo.rb and foo/bar.rb have 'puts' statements in
them, declaring that they were loaded.

Maybe it's irb at fault? Hmm... I created this file:

module Kernel
  alias_method :require_without_me, :require
  def require(*args, &block)
    if args.first == 'test/file/not/loading'
      puts 'Not loading the test file'
    else
      require_without_me(*args, &block)
    end
  end
end

puts 'The following works:'
require 'test/file/not/loading'

puts 'The following does not:'
autoload :Foo, 'test/file/not/loading'
Foo

I'd expect it to end with a constant not found, or module not found.
Instead, I get:

$ ruby1.9 test.rb
The following works:
Not loading the test file
The following does not:
test.rb:17:in `<main>': no such file to load -- test/file/not/loading
(LoadError)

Doesn't look like I typoed.

So, in other words, autoload is either using its own interpretation of
require, or it's hardcoding (in C) a call to the original require, rather
than going through the Ruby code. (I wonder how this would have cooperated
with Rubygems, had that not been included?)

So far, the crushing limitation of autoload is that I can't easily autoload
namespace'd things, without littering the namespace itself with autoload
methods -- or eager-loading all parent namespaces.

Well, that, and I can't seem to customize its behavior _at_all_ other than by
rewriting it from scratch, making it somewhat useless.

3

Hi,

At Fri, 15 Aug 2008 15:57:21 +0900,
David Masover wrote in [ruby-talk:311365]:

autoload :Foo do
  require 'foo'
  Foo.autoload :Bar ...
end

In that simplest form, it would at least centralize all my loading stuff in
one place. In a more meta form, I could run through some search paths I'm
likely to use (in my own app, for instance), and generate an autoload scheme
based on that.

I don't think it makes things simpler, but it sounds
interesting. You can file it in the ITS, if you want.
<http://redmine.ruby-lang.org/projects/ruby/issues?set_filter=1&tracker_id=2&gt;

···

--
Nobu Nakada

4

I reported this bug a long time ago. It is a serious issue with one of
my programs. Because of it I am forced never to use autoload, and tell
all users of my program that they must do the same. I would have
expected this to be fixed for 1.9, and back-ported. It's disappointing
to hear it still remains an issue.

T.

···

On Aug 16, 1:18 am, David Masover <ni...@slaphack.com> wrote:

So, in other words, autoload is either using its own interpretation of
require, or it's hardcoding (in C) a call to the original require, rather
than going through the Ruby code. (I wonder how this would have cooperated
with Rubygems, had that not been included?)

5

I think I've found a solution, for myself -- basically a const_missing hack.

Actually, I cracked open the ActiveSupport source, curious to see how they did
it. And I spent probably an hour or two, staring at the code, before shaking
my head and starting from scratch.

Rails lets you write beautiful code, sometimes, but the Rails source sure
isn't beautiful, most of the time. (In my not-so-humble opinion.)

Still, I can't help but wonder -- if ActiveSupport had to do such backflips to
get it working, maybe there's a serious assumption in my own code?

Let me know if I'm missing something.
I'm still trying to think of ways this could break.

# You can use anything that provides an inflector.
# I'm already using Sequel here, so it works for me.
require 'sequel'

class Module
  def const_missing_with_autoload(name)
    our_name = (self==Object) ? '' : self.name + '::'
    begin
      require (our_name+name.to_s).underscore
      const_get name
    rescue LoadError
      const_missing_without_autoload(name)
    end
  end
  alias_method :const_missing_without_autoload, :const_missing
  alias_method :const_missing, :const_missing_with_autoload
end

···

On Saturday 16 August 2008 07:33:22 Trans wrote:

It is a serious issue with one of
my programs. Because of it I am forced never to use autoload, and tell
all users of my program that they must do the same.

6

Ok, that didn't work. All kinds of edge cases, and I can see why things like
autoload have been written.

So I wrote my own... but there's still one problem I can't seem to solve here:

class AutoLoader < BasicObject
  def initialize parent, mod, file
    @parent, @mod, @file = parent, mod, file
  end
end

# Pushed out here so I can get at Kernel.require.
AutoLoader.send :define_method, :method_missing do |*args, &block|
  @parent.send :remove_const, @mod
  Kernel.require @file
  @parent.const_get(@mod).send *args, &block
end

Typical usage would be:

self.class.const_set :Foo, AutoLoader.new(self.class, :Foo, 'foo')

And it works great -- exactly the way I want -- for _almost_ all cases.
I still can't do this:

module Foo
end

without actually loading foo.rb yet. (Somehow, Ruby knows my BasicObject isn't
a module, without ever tripping the method_missing there.)

Yet the same thing _does_ work with autoload:

autoload :Foo, 'foo'
module Foo; end

For what it's worth, Rails' const_missing has an even worse problem: If I do
the "module Foo" syntax above, I will define Foo, which means const_missing
will never be triggered, which means foo.rb will never be loaded.

It looks like my choices are either to learn to write a (non-portable) C
extension (and maybe still not have it work?), or to create a temporary file,
point autoload at that, and have it do the rest.

That sucks.

Is there any way to know for certain when a file is loaded, when it's autoload
that's doing it, short of modifying the file? (I'm specifically wanting to be
_notified_, so that I can prepare sub-constants to be autoloaded as well --
that is, when Foo is loaded, I want to automatically run Foo.autoload :Bar.)

Is there any way to force autoload to use a different mechanism?

Is there any way, in Ruby, to emulate autoload with actual feature parity?

Or am I really going to have to create a temporary file for each and every
source file I might possibly ever want to use?

···

On Saturday 16 August 2008 12:55:42 David Masover wrote:

On Saturday 16 August 2008 07:33:22 Trans wrote:

> It is a serious issue with one of
> my programs. Because of it I am forced never to use autoload, and tell
> all users of my program that they must do the same.

I think I've found a solution, for myself -- basically a const_missing hack.

7

The things to do is of course to fix the Ruby source to A) use a
callback for autoload and B) to use the normal Kernel.require call.
Then submit a patch and pray the higher ups care enough to check it
in.

Now the question is are you a C coder :wink:

T.

···

On Aug 16, 4:28 pm, David Masover <ni...@slaphack.com> wrote:

That sucks.

Is there any way to know for certain when a file is loaded, when it's autoload
that's doing it, short of modifying the file? (I'm specifically wanting to be
_notified_, so that I can prepare sub-constants to be autoloaded as well --
that is, when Foo is loaded, I want to automatically run Foo.autoload :Bar.)

Is there any way to force autoload to use a different mechanism?

Is there any way, in Ruby, to emulate autoload with actual feature parity?

Or am I really going to have to create a temporary file for each and every
source file I might possibly ever want to use?

8

> Or am I really going to have to create a temporary file for each and every
> source file I might possibly ever want to use?

Actually, I have figured out something that works reasonably well, for now.
Given a path that can be safely autoloaded, it will check for files in there
that look like they could be modules.

It also provides a mixin, which, when included, checks through its existing
paths for possible nested modules and adds them.

I suspect it's twice as long as it needs to be, though, and certainly less
elegant to write.

If anyone's interested, I can clean it up and release it. It's roughly
equivalent to the same functionality in ActiveSupport, except that it's much
shorter (dependencies.rb is around 500 lines, and I'm well under 100), and
requires different kinds of babysitting.

That is: With ActiveSupport, if foo.rb is not loaded, the following:

class Foo; end

will cause foo.rb to _never_ be autoloaded, because const_missing isn't
checked on assigning things to a constant.

However, in my version, to support foo/bar.rb, I have to do at least:

module Foo
  include AutoLoad
end

I can put it in foo.rb, or anywhere that's loaded by the time I need Foo::Bar,
but it is required.

The things to do is of course to fix the Ruby source to A) use a
callback for autoload and B) to use the normal Kernel.require call.
Then submit a patch and pray the higher ups care enough to check it
in.

I'd add a C) Have const_missing actually called in enough places to make it
possible to implement vanilla autoload on top of const_missing.

Having autoload be more flexible in the action it performs would be helpful,
but it'd be even more helpful for it to be flexible in the constants it
applies to.

Now the question is are you a C coder :wink:

Yes and no.

I have written one-line kernel patches, which did things I'm ashamed to admit.
I've cracked open abandoned projects, to try to make them, for example,
compile on 64-bit. I've read the source of poorly-documented parts of
PowerDNS to figure out what it was doing.

To be a programmer on any Unix is to have to be at least somewhat aware of
what C is and what it can do, even if you never use it.

But I have never done any significant development in C. Even if I had, it
would (I assume) still take some time to learn the Ruby codebase and
conventions.

So, someone else needs to do this :frowning:

···

On Saturday 16 August 2008 21:40:32 Trans wrote:

On Aug 16, 4:28 pm, David Masover <ni...@slaphack.com> wrote: