Kernel#autoload vs require_relative

ruby-talk
1

I understand that for require to work relative to the given __FILE__’s
directory one needs to either $LOAD_PATH << File.dirname(__FILE__) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
  autoload :Bar, 'foo/bar.rb'
  autoload :Baz, 'foo/baz.rb'
  autoload :Qux, 'foo/qux.rb'
end

(which doesn’t work without $LOAD_PATH munging), as
opposed to the (currently working) lib/foo.rb with

require_relative 'foo/bar.rb'
require_relative 'foo/baz.rb'
require_relative 'foo/qux.rb'

— Piotr Szotkowski

···

--
If you watch NASA backwards, it’s about a space agency that has no
spaceflight capability, then does low-orbit flights, then lands on moon.
                                                          [Thomas Fuchs]

2

I understand that for require to work relative to the given __FILE__’s
directory one needs to either $LOAD_PATH << File.dirname(__FILE__) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
autoload :Bar, 'foo/bar.rb'
autoload :Baz, 'foo/baz.rb'
autoload :Qux, 'foo/qux.rb'
end

(which doesn’t work without $LOAD_PATH munging),

I wouldn't call having 'lib' in your LOAD_PATH as munging. I'd call that business as usual. Rubygems supports that as a default. All the usual rake tasks and test tools support that as a default. It just seems sane to me... Much more sane than File.dirname(__FILE__) does or require_relative.

(btw, you don't need the ".rb" part--that's just ugly)

as
opposed to the (currently working) lib/foo.rb with

require_relative 'foo/bar.rb'
require_relative 'foo/baz.rb'
require_relative 'foo/qux.rb'

again, ".rb" not needed.

···

On Jun 16, 2011, at 04:29 , Piotr Szotkowski wrote:

3

The question is, are these libs going to loaded anyway regardless of
what is done with the library? If so than you should just go ahead and
load the libs right away with require_relative. Moreover,
require_relative doesn't have to search though all the load paths, so
it will speed up load times. autoload is only useful for a class or
module that is frequently not used in any given run.

···

On Jun 16, 7:29 am, Piotr Szotkowski <chast...@chastell.net> wrote:

I understand that for require to work relative to the given __FILE__’s
directory one needs to either $LOAD_PATH << File.dirname(__FILE__) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
autoload :Bar, 'foo/bar.rb'
autoload :Baz, 'foo/baz.rb'
autoload :Qux, 'foo/qux.rb'
end

(which doesn’t work without $LOAD_PATH munging), as
opposed to the (currently working) lib/foo.rb with

require_relative 'foo/bar.rb'
require_relative 'foo/baz.rb'
require_relative 'foo/qux.rb'

4

Ryan Davis:

I understand that for require to work relative to the given __FILE__’s
directory one needs to either $LOAD_PATH << File.dirname(__FILE__) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
autoload :Bar, 'foo/bar'
autoload :Baz, 'foo/baz'
autoload :Qux, 'foo/qux'
end

(which doesn’t work without $LOAD_PATH munging),

I wouldn't call having 'lib' in your LOAD_PATH as munging.

Ok (and I understand your arguments).

It just seems sane to me... Much more sane than
File.dirname(__FILE__) does or require_relative.

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

What I was trying to ask is whether there couldn’t be
a Kernel#autoload_relative call or a ‘relative: true’
optional parameter to Kernel#autoload to make it symmetric
with the require/require_relative combination. I understand
your answer would be ‘there’s no need’. :slight_smile:

(btw, you don't need the ".rb" part--that's just ugly)

Fully agreed; fixed in the quotation above
(I shouldn’t post before my morning coffee).

— Piotr Szotkowski

···

On Jun 16, 2011, at 04:29 , Piotr Szotkowski wrote:

--
I consider chromatic to be Larry Wall’s Thomas Huxley.
                   [Jarkko Hietaniemi, hates-software]

5

Intransition:

···

On Jun 16, 7:29 am, Piotr Szotkowski <chast...@chastell.net> wrote:

$LOAD_PATH << File.dirname(__FILE__)
module Foo
  autoload :Bar, 'foo/bar'
  autoload :Baz, 'foo/baz'
  autoload :Qux, 'foo/qux'
end

require_relative 'foo/bar'
require_relative 'foo/baz'
require_relative 'foo/qux'

The question is, are these libs going to loaded
anyway regardless of what is done with the library?

In my case it depends on the library in question; I was
looking for an approach that could use a relative-path
Kernel#autoload without touching $LOAD_PATH.

— Piotr Szotkowski
--
At the dentist’s thinking about whether hints of method size can be
seen in intra-class dependency diagrams. Oh no, here comes the drill.
                                                   [Michael Feathers]

6

Piotr Szotkowski:

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

…and now I recalled my particular use-case:

require_relative 'set'

will pull in ‘my’ Set, whereas

require 'set'

will pull in either ‘my’ Set or the stdlib’s Set depending on whether
I prepended or appended the $LOAD_PATH with File.dirname(__FILE__).

// I do agree name clashes are at best avoided and at worst
// thoroughly documented, with require './set' being a (somewhat
// ugly) solution as well, but require_relative’s independence
// from $LOAD_PATH ordering does look appealing to me, hence my
// longing for Kernel#autoload_relative. :slight_smile:

— Piotr Szotkowski

···

--
/* obligatory Perl joke (omitted for brevity) */
                              [Tomasz Dubrownik]

7

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

So, while writing the rest of this (long) post, I think I actually dug up a
solution for you, in the form of the 'autoloader' gem I wrote awhile back:

If everything in your 'lib' hierarchy is normally loaded via autoload, and
follows a decent naming convention, I think this solves your problem. In your
example, somewhere in 'lib/foo.rb', you could do this:

  require 'autoloader'
  AutoLoader << File.dirname(__FILE__)

  module Foo
    include AutoLoader
  end

Ok, there's some __FILE__ ugliness, but in exchange, you automagically get
'autoload' statements generated for everything, and you don't have to modify
your load path.

Caveat: I wrote this awhile ago, when Merb was still a thing. It seems to
still work, but you might still want to read it to understand what it does.
It's not that long:

What I was trying to ask is whether there couldn’t be
a Kernel#autoload_relative call or a ‘relative: true’
optional parameter to Kernel#autoload to make it symmetric
with the require/require_relative combination...

There is one thing I would _much_ rather have than this: Enough hooks into
either the language or the autoload mechanism so that things like this are
possible to do entirely inside Ruby.

That is: Currently, we have const_missing, but it isn't called when someone
tries to define a constant. For example, if I have Rails-style const_missing
hacks, and I want to re-open a previously-defined module, I need to do this:

  Foo::Bar.module_eval do
    ...
  end

instead of:

  module Foo
    module Bar
      ...
    end
  end

The latter will never hit const_missing, and would thus never load foo.rb or
whatever. And sure, if I'm re-opening Bar, I'm probably better off defining a
separate module and forcing Bar to include it. But what if this is inside
foo/bar.rb? Now if I say

require 'foo/bar'

I'll get Foo::Bar, but I won't ever load foo.rb -- whereas, say,

  Foo.module_eval do
    module Bar
      ...
    end
  end

...ew.

Anyway, what I really want is something like:

  autoload :Foo do
    # block to ensure Foo is defined
  end

In this case, autoload is the wrong word, but this would let me write the
ultimate autoload-based replacement for the old Rails const_missing hack. What
I have now is in the 'autoloader' gem:

AutoLoader << '/path/to/lib'

It will crawl 'lib' one level deep, take every '.rb' file, and create an
'autoload' tag for it. But in order to map Foo::Bar to 'foo/bar.rb', I still
need the user to have a foo.rb file which does some extra work:

module Foo
  include AutoLoader
end

I can't think of any way to make this work other than having some callback
fire when the file actually gets loaded. Last I checked, Kernel#autoload would
call 'require' from C when fired, so overloading 'require' doesn't help. And
const_missing isn't a substitute, nor is there any way to duplicate
Kernel#autoload's functionality from Ruby.

I actually think autoloading via require_relative is a bad idea, versus just
adding something to the load path. Some people think autoload itself is a bad
idea. There are probably workarounds enough for both of us. But it does
frustrate me that Kernel#autoload is so inflexible, and it really doesn't seem
like it needs to be.

···

On Thursday, June 16, 2011 11:50:57 AM Piotr Szotkowski wrote:

8

Hm, I do prefer using require_relative over require + $LOAD_PATH

changes; it looks cleaner to me and doesn’t touch shared

globals, but I guess that’s more of a personal preference.

It's also more secure, which is why . was removed in the first place. :slight_smile:

9

I know you know this but it needs to be said for others:

require 'my/set'

···

On Jun 16, 2011, at 10:07 , Piotr Szotkowski wrote:

Piotr Szotkowski:

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

…and now I recalled my particular use-case:

require_relative 'set'

will pull in ‘my’ Set, whereas

require 'set'

will pull in either ‘my’ Set or the stdlib’s Set depending on whether
I prepended or appended the $LOAD_PATH with File.dirname(__FILE__).

10

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

So, while writing the rest of this (long) post, I think I actually dug up a
solution for you, in the form of the 'autoloader' gem I wrote awhile back:

GitHub - masover/autoloader: Crawl a directory at a time, adding files via Kernel#autoload

If everything in your 'lib' hierarchy is normally loaded via autoload, and
follows a decent naming convention, I think this solves your problem. In your
example, somewhere in 'lib/foo.rb', you could do this:

require 'autoloader'
AutoLoader << File.dirname(__FILE__)

module Foo
include AutoLoader
end

Ok, there's some __FILE__ ugliness, but in exchange, you automagically get
'autoload' statements generated for everything, and you don't have to modify
your load path.

I think there is a cure for that:

module AutoLoader
  def self.included(cl)
    # "alc.rb:33:in `<class:X>'"
    file = caller[1][%r{\A(.*?):\d+:in\s}, 1]
    self << File.dirname(file)
  end

  # debug:
  def self.<<(d)
    puts "Adding directory: #{d}"
  end
end

Anyway, what I really want is something like:

autoload :Foo do
# block to ensure Foo is defined
end

In this case, autoload is the wrong word, but this would let me write the
ultimate autoload-based replacement for the old Rails const_missing hack. What
I have now is in the 'autoloader' gem:

AutoLoader << '/path/to/lib'

It will crawl 'lib' one level deep, take every '.rb' file, and create an
'autoload' tag for it. But in order to map Foo::Bar to 'foo/bar.rb', I still
need the user to have a foo.rb file which does some extra work:

module Foo
include AutoLoader
end

In Ruby land we could at least do

auto_module :Foo do
  # no more "include" needed here
end

or even

auto_module :Foo

if there is nothing that needs to be defined here. But then we have a
"require" statement and a file with a single line - that doesn't feel
right.

I can't think of any way to make this work other than having some callback
fire when the file actually gets loaded. Last I checked, Kernel#autoload would
call 'require' from C when fired, so overloading 'require' doesn't help. And
const_missing isn't a substitute, nor is there any way to duplicate
Kernel#autoload's functionality from Ruby.

I actually think autoloading via require_relative is a bad idea, versus just
adding something to the load path. Some people think autoload itself is a bad
idea. There are probably workarounds enough for both of us. But it does
frustrate me that Kernel#autoload is so inflexible, and it really doesn't seem
like it needs to be.

Hm, for me this generally works pretty well: I have a lib directory
which is part of the load path and for larger libraries I have an
initial file (e.g. "foo.rb") where the root namespace is defined
together with a set of autoload directives which then load files like
"foo/class1", "foo/class2" etc.

Basically, as long as there is no fixed relationship between file name
and content, there will always be a level of manual intervention
needed. True, you can have conventions manifest in code (like your
AutoLoader) but as long as there is no predominant convention that
fits all needs we won't have a generalized solution. The advantage of
the current autoload approach is that it makes things explicit.
Downside is still that the relativity issue (with regard to pathnames,
not E=mc2) is still there.

But that can be solved with

class Module
  def autoload_r(const, path)
    file = caller[1][%r{\A(.*?):\d+:in\s}, 1]
    autoload(const, File.join(File.dirname(file), path))
  end
end

Kind regards

robert

···

On Fri, Jun 17, 2011 at 9:33 AM, David Masover <ninja@slaphack.com> wrote:

On Thursday, June 16, 2011 11:50:57 AM Piotr Szotkowski wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

11

> require 'autoloader'
> AutoLoader << File.dirname(__FILE__)
>
> module Foo
> include AutoLoader
> end
>
> Ok, there's some __FILE__ ugliness, but in exchange, you automagically
> get 'autoload' statements generated for everything, and you don't have
> to modify your load path.

I think there is a cure for that:

module AutoLoader
  def self.included(cl)
    # "alc.rb:33:in `<class:X>'"
    file = caller[1][%r{\A(.*?):\d+:in\s}, 1]
    self << File.dirname(file)
  end

  # debug:
  def self.<<(d)
    puts "Adding directory: #{d}"
  end
end

That needs some work. Dirname isn't enough, since right now, things may be
nested arbitrarily deep -- I could have lib/foo/bar.rb which looks similar:

  module Foo
    module Bar
      include AutoLoader
    end
  end

But I clearly still want 'lib' to be the top-level directory for this magic.
That is, I want foo/bar/baz.rb to map to Foo::Bar::Baz, not to Bar::Baz.

I really don't mind the __FILE__ ugliness, given that it only has to happen
once, at the very top of the project. The assumption is that you'd never have
a reason to require anything other than this top-level library, since
everything else is included with autoload, so you'll only get the stuff you
need -- that is, there's no advantage to:

  require 'foo/bar'
  Foo::Bar.do_something!

rather than

  require 'foo'
  Foo::Bar.do_something!

So I have no motivation to get rid of __FILE__, though maybe there should be
some convenience version, maybe something like:

  AutoLoader.add_relative_path 'path/to/lib' # defaults to '.'

In Ruby land we could at least do

auto_module :Foo do
  # no more "include" needed here
end

or even

auto_module :Foo

if there is nothing that needs to be defined here. But then we have a
"require" statement and a file with a single line - that doesn't feel
right.

Yeah, it doesn't really solve the problem. 'include' isn't the problem. The
problem is that this library shouldn't force me to add _any_ special syntax or
calls, let alone special files, to make it work. It should just let me pretend
that my entire directory got slurped -- this was inspired by Rails'
const_missing behavior, but also by Ramaze's "acquire" which just requires
every ruby file in a given path.

Hm, for me this generally works pretty well: I have a lib directory
which is part of the load path and for larger libraries I have an
initial file (e.g. "foo.rb") where the root namespace is defined
together with a set of autoload directives which then load files like
"foo/class1", "foo/class2" etc.

I don't mind this, except for two things:

- I shouldn't have to modify the load path. I should be doing it anyway,
maybe, but it shouldn't be a requirement.

- It's annoying having to explicitly autoload everything, especially when I'm
usually following a convention.

Basically, as long as there is no fixed relationship between file name
and content, there will always be a level of manual intervention
needed.

I like convention over configuration. Not convention instead of configuration
-- there's always manual autoload if I need them, and I suspect the first
thing I'd change about AutoLoader at this point is being able to specify which
files are included and which aren't. But when 99% of the time, the process is:

- Create foo/bar.rb.
   - Create class Foo::Bar in foo/bar.rb.
   - Add autoload line to foo.rb.
- Did I really want it called bar.rb? Maybe baz.rb is a better name.
   - git mv foo/bar.rb foo/baz.rb
   - Change Foo::Bar to Foo::Baz in foo/baz.rb
   - Change autoload line in foo.rb.
   - Update any other references.
- Maybe I want to split it into foo/one.rb and foo/two.rb.
   - create foo/one.rb, foo/two.rb.
   - edit/move some stuff, update references.
   - git add foo/one.rb foo/two.rb
   - git rm foo/baz.rb
   - edit foo.rb and update the autoload line.

This is annoying. Sure, it's going to be annoying no matter what, but anything
I can do to ease this pain is helpful, because I do this kind of thing a _lot_
in the first few hours of a project. Anytime I see a better organization,
_now_ is the time, before I've published it to a dev team (let alone to the
world via Github) and locked myself into an API.

It's also annoyingly redundant and menial. This is something the machine can
do perfectly well for me, so why not let it?

Also worth mentioning: When you're autoloading everything, anything beyond the
top-level "require 'yourgem'" is no longer part of your public API. So, any
naming convention is entirely for your own convenience at this point. I just
did what Rails did and made it convenient to follow that one naming
convention. So when I actually use AutoLoader, I don't have exceptions to that
naming convention, or, really, any reason to.

The advantage of
the current autoload approach is that it makes things explicit.
Downside is still that the relativity issue (with regard to pathnames,
not E=mc2) is still there.

Well, as you demonstrate, that's easily solved. But in my opinion, the other
downside is that it makes things explicit, when there isn't a good reason for
them to be.

At this point, AutoLoader isn't about solving the relativity issue, it's about
DRYing things up a bit.

···

On Friday, June 17, 2011 06:06:33 AM Robert Klemme wrote:

On Fri, Jun 17, 2011 at 9:33 AM, David Masover <ninja@slaphack.com> wrote: