Packages, gems, libraries ... and ruby itself

Ok,

So, I spent a chunk of time working through what I think would be a good
solution to the directory structure/packaging stuff that has again plagued
the ruby-talk mailing list.

Here's what I think...

Taking a complete step back, I see alot of extra stuff inside the ruby
installation directory ... and here's what I think should be there ...

The following is based on something like; /usr/ruby or /usr/local/ruby ( for
linux/*nix users ) or c;\ruby or c:\program files\ruby ( for windows users )

... from the ruby base installation directory ($RUBY_HOME) we should have
the following:

$RUBY_HOME/bin

- which is where executables & scripts should live ( ruby, ri, irb, rdoc,
rdoc_server, etc )

$RUBY_HOME/doc

- which is where ALL documentation should exist ...

$RUBY_HOME/doc/core

- for documentation of the core objects in ruby

$RUBY_HOME/doc/stdlib

- for documentation of the objects outside of core that are included in the
ruby distribution

$RUBY_HOME/doc/...

- each additional package should then get it's own documentation tree so
that they are all easily delineated.

$RUBY_HOME/etc

- for things like irb.conf, gem.conf , gem.db ... whatever configuration
files for default server packages ( like rdoc or gem or whatever else )

$RUBY_HOME/lib

- all ruby source files & external libraries should go here.

$RUBY_HOME/lib/ruby/

- all ruby source files

$RUBY_HOME/lib/ruby/core/

- .rb files pertaining to core functionality

$RUBY_HOME/lib/ruby/stdlib/

- .rb files that implement functionality for the stdlib

$RUBY_HOME/lib/ruby/...

- each additional package of .rb files should get a directory here ( same as
the /doc/... name and purpose )

$RUBY_HOME/lib/ext/

- for all .dll or .so files that are included with Ruby or installed and
required by ruby ...

$RUBY_HOME/tests

- for all unit tests

$RUBY_HOME/tests/core
$RUBY_HOME/tests/stdlib
$RUBY_HOME/tests/...

- breaking down the same way as $RUBY_HOME/lib/ruby or $RUBY_HOME/doc

... beyond this, I do like the idea of small self-contained files similar to
java's .jar files ... I was trying to come up with something cute, but all I
came up with is calling them .rib files ( matching the rake/rinda play on
r-instead-of-l pronunciation ... rib as in "ruby lib"

These .rib files would simply be a .tar.gz file mimicing the structure
proposed above plus the addition of a metadata.yaml file.

.rib files would simply enclose the entire structure of things for
distributable functionality. The metadata.yaml file would include naming,
versioning, dependancy, and other necessary information for making sure that
things work (including rake files for running unit tests, or whatever else
the package creator wanted it to do).

As opposed to what we have for gems now, which frankly takes FOREVER to do
simple installs of packages that are even only a few kilobytes... ( having
to download the entire index EVERYTIME is a severe waste of bandwidth )

The rib server should be a distributed ruby ( drb ) server ( openssl ) that
allows for a query/response way of finding out what's out there, instead of
downloading an ever growing file and having all the work be done on the
client side.

The rib system should be platform sensitive ... meaning all calls would have
platform specific filtering ... no-arch, linux-2.4, linux-2.6, osx, freebsd,
etc ... lessening the amount of data that has to be transferred for any
given call.

The configuration of which rib or multiple rib servers should be called
would be part of the configuration information stored in the $RUBY_HOME/etc
directory.

Using this as a standard would make things easier for all platforms as
everybody would have hard paths to each and every place that they need to go
... and if a specific os or distro doesn't like wher things are , they can
use where they want and just replace the directories I've proposed with
symlinks ... ( so if they want /etc instead of $RUBY_HOME/etc ... just
symlink it ) ...

This solution would give a solid path for unit tests ( which is problematic
under gems installed packages )

Also, I think this would be a good thing for getting all of the rdoc
generated documentation into a single tree... and that would be a VERY good
thing.

... it would still allow for multiple versions of libraries, as you could
install /lib/ruby/my_package-1.3 & /lib/ruby/my_package-1.4 and simply
symlink or stub for /lib/ruby/my_package to the current version. this would
allow require 'my_package' for "latest version functionality" or require
'my_package-#.#' for specific version includes.

I know it's re-inventing the wheel, but I think we've gotten so lost in
trying to create a be-all solution that we're not solving the simplest
portion of the problem ... and that's slowing ruby adoption. And that's what
matters to me.

So, I hope this note doesn't tick anybody off. I really just intend it as a
fresh viewpoint. I've not done the necessary research to see how much of an
impact this would have on the ruby distribution itself ... I think it would
be easy ...

This is just my brain dump ... and I hope it helps in some way ... if not,
I'm sorry for the adding noise into the stream ...

I believe this solution gives us maximum flexibility and allows the os
distro/packaging systems to do what they need to do while not disturbing
what we should be able to reliably expect.

As always, your feedback is expected and appreciated. Thank you in advance
for the amount of time you spent to read and hopefully understand my
thoughts.

j.

···

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

Ok,

So, I spent a chunk of time working through what I think would be a good
solution to the directory structure/packaging stuff that has again plagued
the ruby-talk mailing list.

         [...]

As opposed to what we have for gems now, which frankly takes FOREVER to do
simple installs of packages that are even only a few kilobytes... ( having
to download the entire index EVERYTIME is a severe waste of bandwidth )

To be fair, there is effort to reduce this time to something more
reasonable, and a number of schemes have been looked at (better
usage of the protocols, breaking up the zipped archive to smaller
chunks,...). As to when the problem will disappear, I cannot say.
There is mention of this on the TODO list.

The rib server should be a distributed ruby ( drb ) server ( openssl ) that

That (openssl) would exclude users of ruby in places where
encryption is legally difficult. I mention this solely on the basis
that we seem to be aiming for the widest uptake possible...

         [...]

... and if a specific os or distro doesn't like wher things are , they can
use where they want and just replace the directories I've proposed with
symlinks ... ( so if they want /etc instead of $RUBY_HOME/etc ... just
symlink it ) ...

As mentioned recently, symlinks are not cross-platform, alas.

This solution would give a solid path for unit tests ( which is problematic
under gems installed packages )

Details on the problems encountered may facilitate improvemnts in the
existing system, while development continues on your proposal.

Also, I think this would be a good thing for getting all of the rdoc
generated documentation into a single tree... and that would be a VERY good
thing.

The documentation structure is under consideration as part of the
proposed integration of rubygems.

... it would still allow for multiple versions of libraries, as you could
install /lib/ruby/my_package-1.3 & /lib/ruby/my_package-1.4 and simply
symlink or stub for /lib/ruby/my_package to the current version. this would
allow require 'my_package' for "latest version functionality" or require
'my_package-#.#' for specific version includes.

Austin has already mentioned problems with a suffix based scheme a
couple of times recently.

I know it's re-inventing the wheel, but I think we've gotten so lost in
trying to create a be-all solution that we're not solving the simplest
portion of the problem ... and that's slowing ruby adoption. And that's what
matters to me.

I know there seems to be more heat than light at the moment, but the
rubygems TODO list does cover a lot of these points and more.

So, I hope this note doesn't tick anybody off. I really just intend it as a
fresh viewpoint. I've not done the necessary research to see how much of an
impact this would have on the ruby distribution itself ... I think it would
be easy ...

I suspect that it would be difficult, given the effect of change on
existing software. The scheme looks simple and straightforward,
but then I have a unix background, and people from elsewhere might
not see it as so clear. Also, I think it breaks backward
compatibility, because you plan to move things around significantly.

This is just my brain dump ... and I hope it helps in some way ... if not,
I'm sorry for the adding noise into the stream ...

I believe this solution gives us maximum flexibility and allows the os
distro/packaging systems to do what they need to do while not disturbing
what we should be able to reliably expect.

Well, I don't want to stamp on new ideas before they show promise.
I remain to be convinced about this one...

As always, your feedback is expected and appreciated. Thank you in advance
for the amount of time you spent to read and hopefully understand my
thoughts.

j.

         Hugh

···

On Mon, 3 Oct 2005, Jeff Wood wrote: