Why can't Ruby load .gem files directly?

[Brian_Schroder1 (Brian Schröder)]

If it is a debian user problem that is sad, but I have to say that I
try to avoid any libraries that are not packed as a deb. I do not have
gems installed because the duality of the gem and the debian system is
annoying me.

This may be just an anecdote, but a lot of libraries are loosing at
least me as a user. I have packaged deb's myself and I don't think it
is a very complicated process, so I wonder why it is difficult to go
from .gem to .deb?

brian

···

On 02/10/05, Corey Lawson <corey.ssf.lawson@gmail.com> wrote:

It sounds like a debian user problem, not a Ruby user problem. I think most
Ruby people (who probably are not running ruby on Debian) want Ruby to work
like...Ruby, whether it's on a Mac, Windows, RedHat, Debian, Slolaris,
whatever. Has a similar beef been made about Java on Debian? (Why must Java
apps be deployed as .jar, .ear, war, whatever. Can't they pick just one and
make it work with apt-get?).

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/

[Kevin_Brown]

[*snip*]

It sounds like a debian user problem, not a Ruby user problem. I think most
Ruby people (who probably are not running ruby on Debian) want Ruby to work
like...Ruby, whether it's on a Mac, Windows, RedHat, Debian, Slolaris,
whatever. Has a similar beef been made about Java on Debian? (Why must Java
apps be deployed as .jar, .ear, war, whatever. Can't they pick just one and
make it work with apt-get?).

It's both, because gems should be flexible enough to do both, especially since
doing so provides no particular problems to either group. That way, if my
mom wants to install a gem, she uses the debian package, and if I want to, I
do it the 'real' way.

Ultimately, if this results in an internet-schlong-waving-contest, everyone
loses. Debian also has the option of having a package type that includes a
wget and then a gem-install, but this is very contrary to the fact that once
you download the .deb package, you HAVE the software and that's IT.
Introducing nasty hacks like above mentioned are not good for Debian, and
that's why the debate is currently raging.

Also note that it's not just a Debian problem, but a problem of anyone who
uses .debs, including Ubuntu, Kubuntu, Mephis, and a whole host of other
distros.

···

On Sunday 02 October 2005 10:21, Corey Lawson wrote:

[Kevin_Brown]

Eivind Eklund wrote:
>>>Yes, I don't know the details of the current controversy (ie. why
>>>Debian is having a problem packaging gems)
>>
>>Neither do I. Are they trying to repackage gems as... err.. whatever the
>>apt-get file extension is? If so, why?
>
>They're trying to repackage software that the author assume to be
>Gem-distributed as Debian packages (.deb, IIRC). The reason for this
>is simple: To provide Ruby software on Debian for Debian users in the
>way Debian users are used to. These users do not particularly care
>about Ruby, and definately do not want to have to know one packaging
>system for handling Ruby software, and another for handling Perl, and
>another for handling Java, and another for handling Haskell, and
>another for handling Python, and another way for handling C code, and
>...

Okay (and thanks for the reply). That's weird. If they don't paticularly
care about Ruby, what are they doing installing Ruby gems (usually
libraries)? If they're trying to install an application that happens to
be written in Ruby, then does it not suffice for the .deb to list ruby
and rubygems as dependences, and then for the install step to be `gem
install the-app-dependencies` or whatever?

Nope, because in other debian-ish situations with libraries (be they written
in C and compiled) you can put them in and pull them out seperate of the
application by using apt-get, which makes a LOT of sense.

What would happen when you uninstall the program if packaged as you mention?
You can't remove the gems, and you can't really leave them there either...

···

On Sunday 02 October 2005 10:42, Devin Mullins wrote:

Devin

[Eivind_Eklund]

(Note that some of these are projections from problems I know about it
in the context of FreeBSD, which I'm a developer for, and RPA, which
I've been an advisor for. I'm just projecting to Debian, based on
having read various Debian policy documents and talked to Debian users
and developers.)

There is several reasons, including the deinstall problem that was
mentioned, that Gems start writing places they shouldn't (so write
protection of some areas of the disk don't work), and that Debian
generally do maintenance on single versions, fixing security holes as
they pop up etc so you can install a package, then run apt-get update
and get everything updated. This cascades to the rest of the package
organization.

Also, people expect to be able to find documentation etc in the
"normal places" even for langauges they're not used to, and
applications sometimes depend on executables.

There's a whole host of complexities that come up here when you
actually do packaging and maintenance.

Eivind.

···

On 10/2/05, Devin Mullins <twifkak@comcast.net> wrote:

Eivind Eklund wrote:
Okay (and thanks for the reply). That's weird. If they don't paticularly
care about Ruby, what are they doing installing Ruby gems (usually
libraries)? If they're trying to install an application that happens to
be written in Ruby, then does it not suffice for the .deb to list ruby
and rubygems as dependences, and then for the install step to be `gem
install the-app-dependencies` or whatever?

[Kevin_Brown]

[*snip*]
May I request that we talk about technical issues like grown-up
engineers, not like guests on "Jerry Springer?"

Unnecessary insult, but sure.

Tell me if this is an accurate list of the problems packaging gems:

1. Gems provide more guarantees than Debian packages can

Gems provide guarantees that turn out to be wrong with the way Debian installs
software with its packaging scheme (out of the way of the user, so that you
can install your own stuff in /usr/bin etc).

2. Gems let you install more than one version of the same software,
but Debian packages don't

Debian packages do, but will maintain a symlink to the most recently installed
version. I have python2.2 and 2.4 installed right now, but the symlink
points to 2.4, thus causing problems if python2.2 software assumes it can get
to python with /usr/bin/python.

3. something about Debian wanting software to be in a single directory?

Not quite, but in an out of the way place.

I've tried to bring myself up to speed on these issues by reading
recent threads, but it's very difficult to separate the technical
information from the arguing.

Hope I've helped.

···

On Sunday 02 October 2005 11:06, Joshua Haberman wrote:

On Oct 2, 2005, at 9:10 AM, Eivind Eklund wrote:

Josh

[Eivind_Eklund]

>> Joshua Haberman wrote:
>>> Say that you had two implementations of gems with identical feature
>>> sets, but one used the "gem" command to do everything and one let
>>> you
>>> use .gem files directly. I would *much* prefer using the latter.
>>>
>>
>> Actually, it doesn't sound like they're mutually exclusive. Stick a
>> little flag in the metadata.gz that says "I don't have a compile
>> step!"
>> and then it can be stuck on the $LOAD_PATH directly. Otherwise, upon
>> require-ing something from inside the gem, you'll get an error
>> "Install
>> meeee!!"
>>
>>
>>> Yes, I don't know the details of the current controversy (ie. why
>>> Debian is having a problem packaging gems)
>>>
>>
>> Neither do I. Are they trying to repackage gems as... err..
>> whatever the
>> apt-get file extension is? If so, why?
>>
>
> They're trying to repackage software that the author assume to be
> Gem-distributed as Debian packages (.deb, IIRC). The reason for this
> is simple: To provide Ruby software on Debian for Debian users in the
> way Debian users are used to. These users do not particularly care
> about Ruby, and definately do not want to have to know one packaging
> system for handling Ruby software, and another for handling Perl, and
> another for handling Java, and another for handling Haskell, and
> another for handling Python, and another way for handling C code, and
> ...
>
> Instead, they want they software to Just Work Like All Other Software,
> the way they are used to, and to not have to care about the language
> the software is written in at all.

May I request that we talk about technical issues like grown-up
engineers, not like guests on "Jerry Springer?"

These are user interface issues, and I am trying to explain them in a
way that get the overall top level user issues communicated.
Ruby-talk is the wrong list for the detailed technical discussion.
Also, the technical issues are reasonably easy to solve, as long as
there is acceptance that they need to be solved, and that they need to
be discussed at a technical level.

Tell me if this is an accurate list of the problems packaging gems:

1. Gems provide more guarantees than Debian packages can

*Different* guarantees, and being unable to support the guarantees
that Debian packages give. Specifically, Gems do not support using
the directory layout that Debian users expect. Gems generally go in a
single directory. Some ruby developers end up relying on that single
directory because Gems lack the infrastructure for them to access
things any other way.

I've posted more exact details of the stuff that's necessary to
support to rubygems developer list as references a year and half or so
ago. In the last couple of days, I've posted this as extracted and
minimized text with very concrete lists, to be able to start building
the abstractions necessary. That's under the subject of "Gems and
repackaging, hopefully helpful".

There's also the problem that gems are hard to patch, though that's
resolvable by using the original source as long as the authors are not
led to rely on the RubyGems layout, and end up being just rude to
those trying to give them patches to make this work.

Eivind.

···

On 10/2/05, Joshua Haberman <joshua@reverberate.org> wrote:

On Oct 2, 2005, at 9:10 AM, Eivind Eklund wrote:
> On 10/2/05, Devin Mullins <twifkak@comcast.net> wrote:

[Joshua_Haberman1 (Joshua Haberman)]

Right, but the problem with RubyGems that I am trying to solve is that the interpreter can't load them directly. (see the subject of this email). I don't want to perform an "install" step.

If this functionality was supported, I believe a lot of the problems currently being debated would melt away.

Josh

P.S. Incidentally, I don't see the point of the extra "i386-linux" directory. What does it buy you? I don't keep my C libraries in /usr/lib/i386-linux. If shared objects are on my machine, it is assumed that they match the architecture of my machine.

···

On Oct 2, 2005, at 3:40 PM, Austin Ziegler wrote:

On 10/2/05, Joshua Haberman <joshua@reverberate.org> wrote:

my-gem-dir/
my-gem-dir/foo-1.2.gem
my-gem-dir/foo-1.2-libfoo.so
my-gem-dir/foo-1.2-libbar.so
my-gem-dir/baz-0.3.gem

Except that my-gem-dir/foo-1.2-libfoo.so would *actually* be:
my-gem-dir/i386-linux/foo-1.2-libfoo.so or something like that.

Ruby and RubyGems solves a lot of this already.

[Austin_Ziegler5 (Austin Ziegler)]

Because .deb has no concept of multiple versions installed
simultaneously being available for use. Because .gem doesn't have an
answer to the DATADIR problem (the only *real* problem with the
1-gem-1-dir solution, as everything else is simply SMOP and prefixes).
Anything else is ... going to become impolitic very quickly. If you
want to see a *long* rant by me about it, as well as proposals that
*should* be able to make this work even for Debian, read the ruby-core
threads about integrating RubyGems into Ruby core.

-austin

···

On 10/2/05, Brian Schröder <ruby.brian@gmail.com> wrote:

On 02/10/05, Corey Lawson <corey.ssf.lawson@gmail.com> wrote:
> It sounds like a debian user problem, not a Ruby user problem. I think most
> Ruby people (who probably are not running ruby on Debian) want Ruby to work
> like...Ruby, whether it's on a Mac, Windows, RedHat, Debian, Slolaris,
> whatever. Has a similar beef been made about Java on Debian? (Why must Java
> apps be deployed as .jar, .ear, war, whatever. Can't they pick just one and
> make it work with apt-get?).
If it is a debian user problem that is sad, but I have to say that I
try to avoid any libraries that are not packed as a deb. I do not have
gems installed because the duality of the gem and the debian system is
annoying me.

This may be just an anecdote, but a lot of libraries are loosing at
least me as a user. I have packaged deb's myself and I don't think it
is a very complicated process, so I wonder why it is difficult to go
from .gem to .deb?

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

[Devin_Mullins]

Kevin Brown wrote:

but this is very contrary to the fact that once you download the .deb package, you HAVE the software and that's IT.

Ah. Well, err, to qualify my previous post, couldn't the .deb just include the .gem(s) inside, and then call `gem install thegem(s)`?

Devin

[Devin_Mullins]

Kevin Brown wrote:

Nope, because in other debian-ish situations with libraries (be they written in C and compiled) you can put them in and pull them out seperate of the application by using apt-get, which makes a LOT of sense.

But why would someone who doesn't know or care about Ruby be doing that?

What would happen when you uninstall the program if packaged as you mention? You can't remove the gems, and you can't really leave them there either...

`gem uninstall thegems`

(Yeah, there are probably things you can do with the debian pkg manager that you can't with gems, and I think it's worth fixing that.)

I guess my concern with repackaging is that it introduces duplication.

Devin

[Devin_Mullins]

Okay. Well, those are reasonable.

Devin

Eivind Eklund wrote:

···

(Note that some of these are projections from problems I know about it
in the context of FreeBSD, which I'm a developer for, and RPA, which
I've been an advisor for. I'm just projecting to Debian, based on
having read various Debian policy documents and talked to Debian users
and developers.)

There is several reasons, including the deinstall problem that was
mentioned, that Gems start writing places they shouldn't (so write
protection of some areas of the disk don't work), and that Debian
generally do maintenance on single versions, fixing security holes as
they pop up etc so you can install a package, then run apt-get update
and get everything updated. This cascades to the rest of the package
organization.

Also, people expect to be able to find documentation etc in the
"normal places" even for langauges they're not used to, and
applications sometimes depend on executables.

There's a whole host of complexities that come up here when you
actually do packaging and maintenance.

Eivind.

[Joshua_Haberman1 (Joshua Haberman)]

I apologize -- I misread your tone. It sounded like you were trying to make Debian users sound unreasonable, and their wishes unimportant.

I just read the "gems and repackaging" thread on ruby-core@, and I agree with your position 100%.

Josh

···

On Oct 2, 2005, at 10:34 AM, Eivind Eklund wrote:

On 10/2/05, Joshua Haberman <joshua@reverberate.org> wrote:

On Oct 2, 2005, at 9:10 AM, Eivind Eklund wrote:

They're trying to repackage software that the author assume to be
Gem-distributed as Debian packages (.deb, IIRC). The reason for this
is simple: To provide Ruby software on Debian for Debian users in the
way Debian users are used to. These users do not particularly care
about Ruby, and definately do not want to have to know one packaging
system for handling Ruby software, and another for handling Perl, and
another for handling Java, and another for handling Haskell, and
another for handling Python, and another way for handling C code, and
...

Instead, they want they software to Just Work Like All Other Software,
the way they are used to, and to not have to care about the language
the software is written in at all.

May I request that we talk about technical issues like grown-up
engineers, not like guests on "Jerry Springer?"

These are user interface issues, and I am trying to explain them in a
way that get the overall top level user issues communicated.

[Austin_Ziegler5 (Austin Ziegler)]

But do they? What about on a modern AMD or Intel? You'll have i686-linux
and x86_64-linux on the same box. On 64-bit Windows, this is represented
as "Program Files" (64-bit) and "Program Files-x86" (64-bit).

-austin

···

On 10/2/05, Joshua Haberman <joshua@reverberate.org> wrote:

P.S. Incidentally, I don't see the point of the extra "i386-linux"
directory. What does it buy you? I don't keep my C libraries in /
usr/lib/i386-linux. If shared objects are on my machine, it is assumed
that they match the architecture of my machine.

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

[Corey_Lawson1 (Corey Lawson)]

Well, is there an apt-get for every other OS/Debian besides Debian? Might as
well be a Windows Ruby user being steamed for everything not being packaged
as a .MSI file using the newest Windows Installer, because, well, everything
else comes packaged that way for Windows, so might as well be consistent,
no?

Again, it's possible to package Ruby libraries not as .gem, just like it is
feasible to package a J2EE web app as something besides a .[jwe]ar, but in
the context of the environments, it's just easier to deal with creating the
.[jwe]ar files, and it's just easy enough to use gem to manage gem files.

···

On 10/2/05, Brian Schröder <ruby.brian@gmail.com> wrote:

On 02/10/05, Corey Lawson <corey.ssf.lawson@gmail.com> wrote:
> It sounds like a debian user problem, not a Ruby user problem. I think
most
> Ruby people (who probably are not running ruby on Debian) want Ruby to
work
> like...Ruby, whether it's on a Mac, Windows, RedHat, Debian, Slolaris,
> whatever. Has a similar beef been made about Java on Debian? (Why must
Java
> apps be deployed as .jar, .ear, war, whatever. Can't they pick just one
and
> make it work with apt-get?).
>

If it is a debian user problem that is sad, but I have to say that I
try to avoid any libraries that are not packed as a deb. I do not have
gems installed because the duality of the gem and the debian system is
annoying me.

This may be just an anecdote, but a lot of libraries are loosing at
least me as a user. I have packaged deb's myself and I don't think it
is a very complicated process, so I wonder why it is difficult to go
from .gem to .deb?

brian

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/

[Kevin_Brown]

That was my first thought too. No, because if you distribute the debians as
part of a distro, you're requiring them to have an internet connection to get
them, and in fact there is no way at that point to include said software as
part of a distro that you can install without an internet connection.

Unless they set up a gems repository, and have it ask on the CD, but that gets
really nasty when you start realizing that this all works contrary to the way
apt-get has worked in the past.

Also, my mom (who just unplugged her network cable) wants to know what this
funky error about WGET FAILED FOR GEM INSTALL FROM DEB PACKAGE means. (I'm
pointing out that it doesn't allow users the precious luxury of ignorance,
which will be important if Linux ever wants to world dominate.)

···

On Sunday 02 October 2005 10:45, Devin Mullins wrote:

Kevin Brown wrote:
>but this is very contrary to the fact that once
>you download the .deb package, you HAVE the software and that's IT.

Ah. Well, err, to qualify my previous post, couldn't the .deb just
include the .gem(s) inside, and then call `gem install thegem(s)`?

[gabriele_renzi3 (gabriele renzi)]

Devin Mullins ha scritto:

Kevin Brown wrote:

but this is very contrary to the fact that once you download the .deb package, you HAVE the software and that's IT.

Ah. Well, err, to qualify my previous post, couldn't the .deb just include the .gem(s) inside, and then call `gem install thegem(s)`?

no, since gems make layout assumptions that do not match the debian policy (nor the policy of most package systems on *nix, it seem).
Anyway, there was a huge thread on this on ruby-core that people may like to read, since some of this.

I have faith in the rubygems dev crew, anyway, so that the problems can be addressed before merging it in ruby's HEAD.

[Eivind_Eklund]

>What would happen when you uninstall the program if packaged as you mention?
>You can't remove the gems, and you can't really leave them there either...
>
>
`gem uninstall thegems`

What if other stuff rely on those gems? Like user-written code?

This stuff is tricky to get right, even more tricky than the single
package manager situation.

(Yeah, there are probably things you can do with the debian pkg manager
that you can't with gems, and I think it's worth fixing that.)

I guess my concern with repackaging is that it introduces duplication.

In what way are you thinking of?

I think not repackaging introduce duplication, in the form of
duplicated package managers handling the same machine, and thus
duplicated interfaces the user needs to know.

Having native packages avoid duplication as long as you use only
native packages; using both is of course a problem, and really
ideally, they'd be able to hook together nicely - unfortunately, I'm
uncertain if that is doable in practice (for combined technical and
social reasons.)

Eivind.

···

On 10/2/05, Devin Mullins <twifkak@comcast.net> wrote:

[Devin_Mullins]

Kevin Brown wrote:

Ah. Well, err, to qualify my previous post, couldn't the .deb just
include the .gem(s) inside, and then call `gem install thegem(s)`?
   

That was my first thought too. No, because if you distribute the debians as part of a distro, you're requiring them to have an internet connection to get them, and in fact there is no way at that point to include said software as part of a distro that you can install without an internet connection.

RubyGems can install local gem files quite easily. That was what I was referring to.

Also, my mom (who just unplugged her network cable) wants to know what this funky error about WGET FAILED FOR GEM INSTALL FROM DEB PACKAGE means.

Really? My mom wants to know why it takes 6 minutes to boot up Windows Me.

Devin
(I'm joking. Note the ":P".)

···

On Sunday 02 October 2005 10:45, Devin Mullins wrote:

[Devin_Mullins]

gabriele renzi wrote:

no, since gems make layout assumptions that do not match the debian policy (nor the policy of most package systems on *nix, it seem).
Anyway, there was a huge thread on this on ruby-core that people may like to read, since some of this.

Yeah, I figured that I should just be not-lazy and go read ruby-core.

I have faith in the rubygems dev crew, anyway, so that the problems can be addressed before merging it in ruby's HEAD.

Ditto.

Joshua Haberman wrote:

(References to documentation are fine, I just haven't been able to find much information about how to to do these things).

http://docs.rubygems.org/

Devin

[Devin_Mullins]

Eivind Eklund wrote:

`gem uninstall thegems`
   

What if other stuff rely on those gems? Like user-written code?

OK, point. What does apt-get (or ports) do natively to fix this?

I guess my concern with repackaging is that it introduces duplication.
   

In what way are you thinking of?

For a given libraries, you would have multiple packages (RubyGems, ports, apt-get, portage, rpm, fink, darwinports...) all that contain the same meaning in them -- what to install and how to install it. Much of the same logic comprises each of those packages. I would rather efforts be taken to put as much of the logic in a single package system, and let the other ones be "adapters" to that one.

Further, this causes one of two problems for the library author:
1. He's got to publish in 10 different package formats
2. His library has got to be famous enough that the major packaging systems have put maintainers in charge of his library, and he's got to not mind the delay.

I think not repackaging introduce duplication, in the form of
duplicated package managers handling the same machine, and thus
duplicated interfaces the user needs to know.

Yeah, I see your point here. I would think that an "adapter" technique would solve the UI problem better than static repackaging would. As far as multiple package managers being duplication, well, that's true, but there are, like, twenty web frameworks for Python, and that's likely not going to change. Duplication of effort is a fact of life.

really
ideally, they'd be able to hook together nicely - unfortunately, I'm
uncertain if that is doable in practice (for combined technical and
social reasons.)

That's too bad. Imagine if there existed a single piece of code that allowed apt-get to interact with .gem files and gems.rubyforge.com. That would allow apt-get users to control Ruby libs, and wouldn't require each lib to be repackaged for apt-get specifically.

(It would't obviate all those differences you've pointed out, of course.)

Devin

···

On 10/2/05, Devin Mullins <twifkak@comcast.net> wrote: