Packaging ideas (was: Re: packaged REXML)

Sorry for not replying sooner. I wanted to first do my homework and
dig through Gentoo's docs, but I'm still in the midst of their howto.
I'm also cc:ing to the list as you asked.

In particular, there are a set of global --options that Portage can pass to
the configure phase of installation. This allows you to do things like
globally specify --qt-dir, or --no-x11.

Yes, I've seen mention of the USE directive. Pretty smart.

The great thing is that the ebuild files can be really terse. For example,
unlike RPM's spec, you don't have to specify where every file lives for
installation purposes; Portage discovers this information from the chrooted
install directory.

This is what Debian does, too.

File tracking. Package management. Querying which file belongs to which
package. That sort of thing. You can even have Portage build a binary
distribution from an already installed package, something I'd have expected
RPM to be able to do, but doesn't.

One of the nicest thing I've seen in their docs is the ability to
*not* remove the package before the new is installed. Of course not
every application will be so picky to absolutely need this, but it is
quite elegant.

> > is no dependancy from the tarball to the spec, unlike .rpm, .deb,
>
> Conceptually there is neither, too, in debianized sources. The specs
> reside in the standard directory extracted from the tarball but could
> be kept anywhere else.

What I'm saying is that there is /no/ dependancy at all with Portage. Since
the sources don't have build rules in them (aside from their normal
configure/make/ant/whatever scripts). The sources don't know that they're
going to be installed on Portage.

In contrast, you must 'debianize' the sources to make them work with
Debian.

Sorry, I wasn't clear. `Debianizing' is exactly the act of building
that `spec' directory. You could just (and in fact I've done) take
the upstream tarball, unpack it, drop the debian dir in it, and the
.deb package will be built. The binary package is built by chrooting
into a directory and running whatever installation mechanism the
upstream source had, too.

> FIX: Shift the task of the spec from installing a .tar.gz to building
> a directory structure conforming to the distribution policy, that is
> later handed to the installer and database manager.

Portage's fix is this (I said this above, but I want to be clear):

When portage runs the 'install' phase, it installs into a temp directory; EG
/var/lib/portage/<package>. So you get something like this:

  /var/lib/portage/<package>
    usr/
      bin/
        foo
        bar
      man/
        man1/
          foo.man
          bar.man
      lib/
        libfoobar.so

Portage then indexes that directory, and just copies stuff to their
destination directories.

I've tried to go a step further with rpkg. Basically, the packager
decides the `meaning' of each part by putting it in a specific
directory (ext for extensions, bin for executables, and so on). The
packager does not really know where each part will go in the client's
file system. `bin' files might go in /usr/bin, in /usr/local/bin, in
/home/john/bin... the choice is left to the end user. It's ok, of
course, to go with defaults and have `bin' go into /usr/bin, `doc'
into /usr/share/doc etc., but I'd like to preserve the freedom of the
end user to change that.

Massimiliano

···

On Tue, Jul 02, 2002 at 12:24:02PM -0700, Sean Russell wrote:

Ok, so this is what I'm thinking. Give me access to your sources, and I'll
see if I can provide some code, rather than making you do everything:

I want the following features in a distribution mechanism:

1) The distribution spec completely disassociated from the source tarball. As
we've talked about, this means that I can build a spec for, say, Dave Thomas'
rdoc without Dave knowing that his software is being downloaded and installed
by rpkg.
2) As much introspection on the install as possible. When the distribution
spec is by neccessity large and complex, people don't write them.
3) Support for both precompiled and source archives
4) Patching support
5) CVS support

In as much as it is possible, offload all of the work of compiling and
installing onto the package -- it should know best how to compile and install
itself. The distribution mechanism should just know how to fetch the
package, calculate dependancies, tell the package to compile and install
itself, and manage resources on the client side (tracking installed files and
so on).

(1) and (2) are important because practically, they greatly simplify writing
distribution specs. (3) is, well, core functionality. (4) is important
because it allows third parties to provide improvements to the standard
distribution. (5) is important for terse downloads.

Are these goals achievable within the framework of rpkg?

- --
>.. I'll never forget the first time I ran Windows, but I'm trying...
<|>
/|\
/|

···

Sean Russell wrote:

I want the following features in a distribution mechanism:

Will this take care of conflicts?

If app A needs lib Foo version 1, and the newly installed app B runs
with lib Foo version 2, then will satisfying B’s dependency (lib Foo 2)
break app A? I think a packaging mechanism should be able to deal with
backwards incompatible versions of shared libs; app A uses Foo 1, app B
uses Foo 2, so all dependencies are resolved, without breaking stuff by
introducing conflicts by updating libs.
To uniquely identify libs, URIs might be helpful.

Tobi

···


http://www.pinkjuice.com/

Sounds practically like portage :slight_smile:


Signed,
Holden Glova

···

On Sun, 07 Jul 2002 10:23, Sean Russell wrote:

Ok, so this is what I’m thinking. Give me access to your sources, and I’ll
see if I can provide some code, rather than making you do everything:

I want the following features in a distribution mechanism:

  1. The distribution spec completely disassociated from the source tarball.
    As we’ve talked about, this means that I can build a spec for, say, Dave
    Thomas’ rdoc without Dave knowing that his software is being downloaded and
    installed by rpkg.
  2. As much introspection on the install as possible. When the distribution
    spec is by neccessity large and complex, people don’t write them.
  3. Support for both precompiled and source archives
  4. Patching support
  5. CVS support

In as much as it is possible, offload all of the work of compiling and
installing onto the package – it should know best how to compile and
install itself. The distribution mechanism should just know how to fetch
the package, calculate dependancies, tell the package to compile and
install itself, and manage resources on the client side (tracking installed
files and so on).

(1) and (2) are important because practically, they greatly simplify
writing distribution specs. (3) is, well, core functionality. (4) is
important because it allows third parties to provide improvements to the
standard distribution. (5) is important for terse downloads.

Are these goals achievable within the framework of rpkg?

(1) (2) and (3) won’t put anything upside down. I’m not sure about
what you mean by (4), i.e. patching of installed files? For (5), I’d
take the road of making rpkg totally ignorant about where the package
are coming from, and have it talk to pluggable back-ends managers with
a uniform interface. Then, adding rsync or CVS is just a matter of
writing the appropriate back-end.

I agree with about everything you propose, and I add (6) the ability
to handle multiple unrelated package bases.

I’ll send you the `devel’ code of rpkg. It’s quite dirty and (worst)
it has few tests, but when I started I didn’t know better. :frowning: New
code (version and archive files handling) has full test suites.

Massimiliano

···

On Sun, Jul 07, 2002 at 07:23:08AM +0900, Sean Russell wrote:

Ok, so this is what I’m thinking. Give me access to your sources, and I’ll
see if I can provide some code, rather than making you do everything:

I want the following features in a distribution mechanism:

  1. The distribution spec completely disassociated from the source tarball. As
    we’ve talked about, this means that I can build a spec for, say, Dave Thomas’
    rdoc without Dave knowing that his software is being downloaded and installed
    by rpkg.
  2. As much introspection on the install as possible. When the distribution
    spec is by neccessity large and complex, people don’t write them.
  3. Support for both precompiled and source archives
  4. Patching support
  5. CVS support

In as much as it is possible, offload all of the work of compiling and
installing onto the package – it should know best how to compile and install
itself. The distribution mechanism should just know how to fetch the
package, calculate dependancies, tell the package to compile and install
itself, and manage resources on the client side (tracking installed files and
so on).

(1) and (2) are important because practically, they greatly simplify writing
distribution specs. (3) is, well, core functionality. (4) is important
because it allows third parties to provide improvements to the standard
distribution. (5) is important for terse downloads.

Are these goals achievable within the framework of rpkg?

Tobias Reif wrote:

Will this take care of conflicts?

Yes, and no. I mean, nothing in what I described, per se, will take care of
conflicts, but dependancy resolution and conflict handling are both central
to any package management mechanism worth using. I didn’t bother to put in
requirements for features that I (Sean-POLS) expect to be a basic given :slight_smile:

— SER

Holden Glova wrote:

Are these goals achievable within the framework of rpkg?

Sounds practically like portage :slight_smile:

Are you kidding? It /is/ Portage. I started this discussion with
Massimiliano trying to see if I could get him to add Portage-like features
to rpkg.

The one feature I’m lobbying hard for, the CVS support, I’m also campaigning
for in Portage… check bugs.gentoo.org :slight_smile:

I’ve been using RedHat and Mandrake for years now; I’ve been running Gentoo
on a server for only a couple of weeks, and already I’m considering
converting everything over to Gentoo just for the Portage support. It is
amazing how much difference a decent package management system can make.
That said, I’m really happy with the work Massimiliano has done on rpkg; as
I told him, I was really worried that Ruby would go the CPAN route, which
is probably the worst system I’ve ever encountered.

— SER