[ANN] rpa-base 0.2.0

ruby-talk
1

rpa-base 0.2.0 is now available at http://rpa-base.rubyforge.org .
Many of the most popular libraries/applications as per Rubyforge
statistics (rails, rake, redcloth, activerecord, sqlite, log4r, copland,
ruvi, to name a few) have been packaged for use with rpa-base 0.2.0.

Foreword

···

--------

The Ruby Production Archive (RPA) will provide packages of Ruby
libraries and programs in a form that allows production use, engineered
through a stringent process resembling FreeBSD's or Debian's.

rpa-base is a port/package manager designed to support RPA. Its scope and
purposes are different to those of other systems like RubyGems.

Status
------

Please keep in mind that RPA is at an embryonic stage; this means that
it is impossible to commit to all the long term goals stated in the
manifesto (http://rpa-base.rubyforge.org/wiki/wiki.cgi?RpaManifesto) at
the moment. This doesn't make rpa-base any less usable however.

rpa-base requires Ruby 1.8.[12] (certainly 1.8 at least, it might work
on 1.8.0); it has been tested on several Linux distributions (Debian,
Fedora, older RH, Gentoo, etc), FreeBSD, DragonFly BSD, Mac OSX, win32
(cygwin, 'pragmatic installer', WinXP and 2K).

rpa-base is fairly stable at this stage; it has been tested on several
platforms during the last 2 months. Since rpa-base can self-update,
eventual bugs discovered through the intensive testing associated with
a public release could be solved easily by upgrading rpa-base using
rpa-base itself.

We would appreciate any feedback on rpa-base.
A mailing list has been set up for that purpose:
  http://rubyforge.org/mailman/listinfo/rpa-base-testers

Features

rpa-base is a port/package manager designed to support RPA's client-side
package management. You can think of it as RPA's apt-get + dpkg. It
features the following as of 0.2.0:

* strong dependency management: rpa-base installs dependencies as needed,
   keeps track of reverse dependencies on uninstall, and will remove no
   longer needed dependencies
* atomic (de)installs: operations on the local RPA installation are atomic
   transactions; the system has been designed to survive ruby crashes (OS
   crashes too on POSIX systems)
* parallel installs: you can install several ports in parallel; building
   will be parallelized and the final phase will be serialized properly
* self-hosting: rpa-base installs and updates itself
* modular, extensible design: the 2-phase install is similar to FreeBSD and
   Debian's package creation; rpa-base packages need not be restricted
   to installing everything under a single directory ("1 package, 1 dir"
   paradigm)
* rdoc integration: RDoc documentation for libraries is generated at install
   time (currently disabled on win32)
* ri integration: ri data files are generated for all the libraries managed
   by RPA; you can access this information with ri-rpa
* handling C extensions: if you have the required C toolchain, rpa-base can
   compile extensions as needed
* unit testing: when a library is installed, its unit tests are run; the
   installation is canceled if they don't pass

Several of the above features are illustrated in the screenshots and
animations available at
http://rpa-base.rubyforge.org/wiki/wiki.cgi?Rpa_Base_In_Action

Limitations:

A number of features have been pushed back to 0.3.0:
* full support for binary platform-specific packages
* signed packages/ports
* system-wide configuration system
* better user interface

In practice, the first one is the most limiting at the moment since it means
that win32 users in particular need a working C toolchain to install
extensions. This will soon be addressed.

RPA needs your help

RPA is an ambitious project in need for developers. These are some of the
areas that need to be worked on:
* packaging (new software and package maintenance)
* setting up a permanent repository infrastructure
* cross-compilation and build automation
* website development (should provide package indexes, QA section,
  bugtracking, etc)

Please contact me at <batsman.geo@yahoo.com> (adding RPA to the subject
will help get it past the spam filtering :wink: or via IRC, batsman @
#ruby-lang on freenode.net if you have any interest or for any additional
information on RPA/rpa-base and their goals.

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

2

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

3

Can't reproduce with
ruby 1.9.0 (2004-05-19) [i686-linux]

I'll upgrade to a later CVS and try again...

···

On Tue, Aug 10, 2004 at 02:41:46AM +0900, Joel VanderWerf wrote:

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

4

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

This is very strange, cause

batsman@tux-chan:/tmp/ruby1.9/lib$ grep -r "unrecognized" *
getoptlong.rb: set_error(InvalidOption, "unrecognized option `#{argument}'")
open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

batsman@tux-chan:~/src/rpa/rpa-base$ grep -r "unrecognized" *
lib/rpa/.svn/text-base/open-uri.rb.svn-base: raise ArgumentError, "unrecognized option: #{k}"
lib/rpa/open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

I don't know where that "unrecognized arguments" comes from.

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

Can't reproduce it :frowning:

batsman@tux-chan:/tmp/ruby1.9$ ~/ruby1.9/bin/ruby -v
ruby 1.9.0 (2004-08-09) [i686-linux]
batsman@tux-chan:/tmp/ruby1.9$ ~/ruby1.9/bin/rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Building ri-rpa (0.1-4).
Building dependencies rdoc.
Getting port rdoc from
http://rpa-base.rubyforge.org/ports/rdoc_1.8.1-3.rps\.
100% [========================================] 94208 bytes
Building rdoc (1.8.1-3).
Calculating MD5 digests.
Building package in rdoc_1.8.1-3_i686-pc-linux-gnu.rpa.
Fixed shebang in rpa/tmp/bin/ri-rpa.
Calculating MD5 digests.
Building package in ri-rpa_0.1-4_i686-pc-linux-gnu.rpa.
  Installing ri-rpa
Reusing cached package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/ri-rpa_0.1-4_i686-pc-linux-gnu.rpa.
Installing dependencies rdoc.
Starting lightweight (metadata only) transaction for ri-rpa
Reusing cached package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/rdoc_1.8.1-3_i686-pc-linux-gnu.rpa.
Starting lightweight (metadata only) transaction for rdoc
Checking for file conflicts in rdoc.
Starting transaction for rdoc
Package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/rdoc_1.8.1-3_i686-pc-linux-gnu.rpa unpacked.
Finished transaction for rdoc
Finished lightweight (metadata only) transaction for rdoc
Checking for file conflicts in ri-rpa.
Starting transaction for ri-rpa
Package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/ri-rpa_0.1-4_i686-pc-linux-gnu.rpa unpacked.
Finished transaction for ri-rpa
Starting lightweight (metadata only) transaction for ri-rpa
Finished lightweight (metadata only) transaction for ri-rpa
Finished lightweight (metadata only) transaction for ri-rpa
Committed changes

···

On Tue, Aug 10, 2004 at 02:41:46AM +0900, Joel VanderWerf wrote:
  
--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

5

Mauricio Fernández wrote:

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

This is very strange, cause

batsman@tux-chan:/tmp/ruby1.9/lib$ grep -r "unrecognized" *
getoptlong.rb: set_error(InvalidOption, "unrecognized option `#{argument}'")
open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

batsman@tux-chan:~/src/rpa/rpa-base$ grep -r "unrecognized" *
lib/rpa/.svn/text-base/open-uri.rb.svn-base: raise ArgumentError, "unrecognized option: #{k}"
lib/rpa/open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

I don't know where that "unrecognized arguments" comes from.

Tried to run it in gdb to see where it was when exit() was called, but running it in gdb "fixed" the problem:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting
# gdb ruby
GNU gdb 5.3-22mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
(gdb) r /usr/local/bin/rpa install ri-rpa
Starting program: /home/local/bin/ruby /usr/local/bin/rpa install ri-rpa
Installing ports
Getting port ri-rpa from http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Building ri-rpa (0.1-4).
Building dependencies rdoc.
Getting port rdoc from http://rpa-base.rubyforge.org/ports/rdoc_1.8.1-3.rps\.
100% [========================================] 94208 bytes
Building rdoc (1.8.1-3).

This experience is repeated with other packages (I tried net-ssh):

# ruby /usr/local/bin/rpa install net-ssh
Installing ports
Getting port net-ssh from http://rpa-base.rubyforge.org/ports/net-ssh_0.0.5-1.rp
100% [========================================] 117760 bytes
Error: unrecognized arguments net-ssh aborting
# gdb ruby
GNU gdb 5.3-22mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
(gdb) r /usr/local/bin/rpa install net-ssh
Starting program: /home/local/bin/ruby /usr/local/bin/rpa install net-ssh
Installing ports
Getting port net-ssh from http://rpa-base.rubyforge.org/ports/net-ssh_0.0.5-1.rp
100% [========================================] 117760 bytes
Building net-ssh (0.0.5-1).
Generating RI data files.
Generating RDoc HTML documentation.
...

What could be different about running in gdb? My normal shell is zsh, but bash has the same effect.

Once a package is installed, doing "rpa install" on it again succeeds (detecting no changes).

RPA seems to work fine for me on windows, anyway.

···

On Tue, Aug 10, 2004 at 02:41:46AM +0900, Joel VanderWerf wrote:

6

>>This seems to happen for anything I try to install with rpa:
>>
>># rpa install ri-rpa
>>Installing ports
>>Getting port ri-rpa from
>>http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
>>100% [========================================] 264704 bytes
>>Error: unrecognized arguments ri-rpa aborting

Could you try to run it with --debug, to get a backtrace?
Alternatively, you could try to put a debug statement in rpafrontend.rb around

    307 rescue => e
    308 $stderr.puts "Error: #{e} aborting"
    309 raise if @options.debug
    310 exit 6

(I think this is where the error is detected).

Tried to run it in gdb to see where it was when exit() was called, but
running it in gdb "fixed" the problem:

This experience is repeated with other packages (I tried net-ssh):

What could be different about running in gdb? My normal shell is zsh,
but bash has the same effect.

Once a package is installed, doing "rpa install" on it again succeeds
(detecting no changes).

This makes me think it is somehow related to the ri/rdoc integration
and some shell quoting issue.

RPA seems to work fine for me on windows, anyway.

That's good news at least, win32 was such a PITA to support from the
beginning :wink:

···

On Tue, Aug 10, 2004 at 07:48:54AM +0900, Joel VanderWerf wrote:

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

7

Mauricio Fernández wrote:

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps\.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Found it... there was an install.rb on my lib path that was being loaded
instead of the real install.rb. The problem may be in rpa/base.rb, line 256:

         begin
             Dir.chdir(destdir){ load "install.rb" }
         rescue Exception
             RPA::Install::AtExitHandler.cancel
             raise
         end

If install.rb is not found in destdir, $LOAD_PATH is searched, and who
knows what it'll find there. It might be better to use
File.expand_path("install.rb") in this case.

···

On Tue, Aug 10, 2004 at 07:48:54AM +0900, Joel VanderWerf wrote:

8

Mauricio Fernández <batsman.geo@yahoo.com> wrote in message news:<20040809235924.GA24349@student.ei.uni-stuttgart.de>...

This makes me think it is somehow related to the ri/rdoc integration
and some shell quoting issue.

Mauricio, I tried your rpa-base and I admit I loved it, as it really
reminded me of the simplicity of perl's cpan installs.

I do have a minor bug and 3 questions, thou:

a) First the minor bug. It seems documentation creation requires the
downloading of rpa-ri first, which is fine. However, if you download
other modules first and only rpa-ri later, documentation of those
previous modules is not created and requires uninstalling and
re-installing them again.

b) Also, how exactly do I get ri/rpa-ri to show documentation for the
installed packages? I installed my getopt-declare module and it said
it created the ri docs for it, but I don't seem to be able to access
them.

c) As an indirect contributor of a module to rpa-base, how do I make
sure rpa-base is indeed using the latest version of my module? That
is, how do the maintainer(s) of rpa-base get notified of the existance
of a new version of my library?

d) As someone who might contribute other libraries in the future, how
does rpa-base and rubygems interact (if at all). I really would like
to stick to a single format for installs which would hopefully remain
compatible.

9

Thanks a lot for squashing this bug :slight_smile:
I wonder why install.rb couldn't be found in the tempdir, though...

···

On Tue, Aug 10, 2004 at 02:29:09PM +0900, Joel VanderWerf wrote:

Found it... there was an install.rb on my lib path that was being loaded
instead of the real install.rb. The problem may be in rpa/base.rb, line 256:

        begin
            Dir.chdir(destdir){ load "install.rb" }
        rescue Exception
            RPA::Install::AtExitHandler.cancel
            raise
        end

If install.rb is not found in destdir, $LOAD_PATH is searched, and who
knows what it'll find there. It might be better to use
File.expand_path("install.rb") in this case.

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

10

Mauricio, I tried your rpa-base and I admit I loved it, as it really
reminded me of the simplicity of perl's cpan installs.

I have to say that cpan.pm is NOT the model rpa-base is based on. The
major sources of inspiration are FreeBSD and Debian: I found that
on a technical level, there's very little to be stolen from cpan.pm
(I believe that both RubyGems and rpa-base are more powerful than
cpan.pm); I'm more interested in the CPAN process as a whole, but I still
believe there's more to be learned from FreeBSD/Debian than from CPAN.

I do have a minor bug and 3 questions, thou:

a) First the minor bug. It seems documentation creation requires the
downloading of rpa-ri first, which is fine. However, if you download
other modules first and only rpa-ri later, documentation of those
previous modules is not created and requires uninstalling and
re-installing them again.

This is not accurate :wink: Documentation is always generated, it is not
required to have ri-rpa. You need the latter to access the generated ri
information with

$ ri-rpa -p portname

from the command line. (Note that I don't like the -p portname
mechanism, and will try to solve the issue in a more general way.)
So no need to uninstall/reinstall in general.

Not all libraries/apps have rdoc/ri documentation, but for those that
do, you should be able to find it under
  $prefix/share/doc/rpa0.0/portname/rdoc

Now, if you're often getting messages like
WARNING: RI datafile generation failed
when installing a port (regardless of whether you have ri-rpa installed
or not, since as I said it is not needed to generate the docs actually),
this means there's a problem with your rdoc installation: make sure you
have it installed (apt-get install rdoc1.8 under Debian) and that it
does indeed work. In the future, I might ship rdoc with rpa-base and/or
ri-rpa, to prevent such problems.

b) Also, how exactly do I get ri/rpa-ri to show documentation for the
installed packages? I installed my getopt-declare module and it said
it created the ri docs for it, but I don't seem to be able to access
them.

The rdoc information should be in
  $prefix/share/doc/rpa0.0/getopt-declare/rdoc

You can access the ri information as follows:

$ ri-rpa -p getopt-declare -T DelimScanner
---------------------------------------------------- Class: DelimScanner
     A derivative of StringScanner that can scan for delimited
     constructs in addition to regular expressions.

···

On Tue, Aug 10, 2004 at 05:21:21PM +0900, GGarramuno wrote:

------------------------------------------------------------------------

[...]

Note that there's a number of usability issues with the modified ri I
distributed as 'ri-rpa'; I am planning to work on them and eventually
push a new release through the usual channels (rpa update && rpa install
ri-rpa).

c) As an indirect contributor of a module to rpa-base, how do I make
sure rpa-base is indeed using the latest version of my module? That
is, how do the maintainer(s) of rpa-base get notified of the existance
of a new version of my library?

I was hoping that a combination of the following would work:
* regular inspection of release announcements on ruby-talk, rubyforge
  and RAA
* direct notification from the author
* requests from the users

I have yet to create an 'official channel' for the second; I think I'll
set up a rpa-sw-authors mailing list in Rubyforge, but for the time
being you can just drop me a note (RPA in the subject to get to the
appropriate mailbox quickly :slight_smile:

d) As someone who might contribute other libraries in the future, how
does rpa-base and rubygems interact (if at all). I really would like
to stick to a single format for installs which would hopefully remain
compatible.

There are a number of differences (the most important one being the
approach to the versioning problem), as explained in
http://rpa-base.rubyforge.org/wiki/wiki.cgi?Rpa_FAQ

I am not sure total integration is feasible, but just in case I
contributed the code for the tar/gz package format rpa-base uses to
RubyGems, so at least now the packages are externally similar (there
are still important differences in what is put inside, though :).
rpa-base tries to play nice with other package managers:
* it won't overwrite files it didn't "own" before unless you specify
  --force while installing
* on uninstall, it will not remove the files that have been modified
(it assumes they are now "owned" by another pkg manager, or needed by
some other program)

In practice, if you install a Gem without stubs, the Gem & RPA versions
can coexist easily; require_gem "foo" would use the Gem version and the
normal require 'foo' would take the RPA one.

If you install the Gem with stubs, the 2 following cases can happen:
* installing the Gem before: rpa-base won't let you install the RPA
  version unless you --force, in which case the stubs are overwritten
  and require 'foo' would use the RPAfied version. On uninstall,
  the files which are now owned by rpa-base would be removed; in the
  process, you've lost the stubs that RubyGems used.
* installing the RPAfied version before: when RubyGems is about to
  create the stubs, it will complain saying that the files exist already
  and (IIRC) choose not to overwrite them. Then both packages would
  coexist as usual. If gem overwrote them, on uninstall rpa-base would
  detect that and leave them there.

As for having a unified format, I think that
* to the upstream developer (you), the existence of RPA doesn't make
  much of a difference: you can keep maintaining your gemspecs, cause we
  will maintain the RPAfied version of your software without putting
  more strains on you (of course, you can choose to become a part of
  'we' and help maintain the RPA version, but you're not required to :wink:
* for the end user, having 2 separate commands to install is not that
  inconvenient, as long as the packages can coexist peacefully. As shown
  above, this is mostly the case already

Thanks a lot for the feedback.

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

11

I have to say that cpan.pm is NOT the model rpa-base is based on. The
major sources of inspiration are FreeBSD and Debian: I found that
on a technical level, there's very little to be stolen from cpan.pm
(I believe that both RubyGems and rpa-base are more powerful than
cpan.pm); I'm more interested in the CPAN process as a whole, but I still
believe there's more to be learned from FreeBSD/Debian than from CPAN.

could it preserve the downloaded files then in something like distfiles ?
if the install aborts for some reason it's quicker the second time after
correcting the situation if you have the files locally :slight_smile:

Jani

12

Yes, it will soon cache the .rps (port) files. Currently, only the .rpa
(binary package) files generated locally are cached, but this will change.

···

On Tue, Aug 10, 2004 at 07:19:07PM +0900, Jani Monoses wrote:

> I have to say that cpan.pm is NOT the model rpa-base is based on. The
> major sources of inspiration are FreeBSD and Debian: I found that
> on a technical level, there's very little to be stolen from cpan.pm
> (I believe that both RubyGems and rpa-base are more powerful than
> cpan.pm); I'm more interested in the CPAN process as a whole, but I still
> believe there's more to be learned from FreeBSD/Debian than from CPAN.

could it preserve the downloaded files then in something like distfiles ?
if the install aborts for some reason it's quicker the second time after
correcting the situation if you have the files locally :slight_smile:

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com