RPA and author packages/interactions (forward vision)

ruby-talk
1

Discussions of the full RPA at the bottom.

>
>Then it sounds like that RubyGems needs to adopt the framework of RPA, and
>help improve/adapt it to its needs, e.g. versionsing.
>
>I like RPA. I want to use RPA. But I can't. I have to say "pretty please"
>package this for me. I understand the QA. I think that's great. But I also
>think there needs be a way for beta wares to "get on the line". Gems allows
>that. So I'll just use Gems.

Amen, brother.

From trying both, my impression so far (which does not yet include evaluation
of how C modules are handled) is that rpa is clearly superior for my needs.

But, as well intentioned as rpa's team may be, not having a well
documented and clean way to distribute your own packages, I see it never
catching on.

We're crawling two paths towards this: Helping RubyGems with their
commitment to being repacker friendly (at which point we can easily
provide the RubyGems "package API", so you can use .gem's from the RPA
tools), and looking at an experimental section. The latter would
allow authors to hook the same infrastructure we do, and allow authors
to reasonably quickly get "integrated with the team". With that,
they'll either maintain their own packages by having write access to
the main branch, or feel comfortable that they are better off with
having somebody else do that for them.

I'll try to describe the sociotechnical interactions in this a bit better below.

Mauricio, that web page explaining things really does not explain anything.
Icould not make head or tails of it. Having perhaps the rpa "gemspec"
available for the modules of rpa would be a better approach.

[...]

If RPA's format was documented and was as simple to use as a gemspec
(or, better yet, use gemspec itself as it stands, as it is popular enough
already), I really would not bother with rubygems, imo.

The RPA format is simpler/cleaner than the corresponding gemspec,
usually. However, we prefer to have the RPA format only used in the
RPA repository, in order to be able to change it and do updates across
all packages. With the RubyGems commitment to becoming repackager
friendly (and we try to give as much help as we can with that), we
hope that using the RubyGems format for external author-packaged
packages and the RPA format for RPAfied packages (with their extra
requirements) will give an ideal result. Of course, for end user
convenience, these will need to be available through the same user
interface.

I've previously tried to outline what I'd like RPA to be in terms of
quality etc, in the form of the RPA manifesto and other writings.
What I've not shared with anybody beyond musing about it on IRC,
however, is how achieve this,.and how I see the interactions between
RPA and authors when RPA is "fully functional".

Please understand that what I describe here is necessarily incomplete,
and that we'll have to get there one step at a time. We will also
modify the plan as we gain experience from the steps we take, and as
we get feedback from people using it.

First, let's look at this from the perspective of a developer that
does not use the RPA infrastructure directly.

When he has a release ready, he'll just put out a .gem. The Usual
Processes will catch the gem and add it to the gem list. At this
point, it's easily available through RPA interface too, and install
will integrate with RPA. However, some of the RPA features will not
be available for that (because various metadata and setup is missing
compared to a RPAfied package), and the user will be warned about this
and the lack of quality control.

A little while later (hours to days) the RPAfied source variant of the
package (the port update) will appear in the HEAD of RPA. This will
just require minimal testing; standard difference review from the last
release, testing that it package correctly on at least one platform
(more if we have suitable automation), checking for correct internal
version numbering if a library, and other small QA work. For a core
package (one that we have blessed as being "production quality and
maintained from RPA"), there will be somewhat more extensive QA for
updates. We'll probably also use various test branches etc to
streamline this process.

There will also appear various mail aliases, a bug tracker, etc on the
RPA site, but these are the responsibility of the porter (RPA person)
rather than the original author. The standard procedure is to notify
the original author about these lists and of course allow her to take
some level of control of how they work (automatically gating to other
lists if wanted, etc), but RPA will make sure the well-known list
names are there for every package.

As maintenance is done through RPA, the porter will act as a liason to
the author, ensuring that the author get hold of new patches and
relevant bug reports, and trying to screen out the rest, giving a
"maximally relevant" feed to the author (controlled by the preferences
of the author).

One of the problems with most packaging systems is that they distance
the user from the code. Getting a program from the archive into a
form where it is easy to hack on is noticeable amounts of work; often,
it involves going totally outside the archive and checking code out
from some unknown CVS repository somewhere, and using "some weird
unknown build/install procedure". We want RPA to do the opposite:
Make the distance from the package to being an active project
contributor on the code and/or documentation level *smaller* than it
is when starting from an author release. To help this, we'll try to
normalize the development environment.

To start working on a package, do 'rpa hack <portname>', and get a
checked out version of the port with version control metadata for a
distributed VC (at the moment it looks like we'll use DARCS, but we're
not quite sure. CVS does not work at all well for this task.) With
this, it will also be easy to give your changes back to RPA - probably
just 'rpa offer' from the workspace, or something like that. (We'll
look at the exact user interface later.)

To get started with the software in it, look at the RPA_HACKING file
in that directory. It will be what notes the code review team of RPA
has made about that package, using our standard terminology etc (we'll
try to keep the style reasonably consistent.)

To get on the mailing list for the people working on that software
through RPA, using the mail aliases described above, use 'rpa
subscribe dev <package name>'

For an author that want to work more closely with RPA: You can just
hook yourself onto the infrastructure described above, initially
working in a sandbox branch, later becoming your own porter. You'll
get "free merges" to the software from other RPA committers/porters -
added test coverage, fixed documentation, etc.

Eivind.

···

On Fri, 29 Oct 2004 14:48:59 +0900, GGarramuno <ggarramuno@aol.com> wrote:
--
Hazzle free packages for Ruby?
RPA is available from http://www.rubyarchive.org/

2

Discussions of the full RPA at the bottom.

Wow.. this is good insight into how RPA works.

[snip]

We're crawling two paths towards this: Helping RubyGems with their
commitment to being repacker friendly (at which point we can easily
provide the RubyGems "package API", so you can use .gem's from the RPA
tools), and looking at an experimental section. The latter would
allow authors to hook the same infrastructure we do, and allow authors
to reasonably quickly get "integrated with the team". With that,
they'll either maintain their own packages by having write access to
the main branch, or feel comfortable that they are better off with
having somebody else do that for them.

I think its good with competition between the RPA team and the GEMs team.
It results in faster development (suvival of the fittest). What I like is that
both teams follows two distinct goals, only with little overlap.
Sharing code and specs is good.

[snip]

> If RPA's format was documented and was as simple to use as a gemspec
> (or, better yet, use gemspec itself as it stands, as it is popular enough
> already), I really would not bother with rubygems, imo.

The RPA format is simpler/cleaner than the corresponding gemspec,
usually. However, we prefer to have the RPA format only used in the
RPA repository, in order to be able to change it and do updates across
all packages.

This seems to me as a very wise choice.

With the RubyGems commitment to becoming repackager
friendly (and we try to give as much help as we can with that), we
hope that using the RubyGems format for external author-packaged
packages and the RPA format for RPAfied packages (with their extra
requirements) will give an ideal result. Of course, for end user
convenience, these will need to be available through the same user
interface.

How far is RPA from reaching this point?

I've previously tried to outline what I'd like RPA to be in terms of
quality etc, in the form of the RPA manifesto and other writings.
What I've not shared with anybody beyond musing about it on IRC,
however, is how achieve this,.and how I see the interactions between
RPA and authors when RPA is "fully functional".

The quality of Ruby is good.. This makes people aim for better quality than if
they were stuck with C++. Same with RPA.

Please understand that what I describe here is necessarily incomplete,
and that we'll have to get there one step at a time. We will also
modify the plan as we gain experience from the steps we take, and as
we get feedback from people using it.

AFAIK RPA is already pretty mature.. Im wondering why you say 'incomplete'?

First, let's look at this from the perspective of a developer that
does not use the RPA infrastructure directly.

When he has a release ready, he'll just put out a .gem. The Usual
Processes will catch the gem and add it to the gem list. At this
point, it's easily available through RPA interface too, and install
will integrate with RPA. However, some of the RPA features will not
be available for that (because various metadata and setup is missing
compared to a RPAfied package), and the user will be warned about this
and the lack of quality control.

Ah.. so when a package is first released then its just a .gem, and later
when its RPAfied.. it becomes an real rpa package.
This is nice.

A little while later (hours to days) the RPAfied source variant of the
package (the port update) will appear in the HEAD of RPA. This will
just require minimal testing; standard difference review from the last
release, testing that it package correctly on at least one platform
(more if we have suitable automation), checking for correct internal
version numbering if a library, and other small QA work. For a core
package (one that we have blessed as being "production quality and
maintained from RPA"), there will be somewhat more extensive QA for
updates. We'll probably also use various test branches etc to
streamline this process.

ok

There will also appear various mail aliases, a bug tracker, etc on the
RPA site, but these are the responsibility of the porter (RPA person)
rather than the original author. The standard procedure is to notify
the original author about these lists and of course allow her to take
some level of control of how they work (automatically gating to other
lists if wanted, etc), but RPA will make sure the well-known list
names are there for every package.

One thing that I find frustrating when installing a program, and I want to
report a bug or a feature.. that some projects doesn't have homepages,
nor valid mail adresses (though I haven't yet seen ruby projects like this).
It would be nice if there were a uniform way to report bugs, submit feature
requests and contact the package authors.

Mozilla can send crash-info automaticly to the mozilla-bug database.
Something similar could be useful too.

As maintenance is done through RPA, the porter will act as a liason to
the author, ensuring that the author get hold of new patches and
relevant bug reports, and trying to screen out the rest, giving a
"maximally relevant" feed to the author (controlled by the preferences
of the author).

ok

One of the problems with most packaging systems is that they distance
the user from the code. Getting a program from the archive into a
form where it is easy to hack on is noticeable amounts of work; often,
it involves going totally outside the archive and checking code out
from some unknown CVS repository somewhere, and using "some weird
unknown build/install procedure". We want RPA to do the opposite:
Make the distance from the package to being an active project
contributor on the code and/or documentation level *smaller* than it
is when starting from an author release. To help this, we'll try to
normalize the development environment.

sweet music in my eye.

To start working on a package, do 'rpa hack <portname>', and get a
checked out version of the port with version control metadata for a
distributed VC (at the moment it looks like we'll use DARCS, but we're
not quite sure. CVS does not work at all well for this task.) With
this, it will also be easy to give your changes back to RPA - probably
just 'rpa offer' from the workspace, or something like that. (We'll
look at the exact user interface later.)

Maybe it could be useful to know your reasons, why you have
chosen DARCS over subversion over CVS.
Im not that much into version control.. but I am curious to your reasons.

To get started with the software in it, look at the RPA_HACKING file
in that directory. It will be what notes the code review team of RPA
has made about that package, using our standard terminology etc (we'll
try to keep the style reasonably consistent.)

To get on the mailing list for the people working on that software
through RPA, using the mail aliases described above, use 'rpa
subscribe dev <package name>'

For an author that want to work more closely with RPA: You can just
hook yourself onto the infrastructure described above, initially
working in a sandbox branch, later becoming your own porter. You'll
get "free merges" to the software from other RPA committers/porters -
added test coverage, fixed documentation, etc.

I am strongly considering to become an RPA porter/committer.

···

On Friday 29 October 2004 19:03, Eivind Eklund wrote:

--
Simon Strandgaard

3

I think its good with competition between the RPA team and the GEMs team.
It results in faster development (suvival of the fittest). What I like is that
both teams follows two distinct goals, only with little overlap.
Sharing code and specs is good.

I agree. A reasonable amount of competition (while keeping a good
tone and cooperation where possible) provides incentive in both
directions.

> With the RubyGems commitment to becoming repackager
> friendly (and we try to give as much help as we can with that), we
> hope that using the RubyGems format for external author-packaged
> packages and the RPA format for RPAfied packages (with their extra
> requirements) will give an ideal result. Of course, for end user
> convenience, these will need to be available through the same user
> interface.

How far is RPA from reaching this point?

It's a likely rpa-base 0.3 feature. What blocks us from implementing
it immediately is the multiple version support in RubyGems; this needs
a little bit of code for support in rpa-base. The rest of the
features are trivially implementable atop the present rpa-base.

> Please understand that what I describe here is necessarily incomplete,
> and that we'll have to get there one step at a time. We will also
> modify the plan as we gain experience from the steps we take, and as
> we get feedback from people using it.

AFAIK RPA is already pretty mature.. Im wondering why you say
'incomplete'?

rpa-base (package management) for pure Ruby packages is mature. The
full RPA implementation is not yet in place. The complete
implementation needs to include quality guidelines, testing
guidelines, support infrastructure to make maintenance easy, culture
of maintaining packages that other people abandon (and we've grabbed
as part of the core set), etc.

RPA is really a set of promises, and a culture and set of
infrastructure to deliver on those promises. We are trying as well as
we can to get all of that in place - but as it is, we function mostly
as a very good package repository, with QA on entry, but without
fullfilling the role of adopting packages much.

> There will also appear various mail aliases, a bug tracker, etc on the
> RPA site, but these are the responsibility of the porter (RPA person)
> rather than the original author. The standard procedure is to notify
> the original author about these lists and of course allow her to take
> some level of control of how they work (automatically gating to other
> lists if wanted, etc), but RPA will make sure the well-known list
> names are there for every package.

One thing that I find frustrating when installing a program, and I want to
report a bug or a feature.. that some projects doesn't have homepages,
nor valid mail adresses (though I haven't yet seen ruby projects like this).
It would be nice if there were a uniform way to report bugs, submit feature
requests and contact the package authors.

That's what I'm trying to give above. It goes well beyond the
standard stuff that's done when packaging in all other systems I know
of (Debian does some of it, but not as complete),

Contacting the author may be a bit of a stretch; it's not quite clear
to me if it would be OK for us to generally set up an
<port>-author@rubyarchive.org alias (e.g, types-author@rubyarchive.org
-> eivind@FreeBSD.org) without specific author permission. What do
those of you out there think? Would it be good to do this?

Mozilla can send crash-info automaticly to the mozilla-bug database.
Something similar could be useful too.

That's a good idea. I'm not entirely sure how to implement it,
though. I'll try to keep it in the back of my mind and see what I can
do.

> To start working on a package, do 'rpa hack <portname>', and get a
> checked out version of the port with version control metadata for a
> distributed VC (at the moment it looks like we'll use DARCS, but we're
> not quite sure. CVS does not work at all well for this task.) With
> this, it will also be easy to give your changes back to RPA - probably
> just 'rpa offer' from the workspace, or something like that. (We'll
> look at the exact user interface later.)

Maybe it could be useful to know your reasons, why you have
chosen DARCS over subversion over CVS.
Im not that much into version control.. but I am curious to your reasons.

CVS lacks distributed development, decent branching, and rename support.
Subversion lacks distributed development.

If it weren't for the lack of distributed development (SVK does not
seem a stable option), we'd probably have gone with Subversion.
However, distributed development is a biggie. Without it, we are
effectively taking control away from the author. For some authors
(Alexander Kellett, me, and I assume others) this means s/he no longer
can work efficiently. For spare time projects, I'm very dependent on
offline development time, and I'm very dependent on doing diffs while
working, and doing frequent checkins. Effectively, I cannot do
development of things where I do the main development without having a
local commit line.

So, distributed development is about keeping the authors in control,
and making this easy to use for authors with various habits.

As for renames and branches: Branches are important because we plan to
releases that are maintained over time, and do some cross-maintenance.
This is possible to do with CVS, but it is somewhat painfuil,
especially in the presence of

Renames. Renames are fairly important to us, because we cannot assume
that all authors will follow the standard directory structure. By
normalizing the directory structure, we make it easier for somebody to
hack a new piece of software, and we make it *much* easier for people
that do cross-package maintenance - when somebody comes in and fix a
common security fault everywhere s/he can find it in RPA, for
instance.

> To get started with the software in it, look at the RPA_HACKING file
> in that directory. It will be what notes the code review team of RPA
> has made about that package, using our standard terminology etc (we'll
> try to keep the style reasonably consistent.)
>
> To get on the mailing list for the people working on that software
> through RPA, using the mail aliases described above, use 'rpa
> subscribe dev <package name>'
>
> For an author that want to work more closely with RPA: You can just
> hook yourself onto the infrastructure described above, initially
> working in a sandbox branch, later becoming your own porter. You'll
> get "free merges" to the software from other RPA committers/porters -
> added test coverage, fixed documentation, etc.

I am strongly considering to become an RPA porter/committer.

You'd be welcome to, of course. All the infrastructure isn't in place
yet, but we're approaching where we can take in more people.

And thank you for the reply - getting questioned is good :slight_smile:

Eivind.

···

On Tue, 2 Nov 2004 02:52:28 +0900, Simon Strandgaard <neoneye@adslhome.dk> wrote:
--
Hazzle free packages for Ruby?
RPA is available from http://www.rubyarchive.org/