Perl and the culture of libraries

http://blog.jrock.us/articles/You%20are%20missing%20the%20point%20of%20Perl.pod

Great article on perl and cpan. I was all ready to say "yeah, ruby has
libraries too - maybe not the tens of thousands cpan has, but i can
usually find what i want" until I read this paragraph:

   The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

He's right - there really is nothing in the ruby culture/toolset that
encourages everything shared to be *properly* shared. Hoe and friends
[http://nubyonrails.com/articles/tutorial-publishing-rubygems-with-hoe]
are a great step forward, but their use doesn't seem to be widespread
yet. At the very least, an interesting thing to ponder.

Some ideas I had:

1. rubyforge should encourage an ecosystem of scripts that works with
it; right now I can't even *find* such scripts other than via "gem
search" (I've filed a suggestion with rubyforge about this).
2. a majority of the projects on rubyforge (mine included, i admit)
have no documentation. on the other hand, the *landing page* of a cpan
project is the documentation. rubyforge's default landing page seems
to contain a lot of information that seems more suited to a sidebar.
3. perhaps rubygems could support commands like "gem readme", "gem
changelog", "gem todo" and "gem examples" to encourage people to fill
in such documentation where appropriate

martin

I'm on board.
Where do we start ?

Re-face RubyForge to apply the changes you suggest? Who's maintainer for
RubyForge ?

Peter Fitzgibbons
(847) 687-7646
Email: peter.fitzgibbons@gmail.com
IM GTalk: peter.fitzgibbons
IM Yahoo: pjfitzgibbons
IM MSN: pjfitzgibbons@hotmail.com
IM AOL: peter.fitzgibbons@gmail.com

···

On Tue, Aug 5, 2008 at 7:56 AM, Martin DeMello <martindemello@gmail.com>wrote:

http://blog.jrock.us/articles/You%20are%20missing%20the%20point%20of%20Perl.pod

Great article on perl and cpan. I was all ready to say "yeah, ruby has
libraries too - maybe not the tens of thousands cpan has, but i can
usually find what i want" until I read this paragraph:

  The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

He's right - there really is nothing in the ruby culture/toolset that
encourages everything shared to be *properly* shared. Hoe and friends
[http://nubyonrails.com/articles/tutorial-publishing-rubygems-with-hoe\]
are a great step forward, but their use doesn't seem to be widespread
yet. At the very least, an interesting thing to ponder.

Some ideas I had:

1. rubyforge should encourage an ecosystem of scripts that works with
it; right now I can't even *find* such scripts other than via "gem
search" (I've filed a suggestion with rubyforge about this).
2. a majority of the projects on rubyforge (mine included, i admit)
have no documentation. on the other hand, the *landing page* of a cpan
project is the documentation. rubyforge's default landing page seems
to contain a lot of information that seems more suited to a sidebar.
3. perhaps rubygems could support commands like "gem readme", "gem
changelog", "gem todo" and "gem examples" to encourage people to fill
in such documentation where appropriate

martin

I've always liked the Perl documentation conventions. Particularly
the fact that *every* CPAN manpage starts with a Synopsis which
demonstrates the essential usage of the library in a few lines of
code.

···

On Tue, Aug 5, 2008 at 8:56 AM, Martin DeMello <martindemello@gmail.com> wrote:

  The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

--
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

http://blog.jrock.us/articles/You%20are%20missing%20the%20point%20of%\.\.\.

Great article on perl and cpan. I was all ready to say "yeah, ruby has
libraries too - maybe not the tens of thousands cpan has, but i can
usually find what i want" until I read this paragraph:

There are about 14,000 libraries on CPAN, not "tens of thousands".
There are about 8,000 Ruby libraries between RubyForge and the RAA.

The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

Between RubyForge, RDoc and Rubygems I think we've got a pretty good
culture going. He mostly seems to be complaining about the fact that
some Rails users have a habit of posting code online instead of
packaging it.

He's right

<snip>

He's wrong. He's also a Perl programmer deeply steeped in Perl (has
many modules on CPAN), is not a polyglot programmer as far as I can
tell, and has a book on Catalyst to sell you.

Plus, 212 modules? Oof.

Regards,

Dan, former Perl programmer

···

On Aug 5, 6:56 am, "Martin DeMello" <martindeme...@gmail.com> wrote:

(excerpt quoted from the linked article)

   The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

This is somewhat true, and somewhat not. I think the best way to encourage it
would be to build a public, web-facing index of gems, and evolve some
community pressure. It could be as simple as a rating system -- actually
embarrass people for putting up poorly-documented, poorly-tested code.

Some automated tools could help with that, but this is really a social
project. As you said:

Hoe and friends
[http://nubyonrails.com/articles/tutorial-publishing-rubygems-with-hoe\]
are a great step forward, but their use doesn't seem to be widespread
yet.

Before I start my rant, I'm going to provide my suggestion. I think this is
more important than the rant, and I realize some people will get bored.

I suggest that we work on improving the gem system itself. Specifically, we
need to make it as flexible, powerful, and _easy_ as possible, so that people
stop developing things like Rails Plugins.

Right away, the biggest feature I miss is reverse dependencies. (Not that CPAN
had them...) I want to be able to safely install a gem, test it out, and
remove it, and not have to go hunting in my gems directory for other things
I'd like to remove.

Second biggest would be virtual dependencies -- only second biggest because I
don't know if they exist or not. If not, they should. I should be able to
depend on "any Rack engine", not just Rack itself, and certainly not specific
things like Mongrel or Thin.

Now, there is another disturbing thing about Ruby, at least, coming from a
formerly-Perl perspective:

Perl encourages people to modularize. CPAN packages are (at least, in my
limited experience) consistently named, and the actual namespace within the
language generally matches the name of the package on CPAN.

In Ruby, I've mostly seen polar opposites: Either tiny little gems with few
external dependencies, which will play nice with almost anything -- metaid is
an extreme example -- or HUGE gems like Rails.

Yes, I realize Rails is broken up into several gems. Well, three of them,
really -- actionpack, activerecord, and activesupport.

I'm going to argue that activesupport should be at least 10, maybe 20
different packages -- there are so many little pieces of it that I wish I
could use for a one-off script, without pulling in the whole library. Things
like symbol-to-proc, 3.days.ago, etc. But it's going to add significantly to
the startup time of my script even to require activesupport -- that's bloat.

And ActiveRecord -- why, exactly, is validation included? Why not make it a
mixin? That way, with duck typing, I could pull at least some of the default
Rails validations into another ORM. (validates_uniqueness_of might even work,
if the #find methods are similar.)

Things like Rack are a refreshing step forward -- but there are very few
things like Rack.

In Perl, there's an XML namespace, and most XML things are done through
XML::Parser, which provides a common frontend to several actual XML parsers.
The actual interfaces to those parsers -- that is, the pure Perl one, the
LibXML one, and the expat one -- are all separate modules.

Because of this, new things like XML::SAX can be written to fairly easily plug
into all of those libraries.

In Ruby, well, rfeedparser has its own interfaces to the expat gem, the libxml
gem (whatever their names are), and its own internal (sloppy) parser, in case
the others can't be found. Anyone else who just wants to parse XML is going
to have to do the same thing -- or worse, just pick a library.

As long as I'm ranting, may as well talk about plugins. I know about
gem_plugin -- why isn't it used more? Why, instead, are plugins distributed
as things tightly-bound to Rails? Suppose I like ActiveRecord, and I need
has_many_polymorphs -- if I'm in Merb, there's no clear indication of whether
that will work or not.

Well, that, and plugins are a whole separate (and inferior!) package
management system that I have to learn, in addition to gems. The one
advantage I can buy is that you can then distribute your entire, working app
to a web host who simply has Ruby, no external dependencies required -- but
you'd still have to freeze Rails to your app to accomplish that. Why not
freeze other gems, too?

Here's a crazy idea (not sarcasm; I realize it's quite possibly insane): When
you build a framework like Rails, designed to aid in rapid development of an
app, design it around building components, not whole applications. That is,
design it around building gems -- possibly lots of tiny gems (would one per
model be too much?) -- so that when it comes time to "extract functionality"
from your app, you'll already know what depends on what, and what you'll need
to disentangle.

If you made it this far, thanks for reading. Hope I made some kind of sense.

···

On Tuesday 05 August 2008 07:56:22 Martin DeMello wrote:

He's right - there really is nothing in the ruby culture/toolset that
encourages everything shared to be *properly* shared. Hoe and friends
[http://nubyonrails.com/articles/tutorial-publishing-rubygems-with-hoe\]
are a great step forward, but their use doesn't seem to be widespread
yet. At the very least, an interesting thing to ponder.

That's (partly) because Rubygems makes it so easy to "release" small
"libraries" (really naescent bundles of code that may or may not prove
useful). The benefit is that there's more code out there; the problem
is it dilutes the ecosystem. Not all gems are equally deserving of
the effort required to release them *properly*. Accordingly, the gem
tool should not treat all gems equally as it does now.

On Rubyforge there's a good piece of metadata: the status of the
project (rated 1-5 with associated descriptions like "Beta" and stuff
-- going from distant memory here). I think the ecosystem would
benefit if gems had something like this built in, and maybe a free-
text description of the status, like "Small but useful library; not
really documented yet." These things would help users sift through
the mass of gems, and enable us to get the benefit of an active
ecosystem without the problems of dilution.

Some ideas I had: [...]

3. perhaps rubygems could support commands like "gem readme", "gem
changelog", "gem todo" and "gem examples" to encourage people to fill
in such documentation where appropriate

Excellent. I proposed the same thing (well, the "readme" part,
anyway) on the Rubygems list a long time ago. It didn't get any
support. Now that the platform has proven itself and there are
thousands of gems, surely it's time to consider this and other related
ideas.

A related suggestion I had back then was a second command "gemx" that
would allow other useful (perhaps experimental) commands without
polluting the "gem" command namespace.

Gavin

···

On Aug 5, 10:56 pm, Martin DeMello <martindeme...@gmail.com> wrote:

I think a minimally invasive beginning would be a documentation
skeleton generator. Rdoc is wonderful for what it does, but it doesn't
provide much "convention-over-configuration" support for
non-source-code-based documentation, and if Rails has done anything,
it has proven the value of a good set of conventions.

martin

···

On Tue, Aug 5, 2008 at 6:04 AM, Peter Fitzgibbons <peter.fitzgibbons@gmail.com> wrote:

I'm on board.
Where do we start ?

i have to agree with this. one of the main goals of my main.rb lib was to be able to enable *every* program to have a '--help' option, and to allow writing programs that look like this

Main {

   description "

      stuff about the great program

   "

   def run
     do_stuff
   end

}

because a script that does not declare what it does in the first 10 lines is evil and a script that does not have '--help' doubly so.

a @ http://codeforpeople.com/

···

On Aug 5, 2008, at 8:46 AM, Avdi Grimm wrote:

I've always liked the Perl documentation conventions. Particularly
the fact that *every* CPAN manpage starts with a Synopsis which
demonstrates the essential usage of the library in a few lines of
code.

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

No, he really is right. I'm not coming at this from the perspective of
someone who bought into his entire article, just the one paragraph I
quoted. After which, I went and picked five random packages from
CPAN's category tree, and five from rubyforge's. Of the five from
rubyforge, four had no documentation, and one had a link to an
external site which was either down or no longer available. I'm not
pointing fingers; my own projects are in no better shape. I'm just
saying that we as a community don't have the documentation reflex, and
that the perlers do.

martin

···

On Tue, Aug 5, 2008 at 1:46 PM, Daniel Berger <djberg96@gmail.com> wrote:

He's right

<snip>

He's wrong. He's also a Perl programmer deeply steeped in Perl (has
many modules on CPAN), is not a polyglot programmer as far as I can
tell, and has a book on Catalyst to sell you.

I'm the current maintainer for RubyForge; the site/hardware/domain/etc are all owned by Ruby Central.

Yours,

Tom

···

On Aug 5, 2008, at 9:04 AM, Peter Fitzgibbons wrote:

I'm on board.
Where do we start ?

Re-face RubyForge to apply the changes you suggest? Who's maintainer for
RubyForge ?

I've seen some things posted in this thread (overnight for me) that I
mildly disagree with. So, pardon me while I insert my opinion briefly
without quoting:

1) I'm not convinced that packages and applications need to be treated
differently in this discussion. The size of the documentation depends
on the complexity of the thing being documented, obviously; but the
way the documentation looks and behaves -- at least to the user --
should probably be the same.

2) I'm not convinced that CPAN does documentation any better than
rubyforge/gem does. This isn't a matter of having standard fields or
of rewriting our packaging system. We actually have to get our hands
dirty and write in ... you know ... English.

3) The fact that ls and grep are used by most people without reading
the documentation (assuming that it's true) is not an excuse for ls
and grep not to have documentation. Go ahead and read the man pages
for ls and grep. Notice how well they explain how to use the
commands.

···

On Wed, Aug 6, 2008 at 3:48 AM, David Masover <ninja@slaphack.com> wrote:

On Tuesday 05 August 2008 07:56:22 Martin DeMello wrote:
(excerpt quoted from the linked article)

   The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

This is somewhat true, and somewhat not. I think the best way to encourage it
would be to build a public, web-facing index of gems, and evolve some
community pressure. It could be as simple as a rating system -- actually
embarrass people for putting up poorly-documented, poorly-tested code.

Some automated tools could help with that, but this is really a social
project. As you said:

Hoe and friends
[http://nubyonrails.com/articles/tutorial-publishing-rubygems-with-hoe\]
are a great step forward, but their use doesn't seem to be widespread
yet.

Before I start my rant, I'm going to provide my suggestion. I think this is
more important than the rant, and I realize some people will get bored.

I suggest that we work on improving the gem system itself. Specifically, we
need to make it as flexible, powerful, and _easy_ as possible, so that people
stop developing things like Rails Plugins.

Right away, the biggest feature I miss is reverse dependencies. (Not that CPAN
had them...) I want to be able to safely install a gem, test it out, and
remove it, and not have to go hunting in my gems directory for other things
I'd like to remove.

Second biggest would be virtual dependencies -- only second biggest because I
don't know if they exist or not. If not, they should. I should be able to
depend on "any Rack engine", not just Rack itself, and certainly not specific
things like Mongrel or Thin.

Now, there is another disturbing thing about Ruby, at least, coming from a
formerly-Perl perspective:

Perl encourages people to modularize. CPAN packages are (at least, in my
limited experience) consistently named, and the actual namespace within the
language generally matches the name of the package on CPAN.

In Ruby, I've mostly seen polar opposites: Either tiny little gems with few
external dependencies, which will play nice with almost anything -- metaid is
an extreme example -- or HUGE gems like Rails.

Yes, I realize Rails is broken up into several gems. Well, three of them,
really -- actionpack, activerecord, and activesupport.

I'm going to argue that activesupport should be at least 10, maybe 20
different packages -- there are so many little pieces of it that I wish I
could use for a one-off script, without pulling in the whole library. Things
like symbol-to-proc, 3.days.ago, etc. But it's going to add significantly to
the startup time of my script even to require activesupport -- that's bloat.

And ActiveRecord -- why, exactly, is validation included? Why not make it a
mixin? That way, with duck typing, I could pull at least some of the default
Rails validations into another ORM. (validates_uniqueness_of might even work,
if the #find methods are similar.)

Things like Rack are a refreshing step forward -- but there are very few
things like Rack.

In Perl, there's an XML namespace, and most XML things are done through
XML::Parser, which provides a common frontend to several actual XML parsers.
The actual interfaces to those parsers -- that is, the pure Perl one, the
LibXML one, and the expat one -- are all separate modules.

Because of this, new things like XML::SAX can be written to fairly easily plug
into all of those libraries.

In Ruby, well, rfeedparser has its own interfaces to the expat gem, the libxml
gem (whatever their names are), and its own internal (sloppy) parser, in case
the others can't be found. Anyone else who just wants to parse XML is going
to have to do the same thing -- or worse, just pick a library.

As long as I'm ranting, may as well talk about plugins. I know about
gem_plugin -- why isn't it used more? Why, instead, are plugins distributed
as things tightly-bound to Rails? Suppose I like ActiveRecord, and I need
has_many_polymorphs -- if I'm in Merb, there's no clear indication of whether
that will work or not.

Well, that, and plugins are a whole separate (and inferior!) package
management system that I have to learn, in addition to gems. The one
advantage I can buy is that you can then distribute your entire, working app
to a web host who simply has Ruby, no external dependencies required -- but
you'd still have to freeze Rails to your app to accomplish that. Why not
freeze other gems, too?

Here's a crazy idea (not sarcasm; I realize it's quite possibly insane): When
you build a framework like Rails, designed to aid in rapid development of an
app, design it around building components, not whole applications. That is,
design it around building gems -- possibly lots of tiny gems (would one per
model be too much?) -- so that when it comes time to "extract functionality"
from your app, you'll already know what depends on what, and what you'll need
to disentangle.

If you made it this far, thanks for reading. Hope I made some kind of sense.

--
All you can do is try to know who your friends are as you head off to
the war / Pick a star on the dark horizon and follow the light

Being a relative newcomer to Ruby and coming off of Perl, this is
something I'd love to see in many gems. There are so many gems I'd
love to use but can't figure out by reading the documentation (or lack
thereof), and I don't really want to read through all the source just
to use it in two lines of code.

-- Yanik Magnan • http://r-ch.net

···

On Tue, Aug 5, 2008 at 10:46 AM, Avdi Grimm <avdi@avdi.org> wrote:

On Tue, Aug 5, 2008 at 8:56 AM, Martin DeMello <martindemello@gmail.com> wrote:

  The important thing about Perl is that we have a culture of writing
good libraries. No Perl programmer would write a few lines of code,
post it to a blog, and call it a "library". Everyone feels obligated
to create a CPAN distribution, with documentation (sometimes a bit on
the minimal side, but not everyone is a writer), a test suite, a
Makefile, etc. I'm not sure why, but this always happens. I think it's
because there is a strong convention, and tools that make following
the convention easy.

I've always liked the Perl documentation conventions. Particularly
the fact that *every* CPAN manpage starts with a Synopsis which
demonstrates the essential usage of the library in a few lines of
code.

--
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

And ActiveRecord -- why, exactly, is validation included? Why not make it a
mixin? That way, with duck typing, I could pull at least some of the default
Rails validations into another ORM. (validates_uniqueness_of might even work,
if the #find methods are similar.)

There is an experimental project to pull parts of ActiveRecord (e.g.
validations)
into external mixins that can be shared with other model layers:

http://dev.rubyonrails.org/svn/rails/tags/rel_2-0-1/activemodel/README

"Also note that because this module MUST be able to stand alone (for
use in presenters etc), it does not COMPLETELY supplant
ActiveRecord::Validations. AR-specific stuff like
the :on=>:save|:update|:create option, as well as
validates_uniqueness_of are taken care of by AR:V, albeit by extending
AM:V"
http://www.ruby-forum.com/topic/158973

Here's a crazy idea (not sarcasm; I realize it's quite possibly insane): When
you build a framework like Rails, designed to aid in rapid development of an
app, design it around building components, not whole applications. That is,
design it around building gems -- possibly lots of tiny gems (would one per
model be too much?) -- so that when it comes time to "extract functionality"
from your app, you'll already know what depends on what, and what you'll need
to disentangle.

There have been some forays into leveraging more advantage of gems in rails:

http://ryandaigle.com/articles/2008/4/1/what-s-new-in-edge-rails-gem-dependencies
http://www.intridea.com/2008/6/11/gemplugins-a-brief-introduction-to-the-future-of-rails-plugins

···

On Tue, Aug 5, 2008 at 10:48 PM, David Masover <ninja@slaphack.com> wrote:

Sounds like some changes to rubygems and it's toolset.

Peter Fitzgibbons
(847) 687-7646
Email: peter.fitzgibbons@gmail.com
IM GTalk: peter.fitzgibbons
IM Yahoo: pjfitzgibbons
IM MSN: pjfitzgibbons@hotmail.com
IM AOL: peter.fitzgibbons@gmail.com

···

On Tue, Aug 5, 2008 at 8:23 AM, Martin DeMello <martindemello@gmail.com>wrote:

On Tue, Aug 5, 2008 at 6:04 AM, Peter Fitzgibbons > <peter.fitzgibbons@gmail.com> wrote:
> I'm on board.
> Where do we start ?

I think a minimally invasive beginning would be a documentation
skeleton generator. Rdoc is wonderful for what it does, but it doesn't
provide much "convention-over-configuration" support for
non-source-code-based documentation, and if Rails has done anything,
it has proven the value of a good set of conventions.

martin

RDoc:Usage extracts the comment at the top of your main program as your help message (after stripping the '#' characters).

I'm not sure if it is still working, given the changes to RDoc, but as a concept I still think it's a nice way to do it: you can read the source, or do --help, and get the same information, and all without any extra stuff in your source file.

Dave

···

On Aug 5, 2008, at 11:57 AM, ara.t.howard wrote:

i have to agree with this. one of the main goals of my main.rb lib was to be able to enable *every* program to have a '--help' option, and to allow writing programs that look like this

A critical thing for me is that when programming Perl the POD usually contains sufficient information and *EXAMPLES OF USAGE* to throw the basics of an application by simply cutting and pasting from the POD. The Ruby documentation is wonderful when you know what you are doing and just want to brush up on the details but practically useless if you are new to it. This discourages people who might have an interest in the project but don't have the time or skills to work out what is going on from the source.

Just as an example ANTFARM for just announced, the synopsis from the post was:

NAME

ANTFARM

DESCRIPTION

Passive network mapping tool.

URI

http://antfarm.rubyforge.org
http://rubyforge.org/projects/antfarm

INSTALL

gem install ANTFARM

HISTORY

0.2.0
  - Initial release as open source

SYNOPSIS

$> antfarm help

Not discouraged I went to the project page which was just as helpful. Then I walked away. Now ANTFARM maybe the coolest project there is (it recalled to me the use of ALife 'ants' to optimise network routing which was why it piqued by interest) but I don't have a fucking clue about how to do anything with this project. Other than you initialise it with "antfarm db —initialize". Sometimes people shoot themselves in the foot, others seem to go for the head.

I'm not picking on the ANTFARM team in particular there are plenty of projects, both Ruby and non Ruby, that assume that you are psychic or like wading through the source code as some sort of rite of passage. This project seems to be completely free of vapour judging by the amount of code that has been developed but it is almost as if they are actively trying to drive potential users away.

*EXAMPLES OF USAGE* people, were need more of them!

Hi,

Shadowfirebird wrote:

2) I'm not convinced that CPAN does documentation any better than
rubyforge/gem does. This isn't a matter of having standard fields or of
rewriting our packaging system. We actually have to get our hands dirty
and write in ... you know ... English.>

You are somewhat right. Of course no or little documentation is a general
problem at all, not a ruby or rubyforge/gem specific one. But the main
difference is, how CPAN encourages the developer to provide a
documentation too: The documentation is the first thing you see and
searching on CPAN is searching the documentation. And by reading the
documentation top-down you very quickly (often after a couple of lines)
find out, what the lib/package is about.

Every developer on CPAN uses CPAN himself and sees the benefits it brings
to him when he is using CPAN as a user of libs from other users.

When I try to find something on rubyforge, the first thing that drives me
nuts is, that I find a name and a more, but often less informative
description, and very often no code. That last never happens on CPAN, at
least I have not seen something like that until now.

So after finding out, which projects really exist, you cannot browse the
documentation and only very few of the projects provide a more or less
informative project-homepage. And if they provide a homepage, each looks
different and for each one you have to find out, where the information
is, you are looking for. (There are even people, who use blog-entries as
homepages).

So to decide, whether the project could be useful to me, I have to
install it and read the generated documentation. So my CPAN for ruby is
'gem server'.

But even here the documentation is very different in quality. You can be
very happy, if most of the methods are documented, but even then, it is
very often hard to find out the big picture.

So at the end you spent a lot of time and you are not quite sure, whether
you found the right package since this depends strongly on the quality of
the description and wether they then show up in the search you did.

On the other end, browsing CPAN is real fun, even if you are not
searching for something. And this is because (I think) the way CPAN
presents the packages. If you publish something here, you do not want to
be ashamed for not to provide at least a minimal useful documentation.
Big projects even provide tutorials etc., which provide even more then
just a brief API-documentation.

But when I write all this, it will become more and more clear to me, that
it makes no sense to blame rubyforge for this, since as the name
indicates, it is more ment to be something like sourceforge than CPAN. So
what Ruby is missing is at all, is something, that might be called CRbAN.

Regards,
Thomas

I've seen some things posted in this thread (overnight for me) that I
mildly disagree with. So, pardon me while I insert my opinion briefly
without quoting:

1) I'm not convinced that packages and applications need to be treated
differently in this discussion.

I'll agree with that -- but then, I was the one proposing that applications
should be split into packages.

Perl (again, in my limited experience) seems to have very few "applications"
as such -- rather, everything is structured into some sort of "library", and
the "application" itself is a script of less than 100 lines. mod_perl only
encourages this.

Ruby, on the other hand, makes it very easy to inline things -- you can have a
single file with a few thousand lines of code, representing a somewhat
complex class hierarchy -- and people do (Rails does, often.)

On top of that, barring plugins, Rails tends to encourage putting all
functionality into your app, as app-specific, until you can figure out what
to plugin-ize -- and almost no one namespaces anything. (Example: It may have
been fixed by now, but last time I tried namespace'd models, the default XML
output was invalid.)

2) I'm not convinced that CPAN does documentation any better than
rubyforge/gem does.

It probably doesn't. It does, however, encourage documentation.

Maybe I'm unusual, but the first thing I do when I want a CPAN module is, I
head over to search.cpan.org. This works surprisingly well, and once I do
find a module, the very first thing I see is a README, and the very first
thing in that README is a "Hello, World" example.

Now, other people have talked about the Synopsis. I'm also talking about
the "search" functionality. Part of making this obvious is emphasizing how
people find out about gems.

This isn't a matter of having standard fields or
of rewriting our packaging system. We actually have to get our hands
dirty and write in ... you know ... English.

Hopefully, nothing I've suggested needs a full rewrite...

And I agree. The trick is, how do we embarrass people into writing that
English? We can't force them to, and we can't simply ignore their library
(how would they know?) -- my favorite idea so far is to drag them into the
cold, harsh light of crowd review, and allow ratings and reviews on
Rubyforge.

···

On Wednesday 06 August 2008 04:07:48 Shadowfirebird wrote:

"Gem-based development" sounds very close to what I meant -- but there doesn't
seem to be much mention of how one might go about this as a _process_, only a
few tools that might be useful.

Gem dependencies are nice, but all that does is encourage the use of gems
rather than rolling your own functionality. I don't think this was ever a
problem -- do people actually roll their own HTML parsers to use with Rails?
At least the people I work with seem to be smart enough to use things like
hpricot, or rfeedparser.

Gem plugins is a step in the right direction -- but it's still not
particularly easy, compared to simply developing a plugin as an SVN external
(or with piston) -- or simply adding another model to your existing app.

And if you look at the name, it seems very much like "gem plugins" are simply
an approach to making plugins easier. It's still an approach of plugging
something into an existing app.

I'm not sure what is needed, technologically, to make this happen -- maybe
it's simply better scaffolding -- but I'm thinking that about the only thing
that should be in the application is the routing. Everything else could be
split off into re-usable components -- and at a fine granularity.

Of course, that's my opinion, and it runs directly contrary to the 37signals
approach, which seems to be: Develop a monolithic app first, and don't
abstract anything until you've repeated it at least three times. Re-usable
components, in that model, are created by extracting them from an existing
app.

···

On Saturday 09 August 2008 10:20:27 brabuhr@gmail.com wrote:

There have been some forays into leveraging more advantage of gems in rails:

Not really. For instance, if I had a small program, doc_init.rb, that
simply spat out a file project-doc.txt that had the following

···

On Tue, Aug 5, 2008 at 6:31 AM, Peter Fitzgibbons <peter.fitzgibbons@gmail.com> wrote:

Sounds like some changes to rubygems and it's toolset.

----------------------------------------------
Project Name:

Authors:

Short Description:

Long Description:

Usage:

Notes:

----------------------------------------------

it would (a) kickstart the documentation process and (b) give other
tools a standard set of headings to look for (c) give authors a
psychological incentive to include those sections in their
documentation (all of which are useful). Once this achieves a critical
mass, we can ask the rubyforge maintainers to consider looking for a
project-doc.txt in projects and make that the landing page. That in
turn will make sure people who submit to rubyforge include such a
file. It just requires a bit of initial buy-in.

martin