Suggestions to the Ruby community

bbense+comp.lang.ruby.Sep.06.02@telemark.stanford.edu writes:

I could brute force this by running rdoc on all the installed
ruby src and then grepping the XML… I’d kind of like a more
elegant solution. How were you planning to generalize ri from
rdoc output?

Yes.

Also it would be easy to have an rdoc mode that lets you do

rdoc fred.rb

and have the documentation pop out.

Dave

    • Well as I see it there are two things missing in the ruby doc
      world.
    1. A standard output format

I really don’t believe this is possilbe these days while keeping
everybody happy. ‘Nixers’ prefer man (very handy) or even
/bin/cat/foo.rb (or even /bin/cat/foo.rb | grep “def bar” to get the
args); In Windows, help files are preferred. HTML is a cross platform
compromise.

  1. A standard place to find it.

This is a big issue in a number of ways. There is no real offline
standard location (e.g. /usr/local/lib/Ruby/1.7/docs) and while there is
much documentation online, it is all over the place and finding it can
be a real challenge. Cerainly, there is nothing on the order of CPAN. I
think this is the major point contained in stibbs original posting.

    • rdoc does not solve either of these problems. POD solves them
      by

    a. being trivial

Meaning the inline docs are easy to write?

b. Always being between the =begin and =end part of the
installed module.

This is good for general parsing and, of course for /bin/cat, although
not necessary.

No documentation system is going to work well unless (1) creating the
docs is sufficiently easy for the programmer as to encourage (with a
little peer pressure applied) everyone to do it and (2) the issues of #2
above are effectively addressed. Both of these are organization and
social engineering concerns more than they are technical IMO.

Once you have these two things, you can write all kinds of
document tools. In some ways rdoc is “just too damn smart”,
it takes plain code and produces useful documentation. However,
since the rdoc “src” is scattered all over the original src
code, getting docs on the fly is somewhat difficult.

Well, actually, I see this as a definite advantage to RDoc; a file
totally devoid of any programmer-generated documentation can still be
parsed out to provide a meaningful summary of structure and the
relationships among classes, methods, etc. albeit very general and
incomplete.

An interesting thought, as an aside, would be a kind of
content-management system based on RDoc that would fold comments,
explanations, extensions, etc. back into the source, in the appropriate
places as documentation. This might make post-documentation easier for
the module writer, streamline public documentation projects on major
modules and provide a kind of notes and annotation tool for individual
users of a given module. Just a thought, but, IMO a close to “killer
app” one, or a least a “mini-killer” :slight_smile:

I’ve taken a closer look at rd and it at least knows enough
Ruby to differentiate method lists from regular lists.

Originally, my personal differentiation between RD and RDoc is that the
former is preferable for end-user documentation in the form of man pages
and/or HTML for those who need to get the app up and running quickly
while the latter is better for developers and/or library modules where
the end user is a developer who needs a very thorough and in-depth
understanding of the workings, structure and interrelationships of the
code.

However, I don’t see why RDoc cannot be enhanced to provide a choice of
either, depending upon one’s needs, possibly through command line args?

    • If you want to emulate perldoc, then you could use XML as
      and intermediate format, but IMHO, XML is unreadable and
      there’s the question of where to put it. It also raises
      the bar for tool writers, although I guess XML is now ubiquitous
      enough that is not that much of an issue.
    • I guess I’m just enough of a Unix Luddite that I think there’s
      a significant advantage to having a document format
      that I can use /bin/cat on. The other advantages I see are
  1. Play nice with the people already happily using rdtool.

I think RDoc already does. Is that right, Dave?

  1. Possible to include all the tools required to do documentation
    in a ruby-only code with the standard distribution.

This, I think, is imperative. IIRC Matz has already agreed to include
RDoc as part of the standard distribution.

    • Perhaps, we could clarify the situation with a couple use
      cases, I’d like a tool that could answer these questions.

      “Show me all the installed library objects that have
      a read method.”

      “Show me the names of all the installed classes.”

Well, if RDoc generates XML, then the result can be conditionally parsed
as required. In 'nix you would send the RDoc XML to stdout and pipe that
through your parsing tool like rdoc foo.rb | parsetool.rb -g .read - |
less or whatever. A nice GUI could be created for the Windows users :slight_smile:

    • I could brute force this by running rdoc on all the installed
      ruby src and then grepping the XML… I’d kind of like a more
      elegant solution. How were you planning to generalize ri from
      rdoc output?

Why do that when you can write a parser in Ruby, maybe using REXML? :slight_smile:

Regards,

Kent Starr

···

On Fri, 2002-09-06 at 11:31, bbense+comp.lang.ruby.Sep.06.02@telemark.stanford.edu wrote:

In article 1031334976.12271.675.camel@starcrusher,

    • Well as I see it there are two things missing in the ruby doc
      world.
    1. A standard output format

I really don’t believe this is possilbe these days while keeping
everybody happy. ‘Nixers’ prefer man (very handy) or even
/bin/cat/foo.rb (or even /bin/cat/foo.rb | grep “def bar” to get the
args); In Windows, help files are preferred. HTML is a cross platform
compromise.

    • It certainly is possible. POD shows us the way. I think the
      point I’m trying to make is that POD is an output format.
      It just happens to be so trivial that it’s accepted in the
      perl community to write raw POD. Once you have a well defined
      simple output format, you can translate that to any other format
      required.

    I think we all want to get to the point where when you install
    ruby you can type

     make install-man
    

    or

     make install-html
    

    or even

     make install-winhelp
    
    • If you pick a relatively neutral format that everybody uses
      it’s pretty straightforward to get to this goal. In the ruby
      community we have this really great tool for helping us write
      our version of POD, RDoc. To me the question is whether we
      choose rd or XML as the “intermediate” output format.
  1. A standard place to find it.

This is a big issue in a number of ways. There is no real offline
standard location (e.g. /usr/local/lib/Ruby/1.7/docs) and while there is
much documentation online, it is all over the place and finding it can
be a real challenge. Cerainly, there is nothing on the order of CPAN. I
think this is the major point contained in stibbs original posting.

    • CPAN is not a document archive, it’s a source archive. It just
      happens that the most common place to find perl documentation
      is imbedded in the src.
    • rdoc does not solve either of these problems. POD solves them
      by

    a. being trivial

Meaning the inline docs are easy to write?

    • Write, read, slice and dice into your favorite output format.
      We have a big advantage in that we have this really nifty tool
      to help us already in writing inline docs.

b. Always being between the =begin and =end part of the
installed module.

This is good for general parsing and, of course for /bin/cat, although
not necessary.

No documentation system is going to work well unless (1) creating the
docs is sufficiently easy for the programmer as to encourage (with a
little peer pressure applied) everyone to do it and (2) the issues of #2
above are effectively addressed. Both of these are organization and
social engineering concerns more than they are technical IMO.

    • Well, the only reason that 99% of perl modules have
      documentation is that the easiest way to get started
      writing a module is to run a program that sets up the
      structure for you. One of the things this program does
      is write a dummy doc skeleton with embarassing comments
      like

    "Stub documentation for Foo, created by h2xs. It looks like the
    author of the extension was negligent enough to leave the stub
    unedited. "

    Perhaps we should add this feature to rpkg.

Once you have these two things, you can write all kinds of
document tools. In some ways rdoc is “just too damn smart”,
it takes plain code and produces useful documentation. However,
since the rdoc “src” is scattered all over the original src
code, getting docs on the fly is somewhat difficult.

Well, actually, I see this as a definite advantage to RDoc; a file
totally devoid of any programmer-generated documentation can still be
parsed out to provide a meaningful summary of structure and the
relationships among classes, methods, etc. albeit very general and
incomplete.

    • It’s an advantage if we use it properly. I agree that it’s a
      great tool, there’s nothing even close in the perl world.

An interesting thought, as an aside, would be a kind of
content-management system based on RDoc that would fold comments,
explanations, extensions, etc. back into the source, in the appropriate
places as documentation. This might make post-documentation easier for
the module writer, streamline public documentation projects on major
modules and provide a kind of notes and annotation tool for individual
users of a given module. Just a thought, but, IMO a close to “killer
app” one, or a least a “mini-killer” :slight_smile:

    • That’s an extension of my idea. Rdoc next to the methods with
      a post processing step of putting rd between the =begin =end
      tags.

I’ve taken a closer look at rd and it at least knows enough
Ruby to differentiate method lists from regular lists.

Originally, my personal differentiation between RD and RDoc is that the
former is preferable for end-user documentation in the form of man pages
and/or HTML for those who need to get the app up and running quickly
while the latter is better for developers and/or library modules where
the end user is a developer who needs a very thorough and in-depth
understanding of the workings, structure and interrelationships of the
code.

    • Well, my take is that rd is an output format and RDoc is an
      input one.

[snip]

    • Perhaps, we could clarify the situation with a couple use
      cases, I’d like a tool that could answer these questions.

      “Show me all the installed library objects that have
      a read method.”

      “Show me the names of all the installed classes.”

Well, if RDoc generates XML, then the result can be conditionally parsed
as required. In 'nix you would send the RDoc XML to stdout and pipe that
through your parsing tool like rdoc foo.rb | parsetool.rb -g .read - |
less or whatever. A nice GUI could be created for the Windows users :slight_smile:

    • Certainly, that’s an alternative. It wouldn’t be my first choice.
    • I could brute force this by running rdoc on all the installed
      ruby src and then grepping the XML… I’d kind of like a more
      elegant solution. How were you planning to generalize ri from
      rdoc output?

Why do that when you can write a parser in Ruby, maybe using
REXML? :slight_smile:

    • I was using grep in the “generic” sense, but if people think
      XML on the fly is adequate, then I see at least one big
      problem. Rightly or Wrongly, the file name of a ruby module
      has no relation to the classes included. Perl ( and I think
      Java ) have “solved” this problem by using the convention
      that filenames and modulenames must be in sync[1]. This makes
      finding the embedded documentation much simpler. In the
      current ruby world you would need to run rdoc on every
      installed library to find the answer to “any” question.
      rdoc is fast, but I don’t think it’s that fast. Perhaps,
      the solution is to big a global index for classnames/methods,
      XML is probably the correct solution there.
    • Booker C. Bense

[1]- Well, at least all the classes in a module should be
subclasses of the main module name.

···

W Kent Starr elderburn@mindspring.com wrote:

On Fri, 2002-09-06 at 11:31, >bbense+comp.lang.ruby.Sep.06.02@telemark.stanford.edu wrote:

bbense+comp.lang.ruby.Sep.07.02@telemark.stanford.edu writes:

It certainly is possible. POD shows us the way. I think the
point I’m trying to make is that POD is an output format.
It just happens to be so trivial that it’s accepted in the
perl community to write raw POD. Once you have a well defined
simple output format, you can translate that to any other format
required.

Here’s the start of perlpod.pod in pod format:

=head1 NAME

perlpod - plain old documentation

=head1 DESCRIPTION

A pod-to-whatever translator reads a pod file paragraph by paragraph,
and translates it to the appropriate output format. There are
three kinds of paragraphs:
L<verbatim|/“Verbatim Paragraph”>,
L<command|/“Command Paragraph”>, and
L<ordinary text|/“Ordinary Block of Text”>.

=head2 Verbatim Paragraph

A verbatim paragraph, distinguished by being indented (that is,
it starts with space or tab). It should be reproduced exactly,
with tabs assumed to be on 8-column boundaries. There are no
special formatting escapes, so you can’t italicize or anything
like that. A \ means , and nothing else.

=head2 Command Paragraph

All command paragraphs start with “=”, followed by an
identifier, followed by arbitrary text that the command can
use however it pleases. Currently recognized commands are

  =head1 heading
  =head2 heading
  =item text
  =over N
  =back
  =cut
  =pod
  =for X
  =begin X
  =end X

=over 4

=item =pod

=item =cut

The “=pod” directive does nothing beyond telling the compiler to lay
off parsing code through the next “=cut”. It’s useful for adding
another paragraph to the doc if you’re mixing up code and pod a lot.

=item =head1

=item =head2

Head1 and head2 produce first and second level headings, with the text in
the same paragraph as the “=headn” directive forming the heading description.

And here it is in RDoc:

= NAME

perlpod - plain old documentation

= DESCRIPTION

A pod-to-whatever translator reads a pod file paragraph by paragraph,
and translates it to the appropriate output format. There are
three kinds of paragraphs: verbatim, command, and ordinary text. (RDoc
doesn’t yet support links to section heads. It’s coming)

== Verbatim Paragraph

A verbatim paragraph, distinguished by being indented (that is,
it starts with space or tab). It should be reproduced exactly,
with tabs assumed to be on 8-column boundaries. There are no
special formatting escapes, so you can’t italicize or anything
like that. A \ means , and nothing else.

== Command Paragraph

All command paragraphs start with “=”, followed by an
identifier, followed by arbitrary text that the command can
use however it pleases. Currently recognized commands are

  =head1 heading
  =head2 heading
  =item text
  =over N
  =back
  =cut
  =pod
  =for X
  =begin X
  =end X

[=pod, =cut]
The “=pod” directive does nothing beyond telling the compiler to lay
off parsing code through the next “=cut”. It’s useful for adding
another paragraph to the doc if you’re mixing up code and pod a lot.

[=head1, =head2]
Head1 and head2 produce first and second level headings, with the
text in the same paragraph as the “=headn” directive forming the
heading description.

Doesn’t seem to be much difference in readability to me. Why would I
convert RDoc as an input format to POD as an output format. Wouldn’t I
be better off adding the extra output processors that folks wanted?

Regards

Dave

But I’d get to choose the comments, and you don’t want that, believe
me.

Massimiliano

···

On Sun, Sep 08, 2002 at 12:21:58AM +0900, bbense+comp.lang.ruby.Sep.07.02@telemark.stanford.edu wrote:

    • Well, the only reason that 99% of perl modules have
      documentation is that the easiest way to get started
      writing a module is to run a program that sets up the
      structure for you. One of the things this program does
      is write a dummy doc skeleton with embarassing comments
      like

    "Stub documentation for Foo, created by h2xs. It looks like the
    author of the extension was negligent enough to leave the stub
    unedited. "

    Perhaps we should add this feature to rpkg.

Hi –

Doesn’t seem to be much difference in readability to me. Why would I
convert RDoc as an input format to POD as an output format. Wouldn’t I
be better off adding the extra output processors that folks wanted?

That last may be intended as a rhetorical question, but I’ll go ahead
and answer it anyway:

Yes.

:slight_smile:

I’m honestly not sure why the subject of POD has arisen at all in
connection with Ruby. We have such a nice, integrated, pluggable,
expandable tool in RDoc. Another quote from perlpod(1): “The intent
is simplicity, not power.” With RDoc, we have both :slight_smile:

(I know there are some things to be considered in the rd/RDoc
relationship – I’m not trying to write rd off by enthusing about RDoc
– but I don’t see the relevance of POD to the process.)

David

···

On Sun, 8 Sep 2002, Dave Thomas wrote:


David Alan Black | Register for RubyConf 2002!
home: dblack@candle.superlink.net | November 1-3
work: blackdav@shu.edu | Seattle, WA, USA
Web: http://pirate.shu.edu/~blackdav | http://www.rubyconf.com

In article m2fzwlrc8i.fsf@zip.local.thomases.com,

bbense+comp.lang.ruby.Sep.07.02@telemark.stanford.edu writes:

It certainly is possible. POD shows us the way. I think the
point I’m trying to make is that POD is an output format.
It just happens to be so trivial that it’s accepted in the
perl community to write raw POD. Once you have a well defined
simple output format, you can translate that to any other format
required.

Here’s the start of perlpod.pod in pod format:

[snip]

Doesn’t seem to be much difference in readability to me. Why would I
convert RDoc as an input format to POD as an output format. Wouldn’t I
be better off adding the extra output processors that folks wanted?

    • That’s not what I’m suggesting. What I would like to see is
      that some readable format appear between the =begin =end tags
      in every Ruby library file. It’s the IDEA of POD that I would
      like to see in Ruby. If rdoc just wrote skeleton RDoc between
      =begin and =end I’d be fine with that too. I’m not suggesting
      that we take anything from POD but ideas. I was suggesting that
      this format be rd[1], but anything that preserves that readablity
      would be fine.
    • While some people may write quite extensive documentation, I
      suspect most library writers will stick to few comments in the
      appropriate places since rdoc does such a nice job of turning
      that into fairly reasonable documentation. I want a tool that
      does powerful things with the minimal documentation most
      libraries provide. IMHO, Rdoc already has enough features
      what it needs is stuff to work on, more output forms and
      some powerful searching tools. The hoopiest documentation
      in the world is useless if I don’t know where to find it
      and don’t have the tool to read it[2].
    • Again let’s go back to the use case. What’s your scheme
      for addressing these questions?

    “Show me all the installed classes that have a read method.”

or

“Show me all the installed classes that have the Enumerable
module.”

    • Booker C. Bense

[1]- My arguement for using rd is that many libraries already
support this model of rd formatted docs between =begin , =end.
I’m not interested in writing rd, but if rdoc could take care
of that nasty chore for me, that’s be great. I’d really like
a tool that could search that rd for me.

[2]- I fully expect to lose this battle, I’ve lost it every
time I’ve fought it. Featuritius always wins out over
functionality.

···

Dave Thomas Dave@PragmaticProgrammer.com wrote:

In article Pine.LNX.4.44.0209071321020.5502-100000@candle.superlink.net,

Hi –

Doesn’t seem to be much difference in readability to me. Why would I
convert RDoc as an input format to POD as an output format. Wouldn’t I
be better off adding the extra output processors that folks wanted?

That last may be intended as a rhetorical question, but I’ll go ahead
and answer it anyway:

Yes.

:slight_smile:

I’m honestly not sure why the subject of POD has arisen at all in
connection with Ruby. We have such a nice, integrated, pluggable,
expandable tool in RDoc. Another quote from perlpod(1): “The intent
is simplicity, not power.” With RDoc, we have both :slight_smile:

(I know there are some things to be considered in the rd/RDoc
relationship – I’m not trying to write rd off by enthusing about RDoc
– but I don’t see the relevance of POD to the process.)

    • The only advantage POD has over anything existing currently in
      Ruby is that I always know where to find it. If I always know
      where to look and what I’ll find there, I can write automated
      tools to search it[1]. POD was the Perl community’s answer to
      the question that originally started this thread. All the
      objections to Ruby as a “real” language have been answered
      by both the Perl and Python worlds. It wasn’t that long ago
      that having a career as a Perl programmer seemed as remote
      as being a Ruby programmer does now. I’m not suggesting that
      we use POD as anything other than a model for ideas.
    • While Ruby has a document writing tool in rdoc that
      surpasses anything in the Perl world, it doesn’t have a
      document FINDING tool that is anywhere near as good as
      perldoc. In the long run I really don’t care what format
      the documentation is in as long as I can find it quickly.
    • Booker C. Bense

[1]- “find /usr/local/lib/ruby -type f -exec grep …” lacks
some elegance.

···

dblack@candle.superlink.net wrote:

On Sun, 8 Sep 2002, Dave Thomas wrote:

bbense+comp.lang.ruby.Sep.08.02@telemark.stanford.edu writes:

“Show me all the installed classes that have a read method.”

or

“Show me all the installed classes that have the Enumerable
module.”

RDoc could do both of those once I have the ‘ri’ stuff integrated.

Insofar as any objectives have been formally set by the new effort to make Ruby
documentation more consistent/usable, this aspect - finding doco - is a high
priority.

Regards,
Gavin

···

----- Original Message -----
From: bbense+comp.lang.ruby.Sep.08.02@telemark.stanford.edu

    • The only advantage POD has over anything existing currently in
      Ruby is that I always know where to find it. If I always know
      where to look and what I’ll find there, I can write automated
      tools to search it[1]. POD was the Perl community’s answer to
      the question that originally started this thread. All the
      objections to Ruby as a “real” language have been answered
      by both the Perl and Python worlds. It wasn’t that long ago
      that having a career as a Perl programmer seemed as remote
      as being a Ruby programmer does now. I’m not suggesting that
      we use POD as anything other than a model for ideas.
    • While Ruby has a document writing tool in rdoc that
      surpasses anything in the Perl world, it doesn’t have a
      document FINDING tool that is anywhere near as good as
      perldoc. In the long run I really don’t care what format
      the documentation is in as long as I can find it quickly.
    • Booker C. Bense

Dave, do you realize that you have a great beginning towards a Ruby
Refactoring Browser?

···

On Sunday 08 September 2002 11:41 am, Dave Thomas wrote:

bbense+comp.lang.ruby.Sep.08.02@telemark.stanford.edu writes:

“Show me all the installed classes that have a read method.”

or

“Show me all the installed classes that have the Enumerable
module.”

RDoc could do both of those once I have the ‘ri’ stuff integrated.

In article m27khw77t4.fsf@zip.local.thomases.com,

···

Dave Thomas Dave@PragmaticProgrammer.com wrote:

bbense+comp.lang.ruby.Sep.08.02@telemark.stanford.edu writes:

“Show me all the installed classes that have a read method.”

or

“Show me all the installed classes that have the Enumerable
module.”

RDoc could do both of those once I have the ‘ri’ stuff integrated.

    • So you’re proposing ri’s internal format as the intermediate
      output format and ri’s standard archive as the place to store
      documents. Or am I missing something?
    • Booker C. Bense

Albert Wagner alwagner@tcac.net writes:

“Show me all the installed classes that have the Enumerable
module.”

RDoc could do both of those once I have the ‘ri’ stuff integrated.

Dave, do you realize that you have a great beginning towards a Ruby
Refactoring Browser?

Well… remember RDoc’s analysis is static. However, ripping out RDocs
parsers wouldn’t be too painful: it’s basically a single source file,
made by combining and hacking the one that comes with irb.

Dave

bbense+comp.lang.ruby.Sep.09.02@telemark.stanford.edu writes:

So you’re proposing ri’s internal format as the intermediate
output format and ri’s standard archive as the place to store
documents. Or am I missing something?

Yes, although not the current format.

Albert Wagner alwagner@tcac.net writes:

“Show me all the installed classes that have the Enumerable
module.”

RDoc could do both of those once I have the ‘ri’ stuff integrated.

Dave, do you realize that you have a great beginning towards a Ruby
Refactoring Browser?

Well… remember RDoc’s analysis is static. However, ripping out RDocs
parsers wouldn’t be too painful: it’s basically a single source file,
made by combining and hacking the one that comes with irb.

I know :slight_smile: I just spent several weeks doing just that as an exercise for
myself. I thought it would be cool to have an editor that was truely Ruby
aware, but quickly got in over my head. I don’t have the background: I stay
confused about the distinctions between lexers and parsers.

···

On Sunday 08 September 2002 01:06 pm, Dave Thomas wrote:

Dave