Ruby, on the other hand, has two going on three different
formats, if I’m understanding what Sean is saying correctly.
Sadly yes… however there is a saving grace with rubydoc, it’s
standard is XML and the other utilities export XML… which means
that anyone can write a stylesheet that’ll convert rdoc-> rubydoc
or rd-> rubydoc. These stylesheets will be included in the base
rubydoc installation.
In some ways, though, I still think that this is probably the wrong
approach. IMO, either rdoc or rd needs to die. Ideally, rdoc will
pick up the ability to parse rd comments cleanly (perhaps spitting
out warnings) so that there only needs to be one primary
documentation tool.
Agreed, but I don’t have any interest in what happens in the embedded
documentation space. If something new comes along, great, I’ll use
that too. After rubynet and rubydoc have reached critical mass and I
have free time to work on something else, I may spend some time
flushing out an inline documentation format of my own that fits in
nicely with rubydoc that isn’t XML. XML’s great for machines, but
when it comes to writing stuff in emacs or any text editor, writing
XML from scratch is the perfect way to develop carpal tunnel. See the
ruby-doc@ mailing list for details about what I’ve quasi envisioned,
however I need to support some kind of linking of variables, methods,
and classes. After spending so much time in the libxml source, I
actually kinda like gnome’s doc format… granted it’s a rip off of
javadoc and wherever the inventive folks at Sun ripped it off from.
Okay … so … what’s the point of rubydoc? IMO, rdoc is quite
sufficient to produce the necessary documentation for the code. Is
rubydoc intended to massage that documentation into a nicer format
that could, theoretically, look good in PDF?
To encapsulate and provide a transformable, publishable (www, TeX,
PDF, text, nroff), includable (I REALLY want to be able to offer
documentation sets/examples that are dynamically driven from the
rubynet.org and rubydoc.org sites and also have static content. Think
php.net + PostgreSQL’s idocs + docbook + perldoc) and indexable (from
the CLI: rubydoc [term/class]). Nothing anywhere in the programming
world is coming close to offering that. I as a developer want that.
Others likely do too.
Don’t get me wrong – I’m not overly enamoured of the default rdoc
template, but at the moment I’m just too lazy to change it for my
own purposes and (later) offer it as a replacement template.
Good point, I’ll make a not of that request and let you specify a
different txt/nroff stylesheet in your ~/.rubynet/rubydoc.cfg.
I have to be honest and say that I don’t get why rubydoc as a
tool is necessary. If you’re going to make it an aggregator so that
the user can have the equivalent of a single interface like users of
the ActiveState Perl package do … that’s cool.
Basically, yeah.
If you’re going to make it a super-duper version of ri with Rimport,
even better! (See? I forgot another Ruby documentation tool.) But
if it’s just going to be Yet Another Documentation Tool, I’ve got to
say that I don’t see the point.
Naw, that’s why I’m incessant on standardizing around XML. Any of the
Yet-Another-Ruby-Doc formats can be massaged into rubydoc’s XML spec
with a stylesheet. From there, any number of documents can be
made/indexed.
IMO, rubydoc should be a shell and transformation agent on top of
rdoc, not a whole new documentation system. But that’s just IMO.
Bingo, that’s exactly what it’s going to do.
:-/ Working on stuff by yourself and being in a perpetual state of
over commitment and everywhere-all-at-once kinda bites when you’re
trying to design something “right” and aren’t hacking it together.
Help wanted/appreciated.
That’s not a criticism, Sean. However, I can’t at the moment really
provide any assistance with the code side – job searches tend to
take a lot of time, and I have my own coding projects that I’m
attacking[snip].
It’s alright, I know how that goes. I have to talk fault in it moving
slow since I’m doing this work outside of the limelight of -talk.
I’ve actually completely disengaged from the -talk community and only
read through this mailbox when I get a nudge from someone on IRC
hinting that there’s something worth checking out.
I have joined rubynet-devel, though, so I can possibly provide some
design commentary.
I welcome the addition, there’s a good crew of some 30+ lurkers now.
Put enough of them on a list and someone’s bound to chirp up with a
patch every now and then. 
It supports – at least experimentally and I haven’t been able to
get it to work yet – automatic diagram creation.
Speaking of, does ruby have a DOT interface? I’ve wanted to use
this for class diagramming on rubynet but haven’t looked into it.
Look at rdoc; there’s a dot/ directory in the distribution. I’m not
exactly sure the best way to use it – part of the problem could be
that I use Ruby from Windows, and I don’t have dot itself installed
(: As I said above, rdoc should be the documentation system for
Ruby; rubydoc should be a way of transforming rdoc output into ri
database information (a la Rimport), creating a unified API
reference for Ruby and all the modules installed on the user’s
system, etc.
That’s the short term plan. Glad to know my ideas aren’t existing in
a total void.
else. Want to create PDFs instead of HTML files? Write the
appropriate generator and/or template.
The joy of XSLT and flow objects.
Certainly – and this is probably what rubydoc should do. (Of
course, I’ve now tried rdoc’s CHM – Windows HTMLHelp – output and
have had mixed results with it. It seems to ignore #:nodoc:
directives; I may look at that to provide a patch for Dave when he
gets back, like another patch that I’ve made to a patch that he made
in response to a patch that I gave him.)
So long as rdoc does the right thing when it comes to generating XML,
then the stylesheet should handle this correctly… though I’m not a
Win32 guy atm, but I’m doing an increasing amount of Win32 GUI clients
in FOX so who knows, I’ll likely transgress into the depths of MS hell
again at some point in the not too distant future.
I’m not a yaml lover, personally. Culturally I think YAML exists
as a counter movement to Java/XML and XMLs tendency to get bundled
with Java. I can’t say as I disagree with the dislike of the
Java/M$ developer sentiment, Sun hasn’t done much in the way of
innovative computing in a while and I wish would just curl up and
flop. Tandem makes better hardware anyway. 
I think that your cultural analysis is correct. I still think that
YAML is a useful lightweight format.
No disagreements there, but I just hacked out libxml which seems to do
everything that I need it to at the moment… short of generating DTDs
on the fly, but that’ll come here sometime this week.
Here’s the dilly with supporting multiple files and formats: I
don’t know what your preference is.
One format, one file. Period. Don’t give me the option of multiple
formats and files to describe a single package. I picked YAML
because it appears to be close to the .rubynet_ format that you
specified. Perhaps as follows (note that I use ‘…’ when there
could be other attributes or I’m eliding):
Ooooh!
[snipped the wonderful yaml example]
IMO, in a single file, this provides everything that you would have
in the multiple file form – and it’s easier to edit. XML would be
appropriate, too.
There is no perfect format, IMHO. XML’s good for machines and for
transferring between automated processes. Yaml’s good if you need to
have a human touch down and edit the file and don’t want to barrage
him with a zillion characters that the eye has to parse through.
Multiple plain text files, however, are shell friendly. I honestly
see room for having all three as valid formats for describing a
package. I actually wonder if I couldn’t generate the formats for the
plain text files and the YAML from the XML spec… hrm, that’d be an
interesting exercise in code generation… any XSLT/YAML buffs out
there that’d want to take that on?
I’ll re-present this on rubynet-devel with further commentary in a
couple of days, but I think that there are mistakes being made in
the design as expressed by the package information files.
! Excellent! I look forward to the discussion.
Contrary to my sentiment about YAML, I’ll likely support a YAML
interface for configuring packages just because that’s a format
that some developers prefer. I personally favor having simple and
small files each with a specific format. Makes it easier to
manipulate/create the files with sed(1) and find(1). I’m a die
hard UNIX guy at heart, what can I say. It showing?
As I said before, one format and one file. Too many options or files
will make the project unmaintainable. If you keep the existing
dot-rubynet files, I can guarantee that I won’t be packaging things
that way – it’s too much work for me, the maintainer. (Ideally,
even, you would have a cross-platform GUI- or TUI-based interface
for building and maintaining the file.)
The dot files will get compiled into XML. From there, a GUI can
operate on the XML. The dot files are being generated at the moment
with some crude guesses so for me, it’s actually really nice.
$ rubynet --generate
[edit dot files and delete false positives/add missing entries]
$ rubynet --compile-module
By the time the data hits the rubynet server, the data will be
serialized into an XML file. The dot files are used only by an
author for describing their package, not for use in the rubynet
system. Once things hit the rubynet system, it’s XML. Period. For
those that are curious, binary data is MIME64 encoded in an
element in the rubynet file.
Blechhhh. base64 encoding is evil unless it’s unavoidable. It adds
an unnecessary 30% or more to the size of the file.
No arguments from me here, but do you know of another way to encode
binary data in an XML file? I’d love to use something more efficient,
but don’t know of anything. Fortunately with the compression set at 9
the size drops quite dramatically as it seems there are enough
patterns in the mime encoded file. I’ll add support to compress files
individually before they get encoded in the event that this pattern
doesn’t hold true.
It also seems that here, you’re planning on mixing metadata and
content
Correctomundo, that’s exactly what I’m doing… and I’m being
quasi-clever about it for several reasons. I’m trying to create a
format that fullfills three purposes.
- Is indexable. XML+XPath takes care of this.
- Contains meta data for the module that way a skeleton module can be
distributed instead of the full blown tarball. Think FreeBSD ports
here. The reason for doing this is for ports conformity, and
because I’d like to make this system attractive to commercial
vendors who need to distribute modules and have restricted downloads.
- Can be used as a ubiquitous format that stores all of the packaging
and file content. I’ve got an idea in the back of my head for how
to convert a tarball into a rubynet package, for example. As a
user, I only want to have to download one thing. Think of this a
JAR file on steroids, if you will. While I agree with your next
point that it’s a no-no, by and large, I do think that this is
miles better than what Perl has and certainly better than the
simple zip format that JAR files employ.
– which I consider a major no-no when it comes to data
modeling.
See #3 above.
Frankly, I think that this is a task for which XML is uniquely
UNSUITED. Sure, XPath helps here, but XML will never match the
power of a properly coded relational database for this sort of
problem space.
Agreed… there’s an element of usability though that can’t be matched
if you split things into two files. Hrm… maybe I should just
concatenate a tarball with the rubynet skeleton file with a small
contents that gives the version of rubynet needed and the sizes of the
XML and package… hrm… to be continued on devel@rubynet.org.
-sc
···
–
Sean Chittenden
