Can we attack the 'not enough libraries' thing straight on?

ruby-talk
101

Well, the first and difficult step is to create a plain text file that
maps hypothetical categorisations to existing libraries. I have no
experience whatsoever to do this.

Are you able to create such a mapping?

Gavin

···

On Saturday, January 25, 2003, 4:15:04 PM, Warren wrote:

i dont mind helping u dudes to sort it out

On Sat, 25 Jan 2003 07:00:49 +0900 > Gavin Sinclair gsinclair@soyabean.com.au wrote:

On Saturday, January 25, 2003, 4:49:07 AM, Daniel wrote:

On Fri, Jan 24, 2003 at 11:29:36PM +0900, Gavin Sinclair wrote:

IPC::Msg::foo …
IPC::SysV::quux …

I think this is a good idea, but unlikely to happen anytime soon.

Gavin,

I think it’s a great idea. Why can’t it happen any time soon?
There can be a person or two who could try to sort RAA into appropriate
categories and those people can do this. They don’t have to be the
original developers.

102

Thank you Dave, I was going to bring this up as well. Google indexes
the home pages of these libraries, which contain better descriptions
that the metadata authors place (or may place) in the RAA. And they
typically keep the web pages updated.

-rich

···

On Sunday, January 26, 2003, at 11:20 AM, Dave Thomas wrote:

Mike Campbell wrote:

Careful there… You want the archive to be USABLE, regardless of how
well/poorly it models the real world. Correct or not, people at least
understand hierarchies.

One word: Google.

OK, more than one word.

What happened to all those services that thought they’d organize the
information on the 'net into nice tidy hierarchies? Do you use them to
find information?

For me, the answer is rarely. Sometimes I’ll check one of the lists to
see if there are Ruby resources I haven’t come across, but right now I
actually find it easier to type ‘ruby racc’ into my search bar than to
navigate through the RAA.

Cheers

Dave

103

I certainly think that RAA should have a search feature.
That’s not an argument against making it easier to browse too. I think we
want both features.

···

On Mon, Jan 27, 2003 at 01:20:32AM +0900, Dave Thomas wrote:

Mike Campbell wrote:

Careful there… You want the archive to be USABLE, regardless of how
well/poorly it models the real world. Correct or not, people at least
understand hierarchies.

One word: Google.

OK, more than one word.

What happened to all those services that thought they’d organize the
information on the 'net into nice tidy hierarchies? Do you use them to
find information?

For me, the answer is rarely. Sometimes I’ll check one of the lists to
see if there are Ruby resources I haven’t come across, but right now I
actually find it easier to type ‘ruby racc’ into my search bar than to
navigate through the RAA.

Cheers

Dave

–
Daniel Carrera
Graduate Teaching Assistant. Math Dept.
University of Maryland. (301) 405-5137

104

This expands on what Alan Chen wrote below.

I agree that search functionality is necessary for RAA. Archive
entries should have useful keywords in the description so they can be
found easily. I think this can be done and still support browsing by
category. Browsing by category can also be referred to as “topical
searching”.

I continue to think that structuring the information space that is the
archive would be very useful by making the archive more usable for
users (who will have differing levels of knowledge about what the
information space contains). I also think that one categorization
system should be determined first and then different views of that
categorization system could be implemented that users could select as
they chose. I think this would be valuable to have even if no
experienced Ruby user ever used it (i.e., if only nubies ever looked at
it).

Finally, this discussion is not taking place in a vacuum. We have the
RAA. There are four top-level categories. There are a multitude of
second-level categories (86 in Library, 69 in Application).

Looking at the current Library categories, it seems clear that many
could be grouped into super-categories.

Examples include [Calendar, Date], [Algorithm, Array, Datastructure],
[Net, WWW] and more.

Also, there is substantial duplication of categories, such as:
[graphics, Graphics], [Regex, Regular Expressions] and [devel,
Development].

Also, there are a large number of categories that contain only one
entry and should probably be listed under another more general
category, such as:
[Resource => resource], [Pty => ruby-tpty], [RPM => ruby-rpm], [Shell
=> shell], [Parallel => mpi_ruby] and more.

When one looks at the current Application categories, it is more of the
same, including that some entries under Application probably belong
under Libraries. An example is the following 5 categories under
Application: (1) devel, (2) Devel, (3) Development, (4) development and
(5) Development/Logging.

The next time Bruce Eckel spends 45 minutes examining the Ruby
language, I would like for him to quickly see all of the resources Ruby
has in a form that he can quickly grasp :wink: See

http://www.mindview.net/Etc/FAQ.html#Ruby

The fact that the proposal might later be improved upon or transformed
into something better is wonderful, but should not stop improvements
that can be implemented now.

···

On Sunday, January 26, 2003, at 03:19 PM, Alan Chen wrote:

I liked the proposals which separated the functions of searching in the
archive vs. organizing the code from the archive.

Browsing for a given functionality seems to be best handled through
categories. (or Google search :slight_smile: However, once a given library,
module, or application has been located, I would think that accessing
the code would be via a unique location in a heirarchy.

The problem with code use from multiple categories is that naming
would be ambiguous regarding a given interface or codebase. What if
WWW::XML was the same code as Document::XML, but different from
Application::XML… You might end up loading the same library twice
into different spaces, or you might end up supporting two different
XML apis, etc.

Of course it is always nice to be able to make local decisions which
may be better suited to your particular needs and wants. Maybe if we
provided a remapping mechanism:

Local::Taste =
require_as(‘too/deep/into/the/standard/heirarchy/for/my/taste.rb’)

Other than that, some storage of implementation fields would be nice
too: is it pure ruby, or an Win32 only extension, or a multi-arch
extension, dependent on an external lib, etc… but I’m sure that’s
been
discussed before.

[snip]

105

I think my thoughts on this at the moment are that all the modules in the
world won’t help you unless they’re thoroughly documented.

···

–
#define struct union /* Great space saver */

106

i want to make a distinction here over the word category. these are
hiearchical categories you refer too, also known as a Tree.

i beleive it would be an improvement to allow for a Graph. that is to say, an
RAA entry can belong to multiple categories.

since it seems searching is the most common means of finding a package, and
will increasingly become so as the RAA increases in size, then perhaps the
search functionailty should be placed at the top of the page.

also, note, that mutiple categorization will facilitate seach queries better.

···

On Sunday 26 January 2003 03:49 pm, Mark Wilson wrote:

Finally, this discussion is not taking place in a vacuum. We have the
RAA. There are four top-level categories. There are a multitude of
second-level categories (86 in Library, 69 in Application).

–
tom sawyer, aka transami
transami@transami.net

107

Hi, all,

There are many articles overnight…

Regards to the style topic around Google vs Yahoo vs Wiki
(graph, not tree), we developers had discussed it for the
RAA renewal(Oct. 2002) and choosed the Google style
and keep category simple. That’s why current RAA is.

  • We want project owners to self-organize their category
    for users to grasp projects from broader perspective.
  • We want project owners to put searchable keyword at
    ‘short_description’ or ‘description’ of their entry,
    for users who knows what he/she want exactly.
  • We want users to utilize search box.

But, I realize it while this discussion, things aren’t
going our way for now. So I’m wathing discussions
interestingly.

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking proejcts

and for adding comments of users.

From: Daniel Carrera [mailto:dcarrera@math.umd.edu]
Sent: Monday, January 27, 2003 2:46 AM

I certainly think that RAA should have a search feature.

Maybe I misunderstood you again, but does this English
sentence imply that RAA doesn’t have a search feature now?
Or are you thinking more detailed search e.g. “search by
release date, category and description”? That’s on our
ToDo…

Regards,
// NaHi

108

i beleive it would be an improvement to allow for a Graph. that is to say, an
RAA entry can belong to multiple categories.

A complex graph is harder to browse than a tree.

As for searching, I don’t see how a search would work any better or worse
on a graph, a tree or a flat list.

I would suggest having essentially a tree. We can permit some projects to
be in more than one category, but be conservative about it.

since it seems searching is the most common means of finding a package, and
will increasingly become so as the RAA increases in size, then perhaps the
search functionailty should be placed at the top of the page.

I don’t know how common searching is. When I used CPAN I browsed mostly.
Both features are important.

Now, having a tree will not compromise a search, but having a (complex)
graph will compromise browsing. I vote that we avoid too-complex a
graph.

I agree that the search option should be at the top. It takes almost no
room and it’s very useful.

also, note, that mutiple categorization will facilitate seach queries better.

I would agree only if we have a simple graph (almost a tree). Having too
many things in too many categories will cause the search to produce a
large number of unwated results. As much as I like Google, this is one
thing I find anoying about it.

Take the extreme case, just for illustration, where everything is is under
every single category. If you search for any given category, your results
will contain everything, making the search useless.

I vote that we maintain the graph simple.

···

On Mon, Jan 27, 2003 at 08:25:22AM +0900, Tom Sawyer wrote:

–
Daniel Carrera
Graduate Teaching Assistant. Math Dept.
University of Maryland. (301) 405-5137

109

But, I realize it while this discussion, things aren’t
going our way for now. So I’m wathing discussions
interestingly.

Good to know.

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking proejcts

and for adding comments of users.

This sounds excellent.

From: Daniel Carrera [mailto:dcarrera@math.umd.edu]
Sent: Monday, January 27, 2003 2:46 AM

I certainly think that RAA should have a search feature.

Maybe I misunderstood you again, but does this English
sentence imply that RAA doesn’t have a search feature now?
Or are you thinking more detailed search e.g. “search by
release date, category and description”? That’s on our
ToDo…

The current search could be improved, but it already provides 60+% of
what anyone could possibly want (i.e. a search engine). Some people
seem to be ignorant of RAA’s search engine altogether! (I do think it
should be moved up the top right now; that’s almost a web UI
standard!)

OK, I’ve had my 2c, but I just put another nickel in the slot.

The only thing required to be able to do very effective searches is
some metadata that can be indexed (or brute-force-searched; who
cares?). The simplest and best way to do this is to have some
standard keywords that package owners can apply to their packages. A
good web interface is not hard: either check-boxes or a multi-list.
(Allow extra keywords in a text area.) The same interface can be used
to search for stuff.

Knowing what you can valuably search for means more powerful
searching, and means you can effectively browse as well - search
whatever categories (oops, I mean “keywords”) you are interested in.

This equates, more or less, to multiple categorisation, but without
all the hassle of working out how to implement such a beast.

Easy enough? Good enough?

Gavin

···

On Monday, January 27, 2003, 6:13:31 PM, Hiroshi wrote:

110

Leaving aside broader changes, the following could be implemented
immediately and without revisiting prior design decisions:

  1. As said by others, the search window should be at the top of the
    page, above the ‘What’s New’ section. The scope of the simple search
    should be the entire archive.

  2. The following sub-categories (or keywords) should be combined
    because they are the same:

In Application:

devel, Devel, Development, development => Development

MIdi, midi => MIDI

util, Util, Utility => Utility

In Library:

devel, Development => Development

graphics, Graphics => Graphics

Regex, Regular Expressions => Regular Expressions

111

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking proejcts

and for adding comments of users.

Nahi, when do you plan to implement this? I think this would go a really
long way toward making the RAA a lot more useful. Would be nice to have
an option to search the comments, and of course the Wiki-esque “What pages
link to this one?” feature.

Chad

112

No. This sentence does not imply that RAA doesn’t have a search feature.
It simply says that it’s a good thing. The sentence is a little
ambiguous.

All I was saying was that, though I want better browsing, I still want to
keep the search.

···

On Mon, Jan 27, 2003 at 04:13:31PM +0900, NAKAMURA, Hiroshi wrote:

I certainly think that RAA should have a search feature.

Maybe I misunderstood you again, but does this English
sentence imply that RAA doesn’t have a search feature now?

–
Daniel Carrera
Graduate Teaching Assistant. Math Dept.
University of Maryland. (301) 405-5137

113

I’ve been working on a topic map wiki at www.ruby-doc.org/wiki/topicwiki.rb

It differs from the standard wiki in that it allows more specific annotations of links.
For example, a wiki page about Rimport could contain links to wiki pages about Rexml and Rdoc, with the linking markup indicating
that these are dependencies. Other wiki page links might be annotated to indicate a reference to online documentation, or a
reference to where book information may be found.

In a conventional wiki, one would know that there was a link from one page to another, or a link to an external site, but would not
know what that link meant without some active interpretation of the English text. The use of TM notation makes the wiki content
more accessible to machine parsing, with greater semantic information.

The use of TM notation means the wiki pages can be used to generate an XTM (XML Topic Map) feed:
www.ruby-doc.org/wiki/wtm/RDP.xtm

The XTM could be used to create a browsable index of ruby apps, libs, docs, dependencies, and so on.

As an aside, topic maps have been applied to CPAN

The ruby-doc.org topic map is a bit less ambitious than the CPAN project. I’m trying to balance ease of use with robust topic
mapping. Users shouldn’t have to think /too/ much about topic maps when adding a page or updating information. Still, since
content can be annotated to indicate types of associations among wiki pages and external links, some thought must be used to ensure
reasonable consistency.

The biggest differences between a conventional wiki and a topic map wiki is that, whereas a conventional wiki tends to be itself the
repository of information, a topic map wiki serves mainly to define associations among topics, and to refer to external sources of
information.

So, the main goal of the ruby-doc.org topic map wiki is to map out Ruby documentation and resources, though defining associations
among Ruby apps and libs is easy enough.

It is more like a book index rather than a book. There are already many Ruby resources, but a thorough index is lacking, and the
topic map wiki might help generate such an index.

Some good introduction to topic maps are:

XML.com: What Are Topic Maps?
What Are Topic Maps

The TAO of Topic Maps
http://www.ontopia.net/topicmaps/materials/tao.html

The XTM 1.0 Spec
http://www.topicmaps.org/xtm/1.0/

Now, to my knowledge, there are no Ruby apps for browsing/processing XTM, but there is a good, free Java application (it runs
servlets via a local install of Tomcat) called The Omnigator:
http://www.ontopia.net/download/freedownload.html

and, for a truly whiz-bang interface, TMNav:

The latter uses TouchGraph (http://www.touchgraph.com/) for navigation, to give you an idea of the interface. Even with the
currently sparse XTM produced by the wiki it’s fun to see the topics and associations so neatly displayed. (Something like this
built using Ruby and Fox would be spiffy …)

BTW, I’ve seen a Ruby app called “Topiq” that apparently deals with topic maps, but it doesn’t appear to do anything with XTM, and
it’s coupled to mod_ruby/eruby and SQLite, which has so far prevented me from doing more than perusing the source code.

The topic map wiki is still pretty alpha; the XTM produced is not normalized, default topics types and associations could be better,
and I’m still unsure of the TM markup notation. Plus, defining good topics and associations is often non-trivial.

I’ve considered having the wiki interpret LTM (linear topic map notation) or AsTMa= (acronym for something or other), two
alternative TM notation formats, though their syntax may conflict with conventional wiki notation and ordinary usage of brackets and
parenthesis. Still, there are advantages to using something already well-defined, and avoiding the creation of yet another markup
language.

A wiki makes a good starting point for a community-driven topic map because wiki pages map nicely to topic map topics, and the wiki
concept is easy enough for most people to get their heads around. Ideally, adding TM features shouldn’t break this ease of use and
familiar conceptual model.

James Britt

···

-----Original Message-----
From: NAKAMURA, Hiroshi [mailto:nahi@keynauts.com]
Sent: Monday, January 27, 2003 12:14 AM
To: ruby-talk ML
Subject: Re: Can we attack the ‘not enough libraries’ thing straight on?

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking projects

and for adding comments of users.

114

A complex graph is harder to browse than a tree.

it would still be browsed just like a tree, only some packages would appear
under multiple nodes.

As for searching, I don’t see how a search would work any better or worse
on a graph, a tree or a flat list.

if a package is afixed to a single category although it provides functionality
in other areas, you would not find via a categorical search for these.

Now, having a tree will not compromise a search, but having a (complex)
graph will compromise browsing. I vote that we avoid too-complex a
graph.

perhaps allowing for at most 3 categories a package can belong too?

I agree that the search option should be at the top. It takes almost no
room and it’s very useful.

Take the extreme case, just for illustration, where everything is is under
every single category. If you search for any given category, your results
will contain everything, making the search useless.

I vote that we maintain the graph simple.

there are always trade-offs, arn’t there? :slight_smile:

so how many Rubyists are watching the super-bowl? or actually, how many
aren’t?

···

On Sunday 26 January 2003 04:55 pm, Daniel Carrera wrote:

–
tom sawyer, aka transami
transami@transami.net

115

and it’s not a wooden nickel either! :slight_smile: nicely put.

···

On Monday 27 January 2003 01:23 am, Gavin Sinclair wrote:

OK, I’ve had my 2c, but I just put another nickel in the slot.

The only thing required to be able to do very effective searches is
some metadata that can be indexed (or brute-force-searched; who
cares?). The simplest and best way to do this is to have some
standard keywords that package owners can apply to their packages. A
good web interface is not hard: either check-boxes or a multi-list.
(Allow extra keywords in a text area.) The same interface can be used
to search for stuff.

Knowing what you can valuably search for means more powerful
searching, and means you can effectively browse as well - search
whatever categories (oops, I mean “keywords”) you are interested in.

This equates, more or less, to multiple categorisation, but without
all the hassle of working out how to implement such a beast.

Easy enough? Good enough?

–
tom sawyer, aka transami
transami@transami.net

116

Hi, Gavin,

From: “Gavin Sinclair” gsinclair@soyabean.com.au
Sent: Monday, January 27, 2003 5:23 PM

The current search could be improved, but it already provides 60+% of
what anyone could possibly want (i.e. a search engine). Some people
seem to be ignorant of RAA’s search engine altogether! (I do think it
should be moved up the top right now; that’s almost a web UI
standard!)

Sure. Our test drive are running here.
http://usausa.dyndns.org/~usa/raa/index.html
http://rrr.jin.gr.jp:8080/raa/index.html

RAA/2.3 is scheduled to be released on this weekend (already freezed
features). ToDos and done is listed at
http://rrr.jin.gr.jp/rwiki?cmd=view;name=ToDo232
I’m sorry but The documentation is in Japanese. Good luck.
– NAKAMURA Hiro$hi

OK, I’ve had my 2c, but I just put another nickel in the slot.

Thanks always.

The only thing required to be able to do very effective searches is
some metadata that can be indexed (or brute-force-searched; who
cares?). The simplest and best way to do this is to have some
standard keywords that package owners can apply to their packages. A
good web interface is not hard: either check-boxes or a multi-list.
(Allow extra keywords in a text area.) The same interface can be used
to search for stuff.

Knowing what you can valuably search for means more powerful
searching, and means you can effectively browse as well - search
whatever categories (oops, I mean “keywords”) you are interested in.

This equates, more or less, to multiple categorisation, but without
all the hassle of working out how to implement such a beast.

Easy enough? Good enough?

Seems good, more than enough. We’ll be able to get standard keywords
from Category discussions of this thread, right?

After releasing RAA/2.3, I’ll try some UI.

Regards,
// NaHi

117

Hi, Mark,

From: “Mark Wilson” mwilson13@cox.net
Sent: Tuesday, January 28, 2003 2:00 AM

  1. As said by others, the search window should be at the top of the
    page, above the ‘What’s New’ section. The scope of the simple search
    should be the entire archive.

Definitely.

  1. The following sub-categories (or keywords) should be combined
    because they are the same:

In Application:

devel, Devel, Development, development => Development

MIdi, midi => MIDI

util, Util, Utility => Utility

In Library:

devel, Development => Development

graphics, Graphics => Graphics

Regex, Regular Expressions => Regular Expressions

I don’t like “arbitrary maintenance by others” and let owners to
self-organize category for now but your request seems quite
reasonable. I as a maintainer, should do above changes.

Folks, is there no objection?

Regards,
// NaHi

118

Hi, Chad,

From: “Chad Fowler” chad@chadfowler.com
Sent: Tuesday, January 28, 2003 3:01 AM

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking proejcts

and for adding comments of users.

Nahi, when do you plan to implement this? I think this would go a really
long way toward making the RAA a lot more useful. Would be nice to have
an option to search the comments, and of course the Wiki-esque “What pages
link to this one?” feature.

We just installed tcpwrap and WEBrick and setup a prototype
app server for the wiki. I’ll do my best but at least
you won’t see it at end of Feb.

I use RWiki so it will have search and ‘What pages link to this one?’
features. Bear in mind there’s no WikiName in RD format.

Regards,
// NaHi

119

Hi, Daniel,

From: “Daniel Carrera” dcarrera@math.umd.edu
Sent: Tuesday, January 28, 2003 3:19 AM

I certainly think that RAA should have a search feature.

Maybe I misunderstood you again, but does this English
sentence imply that RAA doesn’t have a search feature now?

No. This sentence does not imply that RAA doesn’t have a search feature.
It simply says that it’s a good thing. The sentence is a little
ambiguous.

I see. Teacher, give me “F” at English reading (and composition). :slight_smile:

Regards,
// NaHi

120

Wow, that’s interesting!

I’ve read one of the documents you link, two to go (plus references
within those). This is a lot to get my head around, but it seems like
good stuff.

Thanks for all this stuff.

Cheers,
Gavin

···

On Saturday, February 1, 2003, 4:54:16 PM, jbritt wrote:

From: NAKAMURA, Hiroshi [mailto:nahi@keynauts.com]

Back to the style issue, we have a plan to introduce

wiki-like feature to RAA. That’s for linking projects

and for adding comments of users.

I’ve been working on a topic map wiki at www.ruby-doc.org/wiki/topicwiki.rb