[ANN] FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled

ruby-talk
21

:fr Leslie Viljoen [mailto:leslieviljoen@gmail.com]
# c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain Test::Unit
# (druby://127.0.0.1:1310)
# c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `
# concat': can't convert nil into Array (TypeError)
# from (druby://127.0.0.1:1310)

fyi, it does not fail here:

C:\Documents and Settings\botp>fri -f plain Test::Unit
------------------------------------------------------ Class: Test::Unit

TEST::UNIT - RUBY UNIT TESTING FRAMEWORK

···

========================================

Introduction
------------
     Unit testing is making waves all over the place, largely due to the
     fact that it is a core practice of XP. While XP is great, unit
     testing has been around for a long time and has always been a good
     idea. One of the keys to good unit testing, though, is not just
     writing tests, but having tests. What's the difference? Well, if
     you just _write_ a test and throw it away, you have no guarantee
     that something won't change later which breaks your code. If, on
     the other hand, you _have_ tests (obviously you have to write them
     first), and run them as often as possible, you slowly build up a
     wall of things that cannot break without you immediately knowing

[snip long text]

C:\Documents and Settings\botp>ver
Microsoft Windows XP [Version 5.1.2600]

C:\Documents and Settings\botp>ruby -v
ruby 1.8.5 (2006-08-25) [i386-mswin32]

kind regards -botp

22

I get some strange responses on Windows. I'll test on Linux shortly.

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain assert
------------------------------------------------------ Breakpoint#assert

[...]

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain assert_not_same
nil

[...]

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain Test::Unit
(druby://127.0.0.1:1310)
c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `
concat': can't convert nil into Array (TypeError)
       from (druby://127.0.0.1:1310)
       c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `merge_in'
       from (druby://127.0.0.1:1310)
       c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.0.1/lib/fastri/ri_index.rb:323:
       in `get_class' from (druby://127.0.0.1:1310)

As far as I can see from this and your other message,
* your Ruby install in Windows is missing the core/stdlib RI documentation;
  that's the reason why Test::Unit::Assertions#assert isn't found when FastRI
  falls back to "nested" matches (i.e. exact method name matches anywhere in
  the hierarchy), and you get Breakpoint#assert instead of 'multiple choices'
* the RI docs in your Linux install might be incomplete;
  Test::Unit::Assertions#assert should have been found before falling back to
  "nested+partial" matches.
* as for the above TypeError: it looks like a bug in
  rdoc/ri/ri_descriptions.rb (ModuleDescription#merge_in), but my copy says:

88 # merge in another class desscription into this one
89 def merge_in(old)
90 merge(@class_methods, old.class_methods)
91 merge(@instance_methods, old.instance_methods)
92 merge(@attributes, old.attributes)
93 merge(@constants, old.constants)
94 merge(@includes, old.includes)
95 if @comment.nil? || @comment.empty?
96 @comment = old.comment
97 else
98 unless old.comment.nil? or old.comment.empty? then
99 @comment << SM::Flow::RULE.new
100 @comment.concat old.comment
101 end
102 end
103 end

The test in line 98 should have prevented the TypeError, but in your backtrace
#concat is called in line 99 (instead of 100 as in my copy). Maybe you've got
an older version that does not include line 98 above?

$ ruby -v
ruby 1.8.5 (2006-08-25) [i686-linux]

···

On Thu, Nov 09, 2006 at 06:05:40PM +0900, Leslie Viljoen wrote:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

23

Should be fixed in 0.1.1, which I just released, as well as the issue with
executable extensions (and all the other problems reported on this thread).
Now the search strategy can be specified, and includes case-independent
variants, so:

$ fri string.split
----------------------------------------------------------- String#split
     str.split(pattern=$;, [limit]) => anArray

···

On Fri, Nov 10, 2006 at 10:58:47AM +0900, Peña, Botp wrote:

# > feature request: i hope fri remove the case sensitivity. eg,
# > C:\Documents and Settings\peñaijm>fri string.split
# > nil
#
# Seems convenient, I'll implement it.

wow, thanks much, Mauricio.

btw, i installed fastri 0.1.0.1 using rubygems on another mswindows pc. The
install went fine but when i started to run fastri-server, it balked. The
program cannot find the HOME var. So what I did was just create it and
equate it like HOME=%HOMEDIR%+%HOMEPATH%. Everything went fine then...
(note, not all win pcs have HOME var set)

------------------------------------------------------------------------
     Divides str into substrings based on a delimiter, returning an
     array of these substrings.
[...]

The less specific you are, the longer a lookup can take, but fri's slowest
search modes are still much faster than ri :slight_smile:

Here's case-independent, nested, partial matching at work:

$ fri writer.rend
----------------------------------------------------- PDF::Writer#render
     render(debug = false)
------------------------------------------------------------------------
     Return the PDF stream as a string.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

24

:fr Mauricio Julio Fernández Pradier
# $ fri string.split
# -----------------------------------------------------------
# String#split
# str.split(pattern=$;, [limit]) => anArray
# --------------------------------------------------------------
# ----------
# Divides str into substrings based on a delimiter, returning an
# array of these substrings.
# [...]

···

#
# The less specific you are, the longer a lookup can take, but
# fri's slowest
# search modes are still much faster than ri :slight_smile:
#
# Here's case-independent, nested, partial matching at work:
#
# $ fri writer.rend
# -----------------------------------------------------
# PDF::Writer#render
# render(debug = false)
# --------------------------------------------------------------
# ----------
# Return the PDF stream as a string.

tried it on my win box, and yes! it works! very fast yet cool indeed.

btw, Mauricio, how do i troubleshoot the fri? is there a log somewhere? I have a problem with one of my linux box running ubuntu6.10, ruby1.8.5, gem0.9, and rails1.1.6. The install went fine, the fri server is running, but when i query thru fri, eg fri String#upcase, the result returns nil; in fact, all queries returns nil. tips again, pls =)

also,
:~# fastri-server
Looking for Ring server...
No Ring server found, starting my own.
fastri-server 0.0.1 (FastRI 0.1.1) listening on druby://127.0.0.1:34136
                                                 -------------->^^^^^^^

How do i make the fastri fixed to a certain port, say 12345? we're using firewalls, so the concern..

thanks and kind regards -botp

25

# Here's case-independent, nested, partial matching at work:
#
# $ fri writer.rend
# -----------------------------------------------------
# PDF::Writer#render
# render(debug = false)
# --------------------------------------------------------------
# ----------
# Return the PDF stream as a string.

tried it on my win box, and yes! it works! very fast yet cool indeed.

btw, Mauricio, how do i troubleshoot the fri? is there a log somewhere?

Not right now, but it will be added (we really want this if fastri-server is
to run as a daemon).

I have a problem with one of my linux box running ubuntu6.10, ruby1.8.5,
gem0.9, and rails1.1.6. The install went fine, the fri server is running,
but when i query thru fri, eg fri String#upcase, the result returns nil; in
fact, all queries returns nil. tips again, pls =)

hmm I guess that you're getting a RiError for some reason; this might tell us
what's happening:

diff -rN -u old-0.1/lib/fastri/ri_service.rb new-0.1/lib/fastri/ri_service.rb
--- old-0.1/lib/fastri/ri_service.rb 2006-11-11 23:13:38.000000000 +0100
+++ new-0.1/lib/fastri/ri_service.rb 2006-11-11 23:13:38.000000000 +0100
@@ -220,6 +220,8 @@
        end
      end
    rescue RiError
+ puts "RiError: #{$!.message}"
+ puts $!.backtrace
      return nil
    end

Does fri --show-matches Array# return anything? (if it does, the problem
lies in the processing of the descriptions, not in the lookup).
Also, you might want to take a look at the index (~/.fastri-index by default)
to verify that it's not empty (it's a Marshal serialization but it's easy to
spot), or run
  fastri-server -b
to rebuild the index and see if it found any methods/classes/modules:
$ fastri-server -b
Indexing RI docs for ParseTree version 1.5.0.
[...]
Indexing RI docs for tidy version 1.1.2.
Indexing RI docs for transaction-simple version 1.3.0.
Building index.
Indexed:
* 11201 methods
* 2304 classes/modules
Needed 6.926496 seconds

also,
:~# fastri-server
Looking for Ring server...
No Ring server found, starting my own.
fastri-server 0.0.1 (FastRI 0.1.1) listening on druby://127.0.0.1:34136
                                                 -------------->^^^^^^^

How do i make the fastri fixed to a certain port, say 12345? we're using
firewalls, so the concern..

I'll implement this. Note that you'll also have to open the port used by the
Rinda Ring (7647).

cheers,

···

On Sat, Nov 11, 2006 at 11:18:53AM +0900, Peña, Botp wrote:
--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

26

:fr Mauricio Julio Fernández Pradier
# fastri-server -b
# to rebuild the index and see if it found any methods/classes/modules:
# $ fastri-server -b
# Indexing RI docs for ParseTree version 1.5.0.
# [...]
# Indexing RI docs for tidy version 1.1.2.
# Indexing RI docs for transaction-simple version 1.3.0.
# Building index.
# Indexed:
# * 11201 methods
# * 2304 classes/modules
# Needed 6.926496 seconds

yes, that was it. i was too dumb to not recreate it. this pc was a new install and ri was just lately installed. sorry for me being too dumb.

i have another concern though.

root@pc4all:/package# fri case
------------------------------------------------------ Multiple choices:
     Regexp#casefold?, String#casecmp
root@pc4all:/package#
root@pc4all:/package# ri case
More than one method matched your request. You can refine
your search by asking for information on one of:
     Symbol#camelcase, Symbol#downcase, Symbol#downcase?, Symbol#upcase,
     Symbol#upcase?, Symbol#camelcase, Symbol#downcase,
     Symbol#downcase?, Symbol#upcase, Symbol#upcase?, String#casecmp,
     String#downcase, String#downcase!, String#swapcase,
     String#swapcase!, String#upcase, String#upcase!, String#camelcase,
     String#downcase?, String#lowercase?, String#upcase?,
     String#uppercase?, String#camelcase, String#downcase?,
     String#lowercase?, String#upcase?, String#uppercase?,
     Regexp#casefold?

which show that ri gets more hits. i think ri is using regex?

# > How do i make the fastri fixed to a certain port, say
# 12345? we're using
# > firewalls, so the concern..

···

#
# I'll implement this. Note that you'll also have to open the
# port used by the
# Rinda Ring (7647).

yes! thanks again.
kind regards -botp

# cheers,
# Mauricio Fernandez - http://eigenclass.org - singular Ruby

27

# fastri-server -b
# to rebuild the index and see if it found any methods/classes/modules:

yes, that was it. i was too dumb to not recreate it. this pc was a new
install and ri was just lately installed. sorry for me being too dumb.

It's not your fault; it should have been created automatically when you ran
fastri-server for the first time. Or did you already have an empty
.fastri-index?

i have another concern though.

root@pc4all:/package# fri case
------------------------------------------------------ Multiple choices:
     Regexp#casefold?, String#casecmp
root@pc4all:/package#
root@pc4all:/package# ri case
More than one method matched your request. You can refine
your search by asking for information on one of:
     Symbol#camelcase, Symbol#downcase, Symbol#downcase?, Symbol#upcase,

[...]

     String#lowercase?, String#upcase?, String#uppercase?,
     Regexp#casefold?

which show that ri gets more hits. i think ri is using regex?

With FastRI, "partial matches" are of the form
   /#{sep_re}#{name}/ for methods and
   /^#{name}/ for namespaces (classes/modules)
(where sep_re is '#', '\.' or '(#|\.)'), so it only tries to complete the
method and '#case' matches Any::NameSpace::AnyClass#case.*, but not .*case,
as it does in ri. It seems to me that method completion is more useful
normally.
If you want, I can add another search method (I don't really know how to name it
though), that behaves like ri. Note that fri will return the results given by
the first successful method:

$ fri -h
[...]
    -O, --order ORDER Specify lookup order.
                                     (default: eEnNpPxX)
                                     Uppercase: case-indep.
                                     e: exact, n: nested, p: partial
                                     x: nested and partial

So if I add the new "match anywhere" method at the end (say we call it 'a' and
'A' for the case-indep. variant, and then 'b'/'B' for nested+"match anywhere"),
doing -O eEnNpPxXaAbB you'll still get
      Regexp#casefold?, String#casecmp
because it will try partial matching (pP) first. So, in order for it to behave
more like ri, you'd need -O eEnNxXaAbB.

# > How do i make the fastri fixed to a certain port, say 12345? we're using
# > firewalls, so the concern..
#
# I'll implement this. Note that you'll also have to open the # port used by
# the # Rinda Ring (7647).

yes! thanks again.

It's already in HEAD.

···

On Mon, Nov 13, 2006 at 01:13:36PM +0900, Peña, Botp wrote:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby