[ANN] FastRI 0.2.0: full-text searching, smarter search strategies

FastRI is an alternative to the ri command-line tool. It is *much* faster, and
also allows you to offer RI lookup services over DRb. FastRI is smarter than
ri, and can find classes anywhere in the hierarchy without specifying the
"full path". FastRI can perform full-text searching. Its RubyGems support is
better than ri's, and it knows which gem a method/class definition came from.

Getting it

···

==========
Additional information, tarballs... at
  http://eigenclass.org/hiki.rb?fastri

FastRI can be installed with RubyGems:
  gem install fastri
(if you get an old version/a 404 error, please allow some time after the
release until the package propagates to the RubyForge mirrors). Please read
below for an important note regarding the RubyGems packages.

Changes since version 0.1.1 (2006-11-10)

Features
--------
* fri can do full-text search (-S, --full-text); try fri -S byte order
* fri can now determine where a method actually came from for core classes
  e.g. fri File.inject -> docs for Enumerable#inject
* you can specify which ports the DRb services must bind to:
    fastri-server -s 192.168.1.2:54321
    fri -s 192.168.1.2:12345
* new search methods: "anywhere" (a) and "anywhere, case-indep." (A)

Acknowledgements

Tomasz Wegrzanowski
* contributed the code that allows fri to find a method in the ancestors
  for core classes

Usage

There are two parts to FastRI:
* the server: fastri-server
* the client: fri

FastRI uses a Rinda Ring to allow servers to be discovered automatically
without needing to indicate the DRb URIs manually. It can work across
machines if you make sure the ring server is bound to the correct interface,
and the ACL permissions are correct.

Examples

$ fastri-server (creates the index on the first run, blocks)

Later, (times measured with a cold cache):
$ time ruby bin/fri -f plain Array#fetch
------------------------------------------------------------ Array#fetch
      array.fetch(index) -> obj
[...]
real 0m0.287s (real 0m0.127s with a hot cache)
user 0m0.048s
sys 0m0.008s

Compare to:
$ time ri -T -f plain Array#fetch
------------------------------------------------------------ Array#fetch
[...]
real 0m10.136s (real ~ 1.5s with a hot cache)
user 0m1.140s
sys 0m0.464s

This illustrates FastRI's ability to locate classes deep in the class
hierarchy:

$ fri Base
------------------------------------------------------ Multiple choices:

      ActionMailer::Base, ActionView::Base, ActionWebService::API::Base,
      ActionWebService::Base, ActionWebService::Client::Base,
      ActiveRecord::Base, MapReduce::ActiveRecord::Base,
      RSS::Maker::Base, Scruffy::Components::Base,
      Scruffy::Formatters::Base, Scruffy::Layers::Base,
      Scruffy::Renderers::Base, Scruffy::Themes::Base

$ fri Themes::Base
------------------------------------------- Class: Scruffy::Themes::Base
      Scruffy::Themes::Base
      Author: Brasten Sager

      Date: August 14th, 2006

Compare to
$ ri Themes::Base .... several seconds later ...
Nothing known about Themes::Base

A small note about RubyGems + FastRI.

RubyGems adds a noticeable overhead to fri, making it run slower than if you
installed it directly from the tarball with setup.rb.

Compare the execution time when installed with RubyGems:
  $ time fri -f plain String > /dev/null

  real 0m0.385s
  user 0m0.244s
  sys 0m0.036s

to the time fri actually takes to run, without the overhead introduced by
RubyGems:
  $ time ruby bin/fri -f plain String > /dev/null

  real 0m0.088s
  user 0m0.040s
  sys 0m0.008s

If you care about those extra 300ms (and there are situations where they will
matter, e.g. when using fri for method completion), get FastRI from the
tarballs.

License

FastRI is licensed under the same terms as Ruby. See LICENSE.

Feedback

Bug reports, patches, comments... are appreciated.
You can contact the author via <mfp@acm.org>. Please add "fastri" to the
subject in order to bypass the spam filters.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

I like fastri :slight_smile:

Did you implement a way to tell fri about the default pager it should use?
On the 0.1.1 announcement thread you said you probably would have done this in 0.2.0.
I tried by setting a PAGER envvar but it didn't work (many unix tools check for that envvar, "PAGER=cat man man" is just an example).

···

--
Gabriele Marrone

Hi,

remote access doesn't work for me. Anyone got this up an running? I did
something like:

fastri-server -a 192.168.100.0/24 -s 192.168.100.163
Looking for Ring server...
No Ring server found, starting my own.
$ fastri-server 0.0.1 (FastRI 0.2.0) listening on
druby://192.168.100.163:35800
ACL:
deny all
allow 127.0.0.1
allow 192.168.100.0/24

Local access works just fine, but from remote I got:
$ fri -s 192.168.100.163 Array
Couldn't initialize DRb and locate the Ring server.

Any ideas?

Regards

Thomas

···

--
Posted via http://www.ruby-forum.com/.

I like fastri :slight_smile:

:slight_smile:

Did you implement a way to tell fri about the default pager it should use?
On the 0.1.1 announcement thread you said you probably would have done this
in 0.2.0. I tried by setting a PAGER envvar but it didn't work (many unix
tools check for that envvar, "PAGER=cat man man" is just an example).

Sorry, it's not in 0.2.0, but you can find it in HEAD; just get
http://eigenclass.org/repos/fastri/head/bin/fri

$ fri -h

Usage: fri [options] <query>
    -s, --bind ADDR Bind to ADDR for incoming DRb connections.
                                     (default: 127.0.0.1)
    -O, --order ORDER Specify lookup order.
                                     (default: eEnNpPxX)
                                     Uppercase: case-indep.
                                     e:exact n:nested p:partial (completion)
                                     x:nested and partial
        --show-matches Only show matching entries.
    -S, --full-text Perform full-text search.
    -F, --full-text-dir DIR Use full-text index in DIR
                                     (default: /home/batsman/.fastri-fulltext)
    -f, --format FMT Format to use when displaying output:
                                        ansi, plain (default: ansi)
    -P, --[no-]pager Use pager.
                                     (default: don't)
        --pager-cmd PAGER Use pager PAGER.
                                     (default: don't)
    -w, --width WIDTH Set the width of the output.
    -h, --help Show this help message

fri will use a pager by default in full-text search mode, (either the one
given to --pager-cmd, ENV["PAGER"], "less", "more" or "pager", in that order)
but *not* in normal mode, unless you ask for it with -P.

You can alias fri="fri -P" to always use a pager, and it will honor the PAGER
env. variable.

I might as well implement option parsing for ENV["FRI_OPTIONS"].

···

On Fri, Nov 17, 2006 at 12:12:52AM +0900, Gabriele Marrone wrote:
                                     a:match method name anywhere
--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Well on a cisco router, setting up an ACL like that would prevent anyone
from accessing it because it would hit the first rule (deny all) and then
stop. Now, I'm a Ruby newbie, and I've never tried what you're trying, but
that's the first thing that I would check. Can you put the deny all at the
end of the ACL list?

···

On 11/21/06, Thomas Nitsche <thomas.nitsche@gmail.com> wrote:

Hi,

remote access doesn't work for me. Anyone got this up an running? I did
something like:

fastri-server -a 192.168.100.0/24 -s 192.168.100.163
Looking for Ring server...
No Ring server found, starting my own.
$ fastri-server 0.0.1 (FastRI 0.2.0) listening on
druby://192.168.100.163:35800
ACL:
deny all
allow 127.0.0.1
allow 192.168.100.0/24

Local access works just fine, but from remote I got:
$ fri -s 192.168.100.163 Array
Couldn't initialize DRb and locate the Ring server.

Any ideas?

Regards

Thomas

--
Posted via http://www.ruby-forum.com/\.

remote access doesn't work for me. Anyone got this up an running? I did
something like:

fastri-server -a 192.168.100.0/24 -s 192.168.100.163

···

On Wed, Nov 22, 2006 at 12:34:29AM +0900, Thomas Nitsche wrote:
                                   ====================

Looking for Ring server...
No Ring server found, starting my own.
$ fastri-server 0.0.1 (FastRI 0.2.0) listening on
druby://192.168.100.163:35800
ACL:
deny all
allow 127.0.0.1
allow 192.168.100.0/24

Local access works just fine, but from remote I got:
$ fri -s 192.168.100.163 Array

         =================

Couldn't initialize DRb and locate the Ring server.

The address given to fri -s ADDR is not the address of the server (that one
is autodiscovered using the Ring), but the address the local DRb service must
bind to (in short, fri also exports a DRb service that will be used by the
remote fastri-server to tell fri where it is).

So, if your server is 192.168.100.163 and your client is running on
192.168.100.164, you'd have to do

(in 192.168.100.163)
$ fastri-server -a 192.168.100.0/24 -s 192.168.100.163

(in 192.168.100.164)
$ fri -s 192.168.100.164 Array

or

$ export FASTRI_ADDR=192.168.100.164
$ fri Array

I'm revisiting that code to see if I can make 'fri Array' work without
specifying the local address, but for the time being the above should work.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

:From => "Mauricio Julio Fernández Pradier"
# On Fri, Nov 17, 2006 at 12:12:52AM +0900, Gabriele Marrone wrote:
# > I like fastri :slight_smile:
# :slight_smile:

Hi Mauricio,

1) it seems that fri does not implem multiple queries yet? eg

C:\temp>fri upcase downcase
---------------------------------------------------------- String#upcase
     str.upcase => new_str

···

------------------------------------------------------------------------
     Returns a copy of _str_ with all lowercase letters replaced with
     their uppercase counterparts. The operation is locale
     insensitive---only characters ``a'' to ``z'' are affected.

        "hEllO".upcase #=> "HELLO"

C:\temp>

yet

C:\temp>ri -f plain -T upcase downcase
---------------------------------------------------------- String#upcase
     str.upcase => new_str
------------------------------------------------------------------------
     Returns a copy of _str_ with all lowercase letters replaced with
     their uppercase counterparts. The operation is locale
     insensitive---only characters ``a'' to ``z'' are affected.

        "hEllO".upcase #=> "HELLO"

-------------------------------------------------------- String#downcase
     str.downcase => new_str
------------------------------------------------------------------------
     Returns a copy of _str_ with all uppercase letters replaced with
     their lowercase counterparts. The operation is locale
     insensitive---only characters ``A'' to ``Z'' are affected.

        "hEllO".downcase #=> "hello"

2) also, can we allow a -T for fri (since some do like pager while others like do not --sometimes. i prefer piping and teeing and all output in one :slight_smile:

C:\temp>fri -f plain -T upcase downcase
c:/ruby/lib/ruby/1.8/optparse.rb:1381:in `complete': invalid option: -T (OptionParser::InvalidOption)

3) it would be nice if queries (if possible) may return related topics. eg, something like, "Related methods: capitalize, downcase, upcase, trim, ltrim, etc...". This is very good for noobies like me. this may be quite a hard work since we may need an index..

thanks for fri. it really is fast even on mswindows.
kind regards -botp

Thx a lot. It works perfectly that way. It's almost sort of RTFM, but I
misunderstood the sentence "Note that FASTRI_ADDR is the *local*
address: the server will be discovered automatically."

Regards,

Thomas

···

--
Posted via http://www.ruby-forum.com/.

1) it seems that fri does not implem multiple queries yet? eg

oh, I'd never used that with ri (actually, I didn't use ri that much, for the
reason you know)

2) also, can we allow a -T for fri (since some do like pager while others
like do not --sometimes. i prefer piping and teeing and all output in one :slight_smile:

In HEAD, you have
$ fri -h
...
    -P, --[no-]pager Use pager.
                                     (default: don't)
        --pager-cmd PAGER Use pager PAGER.
                               (default: don't)

By default, it doesn't use a pager, unless you are doing full-text searches.

3) it would be nice if queries (if possible) may return related topics. eg,
something like, "Related methods: capitalize, downcase, upcase, trim, ltrim,
etc...".

Yes, I'm thinking about the best way to achieve this; the obvious candidate is
latent semantic indexing.

This is very good for noobies like me. this may be quite a hard work since
we may need an index..

You cannot keep playing the newbie card for years :wink: ... you've been around
since Jan 2001 at least! (I found your 1st post, according to blade, in
RubyConf's foundational thread [ruby-talk:9868]!)

···

On Fri, Nov 17, 2006 at 11:08:56AM +0900, Peña, Botp wrote:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby