Open letter to anyone developing a Ruby IDE

ruby-talk
1

It has been said that features like "intellisense" or "autocomplete" are
difficult to implement for dynamic languages such as Ruby.

I'd be happy with an IDE that let me choose a .rb file per project, let's
call it "environment.rb" which would usually be a list of "require"
statements which would establish a run-time state that would be a fair
approximation of what classes and methods would be available to 90% of my
application.

I mean, sure, dynamic languages let us add/remove classes variable and
methods anytime we like, but during the first moments of my application's
startup I usually set up 90% of the classes that I will need during the
remainder of the program's execution.

Adelle.

2

Hello Adelle,

I mean, sure, dynamic languages let us add/remove classes variable and
methods anytime we like, but during the first moments of my application's
startup I usually set up 90% of the classes that I will need during the
remainder of the program's execution.

The difficult thing is not really gathering of information. It is the
way how do you want to present the thousands of methods
found in your files and the runtime library.

Since type interference is quite impossible the only solution i came
up is some kind of a smalltalk class browser and a very clever
interface to find the right method with as less as possible shortcuts.
So any ideas here would be very helpful.

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

3

In article <courier.4249F4B9.0000089A@mars.sisgroup.com.au>, Adelle Hartley
wrote:

It has been said that features like "intellisense" or "autocomplete" are
difficult to implement for dynamic languages such as Ruby.

I'd be happy with an IDE that let me choose a .rb file per project, let's
call it "environment.rb" which would usually be a list of "require"
statements which would establish a run-time state that would be a fair
approximation of what classes and methods would be available to 90% of my
application.

I mean, sure, dynamic languages let us add/remove classes variable and
methods anytime we like, but during the first moments of my application's
startup I usually set up 90% of the classes that I will need during the
remainder of the program's execution.

How do you know which class is associated with a particular variable?

  a = somemethod(1, 2, 3)
  a.<TAB> # what methods are available? We have an approximation of
    # the classes/methods in the system, but no idea what
    # `a' corresponds to.

···

#
    # You could use "Foo.new is an 'atomic' type constraint,
    # everything else bubbles up from these kind of constraints",
    # but how reliable would that be?

.... although if you had decent unit test coverage, you could use the unit
tests to guess what variables probably correspond to.

"When we ran the unit tests, `a' was an instance of Foo, so assume for code
completion that `a' is a Foo."

+ do some simple inference from e.g. Foo.new.

--
Drunk, ignore if nonsensical.

4

Adelle Hartley wrote:

It has been said that features like "intellisense" or "autocomplete" are
difficult to implement for dynamic languages such as Ruby.

I've got *naive* method completion working in a development version of
the jEdit Ruby Editor Plugin. Not that difficult to implement after
all ...

In jEdit you can create file projects using the Project Viewer. I use
JRuby to parse the static contents of Ruby files in all projects and
create a cache of classes/modules/methods.

For completion I infer the possible classes of a variable, based on
what methods have already been called on it. Then I provide a popup of
the combined set of methods available for those classes, i.e. if it
quacks like a duck, it will probably waddle like one two.

Early days yet, but naive type based method completion is on the way.
I hope to have a working version out in a week's time; in the meantime
version 0.1 of the Ruby plugin for jEdit is available at:

Rob

5

I don't know if people who develop FreeRIDE, ArachnoRuby or other
IDEs/text editors that want to include intellisense know Smalltalk, but
the eCompletion
<http://homepage.mac.com/monique_bakker/squeak/eCompletion.html>
"add-on" (for lack of a better word) available in Squeak is quite nice.
Maybe it could be a source of inspiration for Ruby IDE developers.

Vincent

P.S: Lothar, I'm interested by your choice of language for ArachnoRuby;
it's the first Eiffel software (that I know of) that I've used. Any
reason for going with Eiffel instead of something more dynamic like
Ruby>Python>Smalltalk>Lisp? How's developing with Eiffel too? Thanks
for the insights.

6

Uh....

Where exactly does type inference come in for an auto-completion feature for an IDE?

Something so heavyweight as type inferencing is unnecessary for doing 90% of the work of figuring out what the likely completions for a particular chunk of code you're writing.

It would be far easier to just run etags over all the files you require and use those for auto-completion purposes.

vim has a workable solution for this (and I'm told emacs' solution is nearly, if not actually, identical), it simply auto-completes words that have the same beginning above your current location in the text before switching to words from other buffers. (So if I typed b-<TAB>-<TAB>-<TAB> here, I would get 'buffers' then 'before' then 'beginning' as possible completions.)

Extending this by reading files on require lines into etags would be pretty easy, and you could use some weighted sorting to keep commonly completed words only a few presses of tab away.

While this won't get an accurate completion every time, you really don't care. You'll find that 90% of the time the word you want is less than 3 presses of tab away.

···

On 29 Mar 2005, at 17:35, Lothar Scholz wrote:

Hello Adelle,

It has been said that features like "intellisense" or "autocomplete" are
difficult to implement for dynamic languages such as Ruby.

I'd be happy with an IDE that let me choose a .rb file per project, let's
call it "environment.rb" which would usually be a list of "require"
statements which would establish a run-time state that would be a fair
approximation of what classes and methods would be available to 90% of my
application.

I mean, sure, dynamic languages let us add/remove classes variable and
methods anytime we like, but during the first moments of my application's
startup I usually set up 90% of the classes that I will need during the
remainder of the program's execution.

The difficult thing is not really gathering of information. It is the
way how do you want to present the thousands of methods
found in your files and the runtime library.

Since type interference is quite impossible the only solution i came
up is some kind of a smalltalk class browser and a very clever
interface to find the right method with as less as possible shortcuts.
So any ideas here would be very helpful.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

7

Hi,

The difficult thing is not really gathering of information.
It is the way how do you want to present the thousands of
methods found in your files and the runtime library.

Since type interference is quite impossible the only solution
i came up is some kind of a smalltalk class browser and a
very clever interface to find the right method with as less
as possible shortcuts.
So any ideas here would be very helpful.

I'm writing code against an ORM layer, so the attributes of my object
variables are defined in my database schema, not in my source code.

I think the IDE needs to be running a Ruby debugger and capture it's state
(which variables, constants, classes, methods, etc are 'visible' at a
particular point).

The problem with this approach is that when examining a particular line of
code, there may be several routes by which one might reach that line of code
at run-time.

My suggestion presupposes the use of a debugger (which would pick up classes
and attributes that my ORM layer gets from the database), but rather than
having it execute my application, having it execute the first part of my
application (where 90% of my classes are defined) and then stop before my
application actually does anything.

The IDE would take its table of classes+attributes from state of the
debugger, and then combine that with any local definitions in the
file/module/class that I'm working on.

Adelle.

8

"Rob ." <rob.02004@gmail.com> wrote in message

For completion I infer the possible classes of a variable, based on
what methods have already been called on it.

What a nifty and simple heuristic! How well has it worked so far?

Will your completion list also show parameter names (just display, not
insert into buffer)? That would also be a nice touch.

9

Hi!

        a = somemethod(1, 2, 3)
        a.<TAB> # what methods are available? We have an approximation of
                # the classes/methods in the system, but no idea what
                # `a' corresponds to.
                #
                # You could use "Foo.new is an 'atomic' type constraint,
                # everything else bubbles up from these kind of constraints",
                # but how reliable would that be?

What about the IDE doing eval("a.methods")? Won't that cover the
general <expr>.<TAB> case?

Shajith

10

Unless I'm misunderstanding what Lothar is saying, there's no way to
give out only the methods that a particular object responds to. In
VisualWorks Smalltalk, when you use completion, you just get a list of
_ALL_ methods defined in the system beginning with your pattern.
There's nothinig in the code to let you know what class a particular
object is.

11

Eric Hodel <drbrain@segment7.net> writes:

While this won't get an accurate completion every time, you really
don't care. You'll find that 90% of the time the word you want is less
than 3 presses of tab away.

  Lothar probably forgot that arachno ruby has this feature already 8).
  Although it only uses tokens from current buffer only. But you still
  get 80% accuracy with that. Autocomplete is bound to control-space.
  Speeds up typing considerably.

- Ville

12

Hello Eric,

Where exactly does type inference come in for an auto-completion
feature for an IDE?

Something so heavyweight as type inferencing is unnecessary for doing
90% of the work of figuring out what the likely completions for a
particular chunk of code you're writing.

While this won't get an accurate completion every time, you really
don't care. You'll find that 90% of the time the word you want is less
than 3 presses of tab away.

Unfortunately this is not what people expect, at least when they come
from the Java world. Code Completition is a learning tool not
primarily to save some keystrokes. You slowly learn the available
messages for a class while writing your code. Therefore a much better
way then showing a huge list with hundrets of strings list should be
available.

And methods have the heuristic that they share a common
prefix and differ only the last characters. This makes your
(emacs/vi) completition even more useless.

Thats why for example Wing IDE for python is so successful. They even
encourage their users to use type asserts, which is IMHO a good thing.
As a result they can show some precise information.

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

13

Also, this[1] or something similar could probably get you those hints you need, and again be accurate 90% of the time.

http://www.chadfowler.com/index.cgi/Computing/Programming/Ruby/TypeWatching.rdoc,v

···

On 29 Mar 2005, at 21:00, Eric Hodel wrote:

It would be far easier to just run etags over all the files you require and use those for auto-completion purposes.

vim has a workable solution for this (and I'm told emacs' solution is nearly, if not actually, identical), it simply auto-completes words that have the same beginning above your current location in the text before switching to words from other buffers. (So if I typed b-<TAB>-<TAB>-<TAB> here, I would get 'buffers' then 'before' then 'beginning' as possible completions.)

Extending this by reading files on require lines into etags would be pretty easy, and you could use some weighted sorting to keep commonly completed words only a few presses of tab away.

While this won't get an accurate completion every time, you really don't care. You'll find that 90% of the time the word you want is less than 3 presses of tab away.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

14

Eric Hodel wrote:

Extending this by reading files on require lines into etags
would be pretty easy

Zeus for Windows uses ctags as it's source of tags information
and this is used to provide a limited form of auto-completion.

But unfortunately providing good intellisensing is far from
simple. The reason is the information produced buy ctags is
class based and looks very nice in a class browser window.

But the programmer works with instances of those classes and
the hardest part of implementing good intellisensing is
correctly identifying to which class the current instance
belongs.

This is not as simple as it may seem as it depends a lot on
the current scope. For example a local variable, a global
variable, and a member variable can all have the same name
yet all be of different class. But of the three only one will
be correct and which one depends on the current scope.

Jussi Jumppanen
Author of: Zeus for Windows (New version 3.94 out now)
"The C/C++, Cobol, Java, HTML, Python, PHP, Ruby folding editor"
Home Page: http://www.zeusedit.com

15

itsme213 wrote:

"Rob ." wrote:
> For completion I infer the possible classes of a variable, based on
> what methods have already been called on it.

What a nifty and simple heuristic! How well has it worked so far?

It works. Obviously the more unique method names are, the better the
type matching will be. Now I have a class/method cache implemented in
Java the performance issue has gone away. Previously I was doing
multiple calls out to ri from Java, which was very slow.

Outstanding technical issues include keeping the cache uptodate as the
code is being edited, and recognizing parent class methods and mixins.

As to whether it's useful, not sure, haven't done any Ruby development
with such a feature available before. I tend to think that navigation
features like, "find usages" and "find declarations" can prove more
beneficial during development.

Will your completion list also show parameter names (just display, not
insert into buffer)? That would also be a nice touch.

Yes, the completion list shows parameter names - it's inserting them
too at the moment.

16

Shajith wrote:

       a = somemethod(1, 2, 3)
       a.<TAB> # what methods are available? We have an approximation of
               # the classes/methods in the system, but no idea what
               # `a' corresponds to.
               #
               # You could use "Foo.new is an 'atomic' type constraint,
               # everything else bubbles up from these kind of constraints",
               # but how reliable would that be?

What about the IDE doing eval("a.methods")? Won't that cover the
general <expr>.<TAB> case?

What if a is `rm -rf /`? It might not always be possible to execute code as it might have side effects which I think is also the reason for IRB's rather simplistic method completion...

17

Weird, I was just working on this problem this morning in the index
feature for fastcst. I have some code right now which indexes the
source using suffix arrays and then lets you quickly search for stuff in
the source. It could easily be "stolen" and turned into a code
completion thing I think. Here's some of the results of doing
"index .join" on my source:

./lib/fastcst/operation.rb:
        "File.join(target, path)"
./lib/fastcst/command/finish.rb:
        "File.join(@repo.work_dir, UNDO_JOURNAL)"
./lib/fastcst/command/list.rb:
        "\#{readable_path.join('/')} ]\""
./lib/fastcst/command/disp.rb:
        "File.join(@repo.work_dir, MetaData::META_DATA_FILE)"
./lib/fastcst/command/merge.rb:
        "File.join(@repo.work_dir, MetaData::META_DATA_FILE)"

Which is basically all the places where a .join "thing" is used, and
probably enough information to figure out what possible parameters
the .join takes.

The "completion" could then simply be when you start to type a certain
sequence (say "." followed by another char) it would do a search like
this, collapse "similar" stuff and present that as the completion. Neat
thing is this works with any source, not just Ruby, and requires no
parsing.

This is also fairly quick even though I'm storing the index in a
serialized hash table.

Anyway, is this kind of what you're looking for?

Zed

> Hello Adelle,
>
>> It has been said that features like "intellisense" or "autocomplete"
>> are
>> difficult to implement for dynamic languages such as Ruby.
>>

<snip>

Where exactly does type inference come in for an auto-completion
feature for an IDE?

<snip>

···

On Wed, 2005-03-30 at 14:00 +0900, Eric Hodel wrote:

On 29 Mar 2005, at 17:35, Lothar Scholz wrote:

18

But you need to execute the entire program right down to the line where completion is requested for that eval() return anything useful. A program could have many "a" variables. Executing the code is an unrealistic requirement; way too many possible external dependancies.

James Edward Gray II

···

On Mar 30, 2005, at 6:19 AM, Shajith wrote:

Hi!

        a = somemethod(1, 2, 3)
        a.<TAB> # what methods are available? We have an approximation of
                # the classes/methods in the system, but no idea what
                # `a' corresponds to.
                #
                # You could use "Foo.new is an 'atomic' type constraint,
                # everything else bubbles up from these kind of constraints",
                # but how reliable would that be?

What about the IDE doing eval("a.methods")? Won't that cover the
general <expr>.<TAB> case?

19

Hello Eric,

> Where exactly does type inference come in for an auto-completion
> feature for an IDE?

> Something so heavyweight as type inferencing is unnecessary for doing
> 90% of the work of figuring out what the likely completions for a
> particular chunk of code you're writing.

> While this won't get an accurate completion every time, you really
> don't care. You'll find that 90% of the time the word you want is less
> than 3 presses of tab away.

Unfortunately this is not what people expect, at least when they come
from the Java world. Code Completition is a learning tool not
primarily to save some keystrokes. You slowly learn the available
messages for a class while writing your code. Therefore a much better
way then showing a huge list with hundrets of strings list should be
available.

Have you used Readline tab completion in irb? It is far from useful, giving you too many potential completions from too far up the inheritance tree.

And methods have the heuristic that they share a common
prefix and differ only the last characters. This makes your
(emacs/vi) completition even more useless.

In the cases where I've found methods that share a common prefix, I've found it rare that I'm actually calling those methods from regular code. Test::Unit is the best example of this. I have many, many methods starting with test_ but I never call them.

In the cases in Ruby where there are methods that share a common prefix, #each comes to mind, I don't see how this is useless at all.

I type ea<tab> and see "each" then I hit tab again "each_line" and again "each_with_index" which is the one I wanted. I don't see how that's less useful than the ones I've seen in MS' VB editors. (Besides, I'd never tab-complete 'each', all but the 'c' is on the home row of a Dvorak keyboard.)

If you still think its totally useless I'd like to see a series of keystrokes that show how it falls down.

FWIW, ri says there's the following possible each completions:
Core: #each #each_line, #each_with_index, #each_value, #each_pair, #each_key, #each_byte, #each_object, #each_pair
Pathname: #each_entry, #each_filename
YAML: #each_document, #each_node
Vector: #each2

You can only get those last 5 by requiring extra libraries.

···

On 29 Mar 2005, at 21:30, Lothar Scholz wrote:

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

20

Eric Hodel wrote:

http://www.chadfowler.com/index.cgi/Computing/Programming/Ruby/ TypeWatching.rdoc,v

It would be nice to see this somehow integrated in IDE's, but this still means the code needs to be executed somehow before the typehints are available, not?

Regards,

Peter