Ruby and science?

ruby-talk
1

I am really puzzled.

I am a scientist and I have been using ruby for years. I like thinking
and programming in ruby, but feel more and more uneasy. I'll try to
express my feelings below (I am not a native English speaker).

The ruby language and the ruby ecosystem (and the wonderful ruby
community) fit perfectly with small applications, scripting, web
frameworks, and so on. When you come to science, it is another story.
Numerical computing, statistics, whatever, cannot be coded in pure ruby
and you have to rely on specialized libraries. There are tentative gems
(generally obsolete and not maintained) to connect ruby to these
librairies.

Take for instance 'linalg' for linear algebra. It is based on LAPACK.
You need a Fortran to c converter, but f2c was deprecated, was replaced
by g2c, and g2c is deprecated but not replaced, waiting for gfortran...
Moreover, the compilation chain uses uncompatible versions of the c
compiler (or even of ruby itself). Whether on Linux (Ubuntu Lucid) or on
Windows, I had to give up. I understand the situation is not different
on OSX.

The situation is quite similar when you try to connect to the R language
for statistics, or whatever similar. The web is full of old posts asking
for help in similar situations with zero answer.

Am I wrong ? Maybe I am too clumsy. But, after all, a programming
environment is a tool to do your job and deliver. My job is to deliver
science and I end spending my time compiling (I mean failing to compile)
libraries.

The advice I usually get is : shift to Python. A very sad perspective
indeed.

More generally, is there a future for ruby outside its present niches ?

_md

···

--
Posted via http://www.ruby-forum.com/.

2

Am I wrong ? Maybe I am too clumsy. But, after all, a programming
environment is a tool to do your job and deliver. My job is to deliver
science and I end spending my time compiling (I mean failing to compile)
libraries.

If you need the proper tools Right Now, then, well, you have to pick
another tool from your toolbox.

The advice I usually get is : shift to Python. A very sad perspective
indeed.

More generally, is there a future for ruby outside its present niches ?

Of course there is. However, the support (especially in the sciences)
requires two abilities: A solid understanding of the problem space,
and a solid understanding of programming to implement solutions that
fit the problem space.

IOW: scientific Ruby libraries need scientists who can write Ruby code
(and/or C code to wrap binaries or to provide Ruby interfaces to
tools).

I'd love to do my computations for EE in Ruby however, since there's
pretty much no support for the kinds of maths I need to use (linear
algebra, complex numbers), I'm pretty much stuck with Maple. And I
neither have the time, nor the skill, to implement this properly for
other people using Ruby. Unfortunately (though, a complex number
library shouldn't be too difficult to implement).

Though, I wonder:

Numerical computing, statistics, whatever, cannot be coded in pure ruby
and you have to rely on specialized libraries.

Why? Unless you need more accuracy than the IEEE float provides, you
should be fine in Ruby, unless performance is a major issue, where you
can drop down to C code, if need be (for example with Ruby Inline[0]).

Or is the data parsing / presentation / plotting the issue?

[0] rubyinline | software projects | by ryan davis

···

On Thu, Dec 16, 2010 at 11:19 AM, Michel Demazure <michel@demazure.com> wrote:
--
Phillip Gawlowski

Though the folk I have met,
(Ah, how soon!) they forget
When I've moved on to some other place,
There may be one or two,
When I've played and passed through,
Who'll remember my song or my face.

3

Have you seen Ruby/GSL?

  http://rb-gsl.rubyforge.org/

It just got a new release last month, and looks like it has decent
coverage of GSL.

Regards,
Ammar

···

On Thu, Dec 16, 2010 at 12:19 PM, Michel Demazure <michel@demazure.com> wrote:

Take for instance 'linalg' for linear algebra. It is based on LAPACK.
You need a Fortran to c converter, but f2c was deprecated, was replaced
by g2c, and g2c is deprecated but not replaced, waiting for gfortran...
Moreover, the compilation chain uses uncompatible versions of the c
compiler (or even of ruby itself). Whether on Linux (Ubuntu Lucid) or on
Windows, I had to give up. I understand the situation is not different
on OSX.

4

Have you considered trying to use a library like JAMA with JRuby?

http://math.nist.gov/javanumerics/jama/

···

On Thu, Dec 16, 2010 at 3:19 AM, Michel Demazure <michel@demazure.com>wrote:

Take for instance 'linalg' for linear algebra. It is based on LAPACK.
You need a Fortran to c converter, but f2c was deprecated, was replaced
by g2c, and g2c is deprecated but not replaced, waiting for gfortran...
Moreover, the compilation chain uses uncompatible versions of the c
compiler (or even of ruby itself). Whether on Linux (Ubuntu Lucid) or on
Windows, I had to give up. I understand the situation is not different
on OSX.

--
Tony Arcieri
Medioh! Kudelski

5

i worked in science using ruby for many years. it is true that sometimes
various libs (narray, ruby gsl, netcdf, opendap bindings, etc) require some
tweaking to compile and get running. my observation, during the decade or
so doing this work, is that this issue is not exclusive to ruby at all: if
one is going to be a serious scientific programmer then one needs to
understand all sorts of things 'normal' computer scientists don't normally
consider any more: compiler flags, endianness, huge word sizes, specicalized
hardware interfaces, dedicated super computer compiler tool chains, and
generally shoddy source control and distribution mechanisms (here - i wrote
this for f77 10 years ago...) dominate the scientific programming landscape.
my experience has been that this is as true for python as it is for fortran
as it is for ruby. understanding the black art of object files, linkers,
and compiler implementations is just a fact of life when one is pushing the
boundaries of what machines do commonly. it is no reason to give up on
ruby, or any other language for that matter. if you choose, say, R, then
maybe you'll solve certain problems more quickly, but hit roadblocks in
others, like accessing data via some obscure ftp based protocol... needing
to understand complier black art to make computers go fast i can live with,
but hacking makefiles to do common tasks i abhor - pick your poison.

6

Hi Michel,
I think you are completely correct in your criticism of ruby's lack of
support for scientific computing, and complex graphing. I think the
problem is that python had a bit of a lead time (~5 yrs.) before ruby
became popular. During that period, many of the scientists switched
from perl to python and began developing the tools they needed. When
Ruby was popularized with rails, python was already established. So
most of the talent that could solve this problem for the ruby
community is already happily doing science with python and has no
reason to switch to a slightly more elegantly designed language (as
some would argue). I learned some R and use ruby to crunch the numbers
and R to plot them. I think it is a reasonable solution. If I did a
lot of data crunching/graphing, I might write a DSL that allowed R
plotting code to be made via a nicer ruby-like syntax, but so far I
haven't been motivated enough. Alternatively, I would switch to
python, I think it would take a couple of weeks to get up to speed as
the two are quite similar conceptually. I think it would be great if
we had a dedicated scientific community to build the tools in ruby,
but I don't see it happening, because python has already filled the
niche.
Tim

···

On Dec 16, 2:19 am, Michel Demazure <mic...@demazure.com> wrote:

I am really puzzled.

I am a scientist and I have been using ruby for years. I like thinking
and programming in ruby, but feel more and more uneasy. I'll try to
express my feelings below (I am not a native English speaker).

The ruby language and the ruby ecosystem (and the wonderful ruby
community) fit perfectly with small applications, scripting, web
frameworks, and so on. When you come to science, it is another story.
Numerical computing, statistics, whatever, cannot be coded in pure ruby
and you have to rely on specialized libraries. There are tentative gems
(generally obsolete and not maintained) to connect ruby to these
librairies.

Take for instance 'linalg' for linear algebra. It is based on LAPACK.
You need a Fortran to c converter, but f2c was deprecated, was replaced
by g2c, and g2c is deprecated but not replaced, waiting for gfortran...
Moreover, the compilation chain uses uncompatible versions of the c
compiler (or even of ruby itself). Whether on Linux (Ubuntu Lucid) or on
Windows, I had to give up. I understand the situation is not different
on OSX.

The situation is quite similar when you try to connect to the R language
for statistics, or whatever similar. The web is full of old posts asking
for help in similar situations with zero answer.

Am I wrong ? Maybe I am too clumsy. But, after all, a programming
environment is a tool to do your job and deliver. My job is to deliver
science and I end spending my time compiling (I mean failing to compile)
libraries.

The advice I usually get is : shift to Python. A very sad perspective
indeed.

More generally, is there a future for ruby outside its present niches ?

_md

--
Posted viahttp://www.ruby-forum.com/.

7

Hi All,

I know this is an old post... but I´m new to the subject. I don´t know
if people will read this or not. Anyhow, I´ve just implemented a Gem
called MDArray which is available at RubyGem and GitHub
(https://github.com/rbotafogo/mdarray). It implements a multi
dimensional array in the spirit of numpy. It is far from numpy
performance and features, but I´m planning to keep adding features. I
think it should be functional for what it does, but still limited. It
is targeted to JRuby as it uses Java-NetCDF library.

Hope this can help someone... cheers,

Rodrigo

···

--
Posted via http://www.ruby-forum.com/.

8

Phillip Gawlowski wrote in post #968809:

Numerical computing, statistics, whatever, cannot be coded in pure ruby
and you have to rely on specialized libraries.

Why? Unless you need more accuracy than the IEEE float provides, you
should be fine in Ruby, unless performance is a major issue, where you
can drop down to C code, if need be (for example with Ruby Inline[0]).

The issue is internal representation. For instance a matrix 1000 x
100000, even of small ints. You need an internal representation of
Fortran type. Actually, 'narray' does that perfectly, but is very
limited in scope (only one real method : LU decomposition). The good
answer seems an interface to the big tools : MathLab, SciLab,... This is
more or less what PyMaths is about, as I understand well.

More generally, bigs things with many indices cannot be really managed
with towers of enumerators...

_md

···

--
Posted via http://www.ruby-forum.com/\.

9

Ammar Ali wrote in post #968816:

Have you seen Ruby/GSL?

  http://rb-gsl.rubyforge.org/

It just got a new release last month, and looks like it has decent
coverage of GSL.

Good news. I'll look. Thanks !
_md

···

--
Posted via http://www.ruby-forum.com/\.

10

Tony Arcieri wrote in post #968904:

Have you considered trying to use a library like JAMA with JRuby?

JAMA: Java Matrix Package

I am still reluctant to a jump to jruby. I'll first check GSL on MRI
(1.9.2). But thanks for the useful link.

···

--
Posted via http://www.ruby-forum.com/\.

11

And thanks to Rocket Scientists™, it does Fortran too!

···

On Dec 16, 2010, at 02:34 , Phillip Gawlowski wrote:

Why? Unless you need more accuracy than the IEEE float provides, you
should be fine in Ruby, unless performance is a major issue, where you
can drop down to C code, if need be (for example with Ruby Inline[0]).

[0] rubyinline | software projects | by ryan davis

12

Curious if you've used RinRuby and what you think of it?

Jose

···

On Dec 21, 2010, at 12:15 AM, timr <timrandg@gmail.com> wrote:

If I did a
lot of data crunching/graphing, I might write a DSL that allowed R
plotting code to be made via a nicer ruby-like syntax, but so far I
haven't been motivated enough.

==============
Jose Hales-Garcia
UCLA Statistics

13

the two are quite similar conceptually. I think it would be great if
we had a dedicated scientific community to build the tools in ruby,
but I don't see it happening, because python has already filled the
niche.
Tim

mostly true. worth mentioning, however, is that scientific visualization is
having a bit of a crisis, largely because of two facts:

- data is increasingly open and accessed via an api, not stored locally
(kml, opendap/netcdf, various home spun http protocols, etc)

- people want to interactive data that can be shared

that's why there is so much interest in svg/js/canvas libraries: one can
access open data with open protocols and produce interactive charts that can
be shared online using standards-ish tools (see
http://vis.stanford.edu/protovis/ for and example...)

there is new work to do, however, many toolsets are, unfortunately, bound to
to dom. great value can be made by programming toolsets which work in
http/svg and which use open data access standards, but which are not bound
to a browser only runtime environment.

ruby is actually in an excellent position to fill this void. food for
thought.

···

On Tuesday, December 21, 2010 1:15:32 AM UTC-7, Tim Rand wrote:

14

That's true. But it's not like they are much more manageable in for
and while loops, either. But having to deal with enumerations, instead
of "just" using a method call, is cumbersome at best, error prone at
worst, especially when working on the code alone.

···

On Thu, Dec 16, 2010 at 11:54 AM, Michel Demazure <michel@demazure.com> wrote:

The issue is internal representation. For instance a matrix 1000 x
100000, even of small ints. You need an internal representation of
Fortran type. Actually, 'narray' does that perfectly, but is very
limited in scope (only one real method : LU decomposition). The good
answer seems an interface to the big tools : MathLab, SciLab,... This is
more or less what PyMaths is about, as I understand well.

More generally, bigs things with many indices cannot be really managed
with towers of enumerators...

--
Phillip Gawlowski

Though the folk I have met,
(Ah, how soon!) they forget
When I've moved on to some other place,
There may be one or two,
When I've played and passed through,
Who'll remember my song or my face.

15

Michel Demazure wrote in post #968819:

Ammar Ali wrote in post #968816:

Have you seen Ruby/GSL?

  http://rb-gsl.rubyforge.org/

It just got a new release last month, and looks like it has decent
coverage of GSL.

Good news. I'll look. Thanks !
_md

Works ! (Ubuntu Lucid)
_md

···

--
Posted via http://www.ruby-forum.com/\.

16

Doing fairly intensive actuarial type calculations I've found:
1. JRuby is currently(?) and anecdotally somewhat slower than MRI 1.9, but
not so much slower as to be a problem.
2. Integration with Java seems to be easy: I'm not a Java programmer, but
I've found it easy to write Java code to do the number crunching, and mostly
easy to integrate the "compiled" Java code with JRuby. (I say mostly because
at the start I couldn't find a way to compile the Java code in a way that
would reliably work with JRuby, but that was essentially me not
understanding how Java packages really worked. I still don't understand how
Java packages really work, but I've found a way to compile that reliably
works for me with JRuby!) That's a big plus because I definitely don't
understand at the moment how to compile C code and integrate that with MRI
Ruby.

···

On Thu, Dec 16, 2010 at 6:36 PM, Michel Demazure <michel@demazure.com>wrote:

Tony Arcieri wrote in post #968904:
>
> Have you considered trying to use a library like JAMA with JRuby?
>
> JAMA: Java Matrix Package

I am still reluctant to a jump to jruby. I'll first check GSL on MRI
(1.9.2).

17

Ryan Davis wrote in post #968969:

···

On Dec 16, 2010, at 02:34 , Phillip Gawlowski wrote:
>
And thanks to Rocket Scientists, it does Fortran too!

Which means my problem was just finding where to look up !

Actually, is there a place, with a FAQ like : if you want to do this
with ruby, then use that ?

_md

--
Posted via http://www.ruby-forum.com/\.

18

I've had good success with rsruby. It isn't the nicest or terribly complete, but by embedding R it is a LOT faster (10x by my measurements) than calling out to R and running R scripts. Usually I'm just doing:

  RSRuby.instance.eval_R r_src

But there is API to do data conversion back/forth. I found it to be a bit clunky and in some cases buggy, but for the most part I'm usually just using R for visualization, not lots of data transfer.

···

On Dec 21, 2010, at 06:09 , Jose Hales-Garcia wrote:

On Dec 21, 2010, at 12:15 AM, timr <timrandg@gmail.com> wrote:

If I did a
lot of data crunching/graphing, I might write a DSL that allowed R
plotting code to be made via a nicer ruby-like syntax, but so far I
haven't been motivated enough.

Curious if you've used RinRuby and what you think of it?

19

> Tony Arcieri wrote in post #968904:
> > Have you considered trying to use a library like JAMA with JRuby?
> >
> > JAMA: Java Matrix Package
>
> I am still reluctant to a jump to jruby. I'll first check GSL on MRI
> (1.9.2).

Doing fairly intensive actuarial type calculations I've found:
1. JRuby is currently(?) and anecdotally somewhat slower than MRI 1.9, but
not so much slower as to be a problem.

And JRuby is getting faster all the time. It's not clear whether one will
necessarily beat the other.

In particular, I remember hearing discussions of a commandline flag in JRuby
which one could use to disallow altering methods on the core numeric types.
This would basically make Ruby math compile down to Java math. I imagine most
scientific applications wouldn't care about altering the core numeric types,
while most scientific applications would care about fast math.

2. Integration with Java seems to be easy: I'm not a Java programmer,

I am, and it is _trivial_. Every now and then I open up irb and pull in some
Java things, just for fun:

require 'java'
import java.util.ArrayList
a = ArrayList.new
a << 5
a << 'ten'
a.each{|x| puts x}

Basically, you think "How would I want this to work?" And then you "require
'java'" and do it, and it works. It's beautiful.

About the only unintuitive thing I ever found was implementing a Java
interface, and while it's somewhat unintuitive, it's still trivial:

require 'java'
import java.util.PriorityQueue
import java.util.Comparator

# singleton comparator
comp = Class.new {
  include Comparator
  def compare a,b
    a.to_s <=> b.to_s
  end
}.new

pq = PriorityQueue.new 11, comp
pq << 2
pq << 3
pq << 10
pq.remove # => 10
pq.remove # => 2
pq.remove # => 3

Aside from that, connecting to a Java class from Ruby is pretty much: Read the
Javadoc, open up irb, try to use it, watch it do exactly what you think it
should.

Oracle's behavior lately is making me kind of iffy about the future of Java as
a platform, but JRuby is just made of awesome.

That's a big plus because I definitely don't
understand at the moment how to compile C code and integrate that with MRI
Ruby.

ruby-inline is very cool, but it's still not quite as easy as being able to
write a Java class, pretend it's a Ruby class, and have it work.

I'd say, give it a spin. Worst thing that happens is you make the core of your
project portable among Ruby implementations.

···

On Thursday, December 16, 2010 05:15:59 pm Colin Bartlett wrote:

On Thu, Dec 16, 2010 at 6:36 PM, Michel Demazure <michel@demazure.com>wrote:

20

Not quite, but have a look at ruby-toolbox.com (IIRC), which gives an
overview of what's available fir what. And there's the Ruby
Application Archive, of course.

···

On Friday, December 17, 2010, Michel Demazure <michel@demazure.com> wrote:

Actually, is there a place, with a FAQ like : if you want to do this
with ruby, then use that ?

--
Phillip Gawlowski

Though the folk I have met,
(Ah, how soon!) they forget
When I've moved on to some other place,
There may be one or two,
When I've played and passed through,
Who'll remember my song or my face.