========Announcing the creation of a "ruby bounty" list=============
Purpose: connect those who want to work on side projects with those who want
to support them.
Is meant to be a place where you can list projects you'd like to see
completed and would be willing to help pay for.
And also a place for people looking for something to do in their spare
time to go.
(Ok, ok so it's mostly a list of projects I wish I could work on, but
don't have time for, but thought I'd try to make a community resource,
if others want to add to the list or what not.)
JRuby C Extension support - we have an early start at such a library
(http://github.com/wmeissner/jruby-cext\) but we need more C hackers
willing to help us build it out. We will only support the "safe" C
functions that don't let you manipulate objects internals directly or
get direct access to pointers.
Pure-Java Nokogiri - Just what it sounds like...for places where
libxml isn't available or native libraries are forbidden.
Dive in
- Charlie
···
On Thu, Jan 21, 2010 at 12:07 AM, Roger Pack <rogerdpack2@gmail.com> wrote:
Fifth time's a charm (durn spam filters)
========Announcing the creation of a "ruby bounty" list=============
It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.
I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.
- Charlie
···
On Sat, Jan 23, 2010 at 11:49 PM, Roger Pack <rogerpack2005@gmail.com> wrote:
Looks like pure java Nokogiri is something popular--the bounty on it has
already risen to $225
> Looks like pure java Nokogiri is something popular--the bounty on it has
> already risen to $225
It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.
I thought FFI was supposed to solve this problem. Is it not?
I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.
I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api. Think about who you hurt by not letting people reuse
valuable libraries written in C.
···
On Sun, Jan 24, 2010 at 09:15:56PM +0900, Charles Oliver Nutter wrote:
On Sat, Jan 23, 2010 at 11:49 PM, Roger Pack <rogerpack2005@gmail.com> wrote:
> Looks like pure java Nokogiri is something popular--the bounty on it has
> already risen to $225
It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.
I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.
I'm in for $200 for a Java Nokogiri implementation. So is my partner in
crime, Aaron.
Consider it "conscience money" for all the puppies who died as a result of
us writing and maintaining the JRuby FFI port of Nokogiri.
···
On Sun, Jan 24, 2010 at 7:15 AM, Charles Oliver Nutter <headius@headius.com>wrote:
On Sat, Jan 23, 2010 at 11:49 PM, Roger Pack <rogerpack2005@gmail.com> > wrote:
The goal is a version of Nokogiri without any native code dependencies which
runs entirely within the JVM. That doesn't mean it's written in Java or
even necessarily includes any Java code at all: it could be pure Ruby
interfacing with the native Java XML libraries.
···
On Sun, Jan 24, 2010 at 2:55 PM, Ammar Ali <ammarabuali@gmail.com> wrote:
Does a pure java anything qualify as a ruby bounty? Or is it a java bounty
now? Maybe a ruby-envy bounty?
It "solves" it to a certain degree, but when deploying in enterprise
environments it can be quite a hassle. Think: ancient versions of RHEL with
ancient versions of libxml/libxslt. That's not to mention problems with
32-bit vs 64-bit environments: for some reason the 64-bit version of RedHat
didn't package a 32-bit libxslt, but they did provide a 64-bit one.
However, we originally installed a 32-bit JRE. Eep! 64-bit JRE is an
"unofficially supported" configuration for our environment (they typically
run 32-bit JRE on 64-bit RedHat for some reason).
Nokogiri's great and we have everything working now (we had to do some
symlink hackery though). However, it would definitely streamline the
deployment process a lot if we didn't have native code dependencies, and
right now Nokogiri is the only one we have.
···
On Sun, Jan 24, 2010 at 10:29 PM, Aaron Patterson < aaron@tenderlovemaking.com> wrote:
I thought FFI was supposed to solve this problem. Is it not?
On Sun, Jan 24, 2010 at 09:15:56PM +0900, Charles Oliver Nutter wrote:
I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.
I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api. Think about who you hurt by not letting people reuse
valuable libraries written in C.
I thought FFI was supposed to solve this problem. Is it not?
As Tony mentioned, it solves the problem just fine, except that you
still have to have the right libraries available and permission to use
them. Lots of production deployment environments for Java disallow
loading native libraries because they have a much higher likelihood of
crashing the JVM or circumventing its security policies. Google App
Engine, for example, doesn't allow loading native libraries at all,
and of course any small profile Java environments will often be unable
to use native libs as well (like Android).
I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api. Think about who you hurt by not letting people reuse
valuable libraries written in C.
The MRI C API is terrible if you want to implement it for anything
other than MRI. It allows direct pointer access to object internals,
allows you to *write* to objects directly in memory, has absolutely no
consideration for concurrent execution, no abstraction around object
locations in a relocating GC, and no methods or standards for handling
reference lifecycle (that I am aware of). As I've stated before (and
told you) we'd be happy to support a C API that did not include all
the unsafe bits (like accessing array, string, or hash contents
*directly*) and only allows you access to objects via a handle-based
indirection mechanism. It's just a matter of time and effort to
implement it...and there's a bounty to do so
Even then, however, it still wouldn't solve the problems of loading
native libraries on secure environments.
I firmly believe the MRI's C API is the #1 thing holding it back. Why
can't we get a better GC in place? Native extensions. Why can't we
have real concurrent threading? Native extensions. Why is it a pain in
the ass to use MRI on environments without a compiler easily
available, like Windows? Native extensions. When used with plain Java
libraries, JRuby works almost identically across all platforms without
modification, without build tools present, and without segfaults. That
beats supporting the current unsafe C API any day.
- Charlie
···
On Mon, Jan 25, 2010 at 6:29 AM, Aaron Patterson <aaron@tenderlovemaking.com> wrote:
> I thought FFI was supposed to solve this problem. Is it not?
As Tony mentioned, it solves the problem just fine, except that you
still have to have the right libraries available and permission to use
them. Lots of production deployment environments for Java disallow
loading native libraries because they have a much higher likelihood of
crashing the JVM or circumventing its security policies. Google App
Engine, for example, doesn't allow loading native libraries at all,
and of course any small profile Java environments will often be unable
to use native libs as well (like Android).
Charlie, you're making a great case against using FFI.
It sounds like you're recommending that gems containing non-ruby code, like
Nokogiri, need to be written twice: once in C, and once in Java. Is that an
accurate interpretation?
(Unless, of course, you take Eleanor's advice and do everything in pure
Ruby. Hello, REXML! )
···
On Mon, Jan 25, 2010 at 7:37 AM, Charles Oliver Nutter <headius@headius.com>wrote:
On Mon, Jan 25, 2010 at 6:29 AM, Aaron Patterson > <aaron@tenderlovemaking.com> wrote:
> I implore Ruby implementors to support the MRI C api, as it too is part
> of Ruby's api. Think about who you hurt by not letting people reuse
> valuable libraries written in C.
The MRI C API is terrible if you want to implement it for anything
other than MRI. It allows direct pointer access to object internals,
allows you to *write* to objects directly in memory, has absolutely no
consideration for concurrent execution, no abstraction around object
locations in a relocating GC, and no methods or standards for handling
reference lifecycle (that I am aware of). As I've stated before (and
told you) we'd be happy to support a C API that did not include all
the unsafe bits (like accessing array, string, or hash contents
*directly*) and only allows you access to objects via a handle-based
indirection mechanism. It's just a matter of time and effort to
implement it...and there's a bounty to do so
Even then, however, it still wouldn't solve the problems of loading
native libraries on secure environments.
I firmly believe the MRI's C API is the #1 thing holding it back. Why
can't we get a better GC in place? Native extensions. Why can't we
have real concurrent threading? Native extensions. Why is it a pain in
the ass to use MRI on environments without a compiler easily
available, like Windows? Native extensions. When used with plain Java
libraries, JRuby works almost identically across all platforms without
modification, without build tools present, and without segfaults. That
beats supporting the current unsafe C API any day.
The problem with this attitude is that you eschew some great, robust
libraries that are already out there that solve complex problems. Parsing
XML is a bitch. Fortunately, there are already some great libraries to do
this. There's the libxml2 library, which Nokogiri uses, and Java ships with
some great XML libraries to.
Will we ever see a pure Ruby library as robust and powerful as these (all
performance considerations aside)? REXML certainly isn't there yet. Is it
really worth writing a library in pure Ruby when robust libraries already
exist that Ruby can tap into?
···
On Mon, Jan 25, 2010 at 5:36 AM, Eleanor McHugh < eleanor@games-with-brains.com> wrote:
I implore Ruby developers to write in Pure Ruby and demand all these Ruby
Implementors solve their "performance" problems ;p
FFI is much better than writing any C code at all, due to the
security, stability, and portability problems of writing your own C
bindings. If you are permitted to load a given library and that
library is available and you *must* use that library, FFI is the only
logical choice. But it doesn't get around the fact that you need the
library you're binding to be available and loadable on your target
platform. FFI > C bindings, but [platform-independent binary] > FFI.
And that usually means Java-based.
I should also point out that you don't necessarily have to write JVM
libraries in Java; you could also use Scala or Fan or similar
languages, and it would be just as portable (albeit a bit larger due
to the runtime dependency on those languages' runtime libraries).
But yes, at the end of the day, I believe writing stuff in a portable
binary format like JVM bytecode (or CLR bytecode) is a better choice
than writing in a language that has to be recompiled for every target
system. You ought to know that already...would I be working on JRuby
if I believed any differently?
And yes...I'd love to be able to recommend that everyone just use Ruby
for everything. But I don't think it's simply a performance issue;
there's some pretty amazing things you can get for free with a rich
static type system.
- Charlie
···
On Mon, Jan 25, 2010 at 6:25 PM, Mike Dalessio <mike.dalessio@gmail.com> wrote:
Charlie, you're making a great case against using FFI.
)