brett wrote:
I get the following when I try to compile on windows - any
suggestions? Not sure what srcdir should be or where I should set it.
There needs to be a pre-built gem for win32, but we haven't published one yet. I think someone's working on getting it wired up.
- Charlie
Radosław Bułat wrote:
On Sat, Nov 1, 2008 at 6:40 AM, Charles Oliver Nutter > <charles.nutter@sun.com> wrote:
Actually structs are already supported! See the blog post, and I believe
there's some examples shipped with the gem.Your blog is preaty known but for clarity:
http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
Oh right, a link would have been useful. Thank you 
- Charlie
# --- [begin unscientific test] --------
require 'rubygems'
require 'benchmark'
require 'zlib'
require 'ffi'
require 'dl/import'
module Zlib_ffi
extend FFI::Library
attach_function :zlib_version, :zlibVersion, , :string
end
module Zlib_dl
extend DL::Importable
dlload "libz.dylib"
extern "const char* zlibVersion()"
end
puts Zlib.zlib_version
puts Zlib_ffi.zlib_version
puts Zlib_dl.zlibVersion
Benchmark.bm(3) do |bm|
bm.report("ext") { 500_000.times { Zlib.zlib_version } }
bm.report("ffi") { 500_000.times { Zlib_ffi.zlib_version } }
bm.report("dl") { 500_000.times { Zlib_dl.zlibVersion } }
end
# --- [end unscientific test] --------
This gives the following results:
1.2.3
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)
All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed overhead is slightly over 2x for ffi, probably not a big deal unless ffi calls are used in tight loops I guess.
PS: haven't seen any trace of variadic function support in the code.
On 1 nov. 08, at 12:16, Charles Oliver Nutter wrote:
He doesn't have specific numbers for performance at the moment, but the short story is that FFI introduces a bit of overhead; ultimately I believe that the overhead gets lost in the flow of a Ruby application, especially when you're tossing units of work across like SQL queries or arrays.
--
Luc Heinrich - luc@honk-honk.com
We (JRuby and Rubinius contributors) designed in the ability to specify calling convention for a given library bound through FFI. Largely this was needed to support Win32's stdcall, but if Linux/PPC has a different convention and libffi supports it, then FFi will too (though we may have to add it to the list of accepted conventions).
I'll check on the syntax we're supporting and get back to ya.
- Charlie (mobile)
On Nov 3, 2008, at 12:56, Sylvain Joyeux <sylvain.joyeux@polytechnique.org > wrote:
Interesting. On what kind of architectures is the binding part working ? I'm
using dyncall to do the actual interfacing work (http://www.dyncall.org/\) in a
DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
What are you using on your side ?Ruby FFI uses libffi, as does JNA which ships with JRuby. I'm not
certain about libffi specifically. but JNA claims to support OSX (ppc,
x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64), Solaris
(x86, amd64, sparc, sparcv9) and Windows (x86, amd64).I did not know about libffi ... I'll have to look if I should not replace
dyncall by libffi then. Thanks for the info.I'm not sure if linux-ppc support is not provided because it's not
supported or because nobody has a linux-ppc machine to build on.I guess it is a bit of both. Given that libffi is supported on Linux/PPC64, I
guess having it on Linux/PPC should not be that much of a problem -- but still,
I think the calling convention can be slightly different between the two
architectures.Sylvain
Hey, please do. Sounds like it *should* work, at least. I sure don't have a PPC box to test on and I don't think anyone else involved in Ruby FFI does either.
- Charlie
jh+ruby-lang@daria.co.uk wrote:
On Mon, 3 Nov 2008 05:01:11 -0500, Charles Oliver Nutter wrote:
I'm not sure if linux-ppc support is not provided because it's not supported or because nobody has a linux-ppc machine to build on. The latter has been the case for several entries on the list; I myself was the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time, before which there was no shipped support.
There's certainly one way to find out...gem install ffi. Report back here or on ruby-ffi mailing lists what you learn
Ubuntu 8.10 provides libffi on ppc. The gem builds, but a simple ruby
program hangs (rather uses 100% CPU).I can investigate further (e.g building my own libffi), but that may
not happen immediately.
Sean O'halpin wrote:
module NCursesFFI
extend FFI::Library
ffi_lib 'ncurses'
attach_function 'clear', , :int
attach_function 'endwin', , :int
attach_function 'getch', , :int
attach_function 'initscr', , :int
attach_function 'printw', [ :string ], :int
attach_function 'refresh', , :int
endSean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?
It's a fairly tedious (and largely thankless) task to convert a lib
like ncurses. However, I'm in the mood right now so expect a github
repo in the near future Your point about avoiding duplication of
effort is a good one. The main problem in tackling something like this
is fatigue setting in when you've got enough for your immediate
requirement and so ending up with a half-baked conversion (I speak
from experience). I believe another pitfall is trying to rubyify the
interface. I think it's better to provide a thin bridge and then
abstract on top of that. Then you get the benefit of being able to use
existing experience and documentation (with the usual translations)
and others can take or leave your abstraction as they see fit. (I do
think Ruby needs a portable console library but I'd rather build that
on top of ruby versions of ncurses and Win32 console than try to make
ncurses work on Windows. I'm hoping FFI will make that possible.)
I'm wondering if it would be useful to have these FFI-enabled libs
under an ffi/ namespace, e.g. you'd require 'ffi/ncurses' and possibly
include FFI::NCurses. We could put shared structs, constants, etc.
under ffi/include. Any thoughts anyone?
I'd like to also add that the above sample is working on osx
10.5.5/darwin/ppc after a small modification in platform.rb above line
28.ARCH = case CPU.downcase
when /i?86|x86|i86pc|powerpc/
"i386"
when /amd64|x86_64/
"x86_64"I have added "powerpc" to the first "when". btw, i had a clean gem
install.
Excellent news.
Regards,
Sean
On Tue, Nov 4, 2008 at 3:10 PM, Nit Khair <sentinel.2001@gmx.com> wrote:
On Tue, Nov 4, 2008 at 2:38 PM, Ken Bloom <kbloom@gmail.com> wrote:
Nit Khair wrote:
Sean O'halpin wrote:
module NCursesFFI
extend FFI::Library
ffi_lib 'ncurses'
attach_function 'clear', , :int
attach_function 'endwin', , :int
attach_function 'getch', , :int
attach_function 'initscr', , :int
attach_function 'printw', [ :string ], :int
attach_function 'refresh', , :int
endSean,
Going by the above, and the samples of structs on the blog, for a medium sized library such as ncurses, there can be some effort in getting this module ready. Will there be some kind of repository of such modules, so we are not duplicating each others effort ?
I'd recommend folks start releasing FFI-based "thin wrappers" for each library they come across, and then depend on that gem. Don't go over the top...just get the basic bindings in there, structs, tests to make sure it's all working, and release a gem. Then build nice APIs on top of that, separately.
One of my biggest gripes about C extensions in Ruby is that everyone feels the need to release their own wrapper PLUS Ruby-like API in a single gem, when just releasing the wrapper would allow a lot more experimentation. So you end up with multiple people writing their own extensions with their own APIs and little sharing.
If we just get a whole ton of, simple, clean wrappers out there, the APIs will surely follow.
- Charlie
On Tue, Nov 4, 2008 at 2:38 PM, Ken Bloom <kbloom@gmail.com> wrote:
Nit Khair wrote:
ARCH = case CPU.downcase
when /i?86|x86|i86pc|powerpc/
"i386"
when /amd64|x86_64/
"x86_64"
hmm... that code is broken. "x86_64" will match the first regex and result in "i386"
Clifford Heath wrote:
Charles Oliver Nutter wrote:
Actually structs are already supported!
Sorry if you see this twice, posting difficulties.
I'm having a bash at wrapping freetds. This library
has a Context struct which contains three callback
function pointers. POLS says these should look like
this:callback :handle_message, [ :pointer, :pointer, :pointer ], :in # TDSCONTEXT, TSSSOCKET, TDSMESSAGE
callback :handle_int, [ :pointer ], :in # void*class Context < FFI:Struct
layout \
:locale, :pointer, 0, # TDSLOCALE
:parent, :pointer, 4, # void *
:msg_handler, :handle_message, 8, # callback(TDSCONTEXT, TDSSOCKET, TDSMESSAGE)
:err_handler, :handle_message, 12, # callback(TDSCONTEXT, TDSSOCKET, TDSMESSAGE)
:int_handler, :handle_int, 16 # callback(void*)
end Am I on the right track, or is this not possible yet?
There's no support at the moment for callbacks-in-structs, but it's just a bug report (and ideally a patch) away
http://kenai.com/projects/ruby-ffi
- Charlie
Where can I file an issue? I have trouble to build in with ruby1.9 on 64bit.
This gives the following results:
1.2.3
1.2.3
1.2.3
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)
Ubuntu 8.10 64bit
$ ruby --version && ruby ffi_bench.rb
ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
1.2.3.3
1.2.3.3
1.2.3.3
user system total real
ext 0.320000 0.070000 0.390000 ( 0.396774)
ffi 0.770000 0.120000 0.890000 ( 0.895093)
dl 2.090000 0.270000 2.360000 ( 2.365029)
Luc Heinrich wrote:
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed overhead is slightly over 2x for ffi, probably not a big deal unless ffi calls are used in tight loops I guess.
Seems like that pretty well seals the deal for ffi over dl, at the very least. I'm also glad to see FFI wasn't even that bad, especially considering it hasn't received any optimization.
- Charlie
Charles Oliver Nutter wrote:
We (JRuby and Rubinius contributors) designed in the ability to specify calling convention for a given library bound through FFI. Largely this was needed to support Win32's stdcall, but if Linux/PPC has a different convention and libffi supports it, then FFi will too (though we may have to add it to the list of accepted conventions).
I'll check on the syntax we're supporting and get back to ya.
In the body of the module you're binding:
ffi_convention :stdcall # :default is for normal C convention
We could add whatever convention Linux/PPC needs.
- Charlie
> Sean O'halpin wrote:
>> module NCursesFFI
>> extend FFI::Library
>> ffi_lib 'ncurses'
>> attach_function 'clear', , :int
>> attach_function 'endwin', , :int
>> attach_function 'getch', , :int
>> attach_function 'initscr', , :int
>> attach_function 'printw', [ :string ], :int
>> attach_function 'refresh', , :int
>> end> Sean,
> Going by the above, and the samples of structs on the blog, for a medium
> sized library such as ncurses, there can be some effort in getting this
> module ready. Will there be some kind of repository of such modules, so
> we are not duplicating each others effort ?It's a fairly tedious (and largely thankless) task to convert a lib
like ncurses. However, I'm in the mood right now so expect a github
repo in the near future
effort is a good one. The main problem in tackling something like this
is fatigue setting in when you've got enough for your immediate
requirement and so ending up with a half-baked conversion (I speak
from experience). I believe another pitfall is trying to rubyify the
interface. I think it's better to provide a thin bridge and then
abstract on top of that. Then you get the benefit of being able to use
existing experience and documentation (with the usual translations)
and others can take or leave your abstraction as they see fit. (I do
think Ruby needs a portable console library but I'd rather build that
on top of ruby versions of ncurses and Win32 console than try to make
ncurses work on Windows. I'm hoping FFI will make that possible.)
Yeah, first layer of implementation and then abstraction (Rubify) is
the way to go
I'm wondering if it would be useful to have these FFI-enabled libs
under an ffi/ namespace, e.g. you'd require 'ffi/ncurses' and possibly
include FFI::NCurses. We could put shared structs, constants, etc.
under ffi/include. Any thoughts anyone?
Something like the win32utils team does:
gem: win32-service
gem win32-eventlog
require 'win32/service'
require 'win32/eventlog'
All these live inside Win32 namespace
I think these could be named ffi-ncourses but the name implies "FFI
implementation of ncurses" where I will love to have "cross platform
curses" (note the lack of 'n').
These should depend on ruby-ffi gem and we are set to go
My to cents
On Nov 4, 12:51 pm, Sean O'Halpin <sean.ohal...@gmail.com> wrote:
On Tue, Nov 4, 2008 at 3:10 PM, Nit Khair <sentinel.2...@gmx.com> wrote:
>> On Tue, Nov 4, 2008 at 2:38 PM, Ken Bloom <kbl...@gmail.com> wrote:
--
Luis Lavena
Sean O'halpin wrote:
On Tue, Nov 4, 2008 at 3:10 PM, Nit Khair <sentinel.2001@gmx.com> wrote:
attach_function 'initscr', , :int
attach_function 'printw', [ :string ], :int
attach_function 'refresh', , :int
endSean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?It's a fairly tedious (and largely thankless) task to convert a lib
like ncurses. However, I'm in the mood right now so expect a github
repo in the near future
effort is a good one. The main problem in tackling something like this
is fatigue setting in when you've got enough for your immediate
requirement and so ending up with a half-baked conversion (I speak
from experience). I believe another pitfall is trying to rubyify the
interface.
I suppose some of these conversions/wrappers would be "approved" and
sort of become a standard, so we don't have a plethora of them. Then
they could be placed in one repo (after some sort of review). Just a
thought.
Would hate to have find 5 different ncurses wrappers and have to study
each to figure out which to use.
--
Posted via http://www.ruby-forum.com/\.
The good news is that unlike C libraries, if someone bundles their own
wrapper with their own API, you can still get at the wrapper, by finding
the module that imports FFI and just using that.
On Tue, 04 Nov 2008 14:02:50 -0500, Charles Oliver Nutter wrote:
Nit Khair wrote:
Sean O'halpin wrote:
On Tue, Nov 4, 2008 at 2:38 PM, Ken Bloom <kbloom@gmail.com> wrote:
module NCursesFFI
extend FFI::Library
ffi_lib 'ncurses'
attach_function 'clear', , :int
attach_function 'endwin', , :int
attach_function 'getch', , :int
attach_function 'initscr', , :int
attach_function 'printw', [ :string ], :int attach_function
'refresh', , :int
endSean,
Going by the above, and the samples of structs on the blog, for a
medium sized library such as ncurses, there can be some effort in
getting this module ready. Will there be some kind of repository of
such modules, so we are not duplicating each others effort ?I'd recommend folks start releasing FFI-based "thin wrappers" for each
library they come across, and then depend on that gem. Don't go over the
top...just get the basic bindings in there, structs, tests to make sure
it's all working, and release a gem. Then build nice APIs on top of
that, separately.One of my biggest gripes about C extensions in Ruby is that everyone
feels the need to release their own wrapper PLUS Ruby-like API in a
single gem, when just releasing the wrapper would allow a lot more
experimentation. So you end up with multiple people writing their own
extensions with their own APIs and little sharing.If we just get a whole ton of, simple, clean wrappers out there, the
APIs will surely follow.- Charlie
--
Chanoch (Ken) Bloom. PhD candidate. Linguistic Cognition Laboratory.
Department of Computer Science. Illinois Institute of Technology.
http://www.iit.edu/~kbloom1/
Maybe I wasn't clear - what you're proposing is exactly what I
intended to convey. Thin wrapper in a single gem. Any abstractions
should be separate gems.
Regards,
Sean
On Tue, Nov 4, 2008 at 7:02 PM, Charles Oliver Nutter <charles.nutter@sun.com> wrote:
I'd recommend folks start releasing FFI-based "thin wrappers" for each
library they come across, and then depend on that gem. Don't go over the
top...just get the basic bindings in there, structs, tests to make sure it's
all working, and release a gem. Then build nice APIs on top of that,
separately.One of my biggest gripes about C extensions in Ruby is that everyone feels
the need to release their own wrapper PLUS Ruby-like API in a single gem,
when just releasing the wrapper would allow a lot more experimentation. So
you end up with multiple people writing their own extensions with their own
APIs and little sharing.If we just get a whole ton of, simple, clean wrappers out there, the APIs
will surely follow.- Charlie
Charles Oliver Nutter wrote:
There's no support at the moment for callbacks-in-structs, but it's just a bug report (and ideally a patch) away
http://kenai.com/projects/ruby-ffi
Thanks Charlie. I registered at Kenai.com and can log in there
though only by emaill address, username login doesn't work), but
the Bugzilla instalce doesn't recognise the login details at all.
I'll try again later, in case there's a delayed processing step.
I can't see any separate registration for Bugzilla...
Clifford Heath.
Charles Oliver Nutter wrote:
Clifford Heath wrote:
callback :handle_message, [ :poi...
class Context < FFI:Struct
layout \
:msg_handler, :handle_message, 8,There's no support at the moment for callbacks-in-structs
While I'm on a roll, my example above creates a new type (handle_message),
and the ability to define other kinds of types similarly would be excellent.
For example, FreeTDS also has a string struct that contains a char* and a
length, and it uses that type liberally. A typedef would rock, and the above
example would just create a function-pointer typedef.
In a previous life, we implemented a generic structure pack/unpack subsystem.
It had a table for every type containing the alignment and size of that type.
Some architectures need more than one table depending on compile options, but
we supported 30 different architectures using tables whose values were filled
in by the C compiler. The rule of C don't allow re-ordering structure elements,
or at least, the require that two structures that start with the same elements
use the same alignment for those elements.
You can discover the alignment of a type by declaring a struct z { char x; TYPE y; }
and asking for offsetof(struct z, y); Mere byte-counting and padding suffices
to calculate alignment for the members of any structure. Nested structures are
padded to the largest of either the maximum alignment of the members, or some
size specific to the architecture. This can also be populated into the alignment
tables by the C compiler.
If FFI could include such a facility for Struct definitions, that would be magic,
as the current system is almost impossible to use for cross-platform support.
Unless you use the awkward templating system, of course... That requires that
you have the development versions of the target libraries when you want to gem
install a wrapper.
Just some random thoughts, anyhow. I might have time to contribute code to this
effort in December.
Clifford Heath.
Radosław Bułat wrote:
This gives the following results:
1.2.3
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)Ubuntu 8.10 64bit
$ ruby --version && ruby ffi_bench.rb
ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
1.2.3.3
user system total real
ext 0.320000 0.070000 0.390000 ( 0.396774)
ffi 0.770000 0.120000 0.890000 ( 0.895093)
dl 2.090000 0.270000 2.360000 ( 2.365029)
Seems like about the same ratio...probably can be improved too!
- Charlie