SWIG on Solaris problem

Hi folks.

This may be a bit off topic, but it is ruby related.

I can’t seem to get swig to work when my function
has a pointer argument. Here is my simple example:

swig -version

SWIG Version 1.3.9u-20030501-1037
Copyright © 1995-1998
University of Utah and the Regents of the University of California
Copyright © 1998-2001
University of Chicago

Compiled with CC

cat example.c
int fact2(int *n) {
return *n *2;
}

cat example.i
/* example.i /
%module example
%{
/
Put header files here (optional) */
%}

extern int fact2(int *n);

ruby -r example -e ‘puts Example.fact2(2)’
-e:1:in `fact2’: Expected int * (TypeError)
from -e:1

Everything looks correct. I see wrapper code
in example_wrap.c that appears to be wrapping
the int * for me, but no worky.

Any suggestions?

Thanks

···


Jim Freeze

You know if they ever find a way to harness sarcasm as an energy source,
you people are all going to owe me big.
– Bill Paul

Jim Freeze wrote:

I can’t seem to get swig to work when my function
has a pointer argument. Here is my simple example:

swig -version

SWIG Version 1.3.9u-20030501-1037
Copyright (c) 1995-1998
University of Utah and the Regents of the University of California
Copyright (c) 1998-2001
University of Chicago

Compiled with CC

cat example.c
int fact2(int *n) {
return *n *2;
}

cat example.i
/* example.i /
%module example
%{
/
Put header files here (optional) */
%}

extern int fact2(int *n);

ruby -r example -e ‘puts Example.fact2(2)’
-e:1:in `fact2’: Expected int * (TypeError)
from -e:1

Everything looks correct. I see wrapper code
in example_wrap.c that appears to be wrapping
the int * for me, but no worky.

Any suggestions?

First, the instant gratification part of the response :wink:

Modify your SWIG interface file to look like this:

 /* example.i */
 %module example

 %include typemaps.i

 extern int fact2(int *INPUT);

Re-run SWIG, re-compile, etc. and it should work as expected.

Now, for the “why” part. SWIG is no Kreskin, and it can’t guess from
this declaration:

 extern int fact2(int *n);

whether ‘n’ is a pointer to a single integer (as in your case), or a
pointer to an array of integers. For that matter, it can’t tell if ‘n’
is an input-only argument to fact2(), an output-only argument, or maybe
both an input and an output to fact2(). All of this is to say, for many
pointer argument types you will need to give SWIG some kind of hint as
to what’s going on.

For now, just take it for granted that if you include the typemaps.i
file from the standard SWIG library:

 %include typemaps.i

that it will define a number of useful typemaps for dealing with this
situation. You’ve already seen how to handle the case of a pointer
that’s really an input. But what if your function was instead using that
argument as an output, e.g.

 void deepthought(int *answer) {
     *answer = 42;
 }

Here, ‘answer’ is obviously an output-only argument. So you’d use the
OUTPUT typemap from typemaps.i:

 %include typemaps.i

 void deepthought(int *OUTPUT);

When you call this function from Ruby, you don’t pass it any arguments;
its outputs come to you as regular outputs should:

 the_answer = deepthought() # should return 42

Finally, for input-output arguments, use the INOUT typemap:

 /* example.c */
 void triple(int *n) {
     *n = (*n) * 3;
 }

 /* example.i */

 %include typemaps.i

 void triple(int *INOUT);

And to call this function from Ruby:

 tripled_value = triple(5) # should return 15

Hope this helps,

Lyle

In article 20030502170141.A67817@freeze.org,

···

Jim Freeze jim@freeze.org wrote:

Hi folks.

This may be a bit off topic, but it is ruby related.

I can’t seem to get swig to work when my function
has a pointer argument. Here is my simple example:

swig -version

SWIG Version 1.3.9u-20030501-1037
Copyright (c) 1995-1998
University of Utah and the Regents of the University of California
Copyright (c) 1998-2001
University of Chicago

Lyle did a good job of answering your question, but I have one more
suggestion:
swig 1.3.9 is pretty old now, the current version is 1.3.19 and there have
been a lot of improvements especially when it comes to wrapping C++.

Phil

[description snipped]

Thanks Lyle for the fine description. I had %include typemaps.i
and I was just about to go find your presentation at rubyconf.

So, the instant gratification worked… but I was not giving
you the whole story. The real function was:

fact2(int *INPUT, char **str);

After reading your email, I changed this to:

fact2(int *INPUT, char **INPUT);

But compiling and running gives me the error:

test.rb:3:in `fact2’: Expected char ** (TypeError)
from test.rb:3

So, I changed the header description in the .i file (without
changing the actual function definition) to:

fact2(int *INPUT, char *INPUT);

Now, it appears to be getting past the initial function
parameter passing, but I get the error:

ruby test.rb
ld.so.1: /tools/apps/ruby/ruby-1.8.0/bin/ruby: fatal: relocation error:
file ./example.so: symbol fact2: referenced symbol not found
Killed

I’ve seen this error before, but googling did not give me an answer.

···

On Saturday, 3 May 2003 at 6:49:12 +0900, Lyle Johnson wrote:

Jim Freeze wrote:

I can’t seem to get swig to work when my function
has a pointer argument. Here is my simple example:

Modify your SWIG interface file to look like this:

 /* example.i */
 %module example

 %include typemaps.i

 extern int fact2(int *INPUT);

Re-run SWIG, re-compile, etc. and it should work as expected.

Now, for the “why” part. SWIG is no Kreskin, and it can’t guess from
this declaration:

 extern int fact2(int *n);


Jim Freeze

Be braver – you can’t cross a chasm in two small jumps.

 extern int fact2(int *INPUT);

 %include typemaps.i

that it will define a number of useful typemaps for dealing with this
situation. You’ve already seen how to handle the case of a pointer
that’s really an input. But what if your function was instead using that
argument as an output, e.g.

 void deepthought(int *answer) {
     *answer = 42;
 }

Here, ‘answer’ is obviously an output-only argument. So you’d use the
OUTPUT typemap from typemaps.i:

 %include typemaps.i

 void deepthought(int *OUTPUT);

When you call this function from Ruby, you don’t pass it any arguments;
its outputs come to you as regular outputs should:

 the_answer = deepthought() # should return 42

What would happen if I defined several OUTPUT params.
Would I get an array back?

Finally, for input-output arguments, use the INOUT typemap:

 /* example.c */
 void triple(int *n) {
     *n = (*n) * 3;
 }

 /* example.i */

 %include typemaps.i

 void triple(int *INOUT);

And to call this function from Ruby:

 tripled_value = triple(5) # should return 15

What if the function already has a return value.
Can one use an OUTPUT or INOUT argument?

···

On Saturday, 3 May 2003 at 6:49:12 +0900, Lyle Johnson wrote:


Jim Freeze

They’re only trying to make me LOOK paranoid!

Jim Freeze wrote:

So, the instant gratification worked… but I was not giving
you the whole story. The real function was:

fact2(int *INPUT, char **str);

After reading your email, I changed this to:

fact2(int *INPUT, char **INPUT);

But compiling and running gives me the error:

test.rb:3:in `fact2’: Expected char ** (TypeError)
from test.rb:3

Right. The typemaps.i file doesn’t define a typemap for char **, you’ll
need to roll one yourself (I will provide hints :wink:

So, I changed the header description in the .i file (without
changing the actual function definition) to:

fact2(int *INPUT, char *INPUT);

Now, it appears to be getting past the initial function
parameter passing, but I get the error:

ruby test.rb
ld.so.1: /tools/apps/ruby/ruby-1.8.0/bin/ruby: fatal: relocation error:
file ./example.so: symbol fact2: referenced symbol not found
Killed

Right. SWIG interprets this line:

 int fact2(int *INPUT, char *INPUT);

as your saying there’s a real function named fact2() that expects two
arguments, a pointer-to-int and a pointer-to-char. Which of course is
not the case :wink: The real fact2() expects a pointer-to-pointer-to-char as
its second argument.

So the burning question now, is, what is that 2nd argument to fact2()
actually supposed to be? Is it intended to be an array of strings? Put
another way (and perhaps more useful), how would you expect to call this
function from Ruby?

Jim Freeze wrote:

What would happen if I defined several OUTPUT params.
Would I get an array back?

Yes. To consider a few examples, this function:

 void func1(int x);

would return nil, but this function:

 void func2(int *OUTPUT);

and this function:

 void func3(int *OUTPUT, float *OUTPUT);

would return a 2-element array containing an Integer and a Float. If the
function declares a non-void return value, that return value is always
the first element of the array output. For example, this function:

 float func4(int *OUTPUT, int *OUTPUT);

would return a 3-element array containing a Float and two Integers.

– Lyle

Jim Freeze wrote:

you the whole story. The real function was:

fact2(int *INPUT, char **str);

After reading your email, I changed this to:

fact2(int *INPUT, char **INPUT);

But compiling and running gives me the error:

test.rb:3:in `fact2’: Expected char ** (TypeError)
from test.rb:3

Right. The typemaps.i file doesn’t define a typemap for char **, you’ll
need to roll one yourself (I will provide hints :wink:

Good. :slight_smile: I’ll need them.

So, I changed the header description in the .i file (without
changing the actual function definition) to:

fact2(int *INPUT, char *INPUT);

Now, it appears to be getting past the initial function
parameter passing, but I get the error:

ruby test.rb
ld.so.1: /tools/apps/ruby/ruby-1.8.0/bin/ruby: fatal: relocation error:
file ./example.so: symbol fact2: referenced symbol not found
Killed

Right. SWIG interprets this line:

 int fact2(int *INPUT, char *INPUT);

as your saying there’s a real function named fact2() that expects two
arguments, a pointer-to-int and a pointer-to-char. Which of course is
not the case :wink: The real fact2() expects a pointer-to-pointer-to-char as
its second argument.

So the burning question now, is, what is that 2nd argument to fact2()
actually supposed to be? Is it intended to be an array of strings? Put
another way (and perhaps more useful), how would you expect to call this
function from Ruby?

Ok, so, let me change things a bit here. The fact2 code was just
to do a test to see why int* did not work. I am really trying
to wrap the itkDB library. The function is defined as:

dbInit(int *pnArgs, char **pString);

(or dbInit(int *pnArgs, String *pString);
where typdef char * String;)

I expect to call it from ruby as:

require ‘itkdb’
Itkdb.dbInit(1, “some_path”)

So, from what I can tell, the first arg is the number of
Strings in the second String array.
I don’t know why the count has to be passed in as a pointer.
I don’t know if it is being changed.
(I did not write this code, nor can I look at it.)

Is that enough information?

···

On Saturday, 3 May 2003 at 8:29:47 +0900, Lyle Johnson wrote:

Jim Freeze

Good day to avoid cops. Crawl to school.

Jim Freeze wrote:

Ok, so, let me change things a bit here. The fact2 code was just
to do a test to see why int* did not work. I am really trying
to wrap the itkDB library. The function is defined as:

dbInit(int *pnArgs, char **pString);

(or dbInit(int *pnArgs, String *pString);
where typdef char * String;)

I expect to call it from ruby as:

require ‘itkdb’
Itkdb.dbInit(1, “some_path”)

OK, this is getting a little trickier. Try this typemap:

 %module example

 %typemap(in) (int *pnArgs, String *pString) (int temp, int i) {
     Check_Type($input, T_ARRAY);
     temp = RARRAY($input)->len;
     $1 = &temp;
     $2 = (String *) malloc(sizeof(String));
     for (i = 0; i < $1; i++) {
         $2[i] = StringValuePtr(rb_ary_entry($input, i));
     }
 }

 %typemap(freearg) String *pString {
     free((void *) pString);
 }

 void dbInit(int *pnArgs, String *pString);

Hope this helps,

Lyle

Thanks. I’ll give it a try.

···

On Tuesday, 6 May 2003 at 0:18:24 +0900, Lyle Johnson wrote:

Jim Freeze wrote:

Ok, so, let me change things a bit here. The fact2 code was just
to do a test to see why int* did not work. I am really trying
to wrap the itkDB library. The function is defined as:

dbInit(int *pnArgs, char **pString);

(or dbInit(int *pnArgs, String *pString);
where typdef char * String;)

I expect to call it from ruby as:

require ‘itkdb’
Itkdb.dbInit(1, “some_path”)

OK, this is getting a little trickier. Try this typemap:

 %module example

 %typemap(in) (int *pnArgs, String *pString) (int temp, int i) {
     Check_Type($input, T_ARRAY);
     temp = RARRAY($input)->len;
     $1 = &temp;
     $2 = (String *) malloc(sizeof(String));
     for (i = 0; i < $1; i++) {
         $2[i] = StringValuePtr(rb_ary_entry($input, i));
     }
 }

 %typemap(freearg) String *pString {
     free((void *) pString);
 }

 void dbInit(int *pnArgs, String *pString);

Hope this helps,


Jim Freeze

There is a certain impertinence in allowing oneself to be burned for an
opinion.
– Anatole France

Ok, I have been reading the swig docs and I have
a few questions. Other than the code above looking
totally greek to me, the swig manual does its
best to convince me that I never want to write
a typemap. So, why do we have to write a typemap
for something so common as char **? Is it because
I said that I wanted to call dbInit as:

dbInit(n, “some string”)

instead of

dbInit(n, [“some string”]) ?

Second question, do I need to learn how to write typemaps?

Third question, what is the $1 and $2 in the above typemap.

Fourth: Lyle, thanks for all your expert help.

···

On Tuesday, 6 May 2003 at 0:18:24 +0900, Lyle Johnson wrote:

Jim Freeze wrote:

OK, this is getting a little trickier. Try this typemap:

 %module example

 %typemap(in) (int *pnArgs, String *pString) (int temp, int i) {
     Check_Type($input, T_ARRAY);
     temp = RARRAY($input)->len;
     $1 = &temp;
     $2 = (String *) malloc(sizeof(String));
     for (i = 0; i < $1; i++) {
         $2[i] = StringValuePtr(rb_ary_entry($input, i));
     }
 }

 %typemap(freearg) String *pString {
     free((void *) pString);
 }

 void dbInit(int *pnArgs, String *pString);


Jim Freeze

When asked by an anthropologist what the Indians called America before
the white men came, an Indian said simply “Ours.”
– Vine Deloria, Jr.

Ok, I have been reading the swig docs and I have
a few questions. Other than the code above looking
totally greek to me, the swig manual does its
best to convince me that I never want to write
a typemap. So, why do we have to write a typemap
for something so common as char **? Is it because
I said that I wanted to call dbInit as:

dbInit(n, “some string”)

instead of

dbInit(n, [“some string”]) ?

In a nutshell, yes. Basically, the C data structures used by
your application, in this case Cadence’s itkDB, do not (or may
not) map to ruby objects in a manner you will find convenient
or natural to use.

For example, itkDB has places it returns *dbPathStyle as an
output. Without special typemaps, these come back into Ruby
as a generic object that we can pass around, but not really
know what the value is. In C, the results points to an
integer so we have a typemap that tells swig how to map this
type to a ruby Fixnum

 %typemap(ruby,argout) dbPathStyle *OUTPUT ...

Second question, do I need to learn how to write typemaps?

Eventually, yes.

Fourth: Lyle, thanks for all your expert help.

I’d like to thank Lyle for his work on the whole Ruby/SWIG!

Jeff.

···

On Tuesday, May 6, 2003, at 05:47 AM, Jim Freeze wrote:

Jim Freeze wrote:

Ok, I have been reading the swig docs and I have
a few questions. Other than the code above looking
totally greek to me, the swig manual does its
best to convince me that I never want to write
a typemap. So, why do we have to write a typemap
for something so common as char **? Is it because
I said that I wanted to call dbInit as:

dbInit(n, “some string”)

instead of

dbInit(n, [“some string”])?

We should probably add a standard typemap for converting a Ruby Array of
Strings to a char ** (like the one I showed). Your case is a little
unusual in that the first argument is a pointer-to-int, instead of just
an int, but that’s not too much of a problem.

But after taking another look at what I sent you, it doesn’t do quite
what you asked for. (It also had a few bugs). For Ruby, it’s sort-of
redundant to pass in both the length of the Array and the Array itself,
unless you wanted to specifically limit the number of elements used in
the array (e.g. “Here’s an array with 100 elements, but only do
something with the first 15 of them”). The typemap I was shooting for
just expects a single argument (an Array). I’ve further modified it so
that it accepts either a single String, or an Array of Strings:

%typemap(in) (int *pnArgs, String *pString) (int count, int i, VALUE
tmp) {
if (TYPE($input) == T_STRING) {
count = 1;
$1 = &count;
$2 = (String *) malloc(sizeof(String));
$2[0] = StringValuePtr($input);
} else if (TYPE($input) == T_ARRAY) {
count = RARRAY($input)->len;
$1 = &count;
$2 = (String ) malloc(countsizeof(String));
for (i = 0; i < count; i++) {
tmp = rb_ary_entry($input, i);
$2[i] = StringValuePtr(tmp);
}
}
}

%typemap(freearg) String *pString {
free((void *) $1);
}

Second question, do I need to learn how to write typemaps?

For some fairly simple interfaces, you may not need to know how to write
typemaps. But almost anything involving pointers usually requires at
least some knowledge of how typemaps work. This also entails knowing
something about Ruby’s C API (e.g. the code above that deals with
grabbing elements out of a Ruby Array).

Note that even if you weren’t using SWIG, the fundamental problems don’t
go away. If you want wrapper code to interface between Ruby and C/C++,
someone’s got to write it :wink: For me, the big advantage of using SWIG is
that for large interfaces, most of the tedious parts of generating that
wrapper code are taken care of. As I said in my presentation at
RubyConf, though, for small interfaces SWIG may be overkill and you may
be better off just writing the wrapper code yourself, by hand. I will
certainly not try to convince anyone that there isn’t a learning curve
associated with using SWIG :wink:

Third question, what is the $1 and $2 in the above typemap.

The first part of the typemap declaration:

 %typemap(in) (int *pnArgs, String *pString) {
     ...
 }

shows what kinds of argument patterns (in function declarations) that
SWIG is going to try to match for this typemap. In other words, if it
sees any functions that have the arguments “int *pnArgs” and “String
*pString”, with those names, in that order, it’s a match. So each of
these functions would use that typemap:

 void dbInit(int *pnArgs, String *pString);
 void foo(float x, int *pnArgs, String *pString, long y, double z);

but these wouldn’t match:

 void goo(int *pnArgs);
 void boo(String *pString);
 void loo(String *pString, int *pnArgs);
 void bar(int *pnArgs, float y, String *pString);

Inside the typemap code, then, $1 refers to the first value matched (the
int * value) and $2 refers to the second one (the String * value). I
think this is covered in the typemaps section of the SWIG/Ruby docs:

 http://www.swig.org/Doc1.3/Ruby.html

as well as the (more comprehensive) chapter on SWIG typemaps:

 http://www.swig.org/Doc1.3/Typemaps.html

Hope this helps,

Lyle

Jeff Putsch wrote:

In a nutshell, yes. Basically, the C data structures used by
your application, in this case Cadence’s itkDB, do not (or may
not) map to ruby objects in a manner you will find convenient
or natural to use.

I was wondering whose API Jim was working with :wink:

For example, itkDB has places it returns *dbPathStyle as an
output. Without special typemaps, these come back into Ruby
as a generic object that we can pass around, but not really
know what the value is. In C, the results points to an
integer so we have a typemap that tells swig how to map this
type to a ruby Fixnum

%typemap(ruby,argout) dbPathStyle *OUTPUT ...

Ugh. Yes, I guess the upshot of all this is that if your C/C++ “looks” a
lot like the final Ruby interface, you may not need to write too many
typemaps (if any). But the weirder it gets, for whatever reason, the
harder it may be to get SWIG to do something useful with it. As I said
in my last post, this kind of interface will probably present at least
as much of a challenge if you’re writing the code yourself, by hand.

One of the more difficult libraries I’ve dealt with (in terms of
SWIGging it) is one that we use in my company. The library is written in
C, but it must be usable from Fortran. Since Fortran passes all function
arguments by reference (i.e. pointer), the public API for this library
is a lot like the one Jeff is describing – everything, even the inputs,
are pointers instead of “plain old values”.

Remember, however, that wrapping C/C++ libraries with SWIG isn’t an
all-or-nothing proposition. For example, let’s say this itkDB library
has a total of 200 functions in its public interface, but you’re only
interested in wrapping a dozen of those functions. That’s no problem,
just list the functions of interest in your SWIG interface file (as
you’ve already started doing) and ignore the rest. And of course as your
needs change, you can add wrappers for other functions later.