C library extension question

ruby-talk
1

hi, folks -

I am working on a ruby extension for a C library, and having
difficulty of figuring out how to deal with this structure and
wrapping it nicely in Ruby. it is a command dispatch structure, where
you supply an array of function pointers:

/* command dispatch structure */
typedef int (*proxy_cmd)(int trans_id, int nargs, char **args);

struct proxy_commands {
        int cmd_base;
        int cmd_size;
        proxy_cmd * cmd_funcs;
};
typedef struct proxy_commands proxy_commands;

/* end of definition */

Then, as a user you can define an array of function pointers like
this:

proxy_cmd cmds = {
   func_a,
   func_b,
   func_c,
   ...
};

and fill in the structure:

proxy_commands mytabs = {
    CMD_BASE;
    sizeof(cmds)/sizeof(proxy_cmd),
    cmds
}

An example of API will look like this:
register(&mytabs, .... )

What is the best way of wrapping this in Ruby?

From end user perspective, I am thinking the interaction like this:

def func_a ( ...)
end

def func_b( ...)
end

mytabs = [func_a, func_b]
register(mytabs)

If you have experience on writing C extensions, I'd appreciate your
input.
TIA

Oliver

2

/* command dispatch structure */
typedef int (*proxy_cmd)(int trans_id, int nargs, char **args);

struct proxy_commands {
        int cmd_base;
        int cmd_size;
        proxy_cmd * cmd_funcs;
};
typedef struct proxy_commands proxy_commands;

What you're looking for is probably Data_Wrap_Struct and friends. See ruby.h and intern.h. For a simple example of this in action, look at the image_science.rb code (available via gems or on the seattlerb project on rubyforge). It is only 228 lines long currently so it is a pretty easy read.

Then, as a user you can define an array of function pointers like
this:

proxy_cmd cmds = {
   func_a,
   func_b,
   func_c,
   ...
};

and fill in the structure:

proxy_commands mytabs = {
    CMD_BASE;
    sizeof(cmds)/sizeof(proxy_cmd),
    cmds
}

I'd suggest using a ruby array at this stage. Cleaner.

An example of API will look like this:
register(&mytabs, .... )

What is the best way of wrapping this in Ruby?

Well, the BEST way is to not wrap it at all:

def func_a; ...; end
def func_b; ...; end

msgs = [:func_a, :func_b]

msgs.each do |msg|
   send msg
end

Short of that, you'll want to poke around with DataWrapStruct.

···

On Jul 16, 2007, at 12:20 , Oliver wrote:

3

Thanks for the suggestion, I will take a look at the sample lib you
suggested.
However, I don't fully understand the reasoning of "not wrapping it".

Here is my line of thinking:

- the original C function register() expects an array of function
pointers, and those functions are suppose to be user-supplied and
implemented by C.

- Ideally, we should now allow those functions to be implemented by
ruby ... is this possible at all?

thanks

Oliver

···

On Jul 16, 6:53 pm, Ryan Davis <ryand-r...@zenspider.com> wrote:

On Jul 16, 2007, at 12:20 , Oliver wrote:

> /* command dispatch structure */
> typedef int (*proxy_cmd)(int trans_id, int nargs, char **args);

> struct proxy_commands {
> int cmd_base;
> int cmd_size;
> proxy_cmd * cmd_funcs;
> };
> typedef struct proxy_commands proxy_commands;

What you're looking for is probably Data_Wrap_Struct and friends. See
ruby.h and intern.h. For a simple example of this in action, look at
the image_science.rb code (available via gems or on the seattlerb
project on rubyforge). It is only 228 lines long currently so it is a
pretty easy read.

> Then, as a user you can define an array of function pointers like
> this:

> proxy_cmd cmds = {
> func_a,
> func_b,
> func_c,
> ...
> };

> and fill in the structure:

> proxy_commands mytabs = {
> CMD_BASE;
> sizeof(cmds)/sizeof(proxy_cmd),
> cmds
> }

I'd suggest using a ruby array at this stage. Cleaner.

> An example of API will look like this:
> register(&mytabs, .... )

> What is the best way of wrapping this in Ruby?

Well, the BEST way is to not wrap it at all:

def func_a; ...; end
def func_b; ...; end

msgs = [:func_a, :func_b]

msgs.each do |msg|
   send msg
end

Short of that, you'll want to poke around with DataWrapStruct.

4

Thanks for the suggestion, I will take a look at the sample lib you
suggested.
However, I don't fully understand the reasoning of "not wrapping it".

I was just suggesting that thinking laterally might be an alternative. Obviously not if you're using a specific 3rd party library that you need to use. But if you just want an event engine, we're talking a TEENY amount of code in ruby (sans-leaks, sans-cores, sans-icky) compared to C.

Here is my line of thinking:

- the original C function register() expects an array of function
pointers, and those functions are suppose to be user-supplied and
implemented by C.

what users at what level/language?

- Ideally, we should now allow those functions to be implemented by
ruby ... is this possible at all?

I think you'd have to do some extra work to wrap up rb_funcall itself.

···

On Jul 16, 2007, at 16:50 , Oliver wrote:

5

probably I didn't make myself clear:

Under normal usage, the library expects user to supply some functions
in C, such as func_a(), func_b(), through function pointers defined in
above dispatch command structure.

Now, from a wrapper stand point of view, I was hoping that the user-
provided function func_a(), func_b() etc. can be done by ruby as well
- it will be very awkward for a user to use partial c, partial ruby
for the same library. I don't see why rb_funcall() can help in this
case - in a sense, this case calls for a way to map ruby function into
C function, for the sake of using the C library call.

thanks

Oliver

···

On Jul 16, 8:00 pm, Ryan Davis <ryand-r...@zenspider.com> wrote:

On Jul 16, 2007, at 16:50 , Oliver wrote:

> Thanks for the suggestion, I will take a look at the sample lib you
> suggested.
> However, I don't fully understand the reasoning of "not wrapping it".

I was just suggesting that thinking laterally might be an
alternative. Obviously not if you're using a specific 3rd party
library that you need to use. But if you just want an event engine,
we're talking a TEENY amount of code in ruby (sans-leaks, sans-cores,
sans-icky) compared to C.

> Here is my line of thinking:

> - the original C function register() expects an array of function
> pointers, and those functions are suppose to be user-supplied and
> implemented by C.

what users at what level/language?

> - Ideally, we should now allow those functions to be implemented by
> ruby ... is this possible at all?

I think you'd have to do some extra work to wrap up rb_funcall itself.

6

Write func_a, func_b, func_c as C functions that massage their arguments
into Rubyland and call rb_funcall to hit the user-written Ruby functions.
Similarly massage the return values from Ruby back into C values.

···

On 7/16/07, Oliver <fwang2@gmail.com> wrote:

probably I didn't make myself clear:

Under normal usage, the library expects user to supply some functions
in C, such as func_a(), func_b(), through function pointers defined in
above dispatch command structure.

Now, from a wrapper stand point of view, I was hoping that the user-
provided function func_a(), func_b() etc. can be done by ruby as well
- it will be very awkward for a user to use partial c, partial ruby
for the same library. I don't see why rb_funcall() can help in this
case - in a sense, this case calls for a way to map ruby function into
C function, for the sake of using the C library call.

7

I suggested rb_funcall because it is the C entry point into a ruby method. Since the function pointer signature doesn't match up with ruby's, I don't think you can share between C and ruby 1:1. But, you might be able to wrap it up in such a way that you can call through from C into ruby.

I wasn't suggesting the user work in a mix of the two, just you.

Really NonFunctional Basic Visual:

def func_a
   puts "yay!"
end

Workhorse.register :func_a

···

On Jul 16, 2007, at 18:35 , Oliver wrote:

probably I didn't make myself clear:

Under normal usage, the library expects user to supply some functions
in C, such as func_a(), func_b(), through function pointers defined in
above dispatch command structure.

Now, from a wrapper stand point of view, I was hoping that the user-
provided function func_a(), func_b() etc. can be done by ruby as well
- it will be very awkward for a user to use partial c, partial ruby
for the same library. I don't see why rb_funcall() can help in this
case - in a sense, this case calls for a way to map ruby function into
C function, for the sake of using the C library call.

----

void rb_func_wrapper() {
   rb_funcall(rb_cKernel, ID2SYM(...));
}

void rb_register(VALUE sym_name) {
   register(rb_func_wrapper << N & SYM2ID(sym_name)); // or something equally scary
}

----

not that I think you can do that... but it is a thought.

----

Another thought, assuming you have the latitude, is to dynamically generate the C functions for register using something like RubyInline.

8

Hi Oliver,

Does the library let you specify an argument to pass to these
callbacks when you register them somehow? If so, I'd probably define
the callbacks as Procs (or anything with #call) on the ruby side, and
in the C side, register the same function, say run_command(VALUE proc)
for all the commands, passing the Proc as an argument. run_command()
would just use rb_funcall to call the proc's #call.

If not, and you do indeed have to register distinct C functions for
each command, two ideas come to mind:

1. Have a hard limit on the number of commands, and define C functions
func1, func2, ... funcN to call the n-th one.
2. Use RubyInline to dynamically define each C function, load it as a
shared library, and add it to your struct (as Ryan just mentioned).

As for the interface on the ruby side, a few options come to mind.
The more direct translation of the C API:

  command1 = lambda{|*args| ... }
  command2 = lambda{|*args| ... }
  ...
  register(command1, command2, ...)

(You could have #register have symbols denote method calls on Object
too, but I think procs would feel the most natural.)

Define by blocks on a method:

  add_command{|*args| ... }
  ...
  register_commands

Register by block:

  register_commands do |c|
    c.add{|*args| ... }
  end

Hope this gives you some ideas,
George.

···

On 7/17/07, Oliver <fwang2@gmail.com> wrote:

Now, from a wrapper stand point of view, I was hoping that the user-
provided function func_a(), func_b() etc. can be done by ruby as well
- it will be very awkward for a user to use partial c, partial ruby
for the same library. I don't see why rb_funcall() can help in this
case - in a sense, this case calls for a way to map ruby function into
C function, for the sake of using the C library call.

9

Except that he wants the users to write func_[a-c] in ruby, to be invoked by C, via c function pointers. It is that last teeny requirement that I think blows this up.

···

On Jul 16, 2007, at 19:00 , Francis Cianfrocca wrote:

On 7/16/07, Oliver <fwang2@gmail.com> wrote:

probably I didn't make myself clear:

Under normal usage, the library expects user to supply some functions
in C, such as func_a(), func_b(), through function pointers defined in
above dispatch command structure.

Now, from a wrapper stand point of view, I was hoping that the user-
provided function func_a(), func_b() etc. can be done by ruby as well
- it will be very awkward for a user to use partial c, partial ruby
for the same library. I don't see why rb_funcall() can help in this
case - in a sense, this case calls for a way to map ruby function into
C function, for the sake of using the C library call.

Write func_a, func_b, func_c as C functions that massage their arguments
into Rubyland and call rb_funcall to hit the user-written Ruby functions.
Similarly massage the return values from Ruby back into C values.

10

Thanks for the suggestions, Ryan and George.
I will explore along those lines.

Oliver

···

On Jul 16, 10:59 pm, George <george.og...@gmail.com> wrote:

On 7/17/07, Oliver <fwa...@gmail.com> wrote:

> Now, from a wrapper stand point of view, I was hoping that the user-
> provided function func_a(), func_b() etc. can be done by ruby as well
> - it will be very awkward for a user to use partial c, partial ruby
> for the same library. I don't see why rb_funcall() can help in this
> case - in a sense, this case calls for a way to map ruby function into
> C function, for the sake of using the C library call.

Hi Oliver,

Does the library let you specify an argument to pass to these
callbacks when you register them somehow? If so, I'd probably define
the callbacks as Procs (or anything with #call) on the ruby side, and
in the C side, register the same function, say run_command(VALUE proc)
for all the commands, passing the Proc as an argument. run_command()
would just use rb_funcall to call the proc's #call.

If not, and you do indeed have to register distinct C functions for
each command, two ideas come to mind:

1. Have a hard limit on the number of commands, and define C functions
func1, func2, ... funcN to call the n-th one.
2. Use RubyInline to dynamically define each C function, load it as a
shared library, and add it to your struct (as Ryan just mentioned).

As for the interface on the ruby side, a few options come to mind.
The more direct translation of the C API:

  command1 = lambda{|*args| ... }
  command2 = lambda{|*args| ... }
  ...
  register(command1, command2, ...)

(You could have #register have symbols denote method calls on Object
too, but I think procs would feel the most natural.)

Define by blocks on a method:

  add_command{|*args| ... }
  add_command{|*args| ... }
  ...
  register_commands

Register by block:

  register_commands do |c|
    c.add{|*args| ... }
    c.add{|*args| ... }
  end

Hope this gives you some ideas,
George.

11

EventMachine does exactly the same thing to invoke user-written event
handlers. The reactor core is in C. It calls a C stub that uses rb_funcall
to hit Ruby code. EventMachine's HTTP server also does the same thing. The
HTTP server is in C but it calls out to user-written Ruby code to fulfill
web requests.

···

On 7/16/07, Ryan Davis <ryand-ruby@zenspider.com> wrote:

>
> Write func_a, func_b, func_c as C functions that massage their
> arguments
> into Rubyland and call rb_funcall to hit the user-written Ruby
> functions.
> Similarly massage the return values from Ruby back into C values.

Except that he wants the users to write func_[a-c] in ruby, to be
invoked by C, via c function pointers. It is that last teeny
requirement that I think blows this up.