Ruby and E.V.E. Paradox

[GD1 (GD)]

I have been using Ruby for several years. Most importantly I have been using this amazing language as a support language for E.V.E. Paradox, a game suite I have been working on for much of the last three years. Ruby is used extensively in the tools used to build and test E.V.E. Paradox. It has literally saved me hundreds of hours. I use Ruby to generate many thousands of lines of C++ source from configuration files, automate builds, automatically fix problems with other software, generate dynamic web content, manage data, test, maintain the codebase, and so forth. It is reliable, easy to use, intuitive, and a fantastic base for rapid development of tools. To everyone involved in the development of this amazing language, thankyou.

Recently I have upgraded my development machine and with it, all of the support libraries I use with my software. I presently use LUA for embedded scripting in E.V.E. Paradox. After the upgrade I found that several key API calls had been renamed (for no reason AFAICT), and the existing scripts, even when fixed, no longer works. I have found LUA generally difficult to work with, and only use the basics of the language anyway. The documentation is less than ideal. I started entertaining the idea that it would be very, *very* nice if I could use Ruby as my scripting language instead; basically Ruby is solid and powerful, and I have years of experience with it.

This is what I needed to replace LUA:
- Ability to load a script *from*memory*, not from a file (important).
- Run any init code in the script.
- Hook a bunch of local functions (C++) so that the script can call them for information from the main code body.
- Call functions in the script both periodically and in response to events (eg. an update call every 1/60 of a second, in response to mouse clicks, etc).
- Clean up everything entirely when the level is over.
- Be prepared to run everything again later when the next level needing scripting is loaded.
- Ideally have separate contexts so that two separate games running at once (in the same exe) can work in their own scripting environment.

I noticed that Ruby is built with many functions available in a separate dynamic library, which builds easily both under Linux and through MinGW, and has some kind of embeddable interface. I figured there was a chance I could get things going, so I gave myself two days to figure things out and get the basics running.

The process has been a complete nightmare. I have been plagued by constant crashes and uncertain documentation. I have been at this for close to two full days. I am using ruby-1.8.5-p12 built from source, LD_LIBRARY_PATH is fine, and other instances of Ruby have been completely removed from the machine.

This is what I have encountered:

- There doesn't appear to be any way to indicate that Ruby should clean everything up and get to a state where I can start again. I have attempted to work around this by making each script contain a single unique class that I use, but this is a terrible hack.
- How do I load a script from memory? rb_eval_string()? rb_compile_string()? Something else?
- The documentation is flawed. It indicates calling rb_run(), but that can eventually call exit, which is the last thing you want from an embedded language. It mentions rb_apply() as having an argc parameter.
- I am getting inexplicable crashes when calling the garbage collector. Fearing it was my mistake, I eliminated calls repeatedly until I got down to ruby_init() and two calls to rb_gc(). This crashed. I have searched around and noticed people with comparable problems, and did the same thing. I have seen condescending replies with no indication as to why a simple code sequence such as this could cause a crash. Not calling the garbage collector is NOT a solution, as it could be called outside of my control and I don't want my code randomly crashing when it does.
- I managed to mitigate some crashes by moving the Ruby initialisation from inside a function to near the code initialisation such that the stack present on creation will be in place by the time other calls are made. This made a difference! Is the ruby interpreter performing some operations on the real stack that I don't know about? Shouldn't something like this be mentioned somewhere?-
- I wrapped rb_gc() in a rb_protect() call, didn't help.
- The documentation is not at all clear how you protect something from the garbage collector, or indicate it is no longer in use.
- How does one call "Foo.new(args)" on a class from outside of Ruby, where the args have been generated from outside of Ruby (hence no name) meaning rv_eval_string is unavailable, and I'm not sure how rb_funcall()/rb_apply() can be used to call "new" on a class to get an object?
- rb_protect() is a complete mess. Rather than being able to call an arbitrary function with a void pointer, you are stuck calling something that takes a single VALUE arg. This means that if you want to call a general function, you need to resort to tricks like building up a ruby array with the receiver, method string, and real args (also in their own array), pass it through, and deconstruct it on the other end, feeding it into rb_apply() rather than using the more convenient rb_funcall() calls. Writing multiple dedicated callbacks is not an option, as I lose the local context (eg. "this") in C++ when I do that, unless I go to the extra effort to wrap it. Is there some reason it is like this?
- The code runs fine for a while if I don't call the garbage collector. Unfortunately, I get stack depth errors and eventual segfaults if I make repeated calls to the Ruby script (via rb_apply()). Since I need to call an "update" function sixty times a second, minimising my call use is not an option.
- When searching for errors that come up during development (eg. stack size problems, crashes on garbage collection) I get a huge number of hits. I am not alone here, and it seems people are having genuine trouble finding a solution.
- I can't find one example that completely and correctly demonstrates the whole process involved in having C call Ruby then call C, return values through the chain neatly, and clean up. If there was such a reference, it would be outstanding.
- There seem to be external efforts to clean things up, but they aren't making it back into the official code. Is the embedded interface not really supposed to be used for anything but extensions to Ruby?

I wonder if Ruby is at all suitable as a general-purpose embedded scripting language? I am beginning to think that it is not.

The embedded interface and documentation are terrible! I don't mean any offense, and I maintain my stance that Ruby is an outstanding language. I love it. And I do realise that sometimes code takes some time to mature- I've been using Ruby since the 90's and it wasn't always a smooth ride. Is this the problem, it just isn't quite ready for embedding yet? Or I'm not using the right documentation? Or I'm asking too much? Am I doing the wrong thing?

It probably seems odd for someone with such a high opinion of the language to have such a low opinion of the embedded interface. Perhaps it is because Ruby has set my expectations so high that I am that much more easily disappointed.

Some projects seem to have managed it, and they may work, to some degree. But their use of it seems to be fairly basic, rather than counting on it as a core component of the program. VIM has a nice example, although it doesn't seem to worry about garbage collection or cleaning up. As such it doesn't seem complete.

Am I alone in thinking this? Should I be looking at other embeddable languages (Python, for example), or just go back and get the LUA code working again? Am I doing something wrong?

Is there anything I can do to successfully and reliably add support for my favourite scripting language into my project? Or is it simply an unrealistic prospect?

Garth

[Devin_Mullins]

GD wrote:

- Ideally have separate contexts so that two separate games running at once (in the same exe) can work in their own scripting environment.

AFAIK, this is going to be your difficult one. Others with actual knowledge can comment on why, and on workarounds.

Other than that, I'm clueless, and WOW that was a long post.

Devin

[gga]

GD ha escrito:

This is what I needed to replace LUA:
- Ability to load a script *from*memory*, not from a file (important).

Use rb_eval_string_protect().

- Run any init code in the script.

BEGIN {} calls or rb_eval_string_protect() of whatever you need.

- Hook a bunch of local functions (C++) so that the script can call them for
information from the main code body.

Ideally, handle this using SWIG, as either a ruby extension or a pseudo
extension you link in statically.
Avoid doing the binding yourself manually. Binding functions for most
basic C constructs (ints, chars, etc) is VERY easy.
Doing proper binding of classes, it is not as easy, as the GC requires
you to be careful. Best thing for you to do is look at SWIG's output.
SWIG also deals properly now with inheritance of C++ classes into ruby,
including multiple inheritance.

- Call functions in the script both periodically and in response to events (eg.
an update call every 1/60 of a second, in response to mouse clicks, etc).

This may be much trickier with the current ruby C interpreter,
depending on what you are looking for. You can start a C++ thread to
do it, but the ruby C interpreter is not thread safe. So, if you have
ruby code running in two different threads at the same time, bad things
WILL happen.
You can also try to rely on a single thread and let ruby's own light
threads handle this. However, this may not be as efficient as actual
multi-threaded code.
Mind you, ruby is not the only language with issues here. Python is
also pretty bad at this.
LUA and TCL are in a class of their own as far as multithreading
interpreters go.

- Clean up everything entirely when the level is over.

This is what I have encountered:

- There doesn't appear to be any way to indicate that Ruby should clean
everything up and get to a state where I can start again. I have attempted to
work around this by making each script contain a single unique class that I use,
but this is a terrible hack.

ruby_cleanup(0); SHOULD clean up the interpreter. Problem is not so
much cleaning the interpreter but restarting it again.
I did find issues when I did this (back in 1.8.0). I think now issues
are better, but I won't vouch for it being 100% solved.
If all fails, best thing you can do is what you did. Encapsulate all
in its own class and clean that instead. That will still leave the
class/module constants around, but it is better than nothing.

- How do I load a script from memory? rb_eval_string()? rb_compile_string()?
Something else?

Use rb_eval_string_protect().

- The documentation is flawed. It indicates calling rb_run(), but that can
eventually call exit, which is the last thing you want from an embedded
language. It mentions rb_apply() as having an argc parameter.

Again, rb_eval_string_protect() is what you probably want.

- I am getting inexplicable crashes when calling the garbage collector.

This is a probably a bug in your code. Take a look at rubyMEL at
highend3d.com for a relatively simple embedding of MEL into a 3d
application (Maya).
Depending on platform (windows, mainly) you need other initialization
code besides ruby_init().

- The documentation is not at all clear how you protect something from the
garbage collector, or indicate it is no longer in use.

You need to use a mark and a free function respectively, and use ruby's
mysterious Data_Wrap_Struct stuff. This is the pretty hairy part of
exposing a C++ api.
Suggestion: look at SWIG's code. It has a bunch of wrapper functions
to handle it nicely and it is now pretty mature.
For any questions regarding SWIG not clear from the SWIG manual, refer
to FXRuby's excellent wrapping of the Fox library.

- How does one call "Foo.new(args)" on a class from outside of Ruby, where the
args have been generated from outside of Ruby (hence no name) meaning

You call it like any other method, with rb_funcall* like:

VALUE cFoo; // defined somewhere else...
int nargs = 1;
VALUE args = { INT2NUM( 30 ) };
ID new_id = rb_intern("new");
rb_funcall2( cFoo, new_id, nargs, args );

- rb_protect() is a complete mess. Rather than being able to call an arbitrary
function with a void pointer, you are stuck calling something that takes a
single VALUE arg.

No, you are wrong. It is extremely well thought out.
No need for ugly void*, thank god.
Think of VALUE as being a void* (it is really an unsigned long). Ruby
requires sizeof(unsigned long) == sizeof(void*) -- see ruby.h's check.
Just do a simple casting of your pointer to VALUE in your caller and do
the opposite cast from VALUE to whatever in your receiver function.
See how all the RCAST() and similar macros work in ruby.h.

- The code runs fine for a while if I don't call the garbage collector.
Unfortunately, I get stack depth errors and eventual segfaults if I make
repeated calls to the Ruby script (via rb_apply()).

This likely means you wrapped your C++ classes incorrectly or you are
trying to run ruby in a multi-threaded environment.
If the first is true, look and learn from SWIG. If the second is true,
remove the multi-threaded code, create a global mutex that all your
ruby functions should call before doing anything or use a different
scripting language at this time.
Ruby is not thread safe, unlike LUA.

- I can't find one example that completely and correctly demonstrates the whole
process involved in having C call Ruby then call C, return values through the
chain neatly, and clean up. If there was such a reference, it would be outstanding.

There's no real trick to it.
Ruby is fine as an embedded language, just as you remember it is not
safe as a multi-threaded embedded language. YARV may eventually solve
this somewhere in the future. The unofficial fork of Ruby that is
Sydney (and now rubinius) supposedly is multi-thread safe, but is
32-bits only (and I won't vouch for it).
Python is just as bad as Ruby, btw, albeit it avoids some threading
issues by using a global lock every time two threads try to run some
code simultaneously, effectively running only a single thread at a
time. In ruby, you can do the same, but you need to create this global
mutex yourself.
LUA and TCL are different and better in this aspect, as they both can
use different (and very light) interpreter instances to execute code.

Am I alone in thinking this? Should I be looking at other embeddable languages
(Python, for example), or just go back and get the LUA code working again? Am I
doing something wrong?

Truth is... you are looking at the two best languages for embedding.
LUA is admittedly better to embed due its multi-threading aspects,
while Ruby is admittedly better for embedding due to its nice syntax.

Is there anything I can do to successfully and reliably add support for my
favourite scripting language into my project? Or is it simply an unrealistic
prospect?

Follow the advices I gave you and you should be able to get ruby as an
embedded language. Wrapping classes for the garbage collector is the
hairy part, but you can steal all of SWIG's source code for that now
(or use SWIG itself for doing it).
However, if your game engine does require heavy multi-threading in the
scripting language, you are indeed looking at the wrong language.
LUA is still the best language for this, unfortunately.

[gga]

GD ha escrito:

This is what I needed to replace LUA:
- Ability to load a script *from*memory*, not from a file (important).

Use rb_eval_string_protect().

- Run any init code in the script.

BEGIN {} calls or rb_eval_string_protect() of whatever you need.

- Hook a bunch of local functions (C++) so that the script can call them for
information from the main code body.

Ideally, handle this using SWIG, as either a ruby extension or a pseudo
extension you link in statically.
Avoid doing the binding yourself manually. Binding functions for most
basic C constructs (ints, chars, etc) is VERY easy.
Doing proper binding of classes, it is not as easy, as the GC requires
you to be careful. Best thing for you to do is look at SWIG's output.
SWIG also deals properly now with inheritance of C++ classes into ruby,
including multiple inheritance.

- Call functions in the script both periodically and in response to events (eg.
an update call every 1/60 of a second, in response to mouse clicks, etc).

This may be much trickier with the current ruby C interpreter,
depending on what you are looking for. You can start a C++ thread to
do it, but the ruby C interpreter is not thread safe. So, if you have
ruby code running in two different threads at the same time, bad things
WILL happen.
You can also try to rely on a single thread and let ruby's own light
threads handle this. However, this may not be as efficient as actual
multi-threaded code.
Mind you, ruby is not the only language with issues here. Python is
also pretty bad at this.
LUA and TCL are in a class of their own as far as multithreading
interpreters go.

- Clean up everything entirely when the level is over.

This is what I have encountered:

- There doesn't appear to be any way to indicate that Ruby should clean
everything up and get to a state where I can start again. I have attempted to
work around this by making each script contain a single unique class that I use,
but this is a terrible hack.

ruby_cleanup(0); SHOULD clean up the interpreter. Problem is not so
much cleaning the interpreter but restarting it again.
I did find issues when I did this (back in 1.8.0). I think now issues
are better, but I won't vouch for it being 100% solved.
If all fails, best thing you can do is what you did. Encapsulate all
in its own class and clean that instead. That will still leave the
class/module constants around, but it is better than nothing.

- How do I load a script from memory? rb_eval_string()? rb_compile_string()?
Something else?

Use rb_eval_string_protect().

- The documentation is flawed. It indicates calling rb_run(), but that can
eventually call exit, which is the last thing you want from an embedded
language. It mentions rb_apply() as having an argc parameter.

Again, rb_eval_string_protect() is what you probably want.

- I am getting inexplicable crashes when calling the garbage collector.

This is a probably a bug in your code. Take a look at rubyMEL at
highend3d.com for a relatively simple embedding of MEL into a 3d
application (Maya).
Depending on platform (windows, mainly) you need other initialization
code besides ruby_init().

- The documentation is not at all clear how you protect something from the
garbage collector, or indicate it is no longer in use.

You need to use a mark and a free function respectively, and use ruby's
mysterious Data_Wrap_Struct stuff. This is the pretty hairy part of
exposing a C++ api.
Suggestion: look at SWIG's code. It has a bunch of wrapper functions
to handle it nicely and it is now pretty mature.
For any questions regarding SWIG not clear from the SWIG manual, refer
to FXRuby's excellent wrapping of the Fox library.

- How does one call "Foo.new(args)" on a class from outside of Ruby, where the
args have been generated from outside of Ruby (hence no name) meaning

You call it like any other method, with rb_funcall* like:

VALUE cFoo; // defined somewhere else...
int nargs = 1;
VALUE args = { INT2NUM( 30 ) };
ID new_id = rb_intern("new");
rb_funcall2( cFoo, new_id, nargs, args );

- rb_protect() is a complete mess. Rather than being able to call an arbitrary
function with a void pointer, you are stuck calling something that takes a
single VALUE arg.

No, you are wrong. It is extremely well thought out.
No need for ugly void*, thank god.
Think of VALUE as being a void* (it is really an unsigned long). Ruby
requires sizeof(unsigned long) == sizeof(void*) -- see ruby.h's check.
Just do a simple casting of your pointer to VALUE in your caller and do
the opposite cast from VALUE to whatever in your receiver function.
See how all the RCAST() and similar macros work in ruby.h.

- The code runs fine for a while if I don't call the garbage collector.
Unfortunately, I get stack depth errors and eventual segfaults if I make
repeated calls to the Ruby script (via rb_apply()).

This likely means you wrapped your C++ classes incorrectly or you are
trying to run ruby in a multi-threaded environment.
If the first is true, look and learn from SWIG. If the second is true,
remove the multi-threaded code, create a global mutex that all your
ruby functions should call before doing anything or use a different
scripting language at this time.
Ruby is not thread safe, unlike LUA.

- I can't find one example that completely and correctly demonstrates the whole
process involved in having C call Ruby then call C, return values through the
chain neatly, and clean up. If there was such a reference, it would be outstanding.

There's no real trick to it.
Ruby is fine as an embedded language, just as you remember it is not
safe as a multi-threaded embedded language. YARV may eventually solve
this somewhere in the future. The unofficial fork of Ruby that is
Sydney (and now rubinius) supposedly is multi-thread safe, but is
32-bits only (and I won't vouch for it).
Python is just as bad as Ruby, btw, albeit it avoids some threading
issues by using a global lock every time two threads try to run some
code simultaneously, effectively running only a single thread at a
time. In ruby, you can do the same, but you need to create this global
mutex yourself.
LUA and TCL are different and better in this aspect, as they both can
use different (and very light) interpreter instances to execute code.

Am I alone in thinking this? Should I be looking at other embeddable languages
(Python, for example), or just go back and get the LUA code working again? Am I
doing something wrong?

Truth is... you are looking at the two best languages for embedding.
LUA is admittedly better to embed due its multi-threading aspects,
while Ruby is admittedly better for embedding due to its nice syntax.

Is there anything I can do to successfully and reliably add support for my
favourite scripting language into my project? Or is it simply an unrealistic
prospect?

Follow the advices I gave you and you should be able to get ruby as an
embedded language. Wrapping classes for the garbage collector is the
hairy part, but you can steal all of SWIG's source code for that now
(or use SWIG itself for doing it).
However, if your game engine does require heavy multi-threading in the
scripting language, you are indeed looking at the wrong language.
LUA is still the best language for this, unfortunately.
Another thing you might find out an issue when embedding another
language is performance.
LUA (and specially LuaJIT) is damn fast and is several orders of
magnitude faster than ruby (LuaJIT is about 10x-15x faster for most
stuff, and that includes YARV).

[M_Edward_Ed_Borasky1 (M. Edward (Ed) Borasky)]

GD wrote:

[snip]

I wonder if Ruby is at all suitable as a general-purpose embedded scripting language? I am beginning to think that it is not.

The embedded interface and documentation are terrible! I don't mean any offense, and I maintain my stance that Ruby is an outstanding language. I love it. And I do realise that sometimes code takes some time to mature- I've been using Ruby since the 90's and it wasn't always a smooth ride. Is this the problem, it just isn't quite ready for embedding yet? Or I'm not using the right documentation? Or I'm asking too much? Am I doing the wrong thing?

It probably seems odd for someone with such a high opinion of the language to have such a low opinion of the embedded interface. Perhaps it is because Ruby has set my expectations so high that I am that much more easily disappointed.

Some projects seem to have managed it, and they may work, to some degree. But their use of it seems to be fairly basic, rather than counting on it as a core component of the program. VIM has a nice example, although it doesn't seem to worry about garbage collection or cleaning up. As such it doesn't seem complete.

Am I alone in thinking this? Should I be looking at other embeddable languages (Python, for example), or just go back and get the LUA code working again? Am I doing something wrong?

Is there anything I can do to successfully and reliably add support for my favourite scripting language into my project? Or is it simply an unrealistic prospect?

Well ... first of all -- I suspect your project is rather large, and you seem to have factored it into a Ruby piece, a C++ piece, and a Lua piece, with each of those languages doing what it is good at. So, barring a complete re-write in a *single* language, I'd say the most expeditious thing to do would be to fix the Lua code that's broken in Lua, rather than trying to rewrite it in Ruby. I don't know what your time and money constraints are -- if you need to earn a living with E.V.E. Paradox, I'd do whatever gets you on the air in the most "agile" manner.

But if you've got the time, the ambition, the money, etc. and want to rewrite the project in a single language, I think, given today's high-speed low-cost systems, almost any language will do the job, so pick one that you can work productively in. With a little tweaking and maybe a little C, you ought to be able to do everything you're doing now in Ruby instead of a mix of languages. Ruby has a nice set of concurrency primitives, such as monitors, threads, distributed objects and Linda-like tuple spaces.

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

[Vincent_Fourmond]

GD wrote:

Am I alone in thinking this? Should I be looking at other embeddable
languages (Python, for example), or just go back and get the LUA code
working again? Am I doing something wrong?

  Just an idea - have you ever looked at scheme ? It seems a good choice
for embedded scripts, although I never had to work with it so far.

  Vince

···

--
Vincent Fourmond, PhD student
http://vincent.fourmond.neuf.fr/

[GD1 (GD)]

Hey Devin,

Devin Mullins wrote:

GD wrote:

- Ideally have separate contexts so that two separate games running at once (in the same exe) can work in their own scripting environment.

AFAIK, this is going to be your difficult one. Others with actual knowledge can comment on why, and on workarounds.

Looking at the Ruby source, I can see this being difficult. There
are also workarounds on the E.V.E. Paradox side (script merging,
limiting access to one scripted level at a time etc) if needed.
Hence "ideally" rather than "realistically". The other stuff
is pretty critical. Not crashing is also nice!

I've considered forking and using pipe IPC to solve the problem,
but I'd lose the tight integration and also have to learn a lot
more about Windows IPC. I might look into this if the current
approach is infeasible.

Other than that, I'm clueless, and WOW that was a long post.

Indeed, and let it not be said that I didn't give it a darn good
shot. Would you believe that I actually left out a lot of
detail in my post?

Garth

[Martin_DeMello]

What about Io? I haven't used either Io or Lua myself, but they both
seem to offer similar embedding-oriented features, and Io seems like a
more attractive language to work in.

martin

···

On 1/16/07, gga <GGarramuno@aol.com> wrote:

However, if your game engine does require heavy multi-threading in the
scripting language, you are indeed looking at the wrong language.
LUA is still the best language for this, unfortunately.

[GD1 (GD)]

Hey gga,

Thankyou for the reply and information.

gga wrote:
> GD ha escrito:
>
>>This is what I needed to replace LUA:
>>- Ability to load a script *from*memory*, not from a file (important).
>
> Use rb_eval_string_protect().

Excellent, guessed that one at least. Maybe worth a mention in the docs
if that is the "official" solution; it wasn't too clear.

>>- Hook a bunch of local functions (C++) so that the script can call them for
>>information from the main code body.
>
> Ideally, handle this using SWIG, as either a ruby extension or a pseudo
> extension you link in statically.
> Avoid doing the binding yourself manually. Binding functions for most
> basic C constructs (ints, chars, etc) is VERY easy.
> Doing proper binding of classes, it is not as easy, as the GC requires
> you to be careful. Best thing for you to do is look at SWIG's output.
> SWIG also deals properly now with inheritance of C++ classes into ruby,
> including multiple inheritance.

I'm keeping things fairly simple; I mostly just bind functions/methods
and wrap pointers in opaque data structures (eg. using Data_Wrap_Struct
in the Ruby version). My plan was to store a pointer to the calling class
in an opaque structure and store it in the corresponding Ruby class, so
I can reference it when needed.

>>- Call functions in the script both periodically and in response to events (eg.
>>an update call every 1/60 of a second, in response to mouse clicks, etc).
>
> This may be much trickier with the current ruby C interpreter,
> depending on what you are looking for. You can start a C++ thread to
> do it, but the ruby C interpreter is not thread safe. So, if you have
> ruby code running in two different threads at the same time, bad things
> WILL happen.
> You can also try to rely on a single thread and let ruby's own light
> threads handle this. However, this may not be as efficient as actual
> multi-threaded code.
> Mind you, ruby is not the only language with issues here. Python is
> also pretty bad at this.
> LUA and TCL are in a class of their own as far as multithreading
> interpreters go.

Everything is single-threaded, or at least, the other threads don't even
know Ruby exists. Events and updates are handled in the one thread and
can be fed in in a safe and controlled manner. I'm not asking too much
of an embedded language, I'm keeping things fairly gentle. The nasty
stuff is handled in the C++ code.

>>- Clean up everything entirely when the level is over.
>
>>This is what I have encountered:
>>
>>- There doesn't appear to be any way to indicate that Ruby should clean
>>everything up and get to a state where I can start again. I have attempted to
>>work around this by making each script contain a single unique class that I use,
>>but this is a terrible hack.
>
> ruby_cleanup(0); SHOULD clean up the interpreter. Problem is not so
> much cleaning the interpreter but restarting it again.
> I did find issues when I did this (back in 1.8.0). I think now issues
> are better, but I won't vouch for it being 100% solved.
> If all fails, best thing you can do is what you did. Encapsulate all
> in its own class and clean that instead. That will still leave the
> class/module constants around, but it is better than nothing.

Indeed. Wasn't sure about ruby_cleanup, and I've seen a lot of examples
floating around, along with comments that things end up crashing shortly
afterward.

>>- The documentation is flawed. It indicates calling rb_run(), but that can
>>eventually call exit, which is the last thing you want from an embedded
>>language. It mentions rb_apply() as having an argc parameter.
>
> Again, rb_eval_string_protect() is what you probably want.

I have data structures in the C++ side, no names, so rb_eval_string and
company can't be used.

>>- I am getting inexplicable crashes when calling the garbage collector.
>
> This is a probably a bug in your code. Take a look at rubyMEL at
> highend3d.com for a relatively simple embedding of MEL into a 3d
> application (Maya).
> Depending on platform (windows, mainly) you need other initialization
> code besides ruby_init().

I won't be so arrogant to suggest my code has no bugs (haha) but we're
talking about code that has been run through days of heavy automated
testing each release with the current LUA setup being changed to use
Ruby instead, minimally reduced to using only two calls (ruby_init and
rb_gc) being called from multiple different locations and crashing
in different but consistent ways each time. It is entirely possible
that my code is interfering with it in some way, but it is manifesting
as predictable crashes in Ruby only, not in any of the other libraries
I am using. Short version: Looks more like a bug brought out by my
circumstances rather than because of the code itself.

I'm currently trying to get things going under Linux, so no tricky
init stuff yet.

>>- The documentation is not at all clear how you protect something from the
>>garbage collector, or indicate it is no longer in use.
>
> You need to use a mark and a free function respectively, and use ruby's
> mysterious Data_Wrap_Struct stuff. This is the pretty hairy part of
> exposing a C++ api.
> Suggestion: look at SWIG's code. It has a bunch of wrapper functions
> to handle it nicely and it is now pretty mature.
> For any questions regarding SWIG not clear from the SWIG manual, refer
> to FXRuby's excellent wrapping of the Fox library.

I hadn't thought of going back into SWIG. Good idea, thankyou. I had a
play around with it a week or so ago, seemed to work well. No doubt
the generated code would grant some clues.

Should some of the tricks used by SWIG be mentioned on the official
Ruby site, perhaps? Seems odd that an offsite source is considered
(mildly) authoritative.

>>- How does one call "Foo.new(args)" on a class from outside of Ruby, where the
>>args have been generated from outside of Ruby (hence no name) meaning
>
> You call it like any other method, with rb_funcall* like:
>
> VALUE cFoo; // defined somewhere else...
> int nargs = 1;
> VALUE args[] = { INT2NUM( 30 ) };
> ID new_id = rb_intern("new");
> rb_funcall2( cFoo, new_id, nargs, args );

Okay, so just like an object. Good news. Very elegant and Ruby-ish.
Any way to grab the class from the Ruby script? What I really want is
to replace your first line with something like:

VALUE cFoo = rb_grab_class_from_script("ClassDefinedInScript")

Each level defines its own class and I need to create an object of
the correct type. I can't find a call to lookup a class name and
return a VALUE. There is probably one, I just don't know what it is.

>>- rb_protect() is a complete mess. Rather than being able to call an arbitrary
>>function with a void pointer, you are stuck calling something that takes a
>>single VALUE arg.
>
> No, you are wrong. It is extremely well thought out.
> No need for ugly void*, thank god.
> Think of VALUE as being a void* (it is really an unsigned long). Ruby
> requires sizeof(unsigned long) == sizeof(void*) -- see ruby.h's check.
> Just do a simple casting of your pointer to VALUE in your caller and do
> the opposite cast from VALUE to whatever in your receiver function.
> See how all the RCAST() and similar macros work in ruby.h.

The best way to illustrate would be with an example:

VALUE GlueFunc(void *v)
{
   Glue *glue = (Glue *)v;
   return rb_funcall(glue->game->getLevel(), glue->method, glue->arg0);
}

struct Glue
{
   Game *game;
   ID method;
   VALUE arg0;
};

void Game::runWithProtect1(VALUE level, ID method, VALUE arg0)
{
   Glue glue;
   glue.game = this;
   glue.method = method;
   glue.arg0 = arg0;
   VALUE rv = rb_protect_that_uses_void(GlueFunc, &glue, &status);
   if (status)
     reportError();
   return rv;
)

In main code:

ID id_update;
VALUE current_frame;

....

runWithProtect1(id_update, current_frame);

(NB: There would be a large number of calls to runWithProtect(), so I need
to keep the call format simple).

If the glue function must be a VALUE which is an array, what steps
would you suggest were used to ensure the relevant data, include a C++
pointer, were passed through the callback to reach the glue function
GlueFunc? If you're bored, code up a solution that works. I managed to
get something going, minus the "this" pointer, and it was messy. It
involved converting everything into Ruby objects and putting them into
an array, and deconstructing at the other end. Perhaps you have a better
solution, and I'm open to improved ideas.

>>- The code runs fine for a while if I don't call the garbage collector.
>>Unfortunately, I get stack depth errors and eventual segfaults if I make
>>repeated calls to the Ruby script (via rb_apply()).
>
> This likely means you wrapped your C++ classes incorrectly or you are
> trying to run ruby in a multi-threaded environment.
> If the first is true, look and learn from SWIG. If the second is true,
> remove the multi-threaded code, create a global mutex that all your
> ruby functions should call before doing anything or use a different
> scripting language at this time.
> Ruby is not thread safe, unlike LUA.

Only using Ruby from a single thread. I am likely making mistakes
with creation and marking of Ruby objects/classes as I'm not familiar
enough with it nor does the documentation explain it. Thus manually
calling the GC, which has led to crashes even when all I do is init
and call the GC, hence my problems. I can't be making mistakes with
Ruby if I'm not actually loading any scripts or creating any objects
and classes.

>>- I can't find one example that completely and correctly demonstrates the whole
>>process involved in having C call Ruby then call C, return values through the
>>chain neatly, and clean up. If there was such a reference, it would be outstanding.
>
> There's no real trick to it.

I must disagree. If it is easy, there is a link _somewhere_ that
shows a concrete official example of how it should be done. Maybe
I have not found it. If it doesn't exist, if a Guru who intimately
understands the code could put one together and put it in the official
documentation, it would be an enormous help to people using the
embedded interface.

> Ruby is fine as an embedded language, just as you remember it is not
> safe as a multi-threaded embedded language. YARV may eventually solve
> this somewhere in the future. The unofficial fork of Ruby that is
> Sydney (and now rubinius) supposedly is multi-thread safe, but is
> 32-bits only (and I won't vouch for it).
> Python is just as bad as Ruby, btw, albeit it avoids some threading
> issues by using a global lock every time two threads try to run some
> code simultaneously, effectively running only a single thread at a
> time. In ruby, you can do the same, but you need to create this global
> mutex yourself.
> LUA and TCL are different and better in this aspect, as they both can
> use different (and very light) interpreter instances to execute code.

I'm not using Ruby as a multithreaded language; only one thread will
ever talk to it and being MT safe is unimportant to me.

The Python docs look decent, I'd just rather not relearn Python again,
I chose Ruby years ago, I'm good with it, and I like it better than
Python! But I'd rather move on from LUA if I can...

LUA has some very nice elements; there is a call to get a context, a
call to tear it down, so forth. It works. The interface to the controlling
code, however, is less than ideal and poorly documented. Using it is
painful. Error-handling makes me want to gouge my eyes out. The language
is a mere flicker to the brilliance of Ruby. However, it is very well
suited for the task; its design has always been around being an embedded
language. The embedded interface works. It is a well-constructed embedded
language.

My knowledge of TCL is non-existent.

>>Am I alone in thinking this? Should I be looking at other embeddable languages
>>(Python, for example), or just go back and get the LUA code working again? Am I
>>doing something wrong?
>
> Truth is... you are looking at the two best languages for embedding.
> LUA is admittedly better to embed due its multi-threading aspects,
> while Ruby is admittedly better for embedding due to its nice syntax.

Actually, I got LUA up and running in an hour or so. It was very basic,
but a good start that evolved into the current codebase. The documentation
was poor but I got it eventually. I'm not even using the MT stuff. I didn't
even know it handled it well.

I've spent close to two days on embedding Ruby and can't get the basic
things I've described working. The closest I've got is short runs followed
by crashes I can't diagnose. The embedding documentation is all over the
place, lacks examples, and has mistakes. When someone with twenty years
of development experience is having troubles this bad it does suggest there
might be a problem that needs addressing...

It is enormously frustrating as the Ruby language is significantly superior
to the LUA language, but the case is the exact reverse with the embedding
interface- and the LUA embedding interface could be much better! It breaks
my heart to have to struggle so hard to get a powerful language like Ruby
- that I use throughout my project extensively- to work reliably as an
embedded scripting language for the runtime itself. If I wasn't such a
huge fan of Ruby- and I promise you I am difficult to impress with a
language- then I would have given up after the first few frustrating
hours and said to myself "language is not ready yet", then moved on to
something else. I dearly want to be completely wrong. If I can get Ruby
going I can increase the significance of scripting in level design, and
add some really cool features I can't be bothered implementing in
LUA.

>>Is there anything I can do to successfully and reliably add support for my
>>favourite scripting language into my project? Or is it simply an unrealistic
>>prospect?
>
> Follow the advices I gave you and you should be able to get ruby as an
> embedded language. Wrapping classes for the garbage collector is the
> hairy part, but you can steal all of SWIG's source code for that now
> (or use SWIG itself for doing it).
> However, if your game engine does require heavy multi-threading in the
> scripting language, you are indeed looking at the wrong language.
> LUA is still the best language for this, unfortunately.

I'll dig around in the SWIG stuff, this may give me some ideas.

Thanks for the suggestions and input, much appreciated.

Garth

[GD1 (GD)]

Hey Ed,

> Well ... first of all -- I suspect your project is rather large, and you seem to have factored it into a Ruby piece, a C++ piece, and a Lua piece, with each of those languages doing what it is good at. So, barring a complete re-write in a *single* language,

The existing Ruby stuff is involved in the build process, and involves
running Ruby scripts- no embedded Ruby. It works brilliantly. I am
considering replacing the LUA piece with an embedded Ruby piece. Rewriting
the existing scripts would take a day at most, once I get the embedded Ruby
interface and hooks to my code working fine. The "working fine" bit is
the issue.

You are correct regarding each language doing what it is best at, so
hopefully you'll understand when I say a rewrite is simply never going
to happen. The low-level C++ code is optimised to death and the flexible
Ruby configuration and build tools would be overwhelmingly painful to write
in C++. So yeah, I need the best of all worlds. I am using multiple tools
in ways that emphasise their strengths. I just need them to play nice
together.

> I'd say the most expeditious thing to do would be to fix the Lua code that's broken in Lua, rather than trying to rewrite it in Ruby. I don't know what your time and money constraints are -- if you need to earn a living with E.V.E. Paradox, I'd do whatever gets you on the air in the most "agile" manner.

Yes. Can't see that taking more than a day. BUT if Ruby was my scripting
language I can make huge improvements to the scope of scripting in
E.V.E. Paradox. I can move game features out into it. There is a lot of
incredible stuff I could do. I'd love to have Ruby scripting, and the
payoff in time may be worth it. But if the outcome is unattainable, I need
to consider other options.

Garth

[M_Edward_Ed_Borasky1 (M. Edward (Ed) Borasky)]

Vincent Fourmond wrote:

GD wrote:
  

Am I alone in thinking this? Should I be looking at other embeddable
languages (Python, for example), or just go back and get the LUA code
working again? Am I doing something wrong?
    
  Just an idea - have you ever looked at scheme ? It seems a good choice
for embedded scripts, although I never had to work with it so far.

  Vince

Yeah, there are a couple of Scheme-C half-breeds (Scheme interpreter plus Scheme-to-C compilers, Scheme embeddable in C, etc.) that are pretty good and high-performance. Chicken and Gambit are the two that I know the most about. Gambit has an experimental Erlang-like lightweight process capability that might be good for game programming. I know next to nothing about game programming. I don't play the games and the last time I even attempted to write one, the hardware was on the order of 64 kilobyte 6502s and the language of choice was Forth.

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

[GD1 (GD)]

Hey Vincent,

Vincent Fourmond wrote:

Am I alone in thinking this? Should I be looking at other embeddable
languages (Python, for example), or just go back and get the LUA code
working again? Am I doing something wrong?

  Just an idea - have you ever looked at scheme ? It seems a good choice
for embedded scripts, although I never had to work with it so far.

Worth looking into.

I know no scheme at all, so it puts me at a disadvantage. However, a
quick search yields a few different interpreters, some which _might_
be suitable for embedding.

Thanks for that.

Garth

[GD1 (GD)]

Hey Martin,

Never used it. Will look into it, thanks for the suggestion.

Garth

···

What about Io? I haven't used either Io or Lua myself, but they both

> seem to offer similar embedding-oriented features, and Io seems like a
> more attractive language to work in.

[gga]

GD ha escrito:

I'm keeping things fairly simple; I mostly just bind functions/methods
and wrap pointers in opaque data structures (eg. using Data_Wrap_Struct
in the Ruby version). My plan was to store a pointer to the calling class
in an opaque structure and store it in the corresponding Ruby class, so
I can reference it when needed.

That's possible, but it can get ugly. Do what SWIG does. Create a
hash that stores a mapping between the ruby object and the C++ object
(and/or viceversa). Every time you allocate a new class, you enter it
in the hash. Every time the free function for the class is called, the
mapping is removed from the hash.
You can even use ruby's own hashes to do so.
That way, you don't need to pass any pointer, and can access any class
at any time from any function.
Also, if your classes contain or return pointers to other C++ classes,
you need to keep telling the GC about it. See the SWIG Zoo/Animal
tutorial in the Ruby section.

I have data structures in the C++ side, no names, so rb_eval_string and
company can't be used.

This makes no sense to me.

>>- The documentation is not at all clear how you protect something from the
>>garbage collector, or indicate it is no longer in use.
>

Should some of the tricks used by SWIG be mentioned on the official
Ruby site, perhaps? Seems odd that an offsite source is considered
(mildly) authoritative.

Well, all that stuff IS properly documented in the PickAxe, in the
extending Ruby section. The PickAxe is somewhat poor in that it does
cover properly wrapping methods that return new C++ objects (the Zoo
and Animal tutorial in SWIG is a top-notch example of all the headaches
you can run into on wrapping a complex library).

You also have this, if you haven't read it.

http://metaeditor.sourceforge.net/embed/

I found it somewhat confusing, but it is probably another source.

Okay, so just like an object. Good news. Very elegant and Ruby-ish.
Any way to grab the class from the Ruby script? What I really want is
to replace your first line with something like:

VALUE cFoo = rb_grab_class_from_script("ClassDefinedInScript")

Class names are just constants.
If your class is in the global namespace, use:

VALUE cFoo = rb_const_get( rb_cObject, rb_intern("YourClass") );

If not, replace rb_cObject for the proper module name.

Each level defines its own class and I need to create an object of
the correct type. I can't find a call to lookup a class name and
return a VALUE. There is probably one, I just don't know what it is.

>>- rb_protect() is a complete mess. Rather than being able to call an arbitrary
>>function with a void pointer, you are stuck calling something that takes a
>>single VALUE arg.

//--------------------------------------------------------------------------------------------
// g++ -fPIC -shared -I/usr/include/ -I /usr/include/ruby/\
// -I/usr/lib/ruby/1.8/x86_64-linux/ callback.cpp -o callback.so
//
// > irb
// irb> require 'callback.so'
// irb> wrap()
// irb> wrap 'crap', 'hello'
//

#include <ruby.h>
#include <iostream>

#define VALUEFUNC(f) ((VALUE (*)(ANYARGS)) f)

struct CallbackData_t
{
  void* crapola;
  VALUE method;
  VALUE arg0;
};

VALUE wrap_callback( VALUE v )
{
  using namespace std;
  CallbackData_t* data = static_cast< CallbackData_t* >((void*)v);

  cerr << "wrap callback got: " << data->crapola << endl;
  // do any rb_funcall2() or rb_apply() here

  if ( rb_obj_is_kind_of( data->arg0, rb_cString ) )
    cerr << "arg0 is " << StringValueCStr( data->arg0 ) << endl;

  return Qnil;
}

VALUE wrap_protect(int argc, VALUE *argv, VALUE self)
{
  CallbackData_t data;
  data.crapola = (void*)0xDEADBEEF;

  // 0 mandatory arguments, 2 optional
  rb_scan_args( argc, argv, "02", &data.method, &data.arg0 );

  int error;
  VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );
  if ( error )
    rb_raise( rb_eRuntimeError, "Something crappy happened" );
  return result;
}

extern "C" void Init_callback(void)
{
  rb_define_method(rb_cObject, "wrap", VALUEFUNC(wrap_protect), -1);
}
//--------------------------------------------------------------------------------

The Python docs look decent, I'd just rather not relearn Python again,
I chose Ruby years ago, I'm good with it, and I like it better than
Python! But I'd rather move on from LUA if I can...

Python is worse at embedding than Ruby is. So if you have troubles
with Ruby, good luck with Python! On python you don't need to just
take care of garbage collecting, but of reference counting. It is a
royal pain if you are doing it manually like you are.

LUA has some very nice elements; there is a call to get a context, a
call to tear it down, so forth. It works.

Indeed.

My knowledge of TCL is non-existent.

Keep it that way. If you don't like Lua, TCL will scare the shit out
of you. TCL syntax can look pretty but it often does not. It does
have one of the best embedding interface for embedding of any language
(and 20+ years of knowledge you can rely on).

I've spent close to two days on embedding Ruby and can't get the basic
things I've described working.

You should be able to get ruby up and running in about a couple of
hours (that's all it took me). Wrapping complex classes that return
pointers to other classes and the like, however, is hairy. But SWIG is
there to help now.

I dearly want to be completely wrong.

You are.

[GD1 (GD)]

Hey Ed,

M. Edward (Ed) Borasky wrote:

Vincent Fourmond wrote:

GD wrote:

Am I alone in thinking this? Should I be looking at other embeddable
languages (Python, for example), or just go back and get the LUA code
working again? Am I doing something wrong?
    
  Just an idea - have you ever looked at scheme ? It seems a good choice
for embedded scripts, although I never had to work with it so far.

    Vince

Yeah, there are a couple of Scheme-C half-breeds (Scheme interpreter plus Scheme-to-C compilers, Scheme embeddable in C, etc.) that are pretty good and high-performance. Chicken and Gambit are the two that I know the most about. Gambit has an experimental Erlang-like lightweight process capability that might be good for game programming. I know next to nothing about game programming. I don't play the games and the last time I even attempted to write one, the hardware was on the order of 64 kilobyte 6502s and the language of choice was Forth.

This is interesting. Sorry that I didn't notice it when replying
to Vincent's post. I'll check these out.

And Gambit seems to be the name of a great many things. The Gambit
I know of is a companion to Fluent.

I've got a fair few points of reference for other embedded possibilities
now, quite ironic given the location. Anyway, off to research and
experiment.

Garth

[gga]

> What about Io? I haven't used either Io or Lua myself, but they both
> seem to offer similar embedding-oriented features, and Io seems like a
> more attractive language to work in.

Io is, at this point in time, too immature, compared to Lua or Ruby.
Their current "stable" release won't even compile.

[gga]

> What about Io? I haven't used either Io or Lua myself, but they both
> seem to offer similar embedding-oriented features, and Io seems like a
> more attractive language to work in.

Io is, at this point in time, too immature, compared to Lua or Ruby.
Their current "stable" release won't even compile.
And, unfortunately, their developers are a tad insane and use darcs for
revision control, which means you can spend almost an hour retrieving
their current unstable release the first time you check it out.

[GD1 (GD)]

gga,

gga wrote:
> GD ha escrito:
>
>>I'm keeping things fairly simple; I mostly just bind functions/methods
>>and wrap pointers in opaque data structures (eg. using Data_Wrap_Struct
>>in the Ruby version). My plan was to store a pointer to the calling class
>>in an opaque structure and store it in the corresponding Ruby class, so
>>I can reference it when needed.
>
> That's possible, but it can get ugly. Do what SWIG does. Create a
> hash that stores a mapping between the ruby object and the C++ object
> (and/or viceversa). Every time you allocate a new class, you enter it
> in the hash. Every time the free function for the class is called, the
> mapping is removed from the hash.
> You can even use ruby's own hashes to do so.
> That way, you don't need to pass any pointer, and can access any class
> at any time from any function.
> Also, if your classes contain or return pointers to other C++ classes,
> you need to keep telling the GC about it. See the SWIG Zoo/Animal
> tutorial in the Ruby section.

This is an interesting idea but may be overkill for what I'm trying
to do.

>> >>- The documentation is not at all clear how you protect something from the
>> >>garbage collector, or indicate it is no longer in use.
>> >
>>
>>Should some of the tricks used by SWIG be mentioned on the official
>>Ruby site, perhaps? Seems odd that an offsite source is considered
>>(mildly) authoritative.
>
> Well, all that stuff IS properly documented in the PickAxe, in the
> extending Ruby section. The PickAxe is somewhat poor in that it does
> cover properly wrapping methods that return new C++ objects (the Zoo
> and Animal tutorial in SWIG is a top-notch example of all the headaches
> you can run into on wrapping a complex library).

The online version of the PickAxe has been my main reference. It is
missing a bit of info, generally relating to marking and garbage
collection.

My secondary reference...

> You also have this, if you haven't read it.
>
> http://metaeditor.sourceforge.net/embed/
>
> I found it somewhat confusing, but it is probably another source.

... is this one.

This one fills in some of the gaps from the first. It is no literary
masterpiece but it is quite useful.

>>Okay, so just like an object. Good news. Very elegant and Ruby-ish.
>>Any way to grab the class from the Ruby script? What I really want is
>>to replace your first line with something like:
>>
>>VALUE cFoo = rb_grab_class_from_script("ClassDefinedInScript")
>>
>
> Class names are just constants.
> If your class is in the global namespace, use:
>
> VALUE cFoo = rb_const_get( rb_cObject, rb_intern("YourClass") );
>
> If not, replace rb_cObject for the proper module name.

This I didn't know. So class names are basically constants and
can be accessed that way.

> VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );

You can do this!? Is this safe? It would seem to work on the idea
that VALUE is always going to be of sufficient size and structure
to hold a pointer directly. Is there any authoritative source that
says: "directly casting a pointer to a VALUE is perfectly fine and
will continue to be supported".

If so, VALUE can be used (effectively) in the same way as a void*,
so there is no need to convert everything into Ruby objects as
I had thought.

If not, I'm not sure this call can be counted on to work reliably
in the future.

>>The Python docs look decent, I'd just rather not relearn Python again,
>>I chose Ruby years ago, I'm good with it, and I like it better than
>>Python! But I'd rather move on from LUA if I can...
>
> Python is worse at embedding than Ruby is. So if you have troubles
> with Ruby, good luck with Python! On python you don't need to just
> take care of garbage collecting, but of reference counting. It is a
> royal pain if you are doing it manually like you are.

I'll probably give it a shot for interest's sake. Looking closer at the
Python docs that don't seem as strong as first impressions suggest,
although there are some decent examples.

>>I've spent close to two days on embedding Ruby and can't get the basic
>>things I've described working.
>
> You should be able to get ruby up and running in about a couple of
> hours (that's all it took me).

Yes, I should (something basic anyway). Unfortunately, I didn't (*).
This is the problem.

When such difficulties arise, they can suggest:
- it is the user's fault; or
- there is a problem with the documentation; or
- the interface is unintuitive; or
- the library itself is buggy.

I shouldn't have to explain why the likelihood of the first reduces
significantly with the number of external libraries the user has
successfully integrated in the past and the number of years they have
been working in software development. Any of the remaining three are
worth bringing to the attention of the dev gods.

Anyway I'm hoping that someone with the requisite knowledge spots these
posts, thinks "hey, perhaps we could look again at issue X" and
something comes of my experiences here. Or perhaps not, but nobody can
say I never mentioned it. Ultimately, the selfish aspect of my motivation
is that someone spots these posts, and dispenses some form of guru-like
wisdom that instantly solves the problems I am having.

Googling some of the error messages I am seeing and issues I am facing
suggest I may not be alone.

(*) - Technically I _did_ in a couple of hours, but it wasn't stable and
crashed randomly, so I tried to figure out what was going wrong. No luck.

> Wrapping complex classes that return
> pointers to other classes and the like, however, is hairy. But SWIG is
> there to help now.
>
>>I dearly want to be completely wrong.
>
> You are.

That was unwarranted.

Anyway, in this exchange a number of issues have come up that an
experienced Ruby user (definitely not a guru though!) completely new
to the embedded interface tripped up on completely. I would suggest
that a good place for these little tips and tricks that keep coming up
would be somewhere near the official documentation, so that the next
person who comes along to play with embedded Ruby can start off
slightly further ahead than I did.

Garth

[Benjohn_Barnes]

> What about Io? I haven't used either Io or Lua myself, but they
both
> seem to offer similar embedding-oriented features, and Io seems
like a
> more attractive language to work in.

Io is, at this point in time, too immature, compared to Lua or Ruby.
Their current "stable" release won't even compile.
And, unfortunately, their developers are a tad insane and use darcs for
revision control, which means you can spend almost an hour retrieving
their current unstable release the first time you check it out.

I thought Darks seemed pretty awesome when I looked, but very
annoying needing to get hold of ocaml or hascal, or what ever it was to
compile it up. Now, if it were written in Ruby...

[gga]

GD ha escrito:

This is an interesting idea but may be overkill for what I'm trying
to do.

Well, I'd suggest you read swig docs and see if you still think is
overkill for your project. Since you mentioned it is a game engine, I
seriously doubt that it will be overkill.

> VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );

You can do this!? Is this safe?

Yes. Again, read ruby.h. It checks for this to be true... or ruby
won't compile.

Is there any authoritative source that
says: "directly casting a pointer to a VALUE is perfectly fine and
will continue to be supported".

Besides me, you mean? I guess we have a trust issue, here

Well, that relies on a feature of the C/C++ language. As long as you
are using C ruby and we are dealing with 32/64-bit machines, it will
hold true. Again, read ruby.h and config.h.
Besides that, Matz has mentioned this is ok (search the list). His
goal for VALUE was to have it be like void*, but clearer to users and
keeping type safety. Unfortunately, C does not quite allow it to
behave like a void* in all contexts, so sometimes you do need to cast
it to void*.
This is more or less a limitation of C/C++.

If that's not enough for you, I'll try to see if I can convince the
Pope to say something along those lines, but it might be tricky

If so, VALUE can be used (effectively) in the same way as a void*,
so there is no need to convert everything into Ruby objects as
I had thought.

Err... you weren't exposing each and every single little class in your
code, were you? Ouch.
That's not the idea, of course. The stuff you need to convert to Ruby
is only the stuff you need your users to use in Ruby, as it stands to
reason.

I'll probably give it a shot for interest's sake. Looking closer at the
Python docs that don't seem as strong as first impressions suggest,
although there are some decent examples.

Well... what EXACTLY are you trying to achieve, Garret? You started
the thread mentioning you WANTED to get away from Lua and use Ruby for
your game engine.
Personally, within the context of game engines, I must tell you, I
think that's not such a good idea, as Lua is multi-thread safe today
which is definitively something you do want in a game engine (and
LuaJIT is probably *THE* fatest VM I have *EVER* seen for a non-static
typed language).
As such, I warned you before-hand that neither Python nor Ruby are
thread safe.
You did mention you liked Ruby's syntax better than anything else and
you did not care about multi-threading, so I kept helping you with
ruby.
You mentioned you had difficulties embedding Ruby, and I gave you
pointers for you to read on. You did not understand how to use a
couple of functions, and I pointed to you the right stuff you should be
using.
You posted some code that crashed on you, and I gave you similar
working code that works reliably, with compiling instructions to boot.
You questioned whether my code was valid, which means you probably have
not tried running it.
You have also not followed my advice of looking at SWIG and, instead,
now you want to look at Python (!?). Are you pulling my chain?

If you DO want to use Python for wrapping a library, and you are
familiar with C++ template use, I suggest using boost::python and Py++.
It is slightly more efficient than SWIG, albeit it can be hard to
debug if you are not familiar with templates.
If you don't like templates (like myself), I would ALSO recommend you
learn and use SWIG for embedding Python.
Is there any reason why you are avoiding SWIG and not even looking at
the documentation I pointed you to? In case you did not find it, it's
here, btw:
SWIG and Ruby (read: memory management
section)
SWIG Users Manual

You will learn quite a lot about embedding stuff from reading it. My
guess is that the problems you are having with Ruby and perhaps Lua
might be because you are not too familiar with the subtleties that can
arise from exposing classes to scripting languages. SWIG does a really
good job of explaining that (best thing I've seen yet) and has working
and relatively well tested code in about 20+ languages.

The good thing about SWIG is that you can actually use Lua (which you
mentioned you know well already) and expose one or two classes that way
first. Then, you should be able to easily port that code to ruby, by
just changing just a couple of swig files (if any).
Also, if you end up dropping Ruby at some point in time, you should be
able to again port your code easily to whatever other language you
fancy.

(*) - Technically I _did_ in a couple of hours, but it wasn't stable and
crashed randomly, so I tried to figure out what was going wrong. No luck.

Well, if you've wrapped a class or two and they are crashing, your best
bet is posting the code to them to get help. Otherwise, it is unlikely
anyone will be able to help you out much.
Also, it will be hard for anyone to write better docs, if nobody knows
what exactly are you having difficulties with.

That was unwarranted.

True, it did read quite nasty once I read it myself. What I meant was
more like: "Luckily for you, you are wrong".

I would suggest that a good place for these little tips and tricks that keep coming up
would be somewhere near the official documentation, so that the next
person who comes along to play with embedded Ruby can start off
slightly further ahead than I did.

Sure. Feel free to write them up somewhere. In case it was not clear,
any code or snippet of code I posted in this thread is public-domain.