Win32 Extension Issues Wanted!

ruby-talk
1

Everyone. I had a conversation with folks from Microsoft today about
trying to move Ruby’s installer to Visual Studio 2005 and WiX (the MSI
installer). They’re quite serious about trying to get us shifted to
modern compiler technology rather than VC++6 or even MSYS/MINGW. I
can’t say that I disagree with them, either.

However, I indicated that one of the big problems that we had with
Ruby 1.8.2 was the use of VS.NET 2003 (VC++ 7.1) and its general
incompatibility – focussing on errno, because that’s the one item
which stuck in my mind – with earlier and later runtimes, as well as
a big problem of compatibility with MSYS/gcc and any MSVCRT (isascii,
as reported by Ara Howard).

They asked for as much information as I can gather on this matter. If
you have had problems trying to get Ruby or an extension compiled or
running on Windows for any reason, but especially because of Microsoft
runtime DLL differences, please provide me as much information as
possible so that I can pass it on to the VS team at Microsoft.

For what it’s worth, I have Ruby itself compiled with VS 2005,
although some of the extensions are not being built automatically,
from what I can tell. That may be a build script problem. However,
based on earlier reports of runtime version incompatibilities, it was
looking like I was going to have to recompile a lot of code. The
discussion today suggested that as long as function accesses are used
and not variable accesses, things will be okay (e.g., GetError() in
Windows, not errno). The problem, as I pointed out to them, is that
most Unix developers only ever have to worry about a single C
runtime being on their system and therefore don’t need to worry about
errno being in a different runtime DLL. Ideally, they would be able to
give us an external way of getting at the errno from a specific
runtime that may not be “our” runtime (e.g., the runtime with which we
were linked).

What other problems have people had and what can you provide me as
evidence? Also, can I give them your name and email for direct
contact? I will be headed to Europe soon and won’t be able to respond
quickly.

-austin

···

–
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
* austin@halostatue.ca * http://www.halostatue.ca/feed/
* austin@zieglers.ca

2

They asked for as much information as I can gather on this matter. If
you have had problems trying to get Ruby or an extension compiled or
running on Windows for any reason, but especially because of Microsoft
runtime DLL differences, please provide me as much information as
possible so that I can pass it on to the VS team at Microsoft.

I'm not sure if this is the sort of information you're looking
for, but I've run into the occasional extension, and/or library
upon which an extension relies, that essentially requires mingw/msys to build.

I got RMagick (ruby ImageMagick/GraphicsMagick) to build using
nmake/cl.exe, but it was a total hack. Building the extension
requires ./configure. The primary function of ./configure for
the RMagick build, is to analyze the GraphicsMagick/ImageMagick
header files and produce a config header file wherein about
200 configuration #defines are set or cleared, adapting RMagick
to whatever particular build of ImageMagick/GraphicsMagick it's
working with.

(What I did was install the particular version of
GraphicsMagick I wanted, on my linux system, run the RMagick
./configure over there, then bring the generated
rmagick_config.h file back to my windows system. I also hacked
up RMagick's extconf.rb in order to get it to work with
nmake/cl.exe.)

Some other extensions I suspect may require mingw/msys to build
include rcairo/cairo, DataFlow/PureData, ruby-gstreamer....

The more I run into this, the happier I would be to just have
ALL my ruby extensions (and ruby itself) built with mingw/msys.

Regards,

Bill

···

From: "Austin Ziegler" <halostatue@gmail.com>

3

Austin Ziegler wrote:

They asked for as much information as I can gather on this matter. If
you have had problems trying to get Ruby or an extension compiled or
running on Windows for any reason, but especially because of Microsoft
runtime DLL differences, please provide me as much information as
possible so that I can pass it on to the VS team at Microsoft.

The only build issues I've had relate to nmake. It doesn't seem to be entirely compatible with make. For example, the last time I tried to compile bdb or pl-ruby, it choked on the Makefile.

As far as running some of my extensions, sometimes I get random segfaults if I don't build with VC++ 6. The most notorious one is win32-open3.

Note that Heesob had an interesting post that dealt with this issue, but now I can't find the dang thing.

Regards,

Dan

4

Not specifically. However, part of what I really want to try to get
the MS team to help us out with is a situation wereh I have
ImageMagick built with MSYS, RMagick built with Visual Studio 2005 (cl
14) and Ruby built with VC++6 -- and it all Just Works.

Right now, apparently, this is a nightmare and there's "no way in
hell" that it would work. They're interested in helping us; I'm not
about to turn down the help of the compiler writers. Which is why I
need more information.

-austin

···

On 6/26/06, Bill Kelly <billk@cts.com> wrote:

I'm not sure if this is the sort of information you're looking
for, but I've run into the occasional extension, and/or library
upon which an extension relies, that essentially requires
mingw/msys to build.

--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * You are in a maze of twisty little passages, all alike. // halo • statue
               * austin@zieglers.ca

5

Austin Ziegler wrote:
> They asked for as much information as I can gather on this matter. If
> you have had problems trying to get Ruby or an extension compiled or
> running on Windows for any reason, but especially because of Microsoft
> runtime DLL differences, please provide me as much information as
> possible so that I can pass it on to the VS team at Microsoft.
The only build issues I've had relate to nmake. It doesn't seem to be
entirely compatible with make. For example, the last time I tried to
compile bdb or pl-ruby, it choked on the Makefile.

This may actually be a problem with the Makefile for bdb or pl-ruby,
because while nmake is a *variant* of make, most "Unix" makefiles are
actually GNUMake files.

As far as running some of my extensions, sometimes I get random
segfaults if I don't build with VC++ 6. The most notorious one is
win32-open3.

If you're able to collect any information about that, I'd appreciate it.

Note that Heesob had an interesting post that dealt with this issue, but
now I can't find the dang thing.

That would be useful if it can be found.

-austin

···

On 6/27/06, Daniel Berger <djberg96@gmail.com> wrote:
--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * You are in a maze of twisty little passages, all alike. // halo • statue
               * austin@zieglers.ca

6

From: Daniel Berger <djberg96@gmail.com>
Reply-To: ruby-talk@ruby-lang.org
To: ruby-talk@ruby-lang.org (ruby-talk ML)
Subject: Re: Win32 Extension Issues Wanted!
Date: Tue, 27 Jun 2006 21:41:08 +0900

Austin Ziegler wrote:

They asked for as much information as I can gather on this matter. If
you have had problems trying to get Ruby or an extension compiled or
running on Windows for any reason, but especially because of Microsoft
runtime DLL differences, please provide me as much information as
possible so that I can pass it on to the VS team at Microsoft.

The only build issues I've had relate to nmake. It doesn't seem to be entirely compatible with make. For example, the last time I tried to compile bdb or pl-ruby, it choked on the Makefile.

As far as running some of my extensions, sometimes I get random segfaults if I don't build with VC++ 6. The most notorious one is win32-open3.

Note that Heesob had an interesting post that dealt with this issue, but now I can't find the dang thing.

Do you want to find http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/190272 ?

Regards,

Park Heesob

7

imho microsoft would be __extremely__ hard pressed to deliver here: the issue
isn't so much with the compiler itself, it's that any ruby built with ms
compilers is going to produce a badly crippled ruby. specifically i mean that
a ruby that cannot

   ruby extconf && make && make install

is broken. the point here is that it's not so much the compilers themselves
which are the issue, but the compiler toolchain, including a minimal set of
tools to bootstrap a configuration from. it's the lack of tools like make,
yacc, sh, etc that make it impossible to do

   cd gsl-1.8/
   ./configure --prefix=/usr/local && make && sudo make install

   export LD_RUN_PATH=/usr/local/lib/
   export LD_LIBRARY_PATH=/usr/local/lib/

   cd ruby-gsl-1.8/
   ruby extconf.rb && make && sudo make install

eg. a 'usable' ruby compiler should also compile third party libs that the
extensions themselves are for. image magick is a perfect example of this.

now, i'm well aware that mingw/msys is a bit of a mutt - but i personally
think that simply compiling ruby is only half the issue. that ruby must be
able to compile and link further extensions and the libs then depend on to be
a 'real' ruby. so, unless the ms guys are also willing to help us put
together a __free__ compiler toolchain that will work with an extconf.rb (or
simlar) method of bootstrapping extensions __and__ a means to compile the libs
they bootstrap against it seems little will be gained.

hmmm. re-reading your post i see that you're advocating a situation in which
the compiler output itself is compatible and one might use a combination of
mingw/msys and vc++ tools. that would indeed be a step forward - so are you
advocating producing a ruby with vc++ compilers, or are you advocating simply
having the various compilers out there produce binary abi compatible output?

in any case, thanks for thinking of this - it's definitely needed.

cheers.

-a

···

On Tue, 27 Jun 2006, Austin Ziegler wrote:

On 6/26/06, Bill Kelly <billk@cts.com> wrote:

I'm not sure if this is the sort of information you're looking
for, but I've run into the occasional extension, and/or library
upon which an extension relies, that essentially requires
mingw/msys to build.

Not specifically. However, part of what I really want to try to get
the MS team to help us out with is a situation wereh I have
ImageMagick built with MSYS, RMagick built with Visual Studio 2005 (cl
14) and Ruby built with VC++6 -- and it all Just Works.

Right now, apparently, this is a nightmare and there's "no way in
hell" that it would work. They're interested in helping us; I'm not
about to turn down the help of the compiler writers. Which is why I
need more information.

--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

8

Austin Ziegler wrote:

I'm not sure if this is the sort of information you're looking
for, but I've run into the occasional extension, and/or library
upon which an extension relies, that essentially requires
mingw/msys to build.

Not specifically. However, part of what I really want to try to get
the MS team to help us out with is a situation wereh I have
ImageMagick built with MSYS, RMagick built with Visual Studio 2005 (cl
14) and Ruby built with VC++6 -- and it all Just Works.

I'm not sure this is possible. This has come up quite often in the Perl community. The problem is that the C library is responsible for allocating and deallocating certain resources that cannot be shared across different run-times (memory, I/O buffers, thread synchronization objects for multi-threaded run-times, etc.) It can be worked around with proper interfaces. For instance it wouldn't matter what ImageMagick is compiled with because it provides a completely encapsulated interface; it handles its own resources. I'm not familiar with Ruby extensions, but unless Ruby provides the same encapsulation of resource management it is possible that corruptions will occur. E.g. Say you allocate memory from one run-time which gives you a handle that is valid in that run-time; then you try to free it in the other run-time, but in the other run-time it points to some random area of memory, discontinuous with the first. Boom! or worse.

I also seem to recall that there is some problem with redistribution: msvcrt is available everywhere, but the newer run-times can't be distributed or something like that. I'm not sure of the particulars of that issue.

I suggest contacting the folks at ActiveState.com. They have long dealt with this issue with their distributions of Perl, Python, and Tcl/Tk. They still use VC6 because of these issues.

Randy.

···

On 6/26/06, Bill Kelly <billk@cts.com> wrote:

9

I understand this. However, I spent 45 minutes on the phone with the
Visual Studio team yesterday, and they really seem interested in
trying to make things much easier for people to work with. Aside from
the ./configure && make && make install issues that Ara Howard pointed
out, the *most* likely setup with this would probably be Ruby and all
extensions compiled with VS 2005 using base libraries that may have
been compiled with any other compiler and runtime.

Remember: I'm collecting feedback for the VS team, not me directly.
I'm aware of a number of the problems, but am looking for
*documentation* so that we can try to have solutions made.

-austin

···

On 6/27/06, Randy W. Sims <ml-ruby@thepierianspring.org> wrote:

I'm not sure this is possible. This has come up quite often in the Perl
community. The problem is that the C library is responsible for
allocating and deallocating certain resources that cannot be shared
across different run-times (memory, I/O buffers, thread synchronization
objects for multi-threaded run-times, etc.)

--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * You are in a maze of twisty little passages, all alike. // halo • statue
               * austin@zieglers.ca