[ANN] Bootstrapping Ruby with MinGW: selfhosted and test sandbox!

I couldn't agree more. Thanks for all your efforts Luis!!

--andy

<The fact is, actually getting a performant stable interpreter on the
platform would probably do more for the language than any other single
action. Of course OCI download numbers from rubyforge can easily
attest to this.>

Hey Nobu,

Hi,

At Tue, 25 Mar 2008 06:04:52 +0900,
Luis Lavena wrote in [ruby-talk:295564]:

> > Note that binary gems don't work only because they install their
> > binary aspect into the wrong directory (and, if you have the newer
> > versions of rubygems, auto-install a version which doesn't exist so
> > you'd need to specify you want the win32 gem). Once you install it
> > you can easily copy the .so's to the right directory and they will
> > 'probably' work just fine.

> Not that, RubyGems consider each platform by it's own. I snot the same
> darwin8 to cygwin, so mingw32 shouldn't be considered a compatible
> match to mswin32.

Darwin and cygwin are completely different systems, but mingw32
and mswin32 are for the same system.

Not only mingw, gcc is basically intended to be binary
interface compatible with the system native development
environments. So cygwin is the special case in this sense.

I just tried to be dramatic, also exposing that RubyGems will not
consider mingw a compatible platform to mswin32, since it's using
RbConfig::CONFIG['arch'] to determine it's platform.

I'll like to point htat rubygems 'darwin' is compatible with 'darwin7'
and 'darwin8', but version 7 is not compatible with 8, like happens
with buils with VC6 and VC8/9.

> In a overlooked way you could be correct, but that is too simplistic
> way to look things. When you start seeing errors due exception thrown
> inside shared object we can find several and big issues.

> Consider that the code generated by VC6 is not the same MinGW
> generates. It even differs in several places on how it do things.

Binary interface compatible means that they use same calling
convension, same utility functions, same runtime DLL etc, but
not generated code are identical.

You're are correct, but real life prove me wrong.

Even that MinGW generates msvcrt-ruby18.dll library, I cannot put a
pre-compiled extension build with VC6 to run over the MinGW build, it
just crash.

(Tried several times during the testing procedure).

Everything depends on the complexity of things the extensions perform,
but instead of guessing and "hoping" not to crash, I put my statement
on bold, underlined text: extensions are not easily interchangeable
between these two build systems.

To solve this I'll require a lot of deep tracing inside ruby, and I
still want to fix the redirected IO issues I had with Readline and
still noone replied.

--
Nobu Nakada

Thank you for your reply.

Regards,

···

On 25 mar, 03:41, Nobuyoshi Nakada <n...@ruby-lang.org> wrote:

> On Mar 24, 5:54 pm, rogerdpack <rogerpack2...@gmail.com> wrote:

--
Luis Lavena

The CPU usage problem is related to constant pooling. This has been
solved in Ruby SVN repository, and is included in the build I created.

Anyway, the pure readline (InLine) is not a direct match, but the one
you mention could be.

I'll take a look to this during the weekend.

Regards,

···

On Mar 28, 10:52 am, Michal Suchanek <hramr...@centrum.cz> wrote:

On 28/03/2008, Michal Suchanek <hramr...@centrum.cz> wrote:

> On 28/03/2008, Joel VanderWerf <vj...@path.berkeley.edu> wrote:

> > This is all working very smoothly for me, following the recipe.

> > I ended up with a ruby/mingw32 installation which my program can use to
> > dynamically compile its parts that are generated C code, and run 20%-50%
> > faster than using msvc6 and oci. This is also far preferable than
> > requiring users to have msvc6 installed if they make changes that
> > require recompilation of the generated C code.

> > The only slight hiccup was figuring out that I needed to put
> > sandbox/ruby_mingw/bin on my PATH, but that's really pretty obvious.

> > I hope someone is able to fix readline, though...

> I have read something about pure ruby readline on this list. There's
> something on RAA but the linked site is down.

hmm, I found a rreadline-0.5.4.tar.gz on a FreeBSD mirror.
adding the rreadline folder to site_ruby and a writing readline.rb as follows:

require 'rreadline/RReadline'

fixes the 100% CPU usage for me, at least in irb.

--
Luis Lavena

The readline that comes with installer3.zip works fine for me in irb,
it's just the test that hangs. Is that what others are seeing? Maybe
this is a naive question, but could the test be implemented in a
different manner? It seems to me that this is more of an IO
redirection problem than a readline problem.

I certainly don't have any solutions, I'm just trying to understand the problem.

Thanks

Gordon

Hey Nobu,

Hi,

At Tue, 25 Mar 2008 06:04:52 +0900,
Luis Lavena wrote in [ruby-talk:295564]:

Note that binary gems don't work only because they install their
binary aspect into the wrong directory (and, if you have the newer
versions of rubygems, auto-install a version which doesn't exist so
you'd need to specify you want the win32 gem). Once you install it
you can easily copy the .so's to the right directory and they will
'probably' work just fine.

Not that, RubyGems consider each platform by it's own. I snot the same
darwin8 to cygwin, so mingw32 shouldn't be considered a compatible
match to mswin32.

Darwin and cygwin are completely different systems, but mingw32
and mswin32 are for the same system.

Not only mingw, gcc is basically intended to be binary
interface compatible with the system native development
environments. So cygwin is the special case in this sense.

I just tried to be dramatic, also exposing that RubyGems will not
consider mingw a compatible platform to mswin32, since it's using
RbConfig::CONFIG['arch'] to determine it's platform.

I'll like to point htat rubygems 'darwin' is compatible with 'darwin7'
and 'darwin8', but version 7 is not compatible with 8, like happens
with buils with VC6 and VC8/9.

In a overlooked way you could be correct, but that is too simplistic
way to look things. When you start seeing errors due exception thrown
inside shared object we can find several and big issues.

Consider that the code generated by VC6 is not the same MinGW
generates. It even differs in several places on how it do things.

Binary interface compatible means that they use same calling
convension, same utility functions, same runtime DLL etc, but
not generated code are identical.

You're are correct, but real life prove me wrong.

Even that MinGW generates msvcrt-ruby18.dll library, I cannot put a
pre-compiled extension build with VC6 to run over the MinGW build, it
just crash.

(Tried several times during the testing procedure).

Everything depends on the complexity of things the extensions perform,
but instead of guessing and "hoping" not to crash, I put my statement
on bold, underlined text: extensions are not easily interchangeable
between these two build systems.

To solve this I'll require a lot of deep tracing inside ruby, and I
still want to fix the redirected IO issues I had with Readline and
still noone replied.

As you know Luis, I'm desperately trying to make time to come and help you, sort out some of the code i can donate, etc. Sadly, time is lacking. Anyway, it seems this kind of project would be ideal for the Ruby Mendicant project. I know there are a lot of the long-term rubyists who simply say "Oh we don't care about Windows" or "It's a broken platform" / whatever. The fact is, actually getting a performant stable interpreter on the platform would probably do more for the language than any other single action. Of course OCI download numbers from rubyforge can easily attest to this.

···

On 25 Mar 2008, at 13:14, Luis Lavena wrote:

On 25 mar, 03:41, Nobuyoshi Nakada <n...@ruby-lang.org> wrote:

On Mar 24, 5:54 pm, rogerdpack <rogerpack2...@gmail.com> wrote:

--
Nobu Nakada

Thank you for your reply.

Regards,
--
Luis Lavena

The readline that comes with installer3.zip works fine for me in irb,
it's just the test that hangs. Is that what others are seeing? Maybe
this is a naive question, but could the test be implemented in a
different manner? It seems to me that this is more of an IO
redirection problem than a readline problem.

Is the IO redirection with readline, nothing more, nothing less.

The issue is that tests cannot run successfully until that issue get
fixed.

SO I cannot add MinGW build of Ruby to our CI...

I certainly don't have any solutions, I'm just trying to understand the problem.

Don't worry, I still don't understand i neihter :stuck_out_tongue:

Regards,

···

On Mar 28, 2:52 pm, Gordon Thiesfeld <gthiesf...@gmail.com> wrote:
--
Luis Lavena

The CPU usage problem is related to constant pooling. This has been
solved in Ruby SVN repository, and is included in the build I created.

downloading r15977 seems to still have the 100% readline problem. Is
there a better version to use?
-R

···

--
Posted via http://www.ruby-forum.com/\.

> Hey Nobu,

>> Hi,

>> At Tue, 25 Mar 2008 06:04:52 +0900,
>> Luis Lavena wrote in [ruby-talk:295564]:

>>>> Note that binary gems don't work only because they install their
>>>> binary aspect into the wrong directory (and, if you have the newer
>>>> versions of rubygems, auto-install a version which doesn't exist so
>>>> you'd need to specify you want the win32 gem). Once you install it
>>>> you can easily copy the .so's to the right directory and they will
>>>> 'probably' work just fine.

>>> Not that, RubyGems consider each platform by it's own. I snot the
>>> same
>>> darwin8 to cygwin, so mingw32 shouldn't be considered a compatible
>>> match to mswin32.

>> Darwin and cygwin are completely different systems, but mingw32
>> and mswin32 are for the same system.

>> Not onlymingw, gcc is basically intended to be binary
>> interface compatible with the system native development
>> environments. So cygwin is the special case in this sense.

> I just tried to be dramatic, also exposing that RubyGems will not
> considermingwa compatible platform to mswin32, since it's using
> RbConfig::CONFIG['arch'] to determine it's platform.

> I'll like to point htat rubygems 'darwin' is compatible with 'darwin7'
> and 'darwin8', but version 7 is not compatible with 8, like happens
> with buils with VC6 and VC8/9.

>>> In a overlooked way you could be correct, but that is too simplistic
>>> way to look things. When you start seeing errors due exception
>>> thrown
>>> inside shared object we can find several and big issues.

>>> Consider that the code generated by VC6 is not the sameMinGW
>>> generates. It even differs in several places on how it do things.

>> Binary interface compatible means that they use same calling
>> convension, same utility functions, same runtime DLL etc, but
>> not generated code are identical.

> You're are correct, but real life prove me wrong.

> Even thatMinGWgenerates msvcrt-ruby18.dll library, I cannot put a
> pre-compiled extension build with VC6 to run over theMinGWbuild, it
> just crash.

> (Tried several times during the testing procedure).

> Everything depends on the complexity of things the extensions perform,
> but instead of guessing and "hoping" not to crash, I put my statement
> on bold, underlined text: extensions are not easily interchangeable
> between these two build systems.

> To solve this I'll require a lot of deep tracing inside ruby, and I
> still want to fix the redirected IO issues I had with Readline and
> still noone replied.

As you know Luis, I'm desperately trying to make time to come and help
you, sort out some of the code i can donate, etc. Sadly, time is
lacking. Anyway, it seems this kind of project would be ideal for the
Ruby Mendicant project.

It's an interesting idea, but my project is not Google's Summer of
Code (which I think this idea is better suited for).

Unfortunately, M$ support a very bad project for me, because I don't
even own a machine with Windows on it, and have moved all my
commercial work off of the platform too. The only tie I have left to
the Microsoft world I slide underneath via ODBC....

For RubyMendicant, since it isn't really the same as the GSoC (ie, I'm
taking off from my commercial work to do this!),
I really need to pick projects that I have expertise in along with a
strong interest so that people's donations are used most effectively.

Windows unfortunately doesn't fit that bill, even if the project would
be well worth it for someone better suited.

I have selected 3 projects from my ideas list, and am mostly limiting
the potential projects to these for now:
http://rubymendicant.wikidot.com/projects

Though I won't ignore suggested project ideas, I'm asking folks to
rally behind one or more of those projects, since I've already seen
some support for them and have a plan in mind... it's starting to get
too close to the start date to consider entirely new ideas.

I know there are a lot of the long-term
rubyists who simply say "Oh we don't care about Windows" or "It's a
broken platform" / whatever. The fact is, actually getting a
performant stable interpreter on the platform would probably do more
for the language than any other single action. Of course OCI download
numbers from rubyforge can easily attest to this.

I agree this is important. Maybe suggest this as a GSoC project idea.

http://rubycentral.com/projects/gsoc-2008/ideas-for-gsoc-2008

-greg

···

On Mar 25, 11:29 am, James Tucker <jftuc...@gmail.com> wrote:

On 25 Mar 2008, at 13:14, Luis Lavena wrote:
> On 25 mar, 03:41, Nobuyoshi Nakada <n...@ruby-lang.org> wrote:
>>> On Mar 24, 5:54 pm, rogerdpack <rogerpack2...@gmail.com> wrote:

> To solve this I'll require a lot of deep tracing inside ruby, and I
> still want to fix the redirected IO issues I had with Readline and
> still noone replied.

As you know Luis, I'm desperately trying to make time to come and help
you, sort out some of the code i can donate, etc. Sadly, time is
lacking.

I know about you James, and I'm not pointing fingers to anyone, but I
heard lot of folks scream that OCI have problems, but they, even have
plenty of C or Ruby knowledge, don't offer their help.

Anyway, it seems this kind of project would be ideal for the
Ruby Mendicant project.

from what I have seen the Mendicant project is aimed to other kind of
project and not GSoC or targeting Windows.

The only thing I'll like to add is: whatever code you write, please
try be the most cross-platform aware you can :smiley:

I know there are a lot of the long-term
rubyists who simply say "Oh we don't care about Windows" or "It's a
broken platform" / whatever. The fact is, actually getting a
performant stable interpreter on the platform would probably do more
for the language than any other single action. Of course OCI download
numbers from rubyforge can easily attest to this.

Yes, it seems that today, noone ever used or started computing running
over a M$ OS... DOS or even Windows.

All them reject their roots :wink:

I'll keep Windows as my main platform, since I'm quite comfortable
with it. To be honest the only language that give me so many troubles
with it is Ruby (and I can provide a long list of languages and tools
I have used since I started with computers late in the 80's).

Regards,

···

On Mar 25, 12:29 pm, James Tucker <jftuc...@gmail.com> wrote:

On 25 Mar 2008, at 13:14, Luis Lavena wrote:

--
Luis Lavena

It's an interesting idea, but my project is not Google's Summer of
Code (which I think this idea is better suited for).

Unfortunately, M$ support a very bad project for me, because I don't
even own a machine with Windows on it, and have moved all my
commercial work off of the platform too. The only tie I have left to
the Microsoft world I slide underneath via ODBC....

For RubyMendicant, since it isn't really the same as the GSoC (ie, I'm
taking off from my commercial work to do this!),
I really need to pick projects that I have expertise in along with a
strong interest so that people's donations are used most effectively.

Windows unfortunately doesn't fit that bill, even if the project would
be well worth it for someone better suited.

I have selected 3 projects from my ideas list, and am mostly limiting
the potential projects to these for now:
Projects - Ruby Mendicant

Absolutely fair enough :slight_smile:

semi-P.S. Good luck with project.

···

On 25 Mar 2008, at 15:54, Gregory Brown wrote:

Though I won't ignore suggested project ideas, I'm asking folks to
rally behind one or more of those projects, since I've already seen
some support for them and have a plan in mind... it's starting to get
too close to the start date to consider entirely new ideas.

I know there are a lot of the long-term
rubyists who simply say "Oh we don't care about Windows" or "It's a
broken platform" / whatever. The fact is, actually getting a
performant stable interpreter on the platform would probably do more
for the language than any other single action. Of course OCI download
numbers from rubyforge can easily attest to this.

I agree this is important. Maybe suggest this as a GSoC project idea.

http://rubycentral.com/projects/gsoc-2008/ideas-for-gsoc-2008

-greg

Luis Lavena wrote:

I'll keep Windows as my main platform, since I'm quite comfortable
with it. To be honest the only language that give me so many troubles
with it is Ruby (and I can provide a long list of languages and tools
I have used since I started with computers late in the 80's).

Long ago in a galaxy far away, there were *major* differences between the ActiveState version of Perl and other versions, even on Windows. In the end, someone sat on ActiveState about that and it got fixed. I think the same thing needs to happen to Ruby 1.8.x -- Ruby 1.8.x should be Ruby 1.8.x regardless of whether it's Cygwin, MSYS, DOS, Windown 95/98/ME, Windows 2000/XP/Vista, etc.

To solve this I'll require a lot of deep tracing inside ruby, and I
still want to fix the redirected IO issues I had with Readline and
still noone replied.

As you know Luis, I'm desperately trying to make time to come and help
you, sort out some of the code i can donate, etc. Sadly, time is
lacking.

I know about you James, and I'm not pointing fingers to anyone, but I
heard lot of folks scream that OCI have problems, but they, even have
plenty of C or Ruby knowledge, don't offer their help.

Absolutely, I understand. Among other things I'm still setting up my build management servers and so on for tracking external projects like OCI / rubygems / other dependencies.

Having places to go and experiment helps a lot, and also learning to deal with niggly problems, like sorting out relative vs. full path issues and so on (errors in both ruby's makefiles for openssl, and in rubygems test suite, for one example).

I think you've been through some of these fixes already, and I know Eric was looking for you last night regarding some newer stuff with rubygems, but keeping up with everything, whilst running a startup, is kinda crazy. Basically, once I have my build setups stable (if I ever get enough time to do that...), I'll be publishing them so that we can all start working off a similar base, and problems should get more extensively replicable.

Anyway, it seems this kind of project would be ideal for the
Ruby Mendicant project.

from what I have seen the Mendicant project is aimed to other kind of
project and not GSoC or targeting Windows.

The only thing I'll like to add is: whatever code you write, please
try be the most cross-platform aware you can :smiley:

I think the only major 'ruby platform' I don't test on is Cygwin. Whether or not to count that as a *nix or a windows style OS is extremely foggy, as it depends what you're doing (paths vs A[PB]I, etc etc).

I know there are a lot of the long-term
rubyists who simply say "Oh we don't care about Windows" or "It's a
broken platform" / whatever. The fact is, actually getting a
performant stable interpreter on the platform would probably do more
for the language than any other single action. Of course OCI download
numbers from rubyforge can easily attest to this.

Yes, it seems that today, noone ever used or started computing running
over a M$ OS... DOS or even Windows.

All them reject their roots :wink:

I still love my Amiga, it's just a 68030 doesn't cut it these days :wink:

I'll keep Windows as my main platform, since I'm quite comfortable
with it. To be honest the only language that give me so many troubles
with it is Ruby (and I can provide a long list of languages and tools
I have used since I started with computers late in the 80's).

I 'grew up' consulting in the windows world, although I'll relatively happily use most things. I'm still hopeful that one day I'll find an OS I actually like.

I gotta say, the core interpreter compiles pretty easily on most platforms. The ext/ dir and dependencies however...

···

On 25 Mar 2008, at 17:09, Luis Lavena wrote:

On Mar 25, 12:29 pm, James Tucker <jftuc...@gmail.com> wrote:

On 25 Mar 2008, at 13:14, Luis Lavena wrote:

Regards,
--
Luis Lavena

>

> > To solve this I'll require a lot of deep tracing inside ruby, and I
> > still want to fix the redirected IO issues I had with Readline and
> > still noone replied.
>
> As you know Luis, I'm desperately trying to make time to come and help
> you, sort out some of the code i can donate, etc. Sadly, time is
> lacking.

I know about you James, and I'm not pointing fingers to anyone, but I
heard lot of folks scream that OCI have problems, but they, even have
plenty of C or Ruby knowledge, don't offer their help.

For me either OCI or the new installer is good enough. I know there
are problems with the way OCI is built but they did not affect me so
far :stuck_out_tongue:

I see that you have put lots of time into researching how things are
done on Windows, many thanks for that. I do not have much experience
with Windows programming but I hope that now applying fixes and
enhancements to Ruby on Windows will be as easy and streamlined as on
other platforms.

As far as I know for the few scripts I use on Windows I could replace
the OCI with the mingw built ruby.

> I know there are a lot of the long-term
> rubyists who simply say "Oh we don't care about Windows" or "It's a
> broken platform" / whatever. The fact is, actually getting a
> performant stable interpreter on the platform would probably do more
> for the language than any other single action. Of course OCI download
> numbers from rubyforge can easily attest to this.

Yes, it seems that today, noone ever used or started computing running
over a M$ OS... DOS or even Windows.

All them reject their roots :wink:

Actually I started with the ZX Spectrum BASIC IDE(if you may call it
one). Of course, DOS was better than that, and Windows was eventually
better than DOS but before that I found GNU/Linux is even better in
not getting into my way. When I went back to Windows I found that they
are quite hard to work with.
This is changing somewhat with the possibility to use alternate GUI
shell, MSYS, and whatnot but it's still a hassle to set up, and stuff
breaks if you do not use the default settings for everything. I do not
reject Windows, and I still use dosemu but once better alternative
than Linux and BSD appears I will leave them behind as well.

Hate all the OSes equally for their inflexibility and poor design :wink:

I'll keep Windows as my main platform, since I'm quite comfortable
with it. To be honest the only language that give me so many troubles
with it is Ruby (and I can provide a long list of languages and tools
I have used since I started with computers late in the 80's).

Yes, each pick their poison :stuck_out_tongue:

Thanks

Michal

···

On 25/03/2008, Luis Lavena <luislavena@gmail.com> wrote:

On Mar 25, 12:29 pm, James Tucker <jftuc...@gmail.com> wrote:
> On 25 Mar 2008, at 13:14, Luis Lavena wrote: