Ruby compile failing at make ("LIBRUBY_EXTS: command not found")

Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me near what
appears to be the end:

    making ruby
    mkmain.sh: line 2: LIBRUBY_EXTS: command not found
    make: invalid option -- '.'
    make: invalid option -- 'y'
    make: invalid option -- 'a'
    make: invalid option -- 'c'
    Usage: make [options] [target] ...
    Options:

I'm having problems tracking down where this is getting called or why.

jonathan

Jonathan Gold wrote in post #956225:

Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me

Probably not unless you say something about the environment you're
building under: operating system, C compiler version etc.

near what
appears to be the end:

    making ruby
    mkmain.sh: line 2: LIBRUBY_EXTS: command not found

What shell does your OS have? It's possible that a bash-specific syntax
is being used here, and you're using a POSIX shell, like ksh or dash.
That's the case in Ubuntu:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2010-06-09 16:27 /bin/sh -> dash

Having said that, I have the source to ruby-1.9.1-p429 unpacked on my
machine, where it compiled successfully, and I can't find mkmain.sh
anywhere. Can you post the first two lines of that file?

···

--
Posted via http://www.ruby-forum.com/\.

Jonathan Gold wrote in post #956225:
> Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me

Probably not unless you say something about the environment you're
building under: operating system, C compiler version etc.

Reasonable point. It's an Amazon Linux image which is by all accounts really
Centos (poking around on the box confirms).

> near what
> appears to be the end:
>
> making ruby
> mkmain.sh: line 2: LIBRUBY_EXTS: command not found

What shell does your OS have? It's possible that a bash-specific syntax
is being used here, and you're using a POSIX shell, like ksh or dash.
That's the case in Ubuntu:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2010-06-09 16:27 /bin/sh -> dash

I'm using bash.

Having said that, I have the source to ruby-1.9.1-p429 unpacked on my
machine, where it compiled successfully, and I can't find mkmain.sh
anywhere. Can you post the first two lines of that file?

Nor can I. I actually did a 'find / -name "mkmain.sh"' as well as searched the
centos pkg repos for the file and nothing comes up. My colleague suggested that
perhaps mkmain.sh is actually generated by the make process and cleaned up after
the error, leaving me no chance to see it. I grepped around in the ruby build
dirs for mkmain.sh and only see these references:

    $ find . -type f | xargs grep 'mkmain'./Makefile:MKMAIN_CMD = mkmain.sh
    ./Makefile.in:MKMAIN_CMD = mkmain.sh
    ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat

and then searching for the MKMAIN_CMD string:

    $ find . -type f | xargs grep 'MKMAIN_CMD'
    ./Makefile:MKMAIN_CMD = mkmain.sh
    ./Makefile.in:MKMAIN_CMD = mkmain.sh
    ./ChangeLog: * Makefile.in, win32/Makefile.sub (RUNCMD, MKMAIN_CMD):
    macros to
    ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
    ./uncommon.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
    ./uncommon.mk:exts: $(MKMAIN_CMD)
    ./uncommon.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
    ./common.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
    ./common.mk:exts: $(MKMAIN_CMD)
    ./common.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)

but nothing that seems to generate said file that I can tell.

···

On Fri, Oct 22, 2010 at 05:25:28PM +0900, Brian Candler wrote:

--
Posted via http://www.ruby-forum.com/\.

Brian -- I forgot to also give you some other details:

    $ make -v
    GNU Make 3.81
    Copyright (C) 2006 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.
    There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
    PARTICULAR PURPOSE.

    This program built for x86_64-redhat-linux-gnu

and

    $ gcc -v
    Using built-in specs.
    Target: x86_64-amazon-linux
    Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
    --infodir=/usr/share/info --enable-shared --enable-threads=posix
    --enable-checking=release --with-system-zlib --enable-__cxa_atexit
    --disable-libunwind-exceptions --enable-libgcj-multifile
    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --disable-dssi
    --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic
    --host=x86_64-amazon-linux
    Thread model: posix
    gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)

Not sure whether that will matter too much, but there it is.

jonathan

···

On Sat, Oct 23, 2010 at 02:28:42AM +0900, Jonathan Gold wrote:

On Fri, Oct 22, 2010 at 05:25:28PM +0900, Brian Candler wrote:
> Jonathan Gold wrote in post #956225:
> > Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me
>
> Probably not unless you say something about the environment you're
> building under: operating system, C compiler version etc.

Reasonable point. It's an Amazon Linux image which is by all accounts really
Centos (poking around on the box confirms).

>
> > near what
> > appears to be the end:
> >
> > making ruby
> > mkmain.sh: line 2: LIBRUBY_EXTS: command not found
>
> What shell does your OS have? It's possible that a bash-specific syntax
> is being used here, and you're using a POSIX shell, like ksh or dash.
> That's the case in Ubuntu:
>
> $ ls -l /bin/sh
> lrwxrwxrwx 1 root root 4 2010-06-09 16:27 /bin/sh -> dash

I'm using bash.

>
> Having said that, I have the source to ruby-1.9.1-p429 unpacked on my
> machine, where it compiled successfully, and I can't find mkmain.sh
> anywhere. Can you post the first two lines of that file?

Nor can I. I actually did a 'find / -name "mkmain.sh"' as well as searched the
centos pkg repos for the file and nothing comes up. My colleague suggested that
perhaps mkmain.sh is actually generated by the make process and cleaned up after
the error, leaving me no chance to see it. I grepped around in the ruby build
dirs for mkmain.sh and only see these references:

    $ find . -type f | xargs grep 'mkmain'./Makefile:MKMAIN_CMD = mkmain.sh
    ./Makefile.in:MKMAIN_CMD = mkmain.sh
    ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat

and then searching for the MKMAIN_CMD string:

    $ find . -type f | xargs grep 'MKMAIN_CMD'
    ./Makefile:MKMAIN_CMD = mkmain.sh
    ./Makefile.in:MKMAIN_CMD = mkmain.sh
    ./ChangeLog: * Makefile.in, win32/Makefile.sub (RUNCMD, MKMAIN_CMD):
    macros to
    ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
    ./uncommon.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
    ./uncommon.mk:exts: $(MKMAIN_CMD)
    ./uncommon.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
    ./common.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
    ./common.mk:exts: $(MKMAIN_CMD)
    ./common.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)

but nothing that seems to generate said file that I can tell.

>
> --
> Posted via http://www.ruby-forum.com/\.

Brian and others -- I made some progress and think the problem is in
ext/extmk.rb, but won't be able to dive deeper until next week. Will update this
thread when I do, but wanted to thank you for your help so far.

jonathan

···

On Sat, Oct 23, 2010 at 03:01:55AM +0900, Jonathan Gold wrote:

Brian -- I forgot to also give you some other details:

    $ make -v
    GNU Make 3.81
    Copyright (C) 2006 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.
    There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
    PARTICULAR PURPOSE.

    This program built for x86_64-redhat-linux-gnu

and

    $ gcc -v
    Using built-in specs.
    Target: x86_64-amazon-linux
    Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
    --infodir=/usr/share/info --enable-shared --enable-threads=posix
    --enable-checking=release --with-system-zlib --enable-__cxa_atexit
    --disable-libunwind-exceptions --enable-libgcj-multifile
    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --disable-dssi
    --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic
    --host=x86_64-amazon-linux
    Thread model: posix
    gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)

Not sure whether that will matter too much, but there it is.

jonathan

On Sat, Oct 23, 2010 at 02:28:42AM +0900, Jonathan Gold wrote:
> On Fri, Oct 22, 2010 at 05:25:28PM +0900, Brian Candler wrote:
> > Jonathan Gold wrote in post #956225:
> > > Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me
> >
> > Probably not unless you say something about the environment you're
> > building under: operating system, C compiler version etc.
>
> Reasonable point. It's an Amazon Linux image which is by all accounts really
> Centos (poking around on the box confirms).
>
> >
> > > near what
> > > appears to be the end:
> > >
> > > making ruby
> > > mkmain.sh: line 2: LIBRUBY_EXTS: command not found
> >
> > What shell does your OS have? It's possible that a bash-specific syntax
> > is being used here, and you're using a POSIX shell, like ksh or dash.
> > That's the case in Ubuntu:
> >
> > $ ls -l /bin/sh
> > lrwxrwxrwx 1 root root 4 2010-06-09 16:27 /bin/sh -> dash
>
> I'm using bash.
>
> >
> > Having said that, I have the source to ruby-1.9.1-p429 unpacked on my
> > machine, where it compiled successfully, and I can't find mkmain.sh
> > anywhere. Can you post the first two lines of that file?
>
> Nor can I. I actually did a 'find / -name "mkmain.sh"' as well as searched the
> centos pkg repos for the file and nothing comes up. My colleague suggested that
> perhaps mkmain.sh is actually generated by the make process and cleaned up after
> the error, leaving me no chance to see it. I grepped around in the ruby build
> dirs for mkmain.sh and only see these references:
>
> $ find . -type f | xargs grep 'mkmain'./Makefile:MKMAIN_CMD = mkmain.sh
> ./Makefile.in:MKMAIN_CMD = mkmain.sh
> ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
>
> and then searching for the MKMAIN_CMD string:
>
> $ find . -type f | xargs grep 'MKMAIN_CMD'
> ./Makefile:MKMAIN_CMD = mkmain.sh
> ./Makefile.in:MKMAIN_CMD = mkmain.sh
> ./ChangeLog: * Makefile.in, win32/Makefile.sub (RUNCMD, MKMAIN_CMD):
> macros to
> ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
> ./uncommon.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
> ./uncommon.mk:exts: $(MKMAIN_CMD)
> ./uncommon.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
> ./common.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
> ./common.mk:exts: $(MKMAIN_CMD)
> ./common.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
>
> but nothing that seems to generate said file that I can tell.
>
> >
> > --
> > Posted via http://www.ruby-forum.com/\.

It does indeed look like mkmain.sh is generated by ext/extmk.rb (I found that
out just by hand-creating a mkmain.sh that wasn't writeable, forcing a
stacktrace from whichever part of the build attempted to generate it, and that
turned out to be ext/extmk.rb).

The generated mkmain.sh is set to delete itself before executing (see below),
but I added a hook inside extmk.rb to let me get a hold of it first. Here is
what is getting generated as mkmain.sh:

    rm -f $0; exec make LIBRUBY_SO_UPDATE=$(LIBRUBY_EXTS) SETUP=ext/Setup DLDOBJS=ext/extinit.o EXTLIBS=-lncursesw -ltinfo -lcrypto -ldl EXTLDFLAGS=-L/home/jgold/work/opt/openssl-0.9.8g/x86_64-linux-gnu/lib -Wl,-R/home/jgold/work/opt/openssl-0.9.8g/x86_64-linux-gnu/lib -L. -rdynamic -Wl,-export-dynamic ruby

The thing that's causing problems is that the args to the various env vars
aren't being quoted for the shell. I handquoted them and the script works, so I
took a look inside ext/extmk.rb and saw that the method 'sysquote' (line 42) is
not evaluating @quote to true:

    def sysquote(x)
      @quote ||= /os2/ =~ (CROSS_COMPILING || RUBY_PLATFORM)
      @quote ? x.quote : x
    end

If I manually change the file to just hard-set '@quote = true' then the make
completes normally. That being said, the built ruby fails to load external libs,
but before diving too deeply into those problems I figured I'd stop and wait to
see what people think about the mkmain.sh quoting stuff. I suspect that the
problem underlying the former is also caused by the one underlying the former.

Anyone more familiar with the ruby build have an idea as to where RUBY_PLATFORM
gets set or in general as to why the generated script is not being quoted?

I am able to successfully build and use ruby-1.9.1-p0 and ruby-1.9.1-p376.

jonathan

···

On Sat, Oct 23, 2010 at 07:25:20AM +0900, Jonathan Gold wrote:

Brian and others -- I made some progress and think the problem is in
ext/extmk.rb, but won't be able to dive deeper until next week. Will update this
thread when I do, but wanted to thank you for your help so far.

jonathan

On Sat, Oct 23, 2010 at 03:01:55AM +0900, Jonathan Gold wrote:
> Brian -- I forgot to also give you some other details:
>
> $ make -v
> GNU Make 3.81
> Copyright (C) 2006 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.
> There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
> PARTICULAR PURPOSE.
>
> This program built for x86_64-redhat-linux-gnu
>
> and
>
> $ gcc -v
> Using built-in specs.
> Target: x86_64-amazon-linux
> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
> --infodir=/usr/share/info --enable-shared --enable-threads=posix
> --enable-checking=release --with-system-zlib --enable-__cxa_atexit
> --disable-libunwind-exceptions --enable-libgcj-multifile
> --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --disable-dssi
> --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic
> --host=x86_64-amazon-linux
> Thread model: posix
> gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)
>
> Not sure whether that will matter too much, but there it is.
>
> jonathan
>
> On Sat, Oct 23, 2010 at 02:28:42AM +0900, Jonathan Gold wrote:
> > On Fri, Oct 22, 2010 at 05:25:28PM +0900, Brian Candler wrote:
> > > Jonathan Gold wrote in post #956225:
> > > > Anyone have any ideas why ruby compile (1.9.1-p430) is failing for me
> > >
> > > Probably not unless you say something about the environment you're
> > > building under: operating system, C compiler version etc.
> >
> > Reasonable point. It's an Amazon Linux image which is by all accounts really
> > Centos (poking around on the box confirms).
> >
> > >
> > > > near what
> > > > appears to be the end:
> > > >
> > > > making ruby
> > > > mkmain.sh: line 2: LIBRUBY_EXTS: command not found
> > >
> > > What shell does your OS have? It's possible that a bash-specific syntax
> > > is being used here, and you're using a POSIX shell, like ksh or dash.
> > > That's the case in Ubuntu:
> > >
> > > $ ls -l /bin/sh
> > > lrwxrwxrwx 1 root root 4 2010-06-09 16:27 /bin/sh -> dash
> >
> > I'm using bash.
> >
> > >
> > > Having said that, I have the source to ruby-1.9.1-p429 unpacked on my
> > > machine, where it compiled successfully, and I can't find mkmain.sh
> > > anywhere. Can you post the first two lines of that file?
> >
> > Nor can I. I actually did a 'find / -name "mkmain.sh"' as well as searched the
> > centos pkg repos for the file and nothing comes up. My colleague suggested that
> > perhaps mkmain.sh is actually generated by the make process and cleaned up after
> > the error, leaving me no chance to see it. I grepped around in the ruby build
> > dirs for mkmain.sh and only see these references:
> >
> > $ find . -type f | xargs grep 'mkmain'./Makefile:MKMAIN_CMD = mkmain.sh
> > ./Makefile.in:MKMAIN_CMD = mkmain.sh
> > ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
> >
> > and then searching for the MKMAIN_CMD string:
> >
> > $ find . -type f | xargs grep 'MKMAIN_CMD'
> > ./Makefile:MKMAIN_CMD = mkmain.sh
> > ./Makefile.in:MKMAIN_CMD = mkmain.sh
> > ./ChangeLog: * Makefile.in, win32/Makefile.sub (RUNCMD, MKMAIN_CMD):
> > macros to
> > ./win32/Makefile.sub:MKMAIN_CMD = mkmain.bat
> > ./uncommon.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
> > ./uncommon.mk:exts: $(MKMAIN_CMD)
> > ./uncommon.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
> > ./common.mk: @$(RUNCMD) $(MKMAIN_CMD) MAKE=$(MAKE)
> > ./common.mk:exts: $(MKMAIN_CMD)
> > ./common.mk:$(MKMAIN_CMD): $(MKFILES) incs $(PREP) $(RBCONFIG) $(LIBRUBY)
> >
> > but nothing that seems to generate said file that I can tell.
> >
> > >
> > > --
> > > Posted via http://www.ruby-forum.com/\.