Cross-compiling ruby

I want to build ruby in a cross-compile mode in which the results of a compilation are installed in some arbitrary directory, then are placed in an archive, then are unpacked from the archive in their final location on a large number of machines. The --prefix option of the configure script can be used to change the final destination, but it is overloaded to be both the compile-time destination and the runtime search path. In my case, these are distinct. Furthermore, the build must run without root permission (and I would not want to corrupt an existing installation anyway), the final location requires root permission for writing, and the result of "make install" in my sandbox will not be available on most of the machines on which the archive is installed.

To illustrate the problem, I run a build as normal user in /sandbox/ruby/src, and install the results in the /sandbox/ruby/install tree where the bin, lib, share, and other directories are created. I run this procedure:

mkdir -p /sandbox/ruby/install
cd /sandbox/ruby/src
./configure --prefix=/sandbox/ruby/install
make
make install
cd /sandbox/ruby/install
find . -print | cpio -o > /sandbox/ruby.cpio

Now I install the archive on the target machine:

scp /sandbox/ruby.cpio remote:~/ruby.cpio
ssh remote
su
mkdir -p /usr/foo
( cd /usr/foo && cpio -id ) < ruby.cpio

This is a simplified illustration of the method; the actual procedure is more complicated than this. The src tree of the sandbox is actually read-only so the build results are actually deposited elsewhere. Additionally, ruby is only one of several products built in this sandbox, and the whole thing is wrapped in a complicated make-based procedure. There may even come a time when I must build ruby with an actual cross-compiler targeting a different architecture than the build machine, for an embedded system that has no ability to develop software. But the above presents the relevant aspects of the method.

The issue is that programs like gem have hard-coded into them the location into which the "make install" step deposited them. If you run "gem environment", you see that it expects the location of the ruby executable to be /sandbox/ruby/install/bin/ruby, and the search paths are similarly set. Note that the sandbox is not available on the remote machine.

Granting permission to install into /usr/foo on the build machine is not an option because it's a production environment and we can't even temporarily corrupt that location while other procedures are running. And we don't want to copy the world into a chroot environment for the purpose of building this sandbox. (That would not work anyway because ruby isn't the only thing that gets built in that sandbox, and other things require an intact /usr/foo tree.)

So, how do others build and operate ruby in an environment like this?

P.S. Having a tool invoke a prior version of itself to build itself, as is done with ruby 1.9.2, violates a fundamental tenet of release engineering: You shall not unwind history to reproduce yourself. I see from the archives of this list that others have also reported this issue. When will this oversight be fixed?

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

Thanks to an offline pointer, this is the revised build procedure that builds Ruby in a cross-compile mode:

mkdir -p /sandbox/ruby/install
cd /sandbox/ruby/src
./configure --prefix=/usr/foo
make
make install DESTDIR=/sandbox/ruby/install
cd /sandbox/ruby/install/usr/foo
find . -print | cpio -o > /sandbox/ruby.cpio

The procedure to install the contents of the resulting archive remain the same.

ยทยทยท

-----Original Message-----
From: Paul Sander [mailto:psander@tivo.com]
Sent: Thursday, February 17, 2011 9:19 PM
To: ruby-talk ML
Subject: Cross-compiling ruby

I want to build ruby in a cross-compile mode in which the results of a compilation are installed in some arbitrary directory, then are placed in an archive, then are unpacked from the archive in their final location on a large number of machines. The --prefix option of the configure script can be used to change the final destination, but it is overloaded to be both the compile-time destination and the runtime search path. In my case, these are distinct. Furthermore, the build must run without root permission (and I would not want to corrupt an existing installation anyway), the final location requires root permission for writing, and the result of "make install" in my sandbox will not be available on most of the machines on which the archive is installed.

To illustrate the problem, I run a build as normal user in /sandbox/ruby/src, and install the results in the /sandbox/ruby/install tree where the bin, lib, share, and other directories are created. I run this procedure:

mkdir -p /sandbox/ruby/install
cd /sandbox/ruby/src
./configure --prefix=/sandbox/ruby/install
make
make install
cd /sandbox/ruby/install
find . -print | cpio -o > /sandbox/ruby.cpio

Now I install the archive on the target machine:

scp /sandbox/ruby.cpio remote:~/ruby.cpio
ssh remote
su
mkdir -p /usr/foo
( cd /usr/foo && cpio -id ) < ruby.cpio

This is a simplified illustration of the method; the actual procedure is more complicated than this. The src tree of the sandbox is actually read-only so the build results are actually deposited elsewhere. Additionally, ruby is only one of several products built in this sandbox, and the whole thing is wrapped in a complicated make-based procedure. There may even come a time when I must build ruby with an actual cross-compiler targeting a different architecture than the build machine, for an embedded system that has no ability to develop software. But the above presents the relevant aspects of the method.

The issue is that programs like gem have hard-coded into them the location into which the "make install" step deposited them. If you run "gem environment", you see that it expects the location of the ruby executable to be /sandbox/ruby/install/bin/ruby, and the search paths are similarly set. Note that the sandbox is not available on the remote machine.

Granting permission to install into /usr/foo on the build machine is not an option because it's a production environment and we can't even temporarily corrupt that location while other procedures are running. And we don't want to copy the world into a chroot environment for the purpose of building this sandbox. (That would not work anyway because ruby isn't the only thing that gets built in that sandbox, and other things require an intact /usr/foo tree.)

So, how do others build and operate ruby in an environment like this?

P.S. Having a tool invoke a prior version of itself to build itself, as is done with ruby 1.9.2, violates a fundamental tenet of release engineering: You shall not unwind history to reproduce yourself. I see from the archives of this list that others have also reported this issue. When will this oversight be fixed?

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.