Setting LD_LIBRARY_PATH doesn't seem to work?

On a linux system, I’ve developed a extension to STAF (see previous
posts) that links to libSTAF.so to provide various bits of
functionality. However, I had forgotten that I’d set my
LD_LIBRARY_PATH to find said library, so it was always loaded
correctly.

I discovered that it wasn’t loading correctly when I ran some CGI code
I’d written around that extension. So I did what I do in every other
programming language, and set ENV[‘LD_LIBRARY_PATH’] to point to
/usr/local/staf/lib, where the library resides. Alas, this did not
work. In fact, if I unset LD_LIBRARY_PATH in my environment and tried
this, it didn’t work there either.

I’d suspect myself before I’d suspect ruby, but in this case, I think
I’ve covered all my bases. Comments are welcome.

-=Eric

···


Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
– Blair Houghton

why not compile the module using LD_RUN_PATH? relying on LD_LIBRARY_PATH is
evil :wink:

-a

···

On 11 Nov 2002, Eric Schwartz wrote:

On a linux system, I’ve developed a extension to STAF (see previous
posts) that links to libSTAF.so to provide various bits of
functionality. However, I had forgotten that I’d set my
LD_LIBRARY_PATH to find said library, so it was always loaded
correctly.

I discovered that it wasn’t loading correctly when I ran some CGI code
I’d written around that extension. So I did what I do in every other
programming language, and set ENV[‘LD_LIBRARY_PATH’] to point to
/usr/local/staf/lib, where the library resides. Alas, this did not
work. In fact, if I unset LD_LIBRARY_PATH in my environment and tried
this, it didn’t work there either.

I’d suspect myself before I’d suspect ruby, but in this case, I think
I’ve covered all my bases. Comments are welcome.

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

why not compile the module using LD_RUN_PATH? relying on
LD_LIBRARY_PATH is
evil :wink:

I’ve heard this charge levied before; can someone tell me the
inherent evil in it?

I’m not doubting anyone, just curious.

···

=====

Yahoo IM: michael_s_campbell


Do you Yahoo!?
U2 on LAUNCH - Exclusive greatest hits videos
http://launch.yahoo.com/u2

ahoward ahoward@fsl.noaa.gov writes:

why not compile the module using LD_RUN_PATH? relying on LD_LIBRARY_PATH is
evil :wink:

Why is that? In any event, the problem is that when loading C
extensions, ruby doesn’t seem to be respecting the LD_LIBRARY_PATH
setting. Regardless of the probity of the idea, it’s broken, and
needs fixing.

I worked around it by adding /usr/local/staf/lib to /etc/ld.so.conf,
but I don’t think that’s generally a good idea.

-=Eric

···


Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
– Blair Houghton

Why is that? In any event, the problem is that when loading C
extensions, ruby doesn't seem to be respecting the LD_LIBRARY_PATH
setting. Regardless of the probity of the idea, it's broken, and
needs fixing.

Can you give a complete example for this (configuration, OS, ...)

I use to test bdb this script

pigeon% cat a.sh
#!/bin/sh
for dir in `ls -d /home/ts/local/db[234]*`
do
    export LD_LIBRARY_PATH=${dir}/lib
    ruby extconf.rb --with-db-prefix=$dir --with-db-version=
    make unknown
    echo
    echo -n "Waiting ... "
    read
    make test 2>&1 | more
    make distclean
done
pigeon%

It has always worked.

Guy Decoux

why not compile the module using LD_RUN_PATH? relying on
LD_LIBRARY_PATH is
evil :wink:

I’ve heard this charge levied before; can someone tell me the
inherent evil in it?

I’m not doubting anyone, just curious.

this sums it up

http://www.visi.com/~barr/ldpath.html

mkmf uses LD_RUN_PATH to avoid these problems. it sets LDSHARED to be
something like ‘env LD_RUN_PATH=/usr/local/somepkg/lib gcc -shared’ to avoid
the exact problem described earlier in this thread - which is requiring the
user/program to manage it’s own binary dependancies. i think of it like
having instance variable associate with an object : when the search path is
encoded in a shared library that ‘object’ knows it’s own depends. it does not
rely on user code to manage them.

-a

···

On Wed, 13 Nov 2002, Michael Campbell wrote:

=====

Yahoo IM: michael_s_campbell


Do you Yahoo!?
U2 on LAUNCH - Exclusive greatest hits videos
http://launch.yahoo.com/u2

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

Why is that?

see earlier post.

In any event, the problem is that when loading C extensions, ruby doesn’t
seem to be respecting the LD_LIBRARY_PATH setting. Regardless of the
probity of the idea, it’s broken, and needs fixing.

it is not ruby which is broken

eg/c > cat liba.c
int a;

eg/c > cat libb.c
int b;
extern int a;

eg/c > gcc -o liba.so -shared liba.c

eg/c > gcc -o libb.so -shared libb.c -L ./ -la

eg/c > ldd liba.so
libc.so.6 => /lib/libc.so.6 (0x4000b000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)

eg/c > ldd libb.so
liba.so => not found
libc.so.6 => /lib/libc.so.6 (0x4000b000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)

so liba.so has NO depends, and libb.so HAS depends.

now a program to dlopen them :

eg/c > cat dlopen.c
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>

int
main (int argc, char **argv)
{
char *ld_library_path;

/* set LD_LIBRARY_PATH if arg given */
if (argc > 1)
{
  printf ("SET LD_LIBRARY_PATH TO : %s\n", argv[1]);
  setenv ("LD_LIBRARY_PATH", argv[1], 0);
}

/* print LD_LIBRARY_PATH if env has it */
if (ld_library_path = getenv ("LD_LIBRARY_PATH"))
{
  printf ("LD_LIBRARY_PATH : %s\n", ld_library_path);
}

/* try to open liba.so */
if ((dlopen ("liba.so", RTLD_NOW|RTLD_GLOBAL)) == NULL)
{
  printf ("FAILED TO LOAD 'liba.so'\n");
}
else
{
  printf ("SUCCESSFULLY LOADED 'liba.so'\n");
}

/* try to open libb.so - which depends on a*/
if ((dlopen ("libb.so", RTLD_NOW|RTLD_GLOBAL)) == NULL)
{
  printf ("FAILED TO LOAD 'libb.so'\n");
}
else
{
  printf ("SUCCESSFULLY LOADED 'libb.so'\n");
}

return 0;

}

now using it

eg/c > dlopen
FAILED TO LOAD ‘liba.so’
FAILED TO LOAD ‘libb.so’

eg/c > dlopen ./
SET LD_LIBRARY_PATH TO : ./
LD_LIBRARY_PATH : ./
FAILED TO LOAD ‘liba.so’
FAILED TO LOAD ‘libb.so’

eg/c > env LD_LIBRARY_PATH=./ dlopen
LD_LIBRARY_PATH : ./
SUCCESSFULLY LOADED ‘liba.so’
SUCCESSFULLY LOADED ‘libb.so’

so you see, setting LD_LIBRARY_PATH using setenv does not affect dlopen?!
there is probably a reason for this - but it’s non-obvious. using LD_RUN_PATH
or compiling with -rpath always works and is easy to understand and maintain.

it seems as if dlopen uses the environment of the parent process… perhaps
for securtiy? anyone? all i know is that every time i play with dlopen and
LD_LIBRARY_PATH i’ve been confused…

moral : stay awasy from LD_LIBRARY_PATH

-a

···

On 12 Nov 2002, Eric Schwartz wrote:

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================

and from the dlopen man page :


NOTES

 If other objects were link-edited with pathname  when  path-
 name  was  built,  that is, the pathname has dependencies on
 other objects, those objects will automatically be loaded by
 dlopen(). The directory search path used to find both  path-
 name and the other needed objects may be affected by setting
 the  environment variable LD_LIBRARY_PATH, which is analyzed
 once at process startup, and from a runpath  setting  within
 the object from which the call to dlopen() originated. These
 search rules will only be applied to path names that do  not
 contain  an  embedded   '/'.  Objects whose names resolve to
 the same absolute or relative path name may  be  opened  any
 number  of times using  dlopen(); however, the object refer-
 enced will only be loaded once into the address space of the
 current process.

dlopen is the culprit.

moral : do not use LD_LIBRARY_PATH, use runpath encoding.

-a

···

====================================

Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================