<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I think there is some misinformation in here, as well as some correct but misleading information. Some of the documentation is dodgy, so I have taken the time to verify my assertions on my Mac (Intel OS X 10.4.10) and my linux box (Linux 2.6.15 X86_64). </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I will reiterate what I understand the goals to be:</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>1) When creating an executable, specify where the dynamic loader should look for shared libraries at runtime. The intent is to expand the shared library search space beyond the default search path and remove the need to set environment variables. </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>2) When running an executable, override the default. Primarily a requirement for building GHC.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><B>On Linux</B></DIV><DIV><B>------------</B></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><SPAN class="Apple-style-span">An ELF executable can have an embedded list of paths to search for dynamic libraries (the DT_RPATH entry). This can be set by using -rpath with ld. <B>DT_RPATH is deprecated. </B>This list applies to all shared libraries used by the executable (it is not per shared library). There is no default value placed in the DT_RPATH entry. You must use -rpath to set it.</SPAN></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>There is a new entry, DT_RUNPATH. DT_RUNPATH works similarly to DT_RPATH. However, when it is set, DT_RPATH is ignored. DT_RUNPATH is also set using -rpath, but you must use the --enable-new-dtags switch as well. </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>When looking for a shared library, the dynamic linker(ld.so) checks the paths listed in DT_RPATH (unless DT_RUNPATH Is set) , the paths listed in the environment variable LD_LIBRARY_PATH, the paths listed in DT_RUNPATH, the libraries listed in /etc/ld.so.cache, and finally /usr/lib and /lib. It checks in that order and takes the first library found. At least on my linux box, <B>LD_LIBRARY_PATH does NOT override the paths in DT_RPATH</B> even though the documentation implies that it does. LD_LIBRARY_PATH does override DT_RUNPATH.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>You CAN override the search path embedded using DT_RPATH by using the LD_PRELOAD environment variable. This variable contains a *whitespace-separated* list of libraries (not directories to search) to load prior to the search process. The listed libraries are loaded whether or not they are needed to resolve a dependency in the executable.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Finally, an ELF shared library can also have a DT_RPATH entry. This only impacts the search for shared libraries that are dependencies of the shared library and not the executable. As with the DT_RPATH entry in an ELF executable, this is not overridden by LD_LIBRARY_PATH but can be overridden using LD_PRELOAD as above. </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><B>On Mac OS X</B></DIV><DIV><B>------------------</B></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>A Mach-O executable can embed the full path name for each shared library (as well as rules for acceptable substitutes). This is called the "install name" for the library and it is included by default when building an executable. The install name for the library is NOT based on where the static linker (ld) found the library when the executable was built. The static linker (ld) extracts the install name from the shared library when building the executable. The install name of the shared library is set when building the shared library. When you build a shared library you should know where the library is going to be installed so that the install name is set correctly.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>When looking for shared libraries, the dynamic linker (dyld) first scans the directories in DYLD_LIBRARY_PATH, then checks the location in the install name (which is per library), and finally checks the standard locations.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>DYLD_LIBRARY_PATH successfully overrides the the path embedded in the executable.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Caveat 1: LD_LIBRARY_PATH has no runtime impact, but it does impact where the static linker looks for share libraries. It looks first in the directories specified using -L, the the directories in LD_LIBRARY_PATH, and finally in /lib, /usr/lib, & /usr/local/lib. This is particularly confusing because many configure scripts seem to ignore LD_LIBRARY_PATH and you can get inconsistent results from configure and gcc/ld on whether a library is present.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Caveat 2: Mac OS X has a set of compiler/linker switches for dealing with Frameworks (packages of shared libraries and include files). These are installed outside the typical *nix directory structure. These switches act like -I (to gcc) and -L (to ld). If you end up totally confused about where to find something, read up on this. The OpenGL and OpenAL headers and libraries are in Frameworks, for example.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><B>My Conclusions</B></DIV><DIV><B>-----------------------</B></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>For Mac: The -rpath switch is not available on Mac OS X because it is superfluous. The default behavior of embedding a location for each individual shared library is at least as good. Cabal (and the GHC build process) should use their knowledge of the ultimate install location to set the install name when shared libraries are built. In-place compilation can override this with DYLD_LIBRARY_PATH</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>For Linux: On linux, we should be sure to use the --enable-new-dtags switch if we use -rpath. Otherwise we risk having paths that can't be overridden by LD_LIBRARY_PATH. I know we don't want to have an install that *requires* root access, but we should consider adding entries to /etc/ld.so.conf and running ldconfig to eliminate the need to hardwire path names.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Verification</DIV><DIV>---------------</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I wrote a small but heinous C program that called a trivial function in a shared library. Using 2 copies of the shared library, I observed which was used under the various scenarios. </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>On linux, I used "objdump -p" to examine the ELF executables and shared libraries. I used "ldd" to see which libraries were actually being loaded. When I got an unexpected result from ldd (LD_LIBRARY_PATH not overriding DT_RPATH), I went so far as using lsof on the process to see which files were mapped.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>On the Mac, I used "otool -l" to examine the Mach-O executables and shared libraries. I used the environment variable DYLD_PRINT_LIBRARIES=YES to see which libraries were actually being loaded (this forces dyld to print the location of each library as it is loaded).</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>--Reilly Hayes</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><DIV>On Aug 1, 2007, at 6:36 AM, Simon Marlow wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Roman Leshchinskiy wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Simon Marlow wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Roman Leshchinskiy wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Simon Marlow wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Roman Leshchinskiy wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Simon Marlow wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">We had a discussion on glasgow-haskell-users recently about whether to use libtool, and the general concensus was not:</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hmm, I missed that one completely. It's probably too late for me to jump in but I just don't see how the approach you outlined in</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="http://www.haskell.org/pipermail/glasgow-haskell-users/2007-June/012740.html">http://www.haskell.org/pipermail/glasgow-haskell-users/2007-June/012740.html</A><SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">deals with systems which don't *have* an rpath and with users who manually set LD_LIBRARY_PATH. To be entirely honest, I suspect that what you are trying to achieve can't be done portably and reliably.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I don't expect it to be portable.<SPAN class="Apple-converted-space"> </SPAN>So on MacOS X, is it the case that the only way to build a binary that links to a particular shared library outside the system location is to use LD_LIBRARY_PATH?</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">IIUC yes, except that it's called DYLD_LIBRARY_PATH. From man dyld:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN class="Apple-converted-space"> </SPAN>For<SPAN class="Apple-converted-space"> </SPAN>each<SPAN class="Apple-converted-space"> </SPAN>library that a program uses, the dynamic linker looks</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN class="Apple-converted-space"> </SPAN>for it in each directory in DYLD_LIBRARY_PATH<SPAN class="Apple-converted-space"> </SPAN>in<SPAN class="Apple-converted-space"> </SPAN>turn.<SPAN class="Apple-converted-space"> </SPAN>If<SPAN class="Apple-converted-space"> </SPAN>it</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN class="Apple-converted-space"> </SPAN>still<SPAN class="Apple-converted-space"> </SPAN>can't<SPAN class="Apple-converted-space"> </SPAN>find<SPAN class="Apple-converted-space"> </SPAN>the<SPAN class="Apple-converted-space"> </SPAN>library,<SPAN class="Apple-converted-space"> </SPAN>it<SPAN class="Apple-converted-space"> </SPAN>then<SPAN class="Apple-converted-space"> </SPAN>searches DYLD_FALL-</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN class="Apple-converted-space"> </SPAN>BACK_FRAMEWORK_PATH and DYLD_FALLBACK_LIBRARY_PATH in turn.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">But I think on MacOS, the library paths are usually hardcoded in the executable.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">If you can hardcode the library paths in the executable, then that's exactly what -rpath does isn't it?</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">After looking at the documentation I think I sort of understand how the system works. When you build a shared library you can specify its install_name which is the location where it will ultimately be installed. When linking against the library, the linker hardcodes the library's install_name into the executable. When the executable is run, the dynamic loader looks for the library at that location. If it finds the library, it's happy. If not, it will (appently) look in DYLD_LIBRARY_PATH and friends and then in standard locations (/usr/lib etc.) but will ignore DYLD_LIBRARY_PATH for setuid and setgid processes.</DIV> </BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">It sounds like the only way to get the effect of -rpath is to create a wrapper script that sets DYLD_LIBRARY_PATH, then.<SPAN class="Apple-converted-space"> </SPAN>Presumably this is what libtool does.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">This is a bit of a pain.<SPAN class="Apple-converted-space"> </SPAN>We can't support -hardwire-lib-paths on MacOS, and we'll need some jiggery-pokery in the build system to get the same effect - perhaps ghc-inplace sets DYLD_LIBRARY_PATH before invoking the ghc binary.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Cheers,</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN class="Apple-tab-span" style="white-space:pre"> </SPAN>Simon</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Cvs-ghc mailing list</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="mailto:Cvs-ghc@haskell.org">Cvs-ghc@haskell.org</A></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="http://www.haskell.org/mailman/listinfo/cvs-ghc">http://www.haskell.org/mailman/listinfo/cvs-ghc</A></DIV> </BLOCKQUOTE></DIV><BR></BODY></HTML>