Building and installing packages
Simon Marlow
simonmar@microsoft.com
Mon, 30 Apr 2001 16:47:05 +0100
This is a multi-part message in MIME format.
------_=_NextPart_001_01C0D18C.CE4CBCD8
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
> Currently the fptools build system is a little bit inflexible in its
> handling of packages. There are at least the following problems:
>=20
> 1) It's too monolithic, e.g. fptools/ghc/driver/PackageSrc.hs must
> know about *all* packages in fptools/hslibs. Things get even
> trickier when e.g. I want to split up my OpenGL package into a
> package "OpenGL", containing the GL core, and a package "GLUT"
> containing a simple toolkit, *which needs package "OpenGL"*.
> Centralized knowledge about dependencies might be OK=20
> for standard
> libs, but not for a library like this. A situation like=20
> this will
> (hopefully!) become more common, given the lively discussions on
> the library list.
Agreed, and fixing this has been pending for a long time (you may
remember we discussed this a while back - see the attached message).
But looking at the larger picture, the problem is particularly hairy
because of the requirement to separate build and install-time. One
approach is to say "ok, everything we depend on must be identical
between the build machine and the target machine", which works for
systems with a package manager because we can enumerate the dependencies
exactly.
When we're not using a package manager, the only reasonable approach is
not to have any dependencies at all: which is why we ship with a static
libgmp.a in a binary distribution, and for 5.00.1 we're also going to
ship binary dists without readline support in GHCi (this won't affect
RPMs, of course).
These two approaches I can get my head around. However, you're talking
about something in between: allowing certain dependencies to change or
move around between build time and install time. If we're going to
allow this (and I'm not convinced we should), then we need to be
extremely careful and document exactly what's allowed to change.
=20
> 2) "ghc-pkg -l"'s output is plainly wrong: I don't have a package
> win32 or com installed on my Linux box, although=20
> ghc-pkg tells me
> so.
Yup, I'll fix this.
> 3) Is the comma-separated output a good thing? Re-using it in
> e.g. shell scripts would be much easier if it was a=20
> single package
> name per line.
I'll change it as you suggest, if there are no objections?
> 4) Why is the gmp package not "inlined" into the rts package?
No particular reason, it just seemed cleaner that way.
> 5) The distinction between the compile-time and runtime environment
> is blurred, e.g. for a given (C) library there could be=20
> different
> library names and paths on the box where hslibs are=20
> built and the
> box where GHC/hslibs is installed later. Some packages=20
> will surely
> need help from autoconf at install time.
>
> 6) What has become of the plans to make relocateable RPMs=20
> possible?=20
> It's a quite important feature, esp. when there is the=20
> need for a
> multi-stage bootstrap.
It's certainly possible, but I'm no RPM expert. Check out the way we
build binary distributions: we ship a copy of pkgconf (the compiled
PackageSrc.hs) in the tarball, and run it to generate a package.conf at
install time.
> To remedy this situation, PackageSrc should be nuked=20
> completely and the
> following steps should be done for *each* package:
>=20
> 1) Determine things needed for compilation like #include=20
> paths, names
> of #include files, etc. Although I don't have an example yet,
> this might involve help from autoconf. Most dependencies,
> e.g. other packages, are probably static, but one never knows...
>=20
> 2) Use the information from step 1 to build the package.=20
> This step is
> of course absent during installation where we already have a
> compiled package.
>=20
> 3) Determine things needed for installation like paths and names of
> libraries needed for linking with the current package. This
> definitely needs help from autoconf (e.g.: Where are=20
> the X11 libs
> located? Should we link with libMesa or libGL? ...)
>=20
> 4) Use the information from step 1 and 3 to add the package to the
> given GHC (inplace or "real" one).
Ok, I'm agreed that we definitely need to nuke PackageSrc.hs and have
each package register itself. However, I'm still uneasy about
separating (1) and (3). Let's take your examples: if the X11 libs move,
that shouldn't be a problem, as long as they're still for the same
version of X11. libMesa and libGL are a special case: presumably these
are designed so that if you compile against one, you can link to the
other one. But that's certainly not true for most shared libraries -
our recent experiences with glibc show that even a minor version bump
can cause problems.
Cheers,
Simon
------_=_NextPart_001_01C0D18C.CE4CBCD8
Content-Type: message/rfc822
Content-Transfer-Encoding: 7bit
X-MimeOLE: Produced By Microsoft Exchange V6.0.4688.0
Received: from mail2.microsoft.com (INET-VRS-02 [157.54.8.105]) by 157.microsoft.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2651.58) id P26H7P67; Mon, 21 Aug 2000 04:23:01 -0700
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Received: from 128.36.229.215 by mail2.microsoft.com (InterScan E-Mail VirusWall NT); Mon, 21 Aug 2000 04:22:58 -0700 (Pacific Daylight Time)
Received: by www.haskell.org (Postfix) id 3396E101E; Mon, 21 Aug 2000 07:22:57 -0400 (EDT)
content-class: urn:content-classes:message
Subject: RE: Package trouble
Date: Mon, 21 Aug 2000 11:45:11 +0100
Message-ID: <9584A4A864BD8548932F2F88EB30D1C6115700@TVP-MSG-01.europe.corp.microsoft.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: Package trouble
Thread-Index: AcALYirF3h2RBXN6ROKOZOCuioie4g==
From: "Simon Marlow" <simonmar@microsoft.com>
To: "Sven Panne" <Sven.Panne@informatik.uni-muenchen.de>,
<cvs-ghc@haskell.org>,
<cvs-hslibs@haskell.org>
> I've just started adding my HOpenGL stuff into hslibs and immediately
> stumbled over some rather technical details in the driver and the
> package system:
>=20
> * The User's Guide talks about a GHC version number as the second
> element of a packages configuration triple, but the driver only
> expects a pair without it. I'm not sure if either the=20
> docs or the=20
> driver should be fixed.
I originally intended to add this, and then wondered whether it was
actually
a good idea or not. You currently can't use a library built by a
different
version of the compiler, because the .hi file versioning stops you. I
suppose the currently system allows you to install a bogus package (i.e.
one
that you can't use), but the version number in the package config would
be a
pretty weak safety net.
Unless anyone argues strongly in favour of having a version number in
the
package config, I'll update the documentation to remove it.
> * checkConfigAccess in the driver is a nice idea, but totally
> useless in *nix reality due to the completely underspecified
> permission handling in module Directory. Checking if the owner
> has write access is not the right thing to do, as package.conf is
> hopefully not owned by a mere mortal (i.e. user).
I don't understand this comment: surely checking for write access to the
package config file is the right thing to do? If the config file is
owned
by root, then you need to be root to overwrite it. If on your Win2k
system
you are one of the list of users in the ACL allowed to update the
package
config, then checking for write access will tell you that.
> Whoever reads
> this: Please nuke this poor permission handling interface from
> Directory for Haskell-2! It's not the least common denominator of
> file systems, it's simply underspecified and confusing. Modern
> OSes have different categories of users, ACLs, etc. Simply
> ignoring this fact doesn't help here.
Agreed.
> * HOpenGL will be split into 2 packages: `OpenGL' and `GLUT', with
> the latter depending on the former, but currently the build
> system seems unable to handle this in any other way than
> hardwiring this into a large set of known packages. This=20
> can't be=20
> done for OpenGL/GLUT, see next item
Ok, there's currently a gaping hole in the package support - namely how
installation works. =20
For a binary distribtuion, the paths in the package configuration need
to be
adjusted depending on where the packages are actually installed. The
correct way to do this would seem to be
- install the basic compiler tools and package libraries
- autoconf to generate the individual package configs
for each package from templates,
- ghc --add-package for each of the packages.
For an RPM which has paths hardcoded in, the second step can be omitted.
For "make install", we begin with an empty package configuration and
each
package in hslibs should do its own "ghc --add-package".
For building the package config for ghc-inplace, perhaps this can be
done at
"make boot" time.
The basic idea is to remove all the hslibs from PackageSrc.hs (leaving
just
the rts, gmp, and std packages), with each hslib registering itself at
certain points in the compilation cycle. How does that sound?
> * I vaguely remember a discussion about when and how to do package
> configuration for Manuel's GTK stuff, but forgot the details. The
> story for HOpenGL is as follows: The correct include paths for
> compilation and libs for linking could be detected by some
> autoconf magic, but doing this before the package's compilation
> is not enough/wrong: Due to inlining, the include paths on the
> system where the packages are *installed* are needed, too, and
> knowing how to link OpenGL apps on the building system is
> completely wrong. OK, now there's an official OpenGL ABI effort,
> but I don't want to depend on that, and future packages=20
> will have=20
> similar problems. How should we handle this? Distribute=20
> configure=20
> scripts with packages, which write the package configuration
> files? Currently I can't see another way. :-(
Yes, that seems to be the only way to do it.
Cheers,
Simon
------_=_NextPart_001_01C0D18C.CE4CBCD8--