Building system again

[Roman Leshchinskiy]

Hi all,

today, I tried to build the ghc-ndp branch from a fresh tree after two 
weeks off. Ultimately, it sort of worked, but not without some pain. 
I'll describe what I did and the problems I've encountered in the hope 
that it will be helpful.

   darcs get http://darcs.haskell.org/ghc-ndp --partial
   cd ghc-ndp
   chmod u+x darcs-all
   ./darcs-all get

Fails with

== running darcs get --partial 
http://darcs.haskell.org/ghc-ndp/packages/base
Invalid repository:  http://darcs.haskell.org/ghc-ndp/packages/base

This is because darcs-all determines the base url of the repository by 
removing a trailing "ghc". However, our url ends with "ghc-ndp" so this 
doesn't work. The workaround is to change _darcs/prefs/defaultrepo to 
http://darcs.haskell.org/ghc. I assume this is going to be problematic 
for all branches.

   cd libraries
   darcs get http://darcs.haskell.org/packages/ndp
   cd ..
   <set up mk/build.mk>
   autoreconf
   sh boot

This step is new and, of course, I skipped it on first try. The README 
mentions it but why is it necessary at all? Couldn't this be done as 
part of configure?

   configure

Works, but the first line of output is strange:

checking for GHC version date... darcs: getFdStatus: invalid argument 
(Bad file descriptor)

   make

Fails with

Running Haddock for ndp-0.1...
dist/build/tmp/./Data/Array/Parallel/Unlifted.hs:"dist/build/tmp/./Data/Array/Parallel/Unlifted.hs": 
28:4: Parse error
make[1]: *** [doc.library.ndp] Error 1
make: *** [stage1] Error 2

The parse error is because ndp uses some ghc extensions which Haddock 
doesn't understand. But why is Haddock run at all here? IMO, a separate 
make doc step is preferable to running it automatically. In any case, I 
assume it can be turned off but the build documentation doesn't seem to 
say how. Also, IMO Haddock errors should never be treated as fatal by 
the build system.

A related question: is there a way for a package to say that it doesn't 
want to be processed with Haddock? From the Cabal documentation, I'd 
guess the answer is no.

In any case,

   cd compiler
   make stage=2 boot
   make stage=2

seems to give me a working compiler (thanks to Manuel for suggesting this).

In addition to the above, I've noticed the following problems.

1. Is there a way to get a clean tree? Both make clean and make 
distclean fail to clean any subdirectories (compiler, rts) and 
libraries. This is not really what I expect (and not what they used to 
do, IIRC).

Also, my attempt to make distclean a second time failed with

mk/boilerplate.mk:53: mk/config.mk: No such file or directory
make: *** No rule to make target `mk/config.mk'.  Stop.

IMO, cleaning should not require me to rerun configure.

2. Parallel make (i.e., make -j) doesn't work at all for me. It seems to 
try to build all (or at least some) packages simultaneously which causes 
Cabal to be built several times and ultimately falls over completely. 
IIUC, this is because libraries/Makefile does not define dependencies 
between packages (which is understandable) and does not force them to be 
built in the right order (which it used to do, IIRC). This is quite 
unfortunate, as make -j used to be quite a bit faster than plain make on 
my dual-core Mac. It would be less of a win now that the libraries use 
ghc --make, I guess, but would still speed up the compilation of the 
compiler itself.


Finally, I'd like to point out that the recent changes to the build 
system have caused and continue to cause a lot of problems for us. I 
understand that refactoring the build system is a worthwhile task
but it would be really great if it was less disruptive. Perhaps 
significant changes (like running Haddock by default) could be discussed 
or at least publicised. Also, the lack of support for parallel make is 
really unfortunate as it significantly increases turn-around times in 
some cases (and is confusing to novice users, I guess).

Roman