Restricted commandline lenghts

[Esa Ilari Vuokko]

Hi,

Ticket #19 [1], and maybe some big projects in general, need to be able
to invoke ar/ld so that we don't run into issues in commandline lenght.
(And in theory, compiler's as well, but we that'd require the dreaded
deps analysis.)  I think this is major feature we should resolve before
GHC 6.6 goes out.  It affects significantly some libraries, at least
gtk2hs, and I can see real size benefits on some of my own libraries
as well.

I posted patch to fix this in some cases for Windows [2], but upon
talking about it with Duncan, in irc, there's few exta points:
 * Some unixy shells apparently have restrictions as well, so this
   shouldn't be just Windows.
 * On my old implementation, I just split the commandlines by filecount,
   not according to actual space the paths take.  Is this a major
   point?  It can affect the number of tool invokations quite radically.
 * The algorithm should really be parametriseable because it is, I believe,
   in all cases heuristic.

As how to proceed on implementing this, I'm a bit unsure.
 * Because of unicode conversion issues, I am not entirely sure if we
   can accurately know the length of commandline, atleast in Windows.
   Or going for the bad case, 4 bytes per character gives us a lot of
   setback in common case.
 * I think the common way to workaround this is by using xargs with
   some constant number per params (much like my current algos).
 * I think there is a diffrence between complexities, because with
   ar we really want to do append, but with ld we can do a
   tree-style build.  I haven't benchmarked this.
 * This might be useful to generalise into library function(s).

I could use some advice or experiences from other buildsystems.  If nobody
else steps up, I'll probably implement some choice (but I'm only prepared
to test it in Windows.)

[1] http://hackage.haskell.org/trac/hackage/ticket/19
[2] http://www.haskell.org/pipermail/cabal-devel/2006-May/000006.html

Best regards,
--Esa Ilari Vuokko