buying new hardware; want fast validate
marlowsd at gmail.com
Thu May 1 14:46:22 EDT 2008
Duncan Coutts wrote:
> On Wed, 2008-04-30 at 21:01 -0400, Norman Ramsey wrote:
>> Dear Fellow GHC Hackers,
>> I'll be buying new computers soon, and right now the only thing I do
>> for which computer performance is a problem is GHC hacking---especially
>> running 'validate'. I've got a good performance boost on my current
>> machine by using two SATA drives striped in a RAID-0 configuration.
>> I'm wondering what other tricks may be warranted. Specifically
>> I'm interested in answers to the following questions:
>> 1. How much benefit does one get from four cores versus two?
>> I know that the build process runs in parallel thanks to
>> make -j, but I don't know how much parallelism is available
>> in the testing part. Maybe somebody who already has
>> quad core can compare and let us know the results?
> I recently got a quad core amd64 and I run linux. I noticed from the
> system monitor graphs that not that much of the build process can use
> many cores. I was using make -j5.
> The one bit that really does benefit is building the ghc compiler
> itself. That can make full use of all 4 cores for an extended period.
> The ./configure step is painfully serial and building the early tools
> and the libraries is rather serial too. There are lots of little changes
> here that we could do to make it more parallel. One example is that we
> link 20 copies of Setup.hs rather than reusing one copy. Linking is very
> much serial.
> In principle the test suite should be possible to parallelise but I
> don't know much about python and parallelism.
We're pretty close to being able to run the testsuite in parallel, but
we ran up against a Python bug:
perhaps we could work around that bug with a big lock or something.
Unfortunately it only happens a handful of times over a testsuite run,
so it's hard to work on.
This is the low-hanging fruit for speeding up validate right now.
Anyone feel like having a crack at it?
More information about the Cvs-ghc