I'm particularly interested in parallel performance in the >8 core space. (In fact, we saw some regressions from 7.2->7.4 that we never tracked down properly, but maybe can now.)<div><br></div><div>If the buildbot can make it easy to add a new "slave" machine that runs and uploads its result to a central location, then I would be happy to donate a few hours of dedicated time (no other logins) on a 32 core westmere machine, and hopefully other architectures soon.</div>
<div><br></div><div>Maybe, this use case is well-covered by creating a jenkins/travis slave and letting it move the data around? (CodeSpeed looks pretty nice too.)</div><div><br></div><div>Cheers,</div><div> -Ryan</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Dec 5, 2012 at 12:40 AM, Ben Lippmeier <span dir="ltr"><<a href="mailto:benl@ouroborus.net" target="_blank">benl@ouroborus.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"><br>
On 01/12/2012, at 1:42 AM, Simon Peyton-Jones wrote:<br>
<br>
> | > While writing a new nofib benchmark today I found myself wondering<br>
> | > whether all the nofib benchmarks are run just before each release,<br>
><br>
</div><div class="im">> I think we could do with a GHC Performance Tsar. Especially now that Simon has changed jobs, we need to try even harder to broaden the base of people who help with GHC. It would be amazing to have someone who was willing to:<br>
><br>
> * Run nofib benchmarks regularly, and publish the results<br>
><br>
> * Keep baseline figures for GHC 7.6, 7.4, etc so we can keep<br>
> track of regressions<br>
><br>
> * Investigate regressions to see where they come from; ideally<br>
> propose fixes.<br>
><br>
> * Extend nofib to contain more representative programs (as Johan is<br>
> currently doing).<br>
><br>
> That would help keep us on the straight and narrow.<br>
<br>
<br>
</div>I was running a performance regression buildbot for a while a year ago, but gave it up because I didn't have time to chase down the breakages. At the time we were primarily worried about the asymptotic performance of DPH, and fretting about a few percent absolute performance was too much of a distraction.<br>
<br>
However: if someone wants to pick this up then they may get some use out of the code I wrote for it. The dph-buildbot package in the DPH repository should still compile. This package uses <a href="http://hackage.haskell.org/package/buildbox-1.5.3.1" target="_blank">http://hackage.haskell.org/package/buildbox-1.5.3.1</a> which includes code for running tests, collecting the timings, comparing against a baseline, making pretty reports etc. There is then a second package buildbox-tools which has a command line tool for listing the benchmarks that have deviated from the baseline by a particular amount.<br>
<br>
Here is an example of a report that dph-buildbot made:<br>
<br>
<a href="http://log.ouroborus.net/limitingfactor/dph/nightly-20110809_000147.txt" target="_blank">http://log.ouroborus.net/limitingfactor/dph/nightly-20110809_000147.txt</a><br>
<span class="HOEnZb"><font color="#888888"><br>
Ben.<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
_______________________________________________<br>
Glasgow-haskell-users mailing list<br>
<a href="mailto:Glasgow-haskell-users@haskell.org">Glasgow-haskell-users@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/glasgow-haskell-users" target="_blank">http://www.haskell.org/mailman/listinfo/glasgow-haskell-users</a><br>
</div></div></blockquote></div><br></div>