<div dir="ltr"><div>I'm still seeing different numbers from Johan, so I'm just going to throw up my hands and proceed separately for now.</div><div> </div><div>I'm sharing my Excel spreadsheet. It contains:</div>
<div> </div><div> * My 8 Feb run (libraries O2, programs O1, 10 iterations)</div><div> * Johans' first (comparable) set of 7.6.2 -> HEAD numbers (O2, O1, 5 iterations)</div><div> * My 11 Feb run (libraries O2, programs O2, 30 iterations)</div>
<div> * some analysis, indicating that his and my numbers do not correlate</div><div> </div><div>Info about my runs:</div><div> </div><div> * 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.2.1, 7.2.2, 7.4.1, 7.4.2, 7.6.1, 7.6.2, HEAD (ec9477b1e51)</div>
<div> * I using the NoFib make to build the tests, but not to actually execute them.</div><div> </div><div>Here's why I don't simply use the NoFib makefiles for everything. That would run an entire suite for each version: all of 7.0.1, then all of 7.0.2, etc.</div>
<div> </div><div>I wanted better time locality, since my test machine (though quite capable - Marlow and SPJ might know the specs), has a few other users and I've been seeing plenty of noise in the measurements. So now I use NoFib's makefiles to build all of the tests and generate the test-running command, but I cache the run command and then interleave the execution of each GHC version's executable. In other words:</div>
<div> </div><div> using nofib: outer loop = GHC version, inner loop = each program</div><div> my rig: outer loop = each program; inner loop = GHC version</div><div> </div><div>NB I do all the building before entering that nested loop.</div>
<div> </div><div>So I think I end up with less noise in the data.</div><div> </div><div>I have an additional loop between those two loops that controls the iterations. My rigorous data set (11 Feb) uses 30 iterations.</div>
<div> </div><div>Here are the big swings I'm seeing...</div><div> </div><div>Indicated allocation regressions that seem to still affect HEAD:</div><div> </div><div>NB The 'swing' column is the subtraction (of percentage points) between HEAD today and the best job GHC ever did on that program. The 'concrete' column converts 'swing' to the actual difference in allocation.</div>
<div><font face="courier new,monospace">
</font><font face="courier new,monospace">
</font><table width="763" style="width:573pt;border-collapse:collapse" border="0" cellspacing="0" cellpadding="0"><tbody><tr height="20" style="height:15pt"><td style="border:0px black;background-color:transparent"><font size="3"><font color="#000000"></font></font></td>
</tr></tbody></table><font face="courier new,monospace"> 7.0.1 7.0.* 7.2.* 7.4.1 7.4.2 7.6.* HEAD swing concrete<br>atom 1,372,769,928 0.0% 18.5% 18.4% 18.4% 18.4% 18.4% 18% 252,589,667<br>
exp3_8 3,500,231,456 0.0% 68.2% 68.2% 68.2% 68.2% 68.2% 68% 2,387,157,853<br>integrate 1,092,339,840 0.0% -12.3% -13.8% -57.3% 39.0% -11.0% 46% 505,753,346<br>lcss 1,272,240,016 0.0% 22.5% 22.5% 22.5% 22.5% 22.2% 22% 282,437,284<br>
mandel 244,936,399 0.0% -9.8% 15.9% 24.1% 24.1% 24.0% 34% 82,788,503<br>primes 861,817,344 0.0% 57.9% 57.9% 57.9% 57.9% 57.9% 58% 498,992,242<br>rfib 78,240 4.2% 48.6% 49.7% 48.0% 48.8% 48.2% 48% 37,712<br>
tak 91,304 3.4% 24.7% 24.2% 18.7% 15.8% 14.8% 15% 13,513<br>wave4main 151,494,392 0.0% 20.3% 22.5% 22.5% 22.5% 20.3% 20% 30,753,362<br>wheel-sieve1 24,131,096 0.0% -1.6% -1.6% -1.6% 99.2% 99.2% 101% 24,324,145</font><br>
</div><div><div>7.0 to 7.2 and 7.4 to 7.6 seem to be the major players.</div><div> </div><div>Indicated allocation regressions that seem to still affect HEAD:</div><div><font face="courier new,monospace"><br> 7.0.1 7.0.2 7.0.3 7.0.4 7.2.1 7.2.2 7.4.1 7.4.2 7.6.1 7.6.2 HEAD swing concrete<br>
ansi 1.66 0.9% 0.9% -1.0% -0.3% 0.9% -14.1% -16.7% -1.1% 0.5% 5.6% 22.3% 0.37<br>atom 2.83 1.2% 1.4% -0.2% 8.7% 8.0% 11.3% 9.2% 11.5% 8.7% 9.7% 9.9% 0.28<br>
exp3_8 3.40 0.1% 0.1% -0.9% 19.2% 19.7% 20.6% 23.8% 20.3% 20.4% 19.5% 20.4% 0.69<br>genfft 0.90 3.1% 3.1% 2.9% 3.1% 2.0% 5.8% 4.5% 3.0% -2.1% 7.4% 9.5% 0.09<br>
hpg 0.28 0.4% 0.1% -34.3% 4.0% -8.6% -11.8% -52.1% -46.0% -51.2% -16.8% 35.3% 0.10<br>integrate 0.74 -12.2% -12.2% -20.7% -17.0% -14.7% -14.4% -66.7% 58.4% 63.4% -14.5% 52.2% 0.39<br>
knights 1.82 0.2% 0.0% 1.0% 2.8% -3.6% 1.5% 9.0% -7.0% 0.9% 5.1% 12.1% 0.22<br>lcss 2.81 -0.1% -0.5% -6.0% 6.1% 6.3% 6.4% -0.1% 0.0% -0.4% 24.2% 30.2% 0.85<br>
maillist 0.40 -0.5% 0.7% -63.1% 9.5% 9.7% -10.1% -76.2% -74.0% -75.1% -12.6% 63.6% 0.25<br>mandel 0.24 -2.6% -2.4% -1.8% -9.7% -11.3% 3.6% 8.3% 10.9% 14.5% 10.4% 21.7% 0.05<br>
wang 1.00 1.4% 1.2% -3.3% 1.0% 2.0% 1.9% -3.7% -1.3% -1.5% 7.0% 10.7% 0.11</font></div></div><div> </div><div>More noisy, of course :( The major players from allocation show up again. One new difference is 7.6 -> HEAD, where we seem to lose ~50% in hpg and maillist.</div>
<div> </div><div>I am a bit worried because I see no *huge* swings like Johan reported. Here's an example excerpt from the NoFib makefile's log when building the programs:</div><div> </div><div><font face="courier new,monospace">HC_OPTS = -H32m -O -O -Rghc-timing -package array -H32m -hisuf hi -hisuf hi -hcsuf hc -osuf o -package-db /home/t-nicof/installs/ghc-7.7.20130206/lib/ghc-7.7.20130206/package.conf.d -O2 -rtsopts<br>
</font></div><div>Does anything look particularly suspicious there? (I used -package-db just to be safe; haven't ran multiple GHC versions too often.)</div><div> </div><div>I might investigate some of these myself; it depends on how we allocate my internship time.</div>
<div> </div><div>Please feel free to look into what's happening in some of these! Especially if you have some of the major compiler changes between versions in your head... :)</div><div> </div><div>Thanks. HTH.<br></div>
<div class="gmail_quote">On Fri, Feb 8, 2013 at 8:41 PM, Nicolas Frisby <span dir="ltr"><<a href="mailto:nicolas.frisby@gmail.com" target="_blank">nicolas.frisby@gmail.com</a>></span> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid">
<div dir="ltr">I've been running 7.0.1 all through HEAD. I'd like to compare my setup with Johan's before making any claims.</div><div>
</div><div class="gmail_quote"><div class="h5">On Feb 8, 2013 10:33 AM, "Bryan O'Sullivan" <<a href="mailto:bos@serpentine.com" target="_blank">bos@serpentine.com</a>> wrote:<br type="attribution"></div>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"><div class="h5">
</div><div class="h5">On Thu, Feb 7, 2013 at 1:18 PM, Johan Tibell <span dir="ltr"><<a href="mailto:johan.tibell@gmail.com" target="_blank">johan.tibell@gmail.com</a>></span> wrote:<br></div><div class="h5">
</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"><div>
</div><div>I just ran nofib on current HEAD and compared it to 7.6.2 on my 64-bit Linux machine. There are some regressions I think we should look into before a release:</div><div></div></blockquote></div><div class="h5">
<br></div><div class="h5">It would be interesting to see the numbers compared to 7.6.1, if possible, as I've seen some big performance regressions between 7.6.1 and 7.6.2.
<br></div><div class="im">_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/ghc-devs" target="_blank">http://www.haskell.org/mailman/listinfo/ghc-devs</a><br>
<br></div></blockquote></div><div>
</div></blockquote><div class="gmail_extra"><br></div><p> </p><div> </div></div>