[GHC] #9571: nofib should use criterion-style bootstrapping/sampling

[GHC]

#9571: nofib should use criterion-style bootstrapping/sampling
-------------------------------------+-------------------------------------
       Reporter:  ezyang             |                   Owner:
           Type:  feature request    |                  Status:  new
       Priority:  normal             |               Milestone:
      Component:  NoFib benchmark    |                 Version:  7.9
  suite                              |        Operating System:
       Keywords:                     |  Unknown/Multiple
   Architecture:  Unknown/Multiple   |         Type of failure:
     Difficulty:  Unknown            |  None/Unknown
     Blocked By:                     |               Test Case:
Related Tickets:                     |                Blocking:
                                     |  Differential Revisions:
-------------------------------------+-------------------------------------
 As I discovered when investigating situations like #9570, in some cases,
 test-cases in nofib are giving nonsense, and it's hard to tell unless you
 run nofib several times and notice that percentage differences are
 fluctuating up and down.  The quality of the numbers we get for uninformed
 users would be better if we ran some statistical analysis to tell how many
 times to run the benchmark, and if there were lots of outliers (rather
 than just blindly summarizing all the runs using an average.)

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9571>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler