Perf tests which are better than expected on perf builds

Edward Z. Yang ezyang at MIT.EDU
Sat Jul 20 20:26:10 CEST 2013


These tests have been doing better than expected in the nightlies
for some while.

> Unexpected failures:
>    perf/compiler  T3064 [stat too good] (normal)
>    perf/compiler  T3294 [stat too good] (normal)
>    perf/compiler  T5642 [stat too good] (normal)
>    perf/haddock   haddock.Cabal [stat too good] (normal)
>    perf/haddock   haddock.base [stat too good] (normal)

Unfortunately, fixing them is not a simple matter of shifting
the ranges up, since the tests only exceed expectations on
a /perf/ build, so on a normal build such as 'quick', these
tests all pass normally.

I could bump up the upper bounds so that the builder stops bleating
about them; perhaps we could do something more complicated where the
expected performance depends on what level of optimization GHC was built
with (but I don't know how to implement this.)

Thoughts?

Cheers,
Edward




More information about the ghc-devs mailing list