NoFib for benchmarking vs testing
Mon, 17 Dec 2001 16:11:50 +0000
Looking through NoFib some more, I am increasingly getting the impression that
quite a few of the tests aren't particularly useful benchmarks.
Currently NoFib is both a benchmark suite and a test suite.
I think it might be useful to separate those roles out, giving us separate
"test" and "benchmark" modes.
When run in "benchmark" mode, we would ignore tests that don't make good
While this wouldn't involve any significant change to the tests themselves, it
would make it clearer which tests were thought to be useful benchmarks for
people to test optimisations against.
[I don't think it would be particularly useful for me to work out a way for
"awards" to take 10 seconds]