wishlist: testsuite error reporting

[Edward Z. Yang]

This might be useful.  I personally rerun the test suite with make TESTS="failedtest1 failedtest2" fast
or do a find -name FD2* to get to the base directory; I find that the shell buffer is too chatty
to be particularly useful. I will note that, in absence of wanting to show the progress of the
test suite, only emitting output when there is an error is good design (c.f. SimpleTest).

Edward

Excerpts from Dimitrios Vytiniotis's message of Mon Feb 28 06:39:30 -0500 2011:
> Hi, when I am working on a typechecker patch and then run the testsuite, sometimes I get dozens
> of error reports that look like:
> 
> Unexpected failures:
>    FD2(normal)
>    FrozenErrorTests(normal)
>   ... etc
> 
> It would be helpful for me to be able to quickly see in the same line, (i) which directory the
> test is in, (ii) the name of the  3rd parameter of the corresponding all.T entry, and (iii) the exit
> status of ghc when running that test. Something like:
> 
>    FD2                              (normal) (should_compile/) (compile_fail)   (ghc failed)
>    FrozenErrorTests (normal) (should_fail/)            (compile_fail)  (ghc failed)
> 
> For instance, these entries tell me that a program from should_compile/ which was supposed
> to fail still fails, so the problem is likely to be a shuffling in error messages. If I see an entry:
>    MyFile                       (normal) (should_compile/) (compile)  (ghc failed)
> then that's probably very serious.
> 
> All tests deserve attention but this extra information would help me to focus very fast on
> errors that look more important first and also to navigate to these files w/o searching my
> buffer to see where the file is all the time, or what are its all.T parameters.
> 
> Is this something people think is generally useful, or is it already possible, or am I talking
> nonsense here?
> 
> Thanks
> d-