Checkins and test passing

[Simon Marlow]

Malcolm Wallace wrote:
> Ian Lynagh <igloo at earth.li> wrote:
> 
> 
>>I think it would be a nice requirement that for a test to be marked as
>>expected-to-fail there must be an open bug about it.
> 
> 
> Hmm.  Isn't the meaning upside down here?  I thought an "expected
> failure" was a program that is supposed to fail by the rules of the
> language.

The meaning of these terms as we use them here is:

    fail     = "exhibits incorrect behaviour"
    expected = "we know about the bug and don't intend to fix it soon"

the other meaning, that you were expecting, is

    fail     = "exits with a non-zero exit code"
    expected = "exiting with non-zero exit code is the correct behaviour"

We probably aren't consistent with our use of the terminology in the test suite, 
that's something we should fix (eg. there's a compile_fail function which refers 
to the "non-zero exit code" menaing of "fail").

"desired" wouldn't be right - all failures are undesired.

Cheers,
	Simon