[Hackage] #215: Overhaul support for package's tests

[Hackage]

#215: Overhaul support for package's tests
----------------------------+-----------------------------------------------
  Reporter:  duncan         |        Owner:                 
      Type:  task           |       Status:  new            
  Priority:  normal         |    Milestone:                 
 Component:  Cabal library  |      Version:  1.2.3.0        
  Severity:  normal         |   Resolution:                 
  Keywords:                 |   Difficulty:  project(> week)
Ghcversion:  6.8.2          |     Platform:                 
----------------------------+-----------------------------------------------
Comment (by guest):

 > That also means the interface is a Haskell one, not a process text
 stdin/stdout one.
 > That's likely more portable between Haskell implementations and
 crucially it allows
 > us to build the test code with additional profiling or hpc or whatever.

 Really? Is it not possible to build with hpc style stuff if the interface
 is just a command line program? I'm also not overly convinced by the
 argument about portability.

 Why not assume Test-Is: Test, means that Test is a module Main, with main
 in it. You run the tests (perhaps supplying test-arguments: for command
 line arguments), and the error code is the number of tests fail, where
 exitSuccess is equivalent to no failures. The log of the test run could
 also be displayed on hackage.

 Command line programs are simple, and can be written without learning a
 new API. They can also be invoked from outside Cabal. For example, the
 tagsoup program has a tagsoup binary, which has several example programs
 in it. One of those example things is "tagsoup test" which runs the test
 properties. Changing to have Test call system "tagsoup test" would be a
 bit of a shame.

 In general, something does need to be done, but keeping it very simple
 seems like a good idea. The idea of having a test program be an actual
 program with an error code seems to appeal to the Unix ideal of keeping
 things simple and self-contained. I know for certain that Colin would back
 me up on this :-)

 I would also suggest performance testing be given a back seat. In general,
 its hard to do it well, and get meaningful results. A separate speed-check
 tool is being worked on, and that can be done separately from Cabal. If it
 was anyone other than Dons/Duncan having this discussion, no one would
 have considered performance :-)

-- 
Ticket URL: <http://hackage.haskell.org/trac/hackage/ticket/215#comment:3>
Hackage <http://haskell.org/cabal/>
Hackage: Cabal and related projects