Library tests

From HaskellWiki

Contents 1 Goals 2 Status 3 Design 3.1 Abbreviated QuickCheck in Haddock comments 3.2 #ifdefed tests in code 3.3 Separate test programs

1 Goals

1. Help library developers ensure that their changes work.
2. Enable something like BuildBot to ensure that libraries keep working across all the Implementations.
3. Enable test-driven development if a developer wants to use it.
4. Provide a series of testcases for implementation developers.

2 Status

According to Ian Lynagh, SimonM has been moving tests a directory at a time from the testsuite package that comes with ghc to libraries/foo/tests. It looks like he's gotten to unix and network. The tests "assume the package is part of a GHC build tree with the testsuite installed in ../../../testsuite."

Neil Mitchell has a program at http://www.cs.york.ac.uk/fp/darcs/filepath/GenTests.hs to automatically extract tests from Haddock comments like the ones in http://www.cs.york.ac.uk/fp/darcs/filepath/System/FilePath/Version_0_11.hs.

3 Design

I think the tests should be run with runhaskell Setup.hs test. Cabal's going to need some work to make this work portably. JeffreyYasskin 06:39, 21 March 2007 (UTC)

3.1 Abbreviated QuickCheck in Haddock comments

Need to recognise parameters. GenTests.hs currently recognises a small range of special variables. What about declaring the parameter names and associated types in a header comment? For instance

-- > :type xs, ys :: [Int]

this would declare that xs and ys are free variables in any following tests and of the type specified. They can then be recognised at the lexical level.

Alternatively, what about special "forall" syntax such as:

forall (xs, ys :: [Int]) . reverse xs ++ reverse ys == reverse (ys ++ xs)

GenTests currently has flag syntax for Posix and Windows tests (necessary for FilePath). Should this be in a general solution? If so, are other flags required?

Do we need special syntax for

3.2

#ifdef

ed tests in code

http://darcs.haskell.org/packages/base/Data/Map.hs could easily be moved to this form if we replace the comments with . We'll need to extend Haddock to be able to copy function bodies into the generated documentation.

This poses some problems for arranging the compiled output. The #define flags generally don't show up in the name of the output file, but we want to make sure that even if you've run the tests, runhaskell Setup.hs install installs the non-testing libraries.

We could probably re-build all of the libraries in a separate dist/test/ tree with -DTESTING on. If we have A.hs and B.hs where each exports some implementation functions behind , this gives B's tests access to A's private functions. Is that a problem? JeffreyYasskin 06:39, 21 March 2007 (UTC)

3.3 Separate test programs

Should be in the style of http://darcs.haskell.org/packages/network/tests/, but with a driver that doesn't depend on living inside a ghc source tree.

I personally dislike the "golden file" approach that these test use, but the existing driver also supports a run_command_ignore_output test-fn, which we could combine with the exit_code(0) opt-fn to use any testing framework we want. JeffreyYasskin 06:39, 21 March 2007 (UTC)