Simplifier tests (was Re: patch applied (testsuite): Add a new category of "eyeball" tests)

[Simon Marlow]

Simon Peyton-Jones wrote:
> I'm all for this, if Roman and/or Don care to help -- thank you!  A little readme to explain how to add a new test would be good.

The testsuite is well documented on the wiki:

http://hackage.haskell.org/trac/ghc/wiki/Building/RunningTests

> Ian and/or Simon may want to comment on dependencies, or technology (eg Haskell vs Python).

To do this in Haskell would mean adding (back) regex support to the boot 
libraries.  Well, currently you can do a testsuite run with the boot libs, 
in theory the testsuite could have additional dependencies, but it makes 
things simpler if we keep boot deps == testsuite deps.

I'm not completely against adding packages to the boot set, but very extra 
package increases the validation time, which is something we're sensitive to.

I'd be inclined to do this in Python, unless it really is too painful.

Cheers,
	Simon

> Simon
> 
> | -----Original Message-----
> | From: Roman Leshchinskiy [mailto:rl at cse.unsw.edu.au]
> | Sent: 08 February 2008 00:53
> | To: Simon Peyton-Jones
> | Cc: cvs-ghc at haskell.org
> | Subject: Simplifier tests (was Re: patch applied (testsuite): Add a new category of "eyeball" tests)
> |
> | Simon Peyton Jones wrote:
> | > Thu Feb  7 08:22:44 PST 2008  simonpj at microsoft.com
> | >   * Add a new category of "eyeball" tests
> | >
> | >   These tests are hard to do automatically, but they record examples that
> | >   provoked changes to the optimiser.  Each one has notes that says what you
> | >   should expect to see!
> |
> | I think we should try to do them automatically. Usually it's just a
> | matter of dumping the simplifier output and grepping for things. For the
> | tests you added:
> |
> | >     A ./tests/ghc-regress/eyeball/dead1.hs
> |
> | Make sure "bar" doesn't occur in the output of phase 0.
> |
> | >     A ./tests/ghc-regress/eyeball/inline1.hs
> |
> | Make sure "myIndex" doesn't occur in the output of phase 2.
> |
> | >     A ./tests/ghc-regress/eyeball/inline2.hs
> |
> | Count the number of simplifier iterations in -dshow-passes.
> |
> |  >     A ./tests/ghc-regress/eyeball/inline3.hs
> |
> | Count the number of calls to (+#).
> |
> | And so on. I'll try to implement a simple framework. I'm just not sure
> | if it should be in Haskell, Python or shell?
> |
> | Roman
> 
> _______________________________________________
> Cvs-ghc mailing list
> Cvs-ghc at haskell.org
> http://www.haskell.org/mailman/listinfo/cvs-ghc