[Haskell-cafe] Re: [Haskell] [Fwd: Re: Computer Language Shootout]

[Kirsten Chevalier]

On 1/25/07, Neil Mitchell <ndmitchell at gmail.com> wrote:
> We have the nofib suite, it could do with some attention, but it is
> (or was) a pretty good performance tester.
>

Yes, I've spent enough time staring at nofib results that I understand
just how random and un-representative the programs in nofib are. nofib
consists of whatever people have submitted for nofib, and moreover,
its contents are more or less the same now as they were in 2002. A lot
has changed about the ways in which people use Haskell in those past
four years -- in 2002, GHC was the only big Haskell application out
there, and I don't think that's true anymore. So personally, I have
zero faith in nofib results to tell me anything about how helpful
optimizations will be except on a very large scale of granularity.

> The problem is that something like GHC is very complex, with lots of
> transformations. When transformations are firing other
> transformations, which in turn fire other transformations, it doesn't
> take a great deal to disrupt this flow of optimisation and end up with
> a result which doesn't accurately reflect the particular change you
> made. Better knowledge of the end effects on a program isn't
> necessarily going to translate to better knowledge of the
> optimisations effect.

Sorry for being unclear. I agree with your comments on GHC, and one
thing I was suggesting was that somebody should think about profiling
tools for improving our understanding of how those transformations
interact with each other, not just profiling tools for understanding
the end result. Everyone says that optimization is a black art, but I
remain to be convinced that understanding the interactions between
different optimizations isn't a problem that would submit to
appropriate amounts of effort.

>
> Maybe if we had a greater number and variety of optimising compilers
> we'd be able to more freely experiment with optimisation techniques in
> different settings. At the moment GHC is all there is (with Jhc not
> ready for mainstream use yet)
>

I agree that there should be more diversity in compilers, but I think
even sticking to GHC, there's a lot that could be done when it comes
to improving understanding of just what the optimizer is doing.
Anything better than staring at intermediate code would be an
improvement, since time spent staring at intermediate code usually is
time spent narrowing down the 2 lines out of 1000 that are relevant.
Maybe it's possible to design tools that could help with that
narrowing-down process.

Cheers,
Kirsten

--
Kirsten Chevalier* chevalier at alum.wellesley.edu *Often in error, never in doubt
"by God I *KNOW* what this network is for, and you can't have it."--Russ
Allbery