[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite
afalloon at synopsys.com
Fri Jan 26 10:54:03 EST 2007
David Roundy wrote:
> On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote:
>> Kenneth Hoste wrote:
>>> The idea is to gather a bunch of programs written in Haskell, and which
>>> are representative for the Haskell community (i.e. apps, libraries,
>> A While ago I tried to write a Haskell version of John Harrops
>> ray-tracer benchmark
>> (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the
>> performance was not very good (the OCaml version I based it on was at
>> least 10x faster).
>> I would be happy to contribute my code to the benchmark suite if you are
>> interested. Perhaps someone can point out obvious speed-ups that I
>> missed while trying to improve the performance.
> I would think that what we'd want to benchmark would be clean, optimized
> actually-used code. I.e. things like Data.Bytestring, so that we could see
> how compilers differed on important code, or how the code generated on
> different architectures differed. e.g. if jhc beats ghc on amd64, the ghc
> developers would probably be very curious as to why, and how to fix it.
> Code that's not been properly optimized with respect to strictness, etc,
> would fail to focus the tests on important optimizations of the compiler.
> But of course, the benchmark code should also be clean, since we want to
> ensure that our compilers are good enough that we can write useful
> beautiful code that is also fast.
I tried to optimize it, but I couldn't approach the speed of the OCaml
version. I followed the performance tuning advice from the Wiki, and had
even resorted to writing the inner loop calculations using all unboxed
doubles, but without significant improvements. This is exactly the kind
of code that I write most often, and I would love to see improvements in
the optimizations for this kind of numerically intensive code
(especially without having to resort to compiler-specific unboxed
I agree that common libraries like ByteString need to be well
represented, but the original request additionally included programs
that are representative of applications. A ray-tracer (even with a fixed
scene and only one type of scene primitive) is a fairly nice
approximation of a real numerical batch-oriented application while still
being small enough to understand and modify. I expect thats why John
chose it as his benchmark application in the first place.
I think it would also be a good idea to include SPJ's web server (I
think its from an STM paper). A lot of the people outside the Haskell
community will be able to relate better to metrics like pages/second.
More information about the Haskell-Cafe