[commit: nofib] master: Move benchmarks into benchmark/ subdir. (0449cb0)

[Simon Marlow]

On 19/01/2012 20:27, David Terei wrote:
> So I backed out the path changes.

Great, thanks.  I don't want to stand in the way of progress here, I 
completely agree that nofib needs to be overhauled, but I want to be 
sure that we're all on board with the direction first, and we should 
have a heads-up before any major changes that may break existing 
infrastructure.

FWIW Simon and I use nofib quite a lot to spot performance regressions. 
  Even in its current state it is quite useful for that.

> Can we get the 'GHC Build Reports'
> build bot fixed to publish logs please?

I'll look into it.

> I have no idea why nofib is
> failing as I can view the logs. So can you let me know why its failing
> but more importantly can we get the log uploads fixed.
>
> I want to make nofib more useful. I've created a ticket to track this here:
>
> http://hackage.haskell.org/trac/ghc/ticket/5793

I'll add some of my thoughts to the ticket.

> Basically though the recent changes were about cleaning. So removed
> dead code. And a lot of work to get all the benchmarks from Fibon
> working. As part of this I created a new top level directory
> "benchmarks" and put all the benchmarks under it.
>
> We now have the following benchmark groups:
>
> - imaginary
> - spectral
> - real
>
> - gc
> - hackage
> - parallel
> - repa
> - shootout
> - smp
>
> I've left the top 3 on be default as before for now but once I've done
> some more work would like to enable all of these by default.

Ok, but let's keep an eye on the runtime.  Note that I'm doing 4-5 nofib 
runs on each of the nightly builders right now, if we go wild and 
multiply the time for each run then my nightly builds could easily be 
still going the next day.

Cheers,
	Simon


> Cheers,
> David
>
> On 19 January 2012 00:49, Simon Marlow<marlowsd at gmail.com>  wrote:
>> On 18/01/2012 19:16, David Terei wrote:
>>>
>>> On 18 January 2012 05:03, Simon Marlow<marlowsd at gmail.com>    wrote:
>>>>
>>>> On 17/01/2012 19:13, David Terei wrote:
>>>>>
>>>>>
>>>>> Repository : ssh://darcs.haskell.org//srv/darcs/nofib
>>>>>
>>>>> On branch  : master
>>>>>
>>>>>
>>>>>
>>>>> http://hackage.haskell.org/trac/ghc/changeset/0449cb065437fc8014b6669e5f1c2c8f4a926d16
>>>>>
>>>>>> ---------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> commit 0449cb065437fc8014b6669e5f1c2c8f4a926d16
>>>>> Author: David Terei<davidterei at gmail.com>
>>>>> Date:   Tue Jan 17 10:53:12 2012 -0800
>>>>>
>>>>>      Move benchmarks into benchmark/ subdir.
>>>>>
>>>>
>>>> Has there been any discussion of this?  You do know we're using nofib
>>>> quite
>>>> actively, and this is likely to break scripts etc., right?
>>>>
>>>
>>> No sorry. I wasn't aware of any such things, nofib has always seemed
>>> quite unmaintained to me but I guess that impression is wrong? Should
>>> I revert or are the scripts fine.
>>
>>
>> The nofib runs in the nightly builds failed.  Could you back this out
>> please, and then let's discuss what we want to do.
>>
>> Cheers,
>>         Simon