[commit: testsuite] master: Fix remaining test failures on OS X/x86_64 (bea1187)

[Simon Marlow]

On 05/08/2011 13:35, Daniel Fischer wrote:
> On Friday 05 August 2011, 13:51:32, Manuel M T Chakravarty wrote:
>>
>> Yes, I used standard validate settings.  Judging by the timestamps, the
>> Linux numbers are from a while ago.  They may be closer to the (old)
>> limit than they used to be when the limits were set.
>
> In particular the "bytes allocated" have been close to the upper limit for
> some time on my linux boxes.
>
>>
>> Otherwise, I have no idea where the difference comes from.  However, as
>> far as I can remember, the tests never passed on OS X/x86_64, so I
>> don't think it is a recent change.
>>
>
>>>> @@ -27,10 +27,11 @@ test('T1969',
>
>>>>         if_wordsize(64,
>>>>
>>>>             compiler_stats_num_field('bytes allocated', 420000000,
>>>>
>>>> -                                                      550000000)),
>>>> +                                                      580000000)),
>>>>
>
> T1969 stats from a HEAD built yesterday afternoon:
>
>   [("bytes allocated", "537023352")
>   ,("num_GCs", "953")
>   ,("average_bytes_used", "5471552")
>   ,("max_bytes_used", "10244776")
>   ,("num_byte_usage_samples", "9")
>   ,("peak_megabytes_allocated", "28")
>   ,("init_cpu_seconds", "0.00")
>   ,("init_wall_seconds", "0.00")
>   ,("mutator_cpu_seconds", "0.43")
>   ,("mutator_wall_seconds", "0.84")
>   ,("GC_cpu_seconds", "0.25")
>   ,("GC_wall_seconds", "0.25")
>   ]
>
> So bytes allocated was rather close to the limit and is still closer to the
> new upper limit than the lower.
>
>>>>                                       # 17/11/2009:     434,845,560
>>>>                                       (amd64/Linux) # 08/12/2009:
>>>>                                       459,776,680 (amd64/Linux) #
>>>>                                       17/05/2010:     519,377,728
>>>>                                       (amd64/Linux)
>>>>
>>>> +                                    # 05/08/2011:     561,382,568
>>>> (amd64/OS X)
>
>>>>
>>>> @@ -79,16 +80,18 @@ test('T4801',
>>>>
>
>>>>
>>>>         # expected value: 66 (amd64/Linux):
>>>> +      # expected value: 72 (amd64/OS X):
>>>>         if_wordsize(64,
>>>>
>
>>>> -          compiler_stats_num_field('peak_megabytes_allocated', 55,
>>>> 70)), +
>>>> compiler_stats_num_field('peak_megabytes_allocated', 55, 75)),
>
>
>>>>
>>>> +      # expected value: 813714032 (amd64/OS X):
>>>>         if_wordsize(64,
>>>>
>>>>             compiler_stats_num_field('bytes allocated', 600000000,
>>>>
>>>> -                                                      750000000)),
>>>> +                                                      850000000)),
>>>>
>
> T4801 stats:
>
>   [("bytes allocated", "723868224")
>   ,("num_GCs", "1299")
>   ,("average_bytes_used", "10334167")
>   ,("max_bytes_used", "21496744")
>   ,("num_byte_usage_samples", "12")
>   ,("peak_megabytes_allocated", "61")
>   ,("init_cpu_seconds", "0.00")
>   ,("init_wall_seconds", "0.00")
>   ,("mutator_cpu_seconds", "0.45")
>   ,("mutator_wall_seconds", "0.67")
>   ,("GC_cpu_seconds", "0.51")
>   ,("GC_wall_seconds", "0.51")
>   ]
>
> bytes allocated again close to the (old) upper limit, but
> peak_megabytes_allocated was comfortably within the old range.

I get almost identical results on my Linux box here.  Ok, so to 
summarise the T4801 results:

  peak MB:     Linux; 61
               OS X : 72

  allocations: Linux: 72GB
               OS X:  81GB

those differences are large enough to be a concern, I think (>10%).

There are no differences in the RTS that would cause extra allocation on 
OS X.  It seems to be just the compiler tests that need more memory - so 
something in GHC maybe?

I realise it's hard to investigate these things.  But at least these 
tests are giving us the clues that something might be wrong.  If someone 
with time on their hands and access to both OS X and Linux could do some 
testing with identical GHC versions and build settings to help narrow 
down where the differences might lie, that would be useful.  I'd start 
with some simple benchmarks in nofib.

Cheers,
	Simon