[Haskell-cafe] SMP parallelism increasing GC time dramatically

[Tom Thorne]

The speedup is around 6 times on a 12 core machine, which I think is pretty
decent given that the parallelised section is only a part of my code. The
nested parMaps were left over from a previous implementation, I have moved
to using just the inner one, since the outer map doesn't divide the work
equally and causes one thread to do most of the work.

On Mon, Oct 10, 2011 at 4:04 PM, Christopher Brown
<cmb21 at st-andrews.ac.uk>wrote:

> What kinds of speedup are you getting?
>
> 422000 is a lot of sparks, could be due to the fact you are nesting your
> par maps (why do you do this?)
>
> Chris.
>
>
>
> On 10 Oct 2011, at 15:44, Tom Thorne wrote:
>
> thanks! I just tried setting -A32M and this seems to fix the parallel GC
> problems, I now get a speedup with parallel GC on and performance is the
> same as passing -qg. I had tried -H before and it only made things worse,
> but -A seems to do the trick.
>
> I'm still having problems with segmentation faults though. Depending on how
> I apply parMap, and whether I use monad-par or control.parallel, they seem
> to come and go arbitrarily. In a successful run that lasted about 30s in
> total with control.parallel, +RTS -s reports:
> SPARKS: 422712 (394377 converted, 0 pruned)
>
> am I creating too many sparks?
>
> On Mon, Oct 10, 2011 at 3:07 PM, Gregory Collins <greg at gregorycollins.net>wrote:
>
>> On Mon, Oct 10, 2011 at 3:55 PM, Tom Thorne <thomas.thorne21 at gmail.com>
>> wrote:
>> >
>> > Yes I will try to run threadscope on it, I tried it before and the event
>> log output produced about 1.8GB, and then crashed.
>> > Is there any way to tell the RTS to perform GC less often? My code
>> doesn't use too much memory and I'm using fairly hefty machines (e.g one
>> with 48 cores and 128GB of RAM) and so perhaps the default/heuristic
>> settings aren't optimal.
>>
>> Increasing "-A" and "-H" in the RTS options should help with this.
>>
>> G
>> --
>> Gregory Collins <greg at gregorycollins.net>
>>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20111012/7131d1fe/attachment.htm>