heap profiling

[Jesper Louis Andersen]

On Wed, Jun 16, 2010 at 2:52 AM, Evan Laforge <qdunkan at gmail.com> wrote:

> -s stats say GC time is 46%, productivity is 32%.  That's pretty bad,
> right?  And where is the remaining 22% going?

That is indeed awfully bad. First, you should look out for the case
where you create a lot of data and then quickly consume it afterwards,
because it increases GC pressure. You are spending 2/3 of your time
doing something else than running the mutator (i.e., the program). For
biographical profiling, it might take some time in the profiling
areas, so look at the counts in -s which tells you the elapsed time in
the profiling innards.

You should take a look at the retainer profiling as a common problem
I've run into many times is lazy thunks retaining a lot of memory. If
it is retainers, you either need to make your code strict or annotate
it as such.

Finally, you can tune the eden/nursery generation by providing +RTS
-A5m for instance. The default is 0.5 megabytes which I think is way
too low for most work. Increasing it blindly will hurt though as you
will hit L1/L2 cache limits and get worse performance.

My application has had problems with both retainer profiling and
(especially) biographical profiling - but whenever I try to make a
watered-down reproducible example the problem disappears. If you do
manage the get a reproducible bug going, I am interested in that
ticket.

> So, the main question:
>
> I have a program that runs some computation in a monad stack before
> extracting the final result, a list, and passing it on.  When run
> under the heap profiler, there's a huge spike in that phase, which I
> think should be mostly bogus usage, since the final output is so
> relatively small.  When I run under -hb I see big bands for LAG and
> DRAG.

I would look at retainers. Or if you hold on to things you don't use
anymore rather than dropping the reference. In general, the heap
profiler is the way to go for removing space leak problems -

-- 
J.