<div dir="ltr">I haven't looked very closely, but I'm suspicious of this code from "instance Block Piece"<div><br></div><div> ListLike l -> forM l (\obj -> ...)</div><div> >>= (return . mconcat)</div>
<div><br></div><div>The "forM" means that "l" will be traversed once and create an output list, which will then be mconcat'd together. The list has to be created because of the monadic structure imposed by forM, but if the result of the mconcat isn't demanded right away it will be retained as a thunk that references the newly-created list.</div>
<div><br></div><div>I'd suggest that you replace it with something like</div><div><br></div><div> ListLike l -> foldM (\(!acc) obj -> ... >>= return . mappend acc) mempty l</div><div><br></div><div>Here I've justed added a bang pattern to the accumulator. If whatever is being returned has some lazy fields, you may want to change that to use deepseq instead of a bang pattern.</div>
<div><br></div><div>Also, "foo >>= return . bar" is often regarded as a bit of a code smell, it can be replaced with "bar <$> foo" or "bar `liftM` foo", or sometimes something even simpler depending on circumstances (but IMHO sometimes it's more clear to just leave it alone).</div>
<div><br></div><div>The heap profile does look like a space leak. The line</div><div><br></div><div> <StrappedTemplates-0.1.1.0:Text.Strapped.Render.sat_sc1z></div><div><br></div><div>is a thunk (you can tell because it's in '<>' brackets), so whatever is referencing that is not strict enough. Sometimes another heap profile report, e.g. "-hc" or maybe "-hy" will give more useful information that lets you identify what exactly "sat_sc1z" is. You could also try compiling with -ddump-stg, which will dump the intermediate STG output which usually shows those names. But then you'll probably also need to re-run the profile, since the names change between compilations. Also IIRC some of values aren't named until the cmm phase, but that's harder to map back to Haskell so if you can identify the code from stg it's simpler.</div>
<div><br></div><div>If you haven't seen <a href="http://blog.ezyang.com/2011/06/pinpointing-space-leaks-in-big-programs/">http://blog.ezyang.com/2011/06/pinpointing-space-leaks-in-big-programs/</a>, I'd highly recommend it if you need to track down a space leak.</div>
<div><br></div><div>John L.</div><div><br></div><div><div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Aug 7, 2014 at 10:57 AM, Kyle Hanson <span dir="ltr"><<a href="mailto:me@khanson.io" target="_blank">me@khanson.io</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>I was looking at cleaning up my refactoring a core loop of template rendering to go from a loop with many parameters</div>
<div><br></div><div><div>loop :: RenderConfig -> BlockMap -> InputBucket m -> Builder -> [Pieces] -> ExceptT StrapError m Builder</div>
</div><div><br></div><div>to a looped state monad transformer</div><div><br></div><div>loop :: [Pieces] -> RenderT m Builder</div><div><br></div><div><div>newtype RenderT m a = RenderT </div><div> { runRenderT :: ExceptT StrapError (StateT (RenderState m) m) a </div>
<div> } deriving ( Functor, Applicative, Monad, MonadIO )</div>
</div><div><br></div><div><div>data RenderState m = RenderState</div><div> { position :: SourcePos</div><div> , renderConfig :: RenderConfig</div><div> , blocks :: BlockMap</div><div> , bucket :: InputBucket m</div>
<div> }</div></div><div><br></div><div>however, there is a big slow down (about 6-10x) using a StateT. I think it might have something to do with laziness but I am not exactly sure of where to begin in tracking it down. Swapping out the Lazy State to a Strict State helps a little (only a 5x slow down)</div>
<div><br></div><div>You can find some of the processing code here:</div><div><br></div><div><a href="https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c873f188797c0d4f5/src/Text/Strapped/Render.hs#L189" target="_blank">https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c873f188797c0d4f5/src/Text/Strapped/Render.hs#L189</a><br>
</div><div><br></div><div>With my old loop commented out.</div><div><br></div><div>Its messy right now since I am just trying a number of different approaches. I did some more work factoring out the lifts, trying different iterations of foldlM and stuff but that didn't have that much of an effect on performance.</div>
<div><br></div><div>After profiling I see in the StateT, the report has a lot more CAFs and garbage collecting.</div><div><br></div><div>Here is the profiling report from my original version w/o StateT</div><div><a href="http://lpaste.net/108995" target="_blank">http://lpaste.net/108995</a><br>
</div><div><br></div><div>Slow version with StateT</div><div><a href="http://lpaste.net/108997" target="_blank">http://lpaste.net/108997</a><br></div><div><br></div><div>Here is the "makeBucket" function that is referenced (it is the same in both state and nonstate):</div>
<div><br></div><div><a href="https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c873f188797c0d4f5/examples/big_example.hs#L24" target="_blank">https://github.com/hansonkd/StrappedTemplates/blob/321a88168d54943fc217553c873f188797c0d4f5/examples/big_example.hs#L24</a><br>
</div><div><br></div><div>Looking at stacked overflow and the official docs I have gotten an idea of what is going on. The heaps generated between them tells me that a lot more memory is being allocated to lists. These heaps were generated running my render function against a template with nested loops and a list of elements.</div>
<div><br></div><div><a href="http://imgur.com/a/2jOIf" target="_blank">http://imgur.com/a/2jOIf</a></div><div><br></div><div>I am hoping that maybe someone could give me a hint at what to look at next. I've played around with Strictness and refactoring loops to no avail and now am kind of stuck. Any help would be appreciated.</div>
<div><br></div><div>--</div><div>Kyle Hanson</div></div>
<br>_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/haskell-cafe" target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br>
<br></blockquote></div><br></div>