<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.Code, li.Code, div.Code
{mso-style-name:Code;
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New","serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1585265065;
mso-list-type:hybrid;
mso-list-template-ids:-1258111500 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New","serif";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New","serif";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New","serif";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">It sounds as if there are two issues here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Symbol;mso-fareast-language:EN-US"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><b><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">Should GHC unpack a !’d constructor argument if the constructor’s argument has a lot of fields?
</span></b><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">It probably isn’t profitable to unbox very large products, because it doesn’t save much allocation, and might
<i>cause</i> extra allocation at pattern-match sites. So I think the answer is yes. I’ll open a ticket.<b><o:p></o:p></b></span></p>
<p class="MsoListParagraph"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Symbol;mso-fareast-language:EN-US"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><b><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">Is some library (binary? blaze?) creating far too much code in some circumstances?</span></b><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">
I have no idea about this, but it sounds fishy. Simply creating the large worker function should not make things go bad.<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="margin-left:0cm"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">Incidentally, John, using {-# NOUNPACK #-} !Bar would prevent the unpacking while still allowing the field to be strict. It’s manually
controllable.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:0cm"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US">Simon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""> ghc-devs [mailto:ghc-devs-bounces@haskell.org]
<b>On Behalf Of </b>John Lato<br>
<b>Sent:</b> 01 October 2014 00:45<br>
<b>To:</b> Edward Z. Yang<br>
<b>Cc:</b> Joachim Breitner; ghc-devs@haskell.org<br>
<b>Subject:</b> Re: Build time regressions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Edward,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This is possibly unrelated, but the setup seems almost identical to a very similar problem we had in some code, i.e. very long compile times (6+ minutes for 1 module) and excessive memory usage when compiling generic serialization instances
for some data structures.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In our case, I also thought that INLINE functions were the cause of the problem, but it turns out they were not. We had a nested data structure, e.g.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">> data Foo { fooBar :: !Bar, ... }<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">with Bar very large (~150 records).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">even when we explicitly NOINLINE'd the function that serialized Bar, GHC still created a very large helper function of the form:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">> serialize_foo :: Int# -> Int# -> ...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">where the arguments were the unboxed fields of the Bar structure, along with the other fields within Foo. It appears that even though the serialization function was NOINLINE'd, it simply created a Builder, and while combining the Builder's
ghc saw the full structure. Our serializer uses blaze, but perhaps Binary's builder is similar enough the same thing could happen.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Anyway, in our case the fix was to simply remove the bang pattern from the 'fooBar' record field. Then the serialize_foo function takes a Bar as an argument and serializes that. I'm not entirely sure why compilation takes so much longer
otherwise. I've tried dumping the output of each simplifier phase and it clearly gets stuck at a certain point, but I didn't really debug in much detail so I don't recall the details.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If you think this is related, I can investigate more thoroughly.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">John L.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Oct 1, 2014 at 4:54 AM, Edward Z. Yang <<a href="mailto:ezyang@mit.edu" target="_blank">ezyang@mit.edu</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Hello Joachim,<br>
<br>
This was halfway known, but it sounds like we haven't solved<br>
it completely.<br>
<br>
The beginning of the sordid tale was when Cabal HEAD switched<br>
to using derived binary instances:<br>
<a href="https://ghc.haskell.org/trac/ghc/ticket/9583" target="_blank">https://ghc.haskell.org/trac/ghc/ticket/9583</a><br>
<br>
SPJ fixed the infinite loop bug in the simplifier, but apparently<br>
the deriving binary generates a lot of code, meaning a lot of<br>
memory. <a href="https://ghc.haskell.org/trac/ghc/ticket/9630" target="_blank">https://ghc.haskell.org/trac/ghc/ticket/9630</a><br>
hvr's fix was specifically to solve this problem.<br>
<br>
But it sounds like it didn't eliminate the regression entirely?<br>
If there's an unrelated regression, we should suss it out. It would<br>
be helpful if someone could revert just the deriving changes,<br>
and see if this reverts the compilation time.<br>
<br>
Edward<br>
<br>
Excerpts from Joachim Breitner's message of 2014-09-30 13:36:27 -0700:<br>
> Hi,<br>
><br>
> the attached graph shows a noticable increase in build time caused by<br>
><br>
> Update Cabal submodule & ghc-pkg to use new module re-export types<br>
> author Edward Z. Yang <<a href="mailto:ezyang@cs.stanford.edu">ezyang@cs.stanford.edu</a>><br>
> <a href="https://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176f0ae" target="_blank">
https://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176f0ae</a><br>
><br>
> and only halfway mitigated by<br>
><br>
> Update `binary` submodule in an attempt to address #9630<br>
> author Herbert Valerio Riedel <<a href="mailto:hvr@gnu.org">hvr@gnu.org</a>><br>
> <a href="https://git.haskell.org/ghc.git/commit/3ecca02516af5de803e4ff667c8c969c5bffb35f" target="_blank">
https://git.haskell.org/ghc.git/commit/3ecca02516af5de803e4ff667c8c969c5bffb35f</a><br>
><br>
><br>
> I am not sure if the improvement is related to the regression, but in<br>
> any case: Edward, was such an increase expected by you? If not, can you<br>
> explain it? Can it be avoided?<br>
><br>
> Or maybe Cabal just became much larger... +38% in allocations when<br>
> running haddock on it seems to confirm this.<br>
><br>
> Greetings,<br>
> Joachim<br>
><br>
_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/ghc-devs" target="_blank">http://www.haskell.org/mailman/listinfo/ghc-devs</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>