On Fri, Dec 19, 2008 at 7:58 AM, Daniel Kraft <span dir="ltr"><d@domob.eu></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br><div class="Ih2E3d"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Not having looked at your code, I think you are benefiting from<br>
fusion/deforestation in the first three main functions. In this case,<br>
although you may appear to be evaluating the entire list, in fact the<br>
list elements can be discarded as they are generated, so functions like<br>
length and reverse can run using constant space, rather than O(n) space.<br>
</blockquote>
<br></div>
How does reverse work in constant space? At the moment I can't imagine it doing so; that's why I tried it, but of course you could be right.</blockquote><div><br>No, you are correct, reverse is not constant space. However, as Duncan explained, reverse does not force any elements of the list, so even if you had a list of elements which consumed 1Mb each (when fully evaluated), they would not be forced so the memory would look exactly the same.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d"><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
The sort function, however, requires that the entire list is retained,<br>
hence greater memory usage. I also think you are optimistic in the<br>
memory requirements of your 3 million element list. A list of Ints will<br>
take a lot more than 4 bytes per element (on 32-bit machines) because<br>
there's overhead for the list pointers, plus possibly the boxes for the<br>
Ints themselves. I think there are 3 machine words for each list entry<br>
(pointer to this element, pointer to next element, info-table pointer),<br>
and 2 words for each Int, but I'm just guessing:<br>
<a href="http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/HeapObje" target="_blank">http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/HeapObje</a><br>
cts<br>
</blockquote>
<br></div>
Of course that's the case, but the list being 3 million elements, and not, say 100 (which would still fit into memory for a simple C array of ints) I thought would make it possible. Otherwise, how can one handle such amounts in data anyway? Only using arrays?</blockquote>
<div><br>Well, if you know the size beforehand and you know the whole thing needs to fit into memory at the same time, an array is usually a better choice than a list. Lists are more like loops -- i.e. control flow rather than data, whereas Arrays are definitely data. I realize the imprecision of that statement...<br>
<br>However, I am not sure what all this jabber about swapping is. 28 bytes/elt * 3,000,000 elts = 84 Mb, which easily fits into a modern machine's memory.<br><br>There are a lot of traps for the unwary in memory performance though. Depending on how things are defined, you may be computing <i>too</i> lazily, building up thunks when you should just be crunching numbers. Still, swapping on this 3,000,000 element list is absurd, and we should look closer into your example. Post the rest (eg. the instances?)?<br>
<br>Luke<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d"><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
You might get some mileage by suggesting to GHC that your Fraction type<br>
is strict e.g.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
data Fraction = Fraction !Int !Int<br>
</blockquote>
<br>
which might persuade it to unbox the Ints, giving some space savings.<br>
</blockquote>
<br></div>
I already tried so, but this doesn't change anything to the performance. I will however try now to use the provided rational type, maybe this helps.<br>
<br>
Thanks for the answers,<br><font color="#888888">
Daniel</font><div><div></div><div class="Wj3C7c"><br>
<br>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/haskell-cafe" target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br>
</div></div></blockquote></div><br>