<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;"><DIV>--- On Sat, 11/7/09, Don Stewart <<A href="mailto:dons@galois.com">dons@galois.com</A>> wrote:<BR>> General notes:<BR>><BR>> * unpack is almost always wrong.<BR>> * list indexing with !! is almost always wrong.<BR>> * words/lines are often wrong for parsing large files (they build large list structures).<BR>> * toList/fromList probably aren't the best strategy<BR>> * sortBy (comparing snd)<BR>> * use insertWith'</DIV>
<DIV>> Spefically, avoid constructing intermediate lists, when you can process the<BR>> entire file in a single pass. Use O(1) bytestring substring operations like<BR>> take and drop.</DIV>
<DIV> </DIV>
<DIV>Thanks all for the valuable feedback. Switching from Regex.Posix to Regex.PCRE alone reduced the running time to about 6 secs and a few other optimizations suggested on this thread brought it down to about 5 secs ;) </DIV>
<DIV> </DIV>
<DIV>I then set out to profile the code out of curiosity to see where the bulk of the time was being spent and sure enough the culprit turned out to be "unpack". My question therefore is, given a list L1 of type [(ByteString, Int)], how do I print it out so as to eliminate the "chunk, empty" markers associated with a bytestring? The suggestions posted here are along the lines of "mapM_ print L1" but that's far from desirable especially because the generated output is for perusal by non-technical users etc.</DIV>
<DIV> </DIV>
<DIV>Thanks.<BR></DIV></td></tr></table><br>