[Haskell-cafe] Why does this program eat RAM?

John Goerzen jgoerzen at complete.org
Mon Sep 4 23:03:51 EDT 2006


I have the below program, and I'm trying to run it on an input of about
90MB.  It eats RAM like crazy, and I can't figure out why.

I do know that the problem is not my custwords function (as you can see,
I replaced the call to it with a call to the standard words function on
the last line).  It seems to be wordfreq, but I don't know why.

For information about the problem, see
http://changelog.complete.org/posts/535-A-Haskell-solution-to-Lars-Problem.html

Here's the code:

import System.Environment
import Data.List
import Data.Char
import qualified Data.Map as Map

custwords = filter (/= "") . lines . map (conv . toLower)
    where iswordchar x = isAlphaNum x && isAscii x
          conv x = if iswordchar x then x else '\n'

wordfreq inp = Map.toList $ foldl' updatemap (Map.empty::Map.Map String Int) inp
    where updatemap nm word = Map.insertWith updatefunc word 1 nm
          updatefunc _ x = x + 1

freqsort (w1, c1) (w2, c2) = if c1 == c2
                                 then compare w1 w2
                                 else compare c2 c1

showit (word, count) = show count ++ " " ++ word
main = do args <- getArgs
          interact $ unlines . map showit . take (read . head $ args) .
                     sortBy freqsort . wordfreq . words




More information about the Haskell-Cafe mailing list