[Haskell-cafe] Processing of large files
ketil+haskell at ii.uib.no
Mon Nov 1 14:48:44 EST 2004
"Alexander N. Kogan" <alexander at kogan.nnov.ru> writes:
> How should I modify it to make it useful on large file?
> It eats too much memory...
> procFile =
> putStrLn .
> show .
> foldl merge  .
foldl is infamous for building the complete list, before evaluating
anything. Did you try foldr or foldl' instead? Also, you may want to
make 'merge' more strict; I suspect you build lazy tuples that look like
("word",1+1+1+1+...) and only get evaluated at the end.
Using heap profiling will probably give you some hints (well
documented in the GHC manual, ask if you get stuck)
If I haven't seen further, it is by standing in the footprints of giants
More information about the Haskell-Cafe