[Haskell-cafe] Space leak - help needed
bertram.felgenhauer at googlemail.com
Sun Mar 2 23:20:09 EST 2008
Krzysztof Kościuszkiewicz wrote:
> Another story from an (almost) happy Haskell user that finds himself
> overwhelmed by laziness/space leaks.
> I'm trying to parse a large file (>600MB) with a single S-expression
> like structure. With the help of ByteStrings I'm down to 4min processing
> time in constant space. However, when I try to wrap the parse results
> in a data structure, the heap blows up - even though I never actually
> inspect the structure being built! This bugs me, so I come here looking
> for answers.
Note that Parsec has to parse the whole file before it can decide
whether to return a result (Left _) or an error (Right _). ghc would
have to be quite smart to eliminate the creation of the expression
The polyparse library (http://www.cs.york.ac.uk/fp/polyparse/)
offers some lazy parsers, maybe one of those fits your needs.
Text.ParserCombinators.Poly.StateLazy is the obvious candidate.
More information about the Haskell-Cafe