Oh right. Thanks for pointing out. :)<br><br><div class="gmail_quote">On Wed, Aug 5, 2009 at 10:06 AM, Don Stewart <span dir="ltr"><<a href="mailto:dons@galois.com">dons@galois.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
haskellmail:<br>
<div class="im">> Hi all,<br>
><br>
> I've recently came across a problem when processing a large text file (around<br>
> 2G in size).<br>
><br>
> I wrote a Haskell program to count the number of lines in the file.<br>
><br>
><br>
> module Main where<br>
><br>
> import System<br>
> import qualified Data.ByteString.Char8 as S<br>
> -- import Prelude as S<br>
><br>
> main :: IO ()<br>
> main = do { args <- getArgs<br>
> ; case args of<br>
> { [ filename ] -><br>
> do { content <- S.readFile filename<br>
> ; let lns = S.lines content<br>
> ; putStrLn (show $ length lns)<br>
> }<br>
> ; _ -> error "Usage : Wc <file>"<br>
> }<br>
> }<br>
><br>
><br>
> I get this error, if I use the ByteString module,<br>
> ./Wc a.out<br>
> Wc: {handle: a.out}: hGetBuf: invalid argument (illegal buffer size<br>
> (-1909953139))<br>
> Otherwise, it returns me the result.<br>
><br>
> Another observation is that if I reduce the size of the file, the ByteString<br>
> version works too.<br>
><br>
> Is it a known limitation?<br>
><br>
<br>
</div>Yes, you need to use Data.ByteString.Lazy.Char8 to process files larger<br>
than this on a 32 bit machine (you'll have more space on a 64 bit<br>
machine).<br>
<font color="#888888"><br>
-- Don<br>
</font></blockquote></div><br>