<div dir="ltr">unfortunately read file tries to get the file size<div><br></div><div><pre style="color:rgb(0,0,0)"><span class="">readFile</span> <span class="" style="color:red">::</span> <span class="">FilePath</span> <span class="" style="color:red">-></span> <span class="">IO</span> <span class="">ByteString</span>
<a name="line-1030"></a><span class="">readFile</span> <span class="">f</span> <span class="" style="color:red">=</span> <span class="">bracket</span> <span class="" style="color:red">(</span><span class="">openFile</span> <span class="">f</span> <span class="">ReadMode</span><span class="" style="color:red">)</span> <span class="">hClose</span>
<a name="line-1031"></a> <span class="" style="color:red">(</span><span class="" style="color:red">\</span><span class="">h</span> <span class="" style="color:red">-></span> <span class="">hFileSize</span> <span class="">h</span> <span class="">>>=</span> <span class="">hGet</span> <span class="">h</span> <span class="">.</span> <span class="">fromIntegral</span><span class="" style="color:red">)</span></pre>
<pre style="color:rgb(0,0,0)"><span class="" style="color:red"><br></span></pre><pre style="color:rgb(0,0,0)"><span class="" style="color:red">which wont work on a special file, like a socket. which is what i am trying to simulate here.</span></pre>
</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 16, 2013 at 11:28 AM, Clark Gaebel <span dir="ltr"><<a href="mailto:cg.wowus.cg@gmail.com" target="_blank">cg.wowus.cg@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">See the comment for hGetContents:<div><br></div><div>"<span style="line-height:18px;font-size:13px;font-family:sans-serif">This function reads chunks at a time, doubling the chunksize on each read. The final buffer is then realloced to the appropriate size. For files > half of available memory, this may lead to memory exhaustion. Consider using </span><code style="line-height:16px;font-size:13px;margin:0px;padding:0px"><a href="http://hackage.haskell.org/packages/archive/bytestring/0.9.2.1/doc/html/Data-ByteString.html#v:readFile" style="margin:0px;padding:0px;text-decoration:none;color:rgb(171,105,84)" target="_blank">readFile</a></code><span style="line-height:18px;font-size:13px;font-family:sans-serif"> in this case."</span></div>
<div><br></div><div><a href="http://hackage.haskell.org/packages/archive/bytestring/0.9.2.1/doc/html/Data-ByteString-Char8.html#g:31" target="_blank">http://hackage.haskell.org/packages/archive/bytestring/0.9.2.1/doc/html/Data-ByteString-Char8.html#g:31</a></div>
<div><br></div><div>Maybe try lazy bytestrings?</div><span class="HOEnZb"><font color="#888888"><div><br></div><div> - Clark</div></font></span><div class="HOEnZb"><div class="h5"><div><br>On Tuesday, April 16, 2013, Anatoly Yakovenko wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>-- So why does this code run out of memory?</div><div><br></div><div>import Control.DeepSeq</div><div>import System.IO</div><div>import qualified Data.ByteString.Char8 as BS</div><div><br></div>
<div>scanl' :: NFData a => (a -> b -> a) -> a -> [b] -> [a]</div><div>scanl' f q ls = q : (case ls of</div><div> [] -> []</div><div> x:xs -> let q' = f q x</div>
<div> in q' `deepseq` scanl' f q' xs)</div><div><br></div><div><br></div><div>main = do</div><div> file <- openBinaryFile "/dev/zero" ReadMode</div><div> chars <- BS.hGetContents file</div>
<div> let rv = drop 100000000000 $ scanl' (+) 0 $ map fromEnum $ BS.unpack chars</div><div> print (head rv)</div><div><br></div><div>-- my scanl' implementation seems to do the right thing, because </div><div>
<br></div><div>main = print $ last $ scanl' (+) (0::Int) [0..]<br></div><div><br></div><div>-- runs without blowing up. so am i creating a some thunk here? or is hGetContents storing values? any way to get the exception handler to print a trace of what caused the allocation?</div>
<div><br></div></div>
</blockquote></div>
</div></div></blockquote></div><br></div>