[Haskell-cafe] Need some advice around lazy IO

[C K Kashyap]

Thanks everyone,

Dan, MapMI worked for me ...

Regards,
Kashyap


On Mon, Mar 18, 2013 at 12:42 AM, Petr Pudlák <petr.mvd at gmail.com> wrote:

> Hi Kashyap,
>
> you could also use iteratees or conduits for a task like that. The beauty
> of such libraries is that they can ensure that a resource is always
> properly disposed of. See this simple example:
> https://gist.github.com/anonymous/5183107
> It prints the first line of each file given as an argument. After each
> line is printed, the `fileConduit` pipe ensures that the handle is closed.
> It also makes the program nicely composable.
>
> Best regards,
> Petr
>
>
> import Control.Monad
> import Control.Monad.Trans.Class
>
> import Control.Monad.IO.Class
> import Data.Conduit
>
> import Data.Conduit.List
> import System.Environment
>
> import System.IO
>
>
> {- | Accept file paths on input, output opened file handle, and ensure that the
>  - handle is always closed after its downstream pipe finishes whatever work on it. -}
>
> fileConduit :: MonadResource m => IOMode -> Conduit FilePath m Handle
>
> fileConduit mode = awaitForever process
>
>   where
>     process file = bracketP (openFile file mode) closeWithMsg yield
>
>     closeWithMsg h = do
>
>         putStrLn "Closing file"
>
>
>         hClose h
>
> {- | Print the first line from each handle on input. Don't care about the handle. -}
>
> firstLine :: MonadIO m => Sink Handle m ()
>
> firstLine = awaitForever (liftIO . (hGetLine >=> putStrLn))
>
>
> main = do
>
>     args <- getArgs
>
>
>     runResourceT $ sourceList args =$= fileConduit ReadMode $$ firstLine
>
>
>
>
> 2013/3/17 C K Kashyap <ckkashyap at gmail.com>
>
>> Hi,
>>
>> I am working on an automation that periodically fetches bug data from our
>> bug tracking system and creates static HTML reports. Things worked fine
>> when the bugs were in the order of 200 or so. Now I am trying to run it
>> against 3000 bugs and suddenly I see things like - too  many open handles,
>> out of memory etc ...
>>
>> Here's the code snippet - http://hpaste.org/84197
>>
>> It's a small snippet and I've put in the comments stating how I run into
>> "out of file handles" or simply file not getting read due to lazy IO.
>>
>> I realize that putting ($!) using a trial/error approach is going to be
>> futile. I'd appreciate some pointers into the tools I could use to get some
>> idea of which expressions are building up huge thunks.
>>
>>
>> Regards,
>> Kashyap
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20130318/8547d59e/attachment.htm>