I have the following code, which reads several thousand text files and creates some Map data. On Windows XP, it crapped out with the error "resource exhausted: too many open files." So I'm sure I need to make it stricter. I've never tried to do that before, so I'm not sure where the strictness is necessary. Maybe all I need is to switch to strict ByteStrings?<br>
<br>-- Read a bunch of text files. The files are organized in pairs, where the file names in each pair are<br>-- "a<number>.txt" and "f<fnumber>.txt". Example: "a00001.txt" and "f00001.txt".<br>
-- Each file is a list of float values specified in a certain text format (which happens to be <br>-- an output format of the audio-related program Csound)<br>--<br>-- Input:<br>-- [(String,String)] : a list of pairs of file names<br>
-- String : directory in which the files exist<br>readPvsFiles :: [(String,String)] -> String -> IO (Map Int (Map Float Float))<br>readPvsFiles filenames dir = do<br> -- contents :: [(Int,Map Float Float)]<br>
-- where Int is the number in the file name. <br> -- and Map Float Float is a map of numbers in the<br> -- in the first file mapped to numbers in the second file<br> contents <- mapM (oneFilePair dir) filenames<br>
return $ M.fromList contents<br><br>-- Read one file pair.<br>-- <br>-- Input:<br>-- String : directory of the files<br>-- (String,String) : the two file names in the pair<br>-- (Int,Map Float Float) : <br> -- Int is the number in the file name. <br>
-- and Map Float Float is a map of numbers in the<br>
-- in the first file mapped to numbers in the second file<br>
oneFilePair :: String -> (String,String) -> IO (Int,Map Float Float)<br>oneFilePair dir (ffile,afile) = do<br> -- Read the float values from each file (which is a list of floats in<br> -- a text format.<br> fvalues <- readTableValues (dir ++ "/" ++ ffile)<br>
avalues <- readTableValues (dir ++ "/" ++ afile)<br> -- t is the number in the filename.<br> let t = read . take 6 . drop 1 $ ffile<br> return (t, M.fromList $ zip fvalues avalues)<br><br>-- Open the file via readFile, and parse all the text values in it.<br>
readTableValues :: String -> IO [Float]<br>readTableValues s = do<br> b <- readFile s<br> let bs = lines b<br> lenLine = head . drop 1 $ bs<br> n = read (drop 6 lenLine) :: Int<br> -- valueLines :: [String]. This is a list of the lines<br>
-- in the file that have the float values. One float<br> -- value in text format per line.<br> valueLines = take n . drop 25 $ bs<br> return $ map read valueLines<br><br>