<div>I recently ran into a problem writing a program that I hoped I could speed up by running over multiple CPUs. I want non-haskell users to be able to run the tool and take advantage of multiple CPUs. *But* there is a serious slowdown when the RTS is run with -N and some of the CPUs are already busy.</div>
<div><br></div><div>This is already mentioned in the GHC docs, but the problem I experienced was more serious than I expected causing a slowdown of around 2x compared to running with a single CPU. </div><div><br></div><div>
I reproduced the problem with the following code from the haskell wiki:</div><div><br></div><div><div>{-# LANGUAGE BangPatterns #-}</div><div> import Data.Digest.Pure.MD5</div><div> import qualified Data.ByteString.Lazy as L</div>
<div> import System.Environment</div><div> import Control.Concurrent</div><div> import Control.Monad (replicateM_)</div><div> </div><div> main = do</div><div> files <- getArgs</div><div> str <- newEmptyMVar</div>
<div> mapM_ (forkIO . hashAndPrint str) files</div><div> printNrResults (length files) str</div><div> </div><div> printNrResults i var = replicateM_ i (takeMVar var >>= putStrLn)</div><div> </div><div>
hashAndPrint str f = do</div><div> bs <- L.readFile f</div><div> let !h = show $ md5 bs</div><div> putMVar str (f ++ ": " ++ h)</div></div><div><br></div><div><br></div><div>When run on 4 idle CPU cores, I get the following wall clock times:</div>
<div><div> ./run +RTS -N1 : 20.4 sec</div><div> ./run +RTS -N2 : 11.0 sec</div><div> ./run +RTS -N4 : 6.7 sec</div></div><div><br></div><div>When run on the same 4 core machine, but with 2 cores already busy:</div>
<div><div> ./run +RTS -N1 : 23.5 sec</div><div> ./run +RTS -N2 : 14.1 sec</div><div> ./run +RTS -N4 : 57.8 sec <---- Blowout...</div></div><div><br></div><div>This is quite a problem in practice when running on a shared server. Is there anything that can be done to address this?</div>
<div><br></div><div>(I wrote up a few more details here: <a href="http://thunking.drp.id.au/2012/06/slowdown-with-ghc-when-using-multiple.html">http://thunking.drp.id.au/2012/06/slowdown-with-ghc-when-using-multiple.html</a>)</div>
<div><br></div><div>Thanks,</div><div><br></div><div>-- <br>David Powell<br>
</div>