[Haskell-cafe] Parallel combinator, performance advice

[Bulat Ziganshin]

Hello Neil,

Tuesday, April 7, 2009, 2:25:12 PM, you wrote:

> The problem I'm trying to solve is running system commands in
> parallel.

"system commands" means execution of external commands or just system
calls inside Haskell?

> Running a benchmark of issuing 1 million trivial tasks (create,
> modify, read and IO ref) the version without any parallelism is really
> fast (< 0.1 sec), and the version with parallelism is slow (> 10 sec).
> This could be entirely due to space leaks etc when queueing many
> tasks.

i think it's just because use of MVar/Chan is much slower than IORef
activity. once i checked that on 1GHz cpu and got 2 million withMVar-s
per second

i don't understood exactly what you need, but my first shot is
to create N threads executing commands from channel:

para xs = do
  done <- newEmptyMVar
  chan <- newChan
  writeList2Chan chan (map Just xs ++ [Nothing])
  
  replicateM_ numCapabilities $ do
    forkIO $ do
      forever $ do
        x <- readChan chan
        case x of
          Just cmd -> cmd
          Nothing -> putMVar done ()
  takeMVar done
  


-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com