<br><br><div class="gmail_quote">On Fri, Dec 4, 2009 at 12:28 PM, Patrick Caldon <span dir="ltr"><<a href="mailto:patc@pessce.net">patc@pessce.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">Neil Brown wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Patrick Caldon wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I'm looking for the "right" concurrency library/semantics for what should be a reasonably simple problem.<br>
<br>
I have a little simulator:<br>
<br>
runWorldSim :: MTGen -> SimState -> IO SimState<br>
<br>
it takes about a second to run on a PC. It's functional except it whacks the rng, which needs IO. I run 5-10 of these jobs, and then use:<br>
<br>
mergeWorld :: [SimState] -> SimState<br>
<br>
to pick the best features of the runs and build another possible world (state). Then I use this new world to run another 5-10 jobs and so on. I run this through ~20000 iterations.<br>
<br>
It's an obvious place for parallelism.<br>
<br>
I'm looking for a concurrency library with something like:<br>
<br>
forkSequence :: Int -> [IO a] -> IO [a]<br>
<br>
which I could call with something like this:<br>
<br>
forkSequence 4 (take 10 (repeat (runWorldSim g ss)))<br>
<br>
this would construct 4 threads, then dispatch the 10 jobs onto the threads, and pack up the<br>
results into a list I could run through my merger.<br>
</blockquote>
Why particularly do you want to run the 10 jobs on 4 threads? Haskell's run-time is quite good at spreading out the lightweight threads onto all your cores, so the easiest thing to do is run the 10 jobs on 10 (light-weight) threads and let the run-time sort out the rest. <br>
</blockquote>
<br></div>
Thanks so much for that! I'll give it a go.<br>
<br>
Different threads is just because some of the jobs are memory hogs, and I want to minimize the number running simultaneously. I'll see what happens with a runPar-like approach, and use a queue-based approach if it becomes a problem.<div class="im">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So if what you want is a function:<br>
<br>
runPar :: [IO a] -> IO [a]<br>
<br>
you can easily construct this. Shameless plug: my CHP library effectively has this function already, runParallel :: [CHP a] -> CHP [a] (CHP being a slight layer on top of IO). But you can do it just as easily with, say, STM. Here is a version where order doesn't matter (apologies for the point-free style):<br>
<br>
import Control.Concurrent<br>
import Control.Concurrent.STM<br>
import Control.Monad<br>
<br>
modifyTVar :: TVar a -> (a -> a) -> STM ()<br>
modifyTVar tv f = readTVar tv >>= writeTVar tv . f<br>
<br>
runPar :: [IO a] -> IO [a]<br>
runPar ps<br>
= do resVar <- newTVarIO []<br>
mapM_ (forkIO . (>>= atomically . modifyTVar resVar . (:))) ps<br>
atomically $ do res <- readTVar resVar<br>
when (length res < length ps) retry<br>
return res<br>
<br>
If order does matter, you can zip the results with an index, and sort by the index afterwards. If efficiency matters, you can perform other tweaks. But the principle is quite straightforward. Or you can refactor your code to take the IO dependency out of your random number generation, and run the sets of pure code in parallel using the parallel library. If all you are using IO for is random numbers, that's probably the nicest approach.<br>
<br>
</blockquote></div>
Good, fast random numbers are unfortunately necessary - I had a nice implementation using System.Random, but had to rewrite it because performance was poor :( .</blockquote><div><br></div><div>Have you tried this, pure, library? <a href="http://hackage.haskell.org/package/mersenne-random-pure64">http://hackage.haskell.org/package/mersenne-random-pure64</a></div>
<div><br></div><div><br></div><div><a href="http://hackage.haskell.org/package/mersenne-random-pure64"></a></div></div><br>-- <br>Sebastian Sylvan<br>