[Haskell-cafe] Re: Parallel combinator, performance advice
haskell at list.mightyreason.com
Tue Apr 7 11:01:33 EDT 2009
Neil Mitchell wrote:
> I guess the "nested calls to parallel_" bit is the part of the spec
> that makes everything much harder!
Yes. Much more annoying.
But the problem here is generic. To avoid it you must never allow all thread to
block at once.
The parallel_ function is such a job, so you solved this with the 'idempotent'
trick. You solution works by blocking all but 1 thread.
1a) Some worker thread 1 executes parallel_ with some jobs
1b) These get submitted the work queue 'chan'
1c) worker thread 1 starts on those same jobs, ignoring the queue
1d) worker thread 1 reaches the job being processed by thread 2
1e) worker thread 1 blocks until the jobs is finished in modifyMVar
2a) Worker thread 2 grabs a job posted by thread 1, that calls parallel_
2b) This batch of jobs gets submitted to the work queue 'chan'
2c) worker thread 2 starts on those same jobs, ignoring the queue
1d) worker thread 2 reaches the job being processed by thread 3
1e) worker thread 2 blocks until the jobs is finished in modifyMVar
And now only 1 thread is still working, and it has to work in series.
I think I can fix this...
More information about the Haskell-Cafe