Terminate unused worker threads

[Simon Marlow]

On 17/11/2010 18:55, Ryan Newton wrote:
> Hi all,
>
> Apologies for commenting before understanding Capability.c very well.
>   But it seems that this file uses locking quite heavily.  Has there
> been an analysis of whether atomic memory ops and lock free algorithms
> could play any role here?

The locks here are mostly used when handing over the Capability between 
OS threads, or when an OS thread is going idle, and neither of these are 
going to be made faster by using lock-free data structures.  In fact, 
the lock on the Capability is only used to protect a very few fields - 
most of the fields are private to the Capability and require no locking.

However, there is one place that I think a better implementation could 
be used.  Each Capability has a message queue, which needs to behave 
like a multiple-writer single-reader buffer (order is unimportant). 
Currently it's a linked list protected by a lock, and this is bad - 
multiple writers contend for the lock with the reader.  Having the 
reader take the whole buffer in one go was an improvement, but doesn't 
completely solve the problem.  Something I need to be careful of is that 
the Capability should never go idle when the queue is non-empty (we 
don't want a lost wakeup), so presumably the lock is needed when the 
queue goes from empty to non-empty, but otherwise a lock-free operation 
to add items should suffice.

I haven't trawled the literature yet - is there a known data structure 
that would be good here?

Cheers,
	Simon


> Simon mentioned keeping track of the number of items in the queue so as
> not to have to traverse it while holding the lock.  That, for example,
> seems that it could be accomplished with an atomically modified counter.
>
> Cheers,
>    -Ryan
>
> On Wed, Nov 17, 2010 at 10:25 AM, Edward Z. Yang <ezyang at mit.edu
> <mailto:ezyang at mit.edu>> wrote:
>
>     Excerpts from Simon Marlow's message of Wed Nov 17 06:15:46 -0500 2010:
>      > I suggest keeping track of the number of items in the queue.
>
>     Ok, I added a spare_workers_no field to the Capability struct.
>
>      > So I think the main thing missing is a call to workerTaskStop().
>
>     Added.
>
>      > It would be really nice if we could arrange that in the case where we
>      > have too many spare workers, the extra workers exit via the same
>     route,
>      > but I suspect that might not be easy.
>
>     Do you mean, spare workers that have useful tasks to do?
>
>     Edward
>     _______________________________________________
>     Glasgow-haskell-users mailing list
>     Glasgow-haskell-users at haskell.org
>     <mailto:Glasgow-haskell-users at haskell.org>
>     http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
>