<div dir="ltr"><div style>I think you can test this theory with this patch. If a thread is waiting on the task->cond condition variable which is matched up with task->lock, then pthread_cond_destroy will return EBUSY, which must always be a bug in the RTS.</div>
<div style><br></div><div style>Alexander</div><div><br></div><div>diff --git a/rts/posix/OSThreads.c b/rts/posix/OSThreads.c</div><div>index ae31966..0f12830 100644</div><div>--- a/rts/posix/OSThreads.c</div><div>+++ b/rts/posix/OSThreads.c</div>
<div>@@ -91,7 +91,8 @@ initCondition( Condition* pCond )</div><div> void</div><div> closeCondition( Condition* pCond )</div><div> {</div><div>- pthread_cond_destroy(pCond);</div><div>+ int ret = pthread_cond_destroy(pCond);</div>
<div>+ CHECKM(ret == 0, "RTS BUG! Someone is waiting on condvar %d.", ret);</div><div> return;</div><div> }</div><div> </div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jan 21, 2013 at 8:18 AM, Alexander Kjeldaas <span dir="ltr"><<a href="mailto:alexander.kjeldaas@gmail.com" target="_blank">alexander.kjeldaas@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>I just looked at this code and since I don't know the code I can't give you good solutions, but for others watching this thread the links might prove interesting.</div>
<div>
<br></div><div>My main theory is that you do have some other thread in FFI-land while you are fork()ing. The task->cond, task->lock seems to be related to this (see quoted comments below).</div><div><br>
</div><div>Also, pthread_mutex_destroy is undefined if the lock is locked, so I am guessing that the task->lock is somehow locked when it shouldn't be.</div><div><br></div><div><div>It isn't clear from your description whether this is consistently happening on Linux, or whether this only sometimes happens.</div>
<div><br></div><div>The forkProcess() code seems to hold all capabilities during fork, but that does not include FFI-land threads AFAIU.</div><div><br></div></div><div>Assuming that this happens only rarely, I am trying to understand what happens if the thread that is in FFI-land returns to the RTS (in the parent) after fork(), but before the freeTask() in the child. Based on the descriptions I read, it seems likely that this thread will try to inspect task->cap, which requires holding task->lock.</div>
<div><br></div><div>That would in turn make the pthread_mutex_destroy in the child invalid.</div><div><br></div><div><a href="https://github.com/ghc/ghc/blob/master/rts/Task.h#L57" target="_blank">https://github.com/ghc/ghc/blob/master/rts/Task.h#L57</a><br>
</div><div><br></div><div>"""</div><div> ...</div><div><div> When a task is migrated from sleeping on one Capability to another,</div><div> its task->cap field must be modified. When the task wakes up, it</div>
<div> will read the new value of task->cap to find out which Capability</div><div> it belongs to. Hence some synchronisation is required on</div><div> task->cap, and this is why we have task->lock.</div><div>
<br></div><div> If the Task is not currently owned by task->id, then the thread is</div><div> either</div><div><br></div><div> (a) waiting on the condition task->cond. The Task is either</div><div> (1) a bound Task, the TSO will be on a queue somewhere</div>
<div><span style="white-space:pre-wrap"> </span> (2) a worker task, on the spare_workers queue of task->cap.</div><div> ...</div><div>"""<br></div></div><div><br></div><div>freeTask:</div>
<div><a href="https://github.com/ghc/ghc/blob/master/rts/Task.c#L142" target="_blank">https://github.com/ghc/ghc/blob/master/rts/Task.c#L142</a><br></div><div><br></div><div>the comment in freeTask refers to this test:</div>
<div><div><a href="https://github.com/ghc/testsuite/blob/master/tests/concurrent/should_run/conc059.hs" target="_blank">https://github.com/ghc/testsuite/blob/master/tests/concurrent/should_run/conc059.hs</a><br></div><div>
<br></div>
<div>That test calls the RTC from C which then forkIOs off actions that are outstanding when the RTS exits.</div><div><br></div></div><div>in forkProcess, child code</div><div><a href="https://github.com/ghc/ghc/blob/master/rts/Schedule.c#L1837" target="_blank">https://github.com/ghc/ghc/blob/master/rts/Schedule.c#L1837</a></div>
<div><br></div><div>It look like all this code supports the notion that some other thread can be in foreign code during the fork call.</div><div><br></div><div>discardTasksExcept<br></div><div>
<a href="https://github.com/ghc/ghc/blob/master/rts/Task.c#L305" target="_blank">https://github.com/ghc/ghc/blob/master/rts/Task.c#L305</a></div><span class="HOEnZb"><font color="#888888"><div><br></div><div><br></div><div>
Alexander</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Jan 21, 2013 at 12:15 AM, Mark Lentczner <span dir="ltr"><<a href="mailto:mark.lentczner@gmail.com" target="_blank">mark.lentczner@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Sorry to be reviving this thread so long after.... but I seem to be running into similar issues as Michael S. did at the start.<div><br></div><div>In short, I'm using forkProcess with the threaded RTS, and see occasional hangs:</div>
<div class="gmail_extra"><ul><li>I see these only on Linux. On Mac OS X, I never do.</li><li>I'm using GHC 7.4.2</li><li>I noticed the warning in the doc for forkProcess, but assumed I was safe, as I wasn't holding any shared resources at the time of the fork, and no shared resources in the program are used in the child.</li>
<li>WIth gdb, I've traced the hang to here in the run-time: <font face="courier new, monospace">forkProcess > discardTasksExcept > freeTask > closeMutex(&task->lock) > pthread_mutex_destroy</font></li>
</ul>The discussion in this thread leaves me with these questions:</div><div class="gmail_extra"><ul><li>Is there reason to think the situation has gotten better in 7.6 and later?</li><li>Isn't the only reason <b>System.Process</b> is safer because it does an immediate exec in the child? Alas, I really want to just <font face="courier new, monospace">fork()</font> sometimes.</li>
<li>Is it really true that even if my program has no shared resources with the child, that the IO subsystem and FFI system do anyway? Surely the RTS would take care of doing the right thing with those, no?</li><li>
There should be no concern with <font face="courier new, monospace">exec</font> w.r.t. library invariants since <span style="font-family:'courier new',monospace">exec</span> is wholesale replacement - all the libraries will reinitialize. Is there a problem here I'm missing?</li>
</ul>Alas, I've stopped using the threaded RTS until I understand this better.</div><span><font color="#888888"><div class="gmail_extra"><br></div><div class="gmail_extra">- Mark</div></font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>