[Haskell-cafe] forkProcess, forkIO, and multithreaded runtime

[Joey Hess]

Michael Snoyman wrote:
> I think I have a misunderstanding of how forkProcess should be working.
> Ultimately this relates to some bugs in the development version of keter, but
> I've found some behavior in a simple test program which I wouldn't have
> expected either, which may or may not be related.
> 
> With the program at the end of this email, I would expect that, once per
> second, I would get a message printed from each forkIO'd green thread, the
> forked process, and the master process. And if I spawn 8 or less child threads
> that's precisely what happens. However, as soon as I up that number to 9, the
> child process is never run. The process is, however, created, as can be
> confirmed by looking at the process table.
> 
> This only occurs when using the multithreaded runtime. In other words,
> compiling with "ghc --make test.hs" seems to always produce the expected
> output, whereas "ghc --make -threaded test.hs" causes the behavior described
> above. Having looked through the code for the process package a bit, my initial
> guess is that this is being caused by a signal being sent to the child process,
> but I'm not familiar enough with the inner workings to confirm or disprove this
> guess.
> 
> If anyone has any ideas on this, I'd appreciate it.

While I'm not reproducing that behavior here with your test case and
7.4.1, I recently converted a large program to use -threaded (because I
needed to use yesod in it, actually :), and had large quantities of pain
involving forkProcess. It seemed to come down to this easily overlooked
note in the docs:

  forkProcess comes with a giant warning: since any other running threads
  are not copied into the child process, it's easy to go wrong: e.g. by
  accessing some shared resource that was held by another thread in the
  parent.

In my experience, forkProcess often behaves incomprehensibly (to me)
with -threaded, typically resulting in a forked process hanging, and
quite often only some small percentage of the time, which makes it
really hard to track down and try to diagnose what thunk might not be
getting forced until after the fork, or whatever.

I did some analysis and produced a test case for problems caused by
use of forkProcess in parts of MissingH, here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=681621

My understanding is that System.Process avoids these problems by doing
all the setup around forking a command in C code. I've banished
forkProcess from my code base entirely, except for a double fork I need
to daemonize, and I don't even trust that call. :/

-- 
see shy jo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20121015/ad256959/attachment.pgp>