[Haskell-cafe] forkProcess, forkIO, and multithreaded runtime

[Michael Snoyman]

On Mon, Oct 15, 2012 at 6:30 PM, Joey Hess <joey at kitenet.net> wrote:

> Michael Snoyman wrote:
> > I think I have a misunderstanding of how forkProcess should be working.
> > Ultimately this relates to some bugs in the development version of
> keter, but
> > I've found some behavior in a simple test program which I wouldn't have
> > expected either, which may or may not be related.
> >
> > With the program at the end of this email, I would expect that, once per
> > second, I would get a message printed from each forkIO'd green thread,
> the
> > forked process, and the master process. And if I spawn 8 or less child
> threads
> > that's precisely what happens. However, as soon as I up that number to
> 9, the
> > child process is never run. The process is, however, created, as can be
> > confirmed by looking at the process table.
> >
> > This only occurs when using the multithreaded runtime. In other words,
> > compiling with "ghc --make test.hs" seems to always produce the expected
> > output, whereas "ghc --make -threaded test.hs" causes the behavior
> described
> > above. Having looked through the code for the process package a bit, my
> initial
> > guess is that this is being caused by a signal being sent to the child
> process,
> > but I'm not familiar enough with the inner workings to confirm or
> disprove this
> > guess.
> >
> > If anyone has any ideas on this, I'd appreciate it.
>
> While I'm not reproducing that behavior here with your test case and
> 7.4.1, I recently converted a large program to use -threaded (because I
> needed to use yesod in it, actually :), and had large quantities of pain
> involving forkProcess. It seemed to come down to this easily overlooked
> note in the docs:
>
>   forkProcess comes with a giant warning: since any other running threads
>   are not copied into the child process, it's easy to go wrong: e.g. by
>   accessing some shared resource that was held by another thread in the
>   parent.
>
> In my experience, forkProcess often behaves incomprehensibly (to me)
> with -threaded, typically resulting in a forked process hanging, and
> quite often only some small percentage of the time, which makes it
> really hard to track down and try to diagnose what thunk might not be
> getting forced until after the fork, or whatever.
>
> I did some analysis and produced a test case for problems caused by
> use of forkProcess in parts of MissingH, here:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=681621
>
> My understanding is that System.Process avoids these problems by doing
> all the setup around forking a command in C code. I've banished
> forkProcess from my code base entirely, except for a double fork I need
> to daemonize, and I don't even trust that call. :/
>
>
Well, I tried switching my code to forking/execing from C in a very similar
manner to the process package, and it seems to work.

Thanks for the input everyone!

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20121016/2d9203bd/attachment.htm>