Concurrency (was: Re: [GUI] Re: events & callbacks)

[Daan Leijen]

>> I therefore propose another rule:
>>
>> 2d) All haskell code is run in a single OS thread  (the "GUI thread"). Calls to the CGA
>> can be made at any time from any Haskell thread (within the GUI thread).
>
> My vote is a big NO: This means restricting what a Haskell runtime system is allowed to do.

Please think a little bit longer before voting a "big NO". If we want to make
progress we shouldn't rule out a solid and simple solution based on shaky arguments.

I agree that my formulation was a bit too vague, here is a better version:

2e) All haskell call backs are run in a single OS thread  (the "GUI thread"). Calls to the CGA
    can be made at any time from any Haskell thread that run within the GUI thread.

(In particular, this formulation doesn't restrict the Haskell runtime system at all.
I think that you worry about:
1) What if Haskell automatically decides to run Haskell threads in some other OS thread?
   First, I believe that an automatic system would be a mistake, but secondly, such system
   will surely provide a forkIO variant that runs the Haskell thread in the same OS thread.
2) What if I want to call the CGA from some other OS thread? Well, you should just
   use a co-thread to do it: schedule the call from a haskell thread running in the
   OS GUI thread and communicate the result via MVar's.

However, I don't really want to discuss OS threads/Haskell threads here, it is
enough to know that this formulation does *not* rule out future GHC's or future
OS multi threaded  systems. )

>> - The serialization requirements of the backend toolkit are automatically met since
>> all these haskell threads run in the same OS thread.
>
> ... unless the toolkit uses some global state variable (the "current graphics port" in carbon; drawing to one window from multiple _haskell_ threads simultanously would probably be a problem for Win32, too, as Drawing Context state may be modified in the process).

You are right that different Haskell threads can interact with each other in devious
ways by modifying global state... this is called the IO monad :-) There is no way you can solve this problem in general, and for the GUI it may involve a lot of overhead.

*This is the responsibility of the user when using concurrency*. In your example,
it is a classic multiple writers, single resource pattern.

>> - We can implement all advanced event models on top of this model using concurrency.
>
>> - We most surely do *not* want foreign C calls to be run in a different OS thread
>> and don't want the "threadsafe" keyword here. (I am opposed to such extension --
>> complexity without reason - run your OS threads from C yourself!)
>
> We (I have to admit, that means: I) most surely *do* want the "threadsafe" keyword here.
> After all, it is a feature that I have been taking for granted before I switched from C++ to Hasell. Having to think about an implementation detail of Haskell where things work "just right" when you use C++ is scary and definitely the wrong way to go. That's why threadsafe calls are the future.

You may think so, but I believe that it is much better to leave the user in control.
The "threadsafe" will spawn implicit OS threads and I consider this a bad thing.
If I have the ability to spawn OS threads from with Haskell (and when all C calls are
made within the same OS thread), I can get the "threadsafe" behaviour without using
some implicit mechanism -- spawn a new OS thread, run a Haskell thread in there that
makes the C call and wait for its result via an MVar.

I really think that OS thread resources are too complex to be managed automagically
and that you should just provide some primitive calls to manage OS threads. Usage
will maybe show interesting patterns that can be captured in combinators.

(Note that no extensions to the FFI are needed and OS threads can be selectively supported by implementations by implementing a single function to spawn an OS thread:

forkOS 	  :: IO () -> IO OSThreadId                  -- new OS thread
forkIOIn      :: IO () -> OSThreadId -> IO ThreadId      -- new Haskel thread in specific OS thread
getOSThreadId :: IO OSThreadId                           -- get current OS thread id

But again, this really belongs to the FFI mailing list and I don't
really want to discuss this in length.)


>> Now, the big issue here is how to keep those Haskell threads running! [...]
>
> With threadsafe calls, that problem disappears. There'll just be normal synchronization issues like in every other language.

Well, you introduced heavy weight synchronisation that has to be wrapped around
*every* foreign call to the GUI. Furthermore, you imply that a worker thread should
be run in a different OS thread to keep everything reactive. However, OS threads are
rather heavy weight and it may take longer to start one than the processing takes.
In contrast, Haskell threads are extremely light-weight and fast.

Only heavy processing or should/could use OS threads -- not simple GUI components.
OS thread support is beyond the scope of this library, we just have to make
sure that it can work with future extensions -- and my proposal does so.

> I don't like that. Having those "hooks" into the runtime system sounds like much more unnecessary (and implementation-specific) complexity than the "threadsafe" extension. Not having those hooks would waste some CPU time.

Those "hooks" are just haskell calls: "threadsRunning :: IO Bool", nothing special here, should
have been in the API anyway.

> It should of course be possible to implement the CGA on single-OS-thread haskell implementations (using the "idle time" scheme you proposed above) . But we should write the specification in a way that allows taking advantage of "threadsafe".

The specification shouldn't talk about this at all. If someone wants multiple OS threads, it should be possible to use them with the specification, that is all.

> It would be a big mistake to write the specification in a way that is incompatible with a planned feature of the next version of GHC.

I don't think that the proposed rule is incompatible with a "planned feature of the next version of GHC".

I appreciate the work you are doing on supporting OS threads in GHC -- it is interesting
to see how it will work out. However, for the CGA it is enough to run in a single OS
thread. The specification should just allow room for use of the CGA in the context
of multiple OS threads -- but it doesn't have to support all this automatically as
I think that the issues are more subtle than initially appreciated. Evolution, not revolution!

All the best,
  Daan.