Difference between revisions of "Parallelism"
Jump to navigation
Jump to search
(copy from GHC/Concurrency (content to be divorced)) |
(pure /= non-IO) |
||
| (17 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| + | Parallelism is about speeding up a program by using multiple processors. |
||
| − | == Parallel and Concurrent Programming in GHC == |
||
| + | In Haskell we provide two ways to achieve parallelism: |
||
| − | This page contains notes and information about how to write concurrent programs in GHC. |
||
| + | * Pure parallelism, which can be used to speed up non-IO parts of the program. |
||
| + | * Concurrency, which can be used for parallelising IO. |
||
| + | Pure Parallelism (Control.Parallel): Speeding up a pure computation using multiple processors. Pure parallelism has these advantages: |
||
| − | You may be interested in [[GHC/Parallel|parallelism]] (speeding up pure functions) instead. Have a look there too. |
||
| + | * Guaranteed deterministic (same result every time) |
||
| + | * no [[race conditions]] or [[deadlocks]] |
||
| + | [[Concurrency]] (Control.Concurrent): Multiple threads of control that execute "at the same time". |
||
| − | GHC provides multi-scale support for parallel programming, from very fine-grained, small "sparks", to coarse-grained explicit threads and locks, along with other models of concurrent and parallel programming, including actors, CSP-style concurrency, nested data parallelism and Intel Concurrent Collections. Synchronization between tasks is possible via messages, regular Haskell variables, MVar shared state or transactional memory. |
||
| + | * Threads are in the IO monad |
||
| + | * IO operations from multiple threads are interleaved non-deterministically |
||
| + | * communication between threads must be explicitly programmed |
||
| + | * Threads may execute on multiple processors simultaneously |
||
| + | * Dangers: [[race conditions]] and [[deadlocks]] |
||
| + | Rule of thumb: use Pure Parallelism if you can, Concurrency otherwise. |
||
| − | * See "Real World Haskell" [http://book.realworldhaskell.org/read/concurrent-and-multicore-programming.html chapter 24], for an introduction to the most common forms of concurrent and parallel programming in GHC. |
||
| − | * A [http://donsbot.wordpress.com/2009/09/03/parallel-programming-in-haskell-a-reading-list/ reading list for parallelism in Haskell]. |
||
| − | * The [http://stackoverflow.com/questions/3063652/whats-the-status-of-multicore-programming-in-haskell status of parallel and concurrent programming] in Haskell. |
||
| − | |||
| − | The concurrent and parallel programming models in GHC can be divided into the following forms: |
||
| + | == Starting points == |
||
| − | * Very fine grained: parallel sparks and futures, as described in the paper "[http://www.haskell.org/~simonmar/bib/multicore-ghc-09_abstract.html Runtime Support for Multicore Haskell]" |
||
| − | * Fine grained: lightweight Haskell threads, explicit synchronization with STM or MVars. See the paper "Tackling the Awkward Squad" below. |
||
| ⚫ | |||
| ⚫ | |||
| − | * [http://www.cs.kent.ac.uk/projects/ofa/chp/ CHP]: CSP-style concurrency for Haskell. |
||
| + | * '''Control.Parallel'''. The first thing to start with parallel programming in Haskell is the use of par/pseq from the parallel library. Try the Real World Haskell [http://book.realworldhaskell.org/read/concurrent-and-multicore-programming.html chapter on parallelism and concurrency]. The parallelism-specific parts are in the second half of the chapter. |
||
| − | The most important (as of 2010) to get to know are the basic "concurrent Haskell" model of threads using forkIO and MVars, the use of transactional memory via STM, implicit parallelism via sparks and, if you're interested in scientific programming specifically, nested data parallelism in Haskell. |
||
| + | * If you need more control, try Strategies or perhaps the Par monad |
||
| − | + | == Multicore GHC == |
|
| ⚫ | |||
| − | * '''Basic concurrency: forkIO and MVars'''. |
||
| − | * '''Software Transactional Memory''' (STM) is a new way to coordinate concurrent threads. There's a separate [[Software transactional memory|Wiki page devoted to STM]]. |
||
| − | : STM was added to GHC 6.4, and is described in the paper [http://research.microsoft.com/~simonpj/papers/stm/index.htm Composable memory transactions]. The paper [http://research.microsoft.com/~simonpj/papers/stm/lock-free.htm Lock-free data structures using Software Transactional Memory in Haskell] gives further examples of concurrent programming using STM. |
||
| + | == Alternative approaches == |
||
| − | * '''Foreign function interface'''. If you are calling foreign functions in a concurrent program, you need to know about ''bound threads''. They are described in a Haskell workshop paper, [http://research.microsoft.com/~simonpj/Papers/conc-ffi/index.htm Extending the Haskell Foreign Function Interface with Concurrency]. The GHC Commentary [http://darcs.haskell.org/ghc/docs/comm/rts-libs/multi-thread.html Supporting multi-threaded interoperation] contains more detailed explanation of cooperation between FFI calls and multi-threaded runtime. |
||
| ⚫ | |||
| − | * '''Nested Data Parallelism'''. For an approach to exploiting the implicit parallelism in array programs for multiprocessors, see [[GHC/Data Parallel Haskell|Data Parallel Haskell]] (work in progress). |
||
| + | * [https://hackage.haskell.org/package/monad-par monad-par] and [https://hackage.haskell.org/package/lvish LVish] provide Par monads that can structure parallel computations over "monotonic" data structures, which in turn can be used from within purely functional programs. |
||
| − | |||
| ⚫ | |||
| − | === Using concurrency in GHC === |
||
| − | |||
| − | * You get access to concurrency operations by importing the library [http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.html Control.Concurrent]. |
||
| − | |||
| − | * The GHC manual gives a few useful flags that control scheduling (not usually necessary) [http://www.haskell.org/ghc/docs/latest/html/users_guide/sec-using-parallel.html#parallel-rts-opts RTS options]. |
||
| − | |||
| − | === Multicore GHC === |
||
| − | |||
| ⚫ | |||
| − | + | == See also == |
|
| + | * The [[Parallel|parallelism and concurrency portal]] |
||
| − | * The Sun project to improve http://ghcsparc.blogspot.com/ GHC performance on Sparc] |
||
| + | * Parallel [[Parallel/Reading|reading list]] |
||
| − | * A [http://www.well-typed.com/blog/38 Microsoft project to improve industrial applications of GHC parallelism]. |
||
| + | * [[Parallel/Research|Ongoing research in Parallel Haskell]] |
||
| − | * [http://www.haskell.org/~simonmar/bib/bib.html Simon Marlow's publications on parallelism and GHC] |
||
| − | * [http://www.macs.hw.ac.uk/~dsg/gph/ Glasgow Parallel Haskell] |
||
| − | * [http://www.macs.hw.ac.uk/~dsg/gdh/ Glasgow Distributed Haskell] |
||
| − | * http://www-i2.informatik.rwth-aachen.de/~stolz/dhs/ |
||
| − | * http://www.informatik.uni-kiel.de/~fhu/PUBLICATIONS/1999/ifl.html |
||
| − | * [http://www.mathematik.uni-marburg.de/~eden Eden] |
||
Revision as of 08:35, 11 January 2014
Parallelism is about speeding up a program by using multiple processors.
In Haskell we provide two ways to achieve parallelism:
- Pure parallelism, which can be used to speed up non-IO parts of the program.
- Concurrency, which can be used for parallelising IO.
Pure Parallelism (Control.Parallel): Speeding up a pure computation using multiple processors. Pure parallelism has these advantages:
- Guaranteed deterministic (same result every time)
- no race conditions or deadlocks
Concurrency (Control.Concurrent): Multiple threads of control that execute "at the same time".
- Threads are in the IO monad
- IO operations from multiple threads are interleaved non-deterministically
- communication between threads must be explicitly programmed
- Threads may execute on multiple processors simultaneously
- Dangers: race conditions and deadlocks
Rule of thumb: use Pure Parallelism if you can, Concurrency otherwise.
Starting points
- Control.Parallel. The first thing to start with parallel programming in Haskell is the use of par/pseq from the parallel library. Try the Real World Haskell chapter on parallelism and concurrency. The parallelism-specific parts are in the second half of the chapter.
- If you need more control, try Strategies or perhaps the Par monad
Multicore GHC
Since 2004, GHC supports running programs in parallel on an SMP or multi-core machine. How to do it:
- Compile your program using the -threaded switch.
- Run the program with +RTS -N2 to use 2 threads, for example (RTS stands for runtime system; see the GHC users' guide). You should use a -N value equal to the number of CPU cores on your machine (not including Hyper-threading cores). As of GHC v6.12, you can leave off the number of cores and all available cores will be used (you still need to pass -N however, like so: +RTS -N).
- Concurrent threads (forkIO) will run in parallel, and you can also use the par combinator and Strategies from the Control.Parallel.Strategies module to create parallelism.
- Use +RTS -sstderr for timing stats.
- To debug parallel program performance, use ThreadScope.
Alternative approaches
- Nested data parallelism: a parallel programming model based on bulk data parallelism, in the form of the DPH and Repa libraries for transparently parallel arrays.
- monad-par and LVish provide Par monads that can structure parallel computations over "monotonic" data structures, which in turn can be used from within purely functional programs.
- [OLD] Intel Concurrent Collections for Haskell: a graph-oriented parallel programming model.