Personal tools

Pipes

From HaskellWiki

(Difference between revisions)
Jump to: navigation, search
(Added a link to Control.Proxy.Tutorial on Hackage)
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Stub}}
 
 
 
[[Category:Idioms]]
 
[[Category:Idioms]]
  +
[[Category:Libraries]]
  +
[[Category:Packages]]
  +
  +
The pipes library is a clean and powerful stream processing library that lets you build and connect reusable streaming components.
  +
  +
== Libraries ==
  +
  +
[http://hackage.haskell.org/packages/archive/pkg-list.html#cat:pipes Pipes] is the official Hackage category for the pipes ecosystem.
  +
  +
=== pipes ===
  +
  +
The [http://hackage.haskell.org/package/pipes pipes] package is the core library which provides reusable primitives for stream programming. Use these as building blocks for more sophisticated streaming abstractions.
  +
  +
Read the [http://hackage.haskell.org/packages/archive/pipes/4.0.0/doc/html/Pipes-Tutorial.html official pipes tutorial] to learn more.
  +
  +
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Library GitHub]
  +
  +
=== pipes-safe ===
  +
  +
The [http://hackage.haskell.org/package/pipes-safe pipes-safe] package adds resource management and exception safety to the pipes ecosystem. Use pipes-safe to safely acquire and release resources deterministically within a pipeline.
  +
  +
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Safe-Library GitHub]
  +
  +
=== pipes-concurrency ===
  +
  +
The [http://hackage.haskell.org/package/pipes-concurrency pipes-concurrency] package adds concurrency primitives to the pipes ecosystem with built-in deadlock safety. Use these primitives to:
  +
  +
* Build reactive event-driven programs
  +
* Merge and broadcast streams
  +
* Communicate between multiple concurrent pipelines
  +
  +
Read the [http://hackage.haskell.org/packages/archive/pipes-concurrency/2.0.0/doc/html/Pipes-Concurrent-Tutorial.html official pipes-concurrency tutorial] to learn more.
  +
  +
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Concurrency-Library GitHub]
  +
  +
=== pipes-parse ===
  +
  +
The [http://hackage.haskell.org/package/pipes-parse pipes-parse] package defines the generic machinery necessary for common parsing tasks. Use pipes-parse to:
  +
  +
* handle leftovers and end of input,
  +
* interrupt and resume streaming, and
  +
* sub-divide streams without collecting elements in memory.
  +
  +
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Parse-Library GitHub]
  +
  +
== Community-contributed libraries ==
  +
  +
Listed in alphabetical order.
  +
  +
=== pipes-aeson ===
  +
  +
The [http://hackage.haskell.org/package/pipes-aeson pipes-aeson] library allows you to encode and decode JSON values flowing through streams, possibly
  +
interleaving other stream effects while doing it.
  +
  +
Source code is hosted on [https://github.com/k0001/pipes-aeson GitHub]
  +
  +
=== pipes-attoparsec ===
  +
  +
The [http://hackage.haskell.org/package/pipes-attoparsec pipes-attoparsec] library converts [http://hackage.haskell.org/package/attoparsec attoparsec] parsers to pipes for high-performance incremental parsing.
  +
  +
Source code hosted on [https://github.com/k0001/pipes-attoparsec GitHub]
  +
  +
=== pipes-binary ===
  +
  +
The [http://hackage.haskell.org/package/pipes-binary pipes-binary] library allows streams of binary data to be encoded and decoded using the <code>Binary</code> instances from the [http://hackage.haskell.org/package/binary binary] package.
  +
  +
Source code is hosted on [https://github.com/k0001/pipes-binary GitHub]
  +
  +
=== pipes-network ===
  +
  +
The [http://hackage.haskell.org/package/pipes-network pipes-network] library converts server and client sockets to pipes to seamlessly stream data over any network.
  +
  +
Source code hosted on [https://github.com/k0001/pipes-network GitHub]
  +
  +
=== pipes-network-tls ===
  +
  +
The [http://hackage.haskell.org/package/pipes-network-tls pipes-network-tls] allows streaming through TLS-secured network connections, exposing a similar API to the one exposed by [http://hackage.haskell.org/package/pipes-network pipes-network].
  +
  +
Source code hosted on [https://github.com/k0001/pipes-network-tls GitHub]
  +
  +
=== pipes-zlib ===
  +
  +
The [http://hackage.haskell.org/package/pipes-zlib pipes-zlib] enables compression and decompression of strict ByteString streams using the zlib codec.
  +
  +
Source code hosted on [https://github.com/k0001/pipes-zlib GitHub]
  +
  +
== Community ==
  +
  +
Besides the usual Haskell community channels such as the [[Mailing_lists|official mailing lists]] or the [http://reddit.com/r/haskell Haskell subreddit], you can ask for help, suggest improvements, or discuss about the Pipes ecosystem at the [https://groups.google.com/group/haskell-pipes “haskell-pipes” mailing list at Google Groups].
  +
  +
== Announcements ==
  +
  +
In chronological order:
  +
  +
* [http://www.reddit.com/r/haskell/comments/ohjg7/a_new_approach_to_iteratees/ pipes-1.0.0]: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single [http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Category.html Category].
  +
  +
* [http://www.haskellforall.com/2012/05/pipes-20-pipe-finalization.html pipes-2.0.0]: The introduction of Frames, later deprecated in favor of pipes-safe.
  +
  +
* [http://www.haskellforall.com/2012/07/pipes-21-and-index-core-10-indexed.html pipes-2.1.0]: Transition of Frames to indexed monads, later deprecated in favor of pipes-safe.
  +
  +
* [http://www.haskellforall.com/2012/09/pipes-23-bidirectional-pipes.html pipes-2.3.0]: The introduction of bidirectional Proxies, which evolved into modern pipes.
  +
  +
* [http://www.haskellforall.com/2012/10/pipes-24-proxy-transformers-extra.html pipes-2.4.0]: The release of the proxy transformer extension system.
  +
  +
* [http://www.haskellforall.com/2012/10/pipes-25-faster-and-slimmer.html pipes-2.5.0]: Major performance improvements.
  +
  +
* [http://www.haskellforall.com/2012/12/pipes-30-simpler-unified-api.html pipes-3.0.0]: Major API simplification and consolidation.
  +
  +
* [http://www.haskellforall.com/2013/01/pipes-safe-10-resource-management-and.html pipes-safe-1.0.0]: Resource safety and exception management
  +
  +
* [http://www.haskell.org/pipermail/haskell-cafe/2013-March/106752.html pipes-network-0.1.0]: Use TCP network sockets as pipes streams.
  +
  +
* [http://www.haskellforall.com/2013/03/pipes-32-listt-codensity-arrowchoice.html pipes-3.2.0]: ListT integration, Codensity proxy transformer, and ArrowChoice primitives.
  +
  +
* [http://www.haskellforall.com/2013/04/pipes-concurrency-100-reactive.html pipes-concurrency-1.0.0]: Concurrency support for pipes
  +
  +
* [https://groups.google.com/forum/?fromgroups=#!topic/haskell-pipes/O4_5dZ7WwXE pipes-zlib-0.2.0.0]: Zlib compression/decompression support for pipes
  +
  +
* [https://groups.google.com/d/msg/haskell-pipes/XGb-8x5YWY0/41BUbJagnuEJ pipes-network-tls-0.1.0.0]: Stream through TLS-secured network connections.
   
Pipes are a way to handle streaming data; for an introductory tutorial see [http://hackage.haskell.org/packages/archive/pipes/3.2.0/doc/html/Control-Proxy-Tutorial.html Control.Proxy.Tutorial on Hackage].
+
* [http://www.haskellforall.com/2013/06/pipes-parse-100-pushback-delimited.html pipes-parse-1.0.0]: Pushback, delimited parsers, resumable parsing, and lenses
   
  +
* [https://groups.google.com/d/msg/haskell-pipes/3A-RbzCUovw/9GEIDi5WRbwJ pipes-attoparsec-0.2.0.0]: New API, interleaved parsing support, built on top of pipes-parse.
   
== Links ==
+
* [https://groups.google.com/d/msg/haskell-pipes/2CQ9eiPUxjA/kBqcQboPdjoJ pipes-aeson-0.1.0.0]: Encode and decode JSON streams.
   
* [http://hackage.haskell.org/package/pipes The pipe package] on Hackage
+
* [https://groups.google.com/d/msg/haskell-pipes/7I8KcMW3zHU/5Mi8PKbuv2sJ pipes-binary-0.1.0.0]: Encode and decode binary streams using the <code>binary</code> package.
* [http://hackage.haskell.org/packages/archive/pkg-list.html#cat:enumerator Pipes packages] on Hackage
 
   
  +
* [http://www.haskellforall.com/2013/09/pipes-40-simpler-types-and-api.html pipes-4.0.0]: Simpler types and API
   
=== Blog articles ===
+
== Upcoming libraries ==
   
* [http://paolocapriotti.com/blog/2012/05/29/pipes-2.0-vs-pipes-core/ Pipes 2.0 vs Pipes-core]
+
* pipes-bytestring: ByteString support. Source code is hosted at [https://github.com/Gabriel439/Haskell-Pipes-ByteString-Library GitHub]
* [https://pcapriotti.wordpress.com/2012/02/02/an-introduction-to-guarded-pipes/ An introduction to guarded pipes]
 
* [http://www.haskellforall.com/2012/03/haskell-for-purists-pipe-finalization.html Haskell for Purists - Pipe Finalization]
 
* [http://www.haskellforall.com/2012/05/pipes-20-pipe-finalization.html pipes 2.0 - Pipe Finalization]
 
* [https://pcapriotti.wordpress.com/2012/02/04/monoidal-instances-for-pipes/ Monoidal instances for pipes]
 
   
=== See also ===
+
* pipes-text: Text support
* [[Conduit]]
 
* [[Enumerator and iteratee]]
 
* [[Iteratee I/O]]
 
* [[Iteratee]]
 

Revision as of 03:49, 8 September 2013


The pipes library is a clean and powerful stream processing library that lets you build and connect reusable streaming components.

Contents

1 Libraries

Pipes is the official Hackage category for the pipes ecosystem.

1.1 pipes

The pipes package is the core library which provides reusable primitives for stream programming. Use these as building blocks for more sophisticated streaming abstractions.

Read the official pipes tutorial to learn more.

Source code hosted on GitHub

1.2 pipes-safe

The pipes-safe package adds resource management and exception safety to the pipes ecosystem. Use pipes-safe to safely acquire and release resources deterministically within a pipeline.

Source code hosted on GitHub

1.3 pipes-concurrency

The pipes-concurrency package adds concurrency primitives to the pipes ecosystem with built-in deadlock safety. Use these primitives to:

  • Build reactive event-driven programs
  • Merge and broadcast streams
  • Communicate between multiple concurrent pipelines

Read the official pipes-concurrency tutorial to learn more.

Source code hosted on GitHub

1.4 pipes-parse

The pipes-parse package defines the generic machinery necessary for common parsing tasks. Use pipes-parse to:

  • handle leftovers and end of input,
  • interrupt and resume streaming, and
  • sub-divide streams without collecting elements in memory.

Source code hosted on GitHub

2 Community-contributed libraries

Listed in alphabetical order.

2.1 pipes-aeson

The pipes-aeson library allows you to encode and decode JSON values flowing through streams, possibly interleaving other stream effects while doing it.

Source code is hosted on GitHub

2.2 pipes-attoparsec

The pipes-attoparsec library converts attoparsec parsers to pipes for high-performance incremental parsing.

Source code hosted on GitHub

2.3 pipes-binary

The pipes-binary library allows streams of binary data to be encoded and decoded using the Binary instances from the binary package.

Source code is hosted on GitHub

2.4 pipes-network

The pipes-network library converts server and client sockets to pipes to seamlessly stream data over any network.

Source code hosted on GitHub

2.5 pipes-network-tls

The pipes-network-tls allows streaming through TLS-secured network connections, exposing a similar API to the one exposed by pipes-network.

Source code hosted on GitHub

2.6 pipes-zlib

The pipes-zlib enables compression and decompression of strict ByteString streams using the zlib codec.

Source code hosted on GitHub

3 Community

Besides the usual Haskell community channels such as the official mailing lists or the Haskell subreddit, you can ask for help, suggest improvements, or discuss about the Pipes ecosystem at the “haskell-pipes” mailing list at Google Groups.

4 Announcements

In chronological order:

  • pipes-1.0.0: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single Category.
  • pipes-2.0.0: The introduction of Frames, later deprecated in favor of pipes-safe.
  • pipes-2.1.0: Transition of Frames to indexed monads, later deprecated in favor of pipes-safe.
  • pipes-2.3.0: The introduction of bidirectional Proxies, which evolved into modern pipes.
  • pipes-2.4.0: The release of the proxy transformer extension system.
  • pipes-3.0.0: Major API simplification and consolidation.
  • pipes-3.2.0: ListT integration, Codensity proxy transformer, and ArrowChoice primitives.

5 Upcoming libraries

  • pipes-bytestring: ByteString support. Source code is hosted at GitHub
  • pipes-text: Text support