Personal tools

Library/Data encoding

From HaskellWiki

< Library(Difference between revisions)
Jump to: navigation, search
(Development has been moved to github)
(5 intermediate revisions by 3 users not shown)
Line 11: Line 11:
 
* Base64 (<hask>Codec.Binary.Base64</hask>)
 
* Base64 (<hask>Codec.Binary.Base64</hask>)
 
* Base64Url (<hask>Codec.Binary.Base64Url</hask>)
 
* Base64Url (<hask>Codec.Binary.Base64Url</hask>)
  +
* Base85 (<hask>Codec.Binary.Base85</hask>)
  +
* Python string escaping (<hask>Codec.Binary.PythonString</hask>)
  +
* Quoted-Printable (<hask>Codec.Binary.QuotedPrintable</hask>)
  +
* URL encoding (<hask>Codec.Binary.Url</hask>)
 
* Uuencode (<hask>Codec.Binary.Uu</hask>)
 
* Uuencode (<hask>Codec.Binary.Uu</hask>)
  +
* Xxencode (<hask>Codec.Binary.Xx</hask>)
  +
* yEncode (<hask>Codec.Binary.Yenc</hask>)
   
 
In some cases the encodings also specify headers and footers for the encoded data. Implementation of that is left for the user of the library.
 
In some cases the encodings also specify headers and footers for the encoded data. Implementation of that is left for the user of the library.
Line 19: Line 25:
 
=== Main API ===
 
=== Main API ===
   
The module <Codec.Binary.DataEncoding> provides a type that collects the functions for an individual encoding:
+
The module <hask>Codec.Binary.DataEncoding</hask> provides a type that collects the functions for an individual encoding:
   
 
<haskell>
 
<haskell>
Line 87: Line 93:
   
 
Just like Base64 but with a different encoding alphabet. The encoding alphabet is made URL and filename safe by substituting <tt>+</tt> and <tt>/</tt> for <tt>-</tt> and <tt>_</tt> respectively.
 
Just like Base64 but with a different encoding alphabet. The encoding alphabet is made URL and filename safe by substituting <tt>+</tt> and <tt>/</tt> for <tt>-</tt> and <tt>_</tt> respectively.
  +
  +
=== Base85 ===
  +
  +
Implementation as described in the [http://en.wikipedia.org/wiki/Ascii85 Wikipedia article].
  +
  +
=== Python string escaping ===
  +
  +
Implementation of Python's string escaping.
  +
  +
=== Quoted-Printable ===
  +
  +
Implemented as defined in [http://tools.ietf.org/html/rfc2045 RFC 2045].
  +
  +
=== URL encoding ===
  +
  +
Implemented as defined in [http://tools.ietf.org/html/rfc3986 RFC 3986].
   
 
=== Uuencode ===
 
=== Uuencode ===
   
 
Unfortunately uuencode is badly specified and there are in fact several differing implementations of it. This implementation attempts to encode data in the same way as the <tt>uuencode</tt> utility found in [http://www.gnu.org/software/sharutils/ GNU's sharutils]. The workings of <hask>chop</hask> and <hask>unchop</hask> also follow how sharutils split and unsplit encoded lines.
 
Unfortunately uuencode is badly specified and there are in fact several differing implementations of it. This implementation attempts to encode data in the same way as the <tt>uuencode</tt> utility found in [http://www.gnu.org/software/sharutils/ GNU's sharutils]. The workings of <hask>chop</hask> and <hask>unchop</hask> also follow how sharutils split and unsplit encoded lines.
  +
  +
=== Xxencode ===
  +
  +
Implemented as described in the [http://en.wikipedia.org/wiki/Xxencode Wikipedia article].
   
 
=== yEncoding ===
 
=== yEncoding ===
Line 108: Line 134:
 
== Contributing ==
 
== Contributing ==
   
Retrieve the source code using darcs like this:
+
The source is hosted on [https://github.com/magthe/dataenc/ github] and can be downloaded using git:
   
darcs get http://code.haskell.org/dataenc/devo dataenc
+
git clone https://github.com/magthe/dataenc.git
   
Send any patches to magnus@therning.org
+
Patches can be sent to magnus@therning.org, but I suggest using github's pull requests if possible.

Revision as of 07:37, 31 January 2011

Data Encodings (dataenc): A collection of data encoding algorithms.

Contents

1 Data encodings library

The data encodings library strives to provide implementations in Haskell of every major data encoding, and a few minor ones as well. Currently the following encodings are implemented:

  • Base16 (
    Codec.Binary.Base16
    )
  • Base32 (
    Codec.Binary.Base32
    )
  • Base32Hex (
    Codec.Binary.Base32Hex
    )
  • Base64 (
    Codec.Binary.Base64
    )
  • Base64Url (
    Codec.Binary.Base64Url
    )
  • Base85 (
    Codec.Binary.Base85
    )
  • Python string escaping (
    Codec.Binary.PythonString
    )
  • Quoted-Printable (
    Codec.Binary.QuotedPrintable
    )
  • URL encoding (
    Codec.Binary.Url
    )
  • Uuencode (
    Codec.Binary.Uu
    )
  • Xxencode (
    Codec.Binary.Xx
    )
  • yEncode (
    Codec.Binary.Yenc
    )

In some cases the encodings also specify headers and footers for the encoded data. Implementation of that is left for the user of the library.

2 The API

2.1 Main API

The module
Codec.Binary.DataEncoding
provides a type that collects the functions for an individual encoding:
data DataCodec = DataCodec {
    encode :: [Word8] -> String,
    decode :: String -> Maybe [Word8],
    decode' :: String -> [Maybe Word8],
    chop :: Int -> String -> [String],
    unchop :: [String] -> String
}

It also exposes instances of this type for each encoding:

base16 :: DataCodec
base32 :: DataCodec
base32Hex :: DataCodec
base64 :: DataCodec
base64Url :: DataCodec
uu :: DataCodec

NB There is no instance for yEncoding since the functions in that module have slightly different type signatures.

2.2 Secondary API

Each individual encoding module is also exposed and offers four functions:

encode :: [Word8] -> String
decode :: String -> Maybe [Word8]
decode' :: String -> [Maybe Word8]
chop :: Int -> String -> [String]
unchop :: [String] -> String

3 Description of the encodings

3.1 Base16

Implemented as it's defined in RFC 4648.

Each four bit nibble of an octet is encoded as a character in the set 0-9,A-F.

3.2 Base32

Implemented as it's defined in RFC 4648.

Five octets are expanded into eight so that only the five least significant bits are used. Each is then encoded into a 32-character encoding alphabet.

3.3 Base32Hex

Implemented as it's defined in RFC 4648.

Just like Base32 but with a different encoding alphabet. Unlike Base64 and Base32, data encoded with Base32Hex maintains its sort order when the encoded data is compared bit wise.

3.4 Base64

Implemented as it's defined in RFC 4648.

Three octets are expanded into four so that only the six least significant bits are used. Each is then encoded into a 64-character encoding alphabet.

3.5 Base64Url

Implemented as it's defined in RFC 4648.

Just like Base64 but with a different encoding alphabet. The encoding alphabet is made URL and filename safe by substituting + and / for - and _ respectively.

3.6 Base85

Implementation as described in the Wikipedia article.

3.7 Python string escaping

Implementation of Python's string escaping.

3.8 Quoted-Printable

Implemented as defined in RFC 2045.

3.9 URL encoding

Implemented as defined in RFC 3986.

3.10 Uuencode

Unfortunately uuencode is badly specified and there are in fact several differing implementations of it. This implementation attempts to encode data in the same way as the uuencode utility found in GNU's sharutils. The workings of
chop
and
unchop
also follow how sharutils split and unsplit encoded lines.

3.11 Xxencode

Implemented as described in the Wikipedia article.

3.12 yEncoding

Implemented as it's defined in the 1.3 draft.

4 Downloading

The current release is available from HackageDB.

See #Contributing below for how to get the development version.

5 Example of use

The package omnicodec contains two command line tools for encoding and decoding data.

6 Contributing

The source is hosted on github and can be downloaded using git:

   git clone https://github.com/magthe/dataenc.git

Patches can be sent to magnus@therning.org, but I suggest using github's pull requests if possible.