[Haskell-cafe] Data.Binary and little endian encoding

[Duncan Coutts]

On Thu, 2009-05-14 at 20:46 -0700, David Leimbach wrote:
> 
> 
> On Thu, May 14, 2009 at 8:40 PM, Don Stewart <dons at galois.com> wrote:
>         leimy2k:
>         
>         > I actually need little endian encoding... wondering if
>         anyone else hit this
>         > with Data.Binary. (because I'm working with Bell Lab's 9P
>         protocol which does
>         > encode things on the network in little-endian order).
>         >
>         > Anyone got some "tricks" for this?
>         
>         
>         Yes!
>         There are big, little and host-endian primitives in the
>         Get/Put monads.
>         
>         
>          http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/Data-Binary-Put.html#v%3AputWord16le
>         
>         You can use these to build encoders directly.
> 
> 
> Cool... I just have to write my own encoder and decoder now.
> 
> 
> As a request could we get encodeLe decodeLe for a later version of
> this library?  :-)  That'd be totally awesome.

The thing you're missing (and which admittedly is not clear) is that the
binary package has two parts. One is a layer where you get full control
over the binary representation. The other is a portable serialisation
layer for pickling and unpickling Haskell values. That pickling layer
(ie the Binary class) is not for working with externally-defined binary
formats.

It might seem like we could co-opt the Binary class for this purpose eg
by parametrising by a dozen things like endian, padding, etc etc but I
don't think it scales or is sufficiently flexible (and it'd be slow).

What is missing in the binary package is a nice set of combinators for
using the low level layer to easily construct parsers for
externally-defined formats. That's what you'd want for your P9 protocol.

To reduce confusion we should also split the Haskell picking layer from
the lower layer.

This has been on our TODO list for some time. It needs to be done pretty
carefully however and we've not really had the time.

Duncan