[Haskell-cafe] data.binary get reading beyond end of input bytestring?

Jason Dagit dagit at codersbase.com
Thu Jul 29 13:55:25 EDT 2010


On Thu, Jul 29, 2010 at 10:35 AM, Jason Dagit <dagit at codersbase.com> wrote:

>
>
> On Thu, Jul 29, 2010 at 6:15 AM, Duncan Coutts <
> duncan.coutts at googlemail.com> wrote:
>
>>
>>
>> No idea what WrappedByteString is.
>>
>
> WrappedByteString is a newtype wrapper around ByteString that has a phantom
> type.  This allows instances of to be written such that ByteString can be
> used with the iteratee library.  You can see the source here if you're
> interested:
>
> http://hackage.haskell.org/packages/archive/iteratee/0.3.5/doc/html/src/Data-Iteratee-WrappedByteString.html
>
>
>
>>
>> It would look like attoparsec's resumable parser:
>>
>> data Result a = Fail !ByteString
>>              | Partial (ByteString -> Result a)
>>              | Done !ByteString a
>>
>> runGet :: Get a -> ByteString -> Result a
>>
>> Point is you feed it strict bytestring chunks. Then decoding a lazy
>> bytestring can be implemented on top easily, as can decoding a sequence
>> lazily.
>>
>
> Like attoparsec you'll probably want to write some other utility functions
> for working with Results.  Attoparsec defines feed, parseWith, maybeResult,
> and eitherResult.  I think you'll want something similar here.
>

And I forgot to mention....

Given those constructors for Result, how will you decode a sequence lazily?
 I agree you can consume the input lazily but, decodings won't produce any
values until you hit Done. This is something I noticed with attoparsec as
well.  You can feed it incrementally, but it doesn't produce output until it
reaches a Done state.  In one project, were the input was line based
records, I used ByteString's hGetLine to read individual lines and then
parse them individually.  That gave me a Result for each line and I could
build incremental processing of records on top of that.

Perhaps I was using attoparsec incorrectly, but as far as I can tell it
doesn't have a way to incrementally produce results.  I think what needs to
be added is a way to 'yield' and 'resume'.  What I can't figure out, is how
to meaningfully return 'partial results' in general.  In cases where the top
level combinator is something like manyTill it makes sense, but in other
cases it's unclear to me.  And I think you'd need to add a constructor above
that is some variation of "PartiallyDone a ByteString (ByteString -> Result
a)".

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100729/f04913df/attachment.html


More information about the Haskell-Cafe mailing list