[Haskell-cafe] Re: Do I need to roll my own?

[David Leimbach]

On Wed, Mar 31, 2010 at 2:12 PM, John Lato <jwlato at gmail.com> wrote:

> Hi Dave,
>
> > From: David Leimbach <leimy2k at gmail.com>
> >
> > I'm looking at iteratee as a way to replace my erroneous and really
> > inefficient lazy-IO-based backend for an expect like Monad DSL I've been
> > working for about 6 months or so now on and off.
> >
> > The problem is I want something like:
> >
> > expect "some String"
> > send "some response"
> >
> > to block or perhaps timeout, depending on the environment, looking for
> "some
> > String" on an input Handle, and it appears that iteratee works in a very
> > fixed block size.  While a fixed block size is ok, if I can put back
> unused
> > bytes into the enumerator somehow (I may need to put a LOT back in some
> > cases, but in the common case I will not need to put any back as most
> > expect-like scripts typically catch the last few bytes of data sent
> before
> > the peer is blocked waiting for a response...)
>
> I'm quite sure I don't know what you're trying to do.  The only time I
> can think of needing this is if you're running an iteratee on a file
> handle, keeping the handle open, then running another iteratee on it.
> Is this what you're doing?  If so, I would make a new run function:
>
> runResidue :: (Monad m, SC.StreamChunk s el) => IterateeG s el m a -> m (a,
> s)
> runResidue iter = runIter iter (EOF Nothing) >>= \res ->
>  case res of
>    Done x s -> return (x, s)
>    Cont _ e -> error $ "control message: " ++ show e
>
> This function will return the unused portion of the stream, then you
> can do this:
>
> enumResidue :: Handle -> s -> EnumeratorGM s el m a
> enumResidue h s = enumPure1Chunk s >. enumHandle h
>
> Is this what you need?  If I'm completely wrong about what you're
> trying to do (or you're using multiple threads), there are other
> options.
>

The problem is I am not sure what it was I needed to get started to begin
with.  For a moment it seemed that I could be throwing out data that's been
read, but not yet fed to an iteratee step.  If that's not the case, I'll
never need to put back any characters. The attoparsec-iteratee example
posted, plus some experimentation has led me to believe I don't need to
worry about this sort of thing for the kind of processing I'm looking to do.


>
> You also may want to look at iteratee-HEAD.  The implementation has
> been cleaned up a lot, the block sizes are user-specified, and there's
> an exception-based, user-extensible mechanism for iteratees to alter
> enumerator behavior.
>
>
That's very compelling.

Here's the properties of the system I'm trying to build (in fact I've built
this system with Haskell already months ago, but trying to evaluate if
iteratee can fix problems I've got now)

1. Must have an expect-like language to a subprocess over a pair of Handles
such that I can query what is normally a command line interface as a polling
refresher thread to a cache.  Note that there may be many sub-processes with
a poller/cache (up to 6 so far).  Data produced is dumped as records to
stdout from each thread such that the process that spawned this haskell
program can parse those records and update it's view of this particular part
of the world.

2. Must be able to interleave commands between polled record data from the
processes in 1.  These commands come in over this process's stdin from the
program that started the Haskell program.

3. The polling process in 1, must be able to respond to a timeout situation.
 In this system, a cable could become disconnected or a part of the system
could become unavailable, or a part of the system underneath could become
unreliable and require a restart to guarantee the serviceability of the
whole system.

I have 1 and 2 working fine in a current iteration of the system, but
because we're dealing with a pretty complex stack, 3 is really necessary
too, and I've conquered my timeout issues from earlier in a reasonable
enough way.  The problem is that the timeout "handler" apparently runs into
problems that I think stem from lazyIO not having been evaluated on a handle
yet, but the timeout handler has invalidated that handle, and then that
thunk, which was partially applied, is now talking to some broken value.

As far as I know, I can't go back and prevent previously bound thunks that
have the wrong handle from executing, if that is truly what's happening.
 What I'd like to do is prevent that situation from ever happening to begin
with, or at least rule it out.

I'm hoping version 2 of this system to be based on iteratee and avoid this
sort of problem at all.

The alternative is to write this in another language, but that throws out a
lot of nice and simple code that's present in the non IO bits of this code,
for dealing with parsing and data serialization.

Dave


> Sincerely,
> John
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100331/fcea99dd/attachment-0001.html