Hi John,<div><br></div><div>Thanks for creating a competitor to the iteratee library. I think iteratees are an important abstraction, but there are some things about the iteratee library that I'm not fond of, despite John Lato doing a great job. I think having a bit of healthy competition to explore the design space is excellent.</div>
<div><br></div><div>I have questions for you below.<br><br><div class="gmail_quote">On Wed, Aug 18, 2010 at 9:31 PM, John Millikin <span dir="ltr"><<a href="mailto:jmillikin@gmail.com">jmillikin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Most of you have probably read Oleg's essays on using left-fold<br>
enumerators for incremental IO. In short, by encapsulating monadic<br>
left-folds in an "Iteratee" type, incremental pure processing is<br>
possible without using lazy IO. Sources to read:<br>
<br></blockquote><div>[snip] </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
While I appreciate Mr. Lato's development of the package, I find it<br>
far too large, and its documentation too sparse, to effectively use.<br>
To correct this, I've written the "enumerator" package. It is also<br>
derived from Oleg's IterateeM.hs , but with a simplified API and<br>
significantly reduced dependency list.<br></blockquote><div><br></div><div>I don't mind the dependency list, but I was mildly concerned that iteratee appears to work only on unix and that the API is a bit rough.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Hackage entry: <a href="http://hackage.haskell.org/package/enumerator" target="_blank">http://hackage.haskell.org/package/enumerator</a><br>
Haddock docs: <a href="http://ianen.org/haskell/enumerator/api-docs/" target="_blank">http://ianen.org/haskell/enumerator/api-docs/</a><br>
Source code (literate PDF): <a href="http://ianen.org/haskell/enumerator/enumerator.pdf" target="_blank">http://ianen.org/haskell/enumerator/enumerator.pdf</a><br>
<br>
darcs get <a href="http://ianen.org/haskell/enumerator/" target="_blank">http://ianen.org/haskell/enumerator/</a><br>
<br>
Additionally, I've included examples of using enumerators to implement<br>
simplified versions of the "cat" and "wc" utilities. These should<br>
serve as a useful starting point for anybody who wants to use<br>
enumerators in their own code:<br>
<br>
<a href="http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs" target="_blank">http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs</a><br>
<a href="http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs" target="_blank">http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs</a></blockquote>
<div><br></div><div>The main reason I would use iteratees is for performance reasons. To help me, as a potential consumer of your library, could you please provide benchmarks for comparing the performance of enumerator with say, a) iteratee, b) lazy/strict bytestring, and c) Prelude functions?</div>
<div><br></div><div>I'm interested in both max memory consumption and run-times. Using criterion and/or progression to get the run-times would be icing on an already delicious cake!</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
<br>
There are already a few libraries using the existing "iteratee"<br>
package (snap, attoparsec-iteratee, hexpat-iteratee); I am very<br>
interested in advice from the authors of these libraries. In<br>
particular, are any of the removed features (ListLike,<br>
WrappedByteString, seeking) something your libraries depend on? Are<br>
there any useful combinators you'd like to see included?<br></blockquote><div><br></div><div>The only reason iteratee provides WrappedByteString is because the type class used to abstract over the stream type requires something with kind * -> * and ByteString has kind *. The extra wrapping just adds an ignored phantom type to bytestrings. So if you don't require specific kinds I don't think you'd need to provide a WrappedByteString.</div>
<div><br></div><div>ListLike is possibly nice, but in the type indexed iteratee implementation that I started (but could not finish due to some issues with the type indexing) I didn't use it. ListLike doesn't support type threaded lists at all. On a side note, in my type threaded iteratee library, I initially elided StreamChunk but later added something similar in because I found it useful. I can't recall of the top of my head what the reasoning was, but I could dig deeper if it interests you. I was also following a fairly faithful re-implementation of John Lato's implementation, just with type indexing. I should probably post my partial library regardless. Perhaps others can find ways around the bits I was stuck on.</div>
<div><br></div><div>I can see seeking as being important as your library moves into new domains of use. Particularly when reading large binary streams when the data is sparse.</div><div><br></div><div>Thanks and congrats!</div>
<div>Jason</div><div><br></div><div><br></div></div></div>