<div dir="ltr">John,<div><br></div><div>This package looks very promising. I used iteratee for the yaml package, but I had many of the concerns that you have mentioned below. Version 0.2 of persistent is going to have some form of an enumerator interface for getting the results of a query, and I eventually decided that iteratee was introducing too much complexity to be a good candidate. However, I was able to port the package[1] over to enumerator in about half an hour; I especially benefited from your example applications.<br>
<br></div><div>The only concern that I had was the possible inefficiency of representing all chunks as a list. In the case of persistent, the enumerator will *always* generate a one-lengthed list, and the most common operation is selectList, which returns all results as a list. If I used your consume function, I believe there would be a *lot* of list traversals. Instead, selectList[2] uses ([a] -> [a]) for building up the result internally. I haven't really thought the issue through fully, so I can recommend anything better. Perhaps more importantly, the simplification introduced by just dealing with lists is well received.</div>
<div><br></div><div>Keep up the good work, I look forward to seeing more about enumerator.</div><div><br></div><div>Michael</div><div><br></div><div>[1] <a href="http://github.com/snoyberg/persistent/tree/enumerator">http://github.com/snoyberg/persistent/tree/enumerator</a></div>
<div>[2] <a href="http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base.hs#L322">http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base.hs#L322</a></div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div>
<a href="http://github.com/snoyberg/persistent/tree/enumerator"></a><br><div class="gmail_quote">On Thu, Aug 19, 2010 at 7:31 AM, John Millikin <span dir="ltr"><<a href="mailto:jmillikin@gmail.com">jmillikin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Most of you have probably read Oleg's essays on using left-fold<br>
enumerators for incremental IO. In short, by encapsulating monadic<br>
left-folds in an "Iteratee" type, incremental pure processing is<br>
possible without using lazy IO. Sources to read:<br>
<br>
Oleg: Streams and Iteratees < <a href="http://okmij.org/ftp/Streams.html" target="_blank">http://okmij.org/ftp/Streams.html</a> ><br>
Magnus Therning: Trying to work out iteratees <<br>
<a href="http://therning.org/magnus/archives/735" target="_blank">http://therning.org/magnus/archives/735</a> ><br>
cdsmith: Iteratees Step By Step (Part 1) <<br>
<a href="http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/" target="_blank">http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/</a><br>
><br>
John Millikin (me): Understanding Iteratees <<br>
<a href="http://ianen.org/articles/understanding-iteratees/" target="_blank">http://ianen.org/articles/understanding-iteratees/</a> ><br>
<br>
Currently, the primary package for left-fold enumerators is John<br>
Lato's "iteratee". It is based on Oleg's original code, extended to<br>
support various forms of containers, platform-specific IO, and codecs<br>
for the WAV and TIFF formats.<br>
<br>
While I appreciate Mr. Lato's development of the package, I find it<br>
far too large, and its documentation too sparse, to effectively use.<br>
To correct this, I've written the "enumerator" package. It is also<br>
derived from Oleg's IterateeM.hs , but with a simplified API and<br>
significantly reduced dependency list.<br>
<br>
Hackage entry: <a href="http://hackage.haskell.org/package/enumerator" target="_blank">http://hackage.haskell.org/package/enumerator</a><br>
Haddock docs: <a href="http://ianen.org/haskell/enumerator/api-docs/" target="_blank">http://ianen.org/haskell/enumerator/api-docs/</a><br>
Source code (literate PDF): <a href="http://ianen.org/haskell/enumerator/enumerator.pdf" target="_blank">http://ianen.org/haskell/enumerator/enumerator.pdf</a><br>
<br>
darcs get <a href="http://ianen.org/haskell/enumerator/" target="_blank">http://ianen.org/haskell/enumerator/</a><br>
<br>
Additionally, I've included examples of using enumerators to implement<br>
simplified versions of the "cat" and "wc" utilities. These should<br>
serve as a useful starting point for anybody who wants to use<br>
enumerators in their own code:<br>
<br>
<a href="http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs" target="_blank">http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs</a><br>
<a href="http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs" target="_blank">http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs</a><br>
<br>
There are already a few libraries using the existing "iteratee"<br>
package (snap, attoparsec-iteratee, hexpat-iteratee); I am very<br>
interested in advice from the authors of these libraries. In<br>
particular, are any of the removed features (ListLike,<br>
WrappedByteString, seeking) something your libraries depend on? Are<br>
there any useful combinators you'd like to see included?<br>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/haskell-cafe" target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br>
</blockquote></div><br></div></div>