<br><br><div class="gmail_quote">On Tue, Aug 24, 2010 at 12:49 AM, Heinrich Apfelmus <span dir="ltr"><<a href="mailto:apfelmus@quantentunnel.de">apfelmus@quantentunnel.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">Jason Dagit wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
>From a purely practical viewpoint I feel that treating the chunking<br>
as an abstraction leak might be missing the point. If you said, you<br>
wanted the semantics to acknowledge the chunking but be invariant<br>
under the size or number of the chunks then I would be happier.<br>
<br>
I use iteratees when I need to be explicit about chunking and when I<br>
don't want the resources to "leak outside" of the stream processing.<br>
If you took those properties away, I wouldn't want to use it anymore<br>
because then it would just be an inelegant way to do things.<br>
</blockquote>
<br></div>
I'm curious, can you give an example where you want to be explicit about<br>
chunking? I have a hard time imagining an example where chunking is<br>
beneficial compared to getting each character in sequence. Chunking<br>
seems to be common in C for reasons of performance, but how does that<br>
apply to Haskell?<br></blockquote><div><br></div><div>It applies to Haskell for the same reasons, as far as I can tell. You want it to manage performance characteristics. See my example below. If you wrote it using chunking you wouldn't need lazy io (which is argued quite well in other place to be bad and I assume you've read the arguments and more or less agree). Furthermore, wouldn't iteratees force you to implement something equivalent to either option #1 or #2, but #3 wouldn't be possible?</div>
<div><br></div><div>I think it basically comes down to this: We replace lazy io with explicit chunking because lazy io is unsafe, but explicit chunking can be safe.</div><div><br></div><div>So, if you had a lazy pure generator you wouldn't need chunking, although perhaps the iteratee style would help avoid accidental space leaks that happen from referencing the stream elements outside of the fold (like #3 below).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
<br>
On the matter of leaking resources outside the stream processing,<br>
Iteratee does not give you any guarantees, it's only a stylistic aid<br>
(which can be powerful, of course). For instance, the following Iteratee<br>
returns the whole stream as a list:<br></blockquote><div><br></div><div>I think your example is fine. I consider it a misbehaving iteratee, in the same way that returning any large structure would be misbehaving in this context. I think, if the iteratee returns something large that's different than letting things "leak out". It's like a difference of scope. A well-behaved iteratee will reduce the input to reasonable return value. What would be bad, is if other bits of code could reference parts of the stream, while the iteratee is looking at it, and hold on to it. That would cause a space leak. An example of this bad behavior, would be to use readFile to read a file. Then compute two things: a) sum of the bytes in the file as Int32, b) length (in number of characters) of the file.</div>
<div><br></div><div>Supposing we use lazy io (Prelude.readFile): </div><div> 1) read the file, compute (a), close the file, read the file, compute (b), and finally close the file. You can do so in constant space.</div><div>
2) read the file, use one pass to calculate both (a) and (b) at the same time, then close the file. You can do so in constant space.</div><div> 3) read the file, use one pass to compute (a) followed by a pass to compute (b), then close the file. The space used will be O(filesize).</div>
<div><br></div><div>I consider option #3 to be letting the elements of the stream "leak out". The computation in (b) references them and thus the garbage collector doesn't free them between (a) and (b), and the optimizer cannot fuse (a) and (b) in all cases.</div>
<div><br></div><div>There is a fourth option, and that is to use strict io but then each of the above takes space O(filesize).</div><div><br></div><div>I hope that makes sense. It's getting late here and I could be talking non-sense, but I have tried the above 3 cases in the past and as best as I can recall those were my findings.</div>
<div><br></div><div>Thanks,</div><div>Jason</div></div>