<div dir="ltr"><br><br><div class="gmail_quote">On Wed, Aug 18, 2010 at 2:39 PM, Johan Tibell <span dir="ltr"><<a href="mailto:johan.tibell@gmail.com">johan.tibell@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="gmail_quote"><div class="im">On Wed, Aug 18, 2010 at 2:12 AM, John Meacham <span dir="ltr"><<a href="mailto:john@repetae.net" target="_blank">john@repetae.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<ranty thing to follow><br>
That said, there is never a reason to use UTF-16, it is a vestigial<br>
remanent from the brief period when it was thought 16 bits would be<br>
enough for the unicode standard, any defense of it nowadays is after the<br>
fact justification for having accidentally standardized on it back in<br>
the day.</blockquote></div><div><br>This is false. Text uses UTF-16 internally as early benchmarks indicated that it was faster. See Tom Harper's response to the other thread that was spawned of this thread by Ketil.<br>
</div>
</div><br>Text continues to be UTF-16 today because<br><br> * no one has written a benchmark that shows that UTF-8 would be faster *for use in Data.Text*, and<br> * no one has written a patch that converts Text to use UTF-8 internally.<br>
<br>I'm quite frustrated by this whole discussion; there's lots of talking, no coding, and only a little benchmarking (of web sites, not code). This will get us nowhere.<br><br></blockquote><div>Here's my response to the two points:</div>
<div><br></div><div>* I haven't written a patch showing that Data.Text would be faster using UTF-8 because that would require fulfilling the second point (I'll get to in a second). I *have* shown where there are huge performance differences between text and ByteString/String. Unfortunately, the response has been "don't use bytestring, it's the wrong datatype, text will get fixed," which is quite underwhelming.</div>
<div><br></div><div>* Since the prevailing attitude has been such a disregard to any facts shown thus far, it seems that the effort required to learn the internals of the text package and attempt a patch would be wasted. In the meanwhile, Jasper has released blaze-builder which does an amazing job at producing UTF-8 encoded data, which for the moment is my main need. As much as I'll be chastised by the community, I'll stick with this approach for the moment.</div>
<div><br></div><div>Now if you tell me that text would consider applying a UTF-8 patch, that would be a different story. But I don't have the time to maintain a separate UTF-8 version of text. For me, the whole point of this discussion was to determine whether we should attempt porting to UTF-8, which as I understand it would be a rather large undertaking.</div>
<div><br></div><div>Michael</div></div></div>