bulatz at HotPOP.com
Fri Feb 3 04:53:41 EST 2006
Friday, February 03, 2006, 3:39:38 AM, you wrote:
>> Got a unicode-compliant compiler?
JM> sure do :)
JM> but it currently doesn't recognize any unicode characters as possible
are you read this? :)
> Add support for UTF-8 source files
> GHC finally has support for full Unicode in source files. Source
> files are now assumed to be UTF-8 encoded, and the full range of
> Unicode characters can be used, with classifications recognised
> the implementation from Data.Char. This incedentally means that
> the stage2 compiler will recognise Unicode in source files, because
> was too lazy to port the unicode classifier code into libcompat.
> Additionally, the following synonyms for keywords are now
> forall symbol (U+2200) forall
> right arrow (U+2192) ->
> left arrow (U+2190) <-
> horizontal ellipsis (U+22EF) ..
> there are probably more things we could add here.
> This will break some source files if Latin-1 characters are being
> In most cases this should result in a UTF-8 decoding error. Later
> if we want to support more encodings (perhaps with a pragma to
> the encoding), I plan to do it by recoding into UTF-8 before
> Internally, there were some pretty big changes:
> - FastStrings are now stored in UTF-8
> - Z-encoding has been moved right to the back end. Previously we
> used to Z-encode every identifier on the way in for simplicity,
> and only decode when we needed to show something to the user.
> Instead, we now keep every string in its UTF-8 encoding, and
> Z-encode right before printing it out. To avoid Z-encoding the
> same string multiple times, the Z-encoding is cached inside the
> FastString the first time it is requested.
> This speeds up the compiler - I've measured some definite
> improvement in parsing at least, and I expect compilations
> to be faster too. It also cleans up a lot of cruft from the
> OccName interface. Z-encoding is nicely hidden inside the
> Outputable instance for Names & OccNames now.
> - StringBuffers are UTF-8 too, and are now represented as
> - I've put together some test cases, not by any means exhaustive,
> but there are some interesting UTF-8 decoding error cases that
> aren't obvious. Also, take a look at unicode001.hs for a demo.
Bulat mailto:bulatz at HotPOP.com
More information about the Haskell-prime