[Haskell-cafe] Parsec question

[Tomasz Zielonka]

On Fri, 19 Nov 2004 14:28:07 +0000 (UTC), John Goerzen
<jgoerzen at complete.org> wrote:
> Hi,
>
> I'm porting a parser over from an OCamllex/Ocamlyacc version and I'm
> using Parsec for both the tokenizer and the resulting token stream parser.
>
> I have both of them working fine, but now my question is: how do I
> combine them?  I can't quite figure out how to say "take the output from
> this GenParser Char () Tok and use it as the input for this
> GenParser Tok () ParserOutput".
>
> Well, I have figured out an ugly way involving manually starting up both
> parsers, but it makes position calculations very complex.

Did you try to pair tokens with their SourcePos'es?

What I did in my program was this:

tokenize :: CharParser st tok -> CharParser st [(SourcePos, tok)]
tokenize tp = do
    l <- many t
    eof
    return l
  where
    t = do
        pos <- getPosition
        tok <- tp
        return (pos, tok)

token :: (Show tok) => (tok -> Maybe a) -> GenParser (SourcePos, tok) st a
token f = Parsec.token (show . snd) fst (f . snd)

This has a space leak, but I didn't care, because my inputs are at
most 10kb long.
You can do a lazy version with getInput.

BTW, it would be useful if Parsec allowed to throw a ParseError from one type
of parser in another parser. It would help me in a situation, where I parse the
whole file with one parser, but I retokenize some parts of it and parse them
with another parser type. Right now I am using this workaround, but it doesn't
work well:

        cs <- chars
        pos <- getPosition
        x <- case p pos cs of
                Left err -> do
                    setPosition (errorPos err)
                    fail (messageString (head (errorMessages err)))
                Right x -> return x

How about making an instance for

  instance Control.Monad.Error.MonadError ParseError (GenParser tok st)

PS. I guess I could do this myself.

Best regards,
Tomasz