Advance notice that I'd like to make Cabal depend on parsec

[Simon Peyton-Jones]

Is it essential, or even sensical, that the serialization format GHC needs for storing package info bear any relation to the human authored form? If not, the split out of the package types could be accomplished in a way where GHC uses simple show/read(P) style serialization for storage of package info, where as cabal-lib would use a lovely parsec parser for humans. I'd like this approach.

Good idea -- esp if it makes the packaging story simpler.  GHC already uses a binary format for interface files, so there’s no good reason to use a human-readable format for package data base stuff.  For interface files you can read them with ghc --show-iface, and as Ian remarks something similar is already true for the package data base.

Simon

From: ghc-devs-bounces at haskell.org [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Mark Lentczner
Sent: 17 March 2013 16:57
To: dag.odenhall at gmail.com
Cc: Haskell Libraries; cabal-devel; Duncan Coutts; ghc-devs at haskell.org; Antoine Latter
Subject: Re: Advance notice that I'd like to make Cabal depend on parsec

This thread is raising all sorts of questions for me:

Is it essential, or even sensical, that the serialization format GHC needs for storing package info bear any relation to the human authored form? If not, the split out of the package types could be accomplished in a way where GHC uses simple show/read(P) style serialization for storage of package info, where as cabal-lib would use a lovely parsec parser for humans. I'd like this approach.

The issue of putting the yet one more HP package into GHC's core packages is increasing the exposure of the difficulty of the current GHC/HP relationship. See also threads in HP's mailing list for why can't we bump some packages in GHC's core set for the next HP release. The split arrangement is strange because we have two groups making up what is in the HP, but they have different processes and aims. The complex technical relationship between the moving parts only heightens the difficulty.

Perhaps the major cause is that because GHC is shipped as a library itself, it exposes all it's package dependencies. And as it is a large, and growing, piece of software, the list only wants to grow. But I wonder how often GHC is used as a library itself? If not often, then perhaps GHC should be shipped as two parts: Just a compiler (plus the small number of packages that the compiler forces), and ghc-lib as an optional, even separate, package - perhaps one with even a traditional way of depending on other packages. In otherwords, users that wanted to incorporate the ghc-lib into their programs would depend, and download, and configure, and build, ghc-lib indpenendant of the GHC binaries installed on their system. Perhaps then, GHC, the compiler, built from ghc-lib, would be bootstrapped not from the past compiler, but from the past HP.....

Okay, perhaps that is all just fantasy. But, no other programming system operates the way we do. They all fall into one of two camps:

  *   The dominant implementation is maintained, built, and shipped along with a large collection of "common packages". Examples: Python, Ruby, PHP, Java.
  *   The dominant implementation is shipped as a bare tool, and large common libraries are maintained and shipped independently. Examples: C++ (think g++ and boost), JavaScript (think browsers, and jQuery).
We are in the middle and, I think, experiencing growing pains because of it.

- Mark

On Sat, Mar 16, 2013 at 3:42 PM, dag.odenhall at gmail.com<mailto:dag.odenhall at gmail.com> <dag.odenhall at gmail.com<mailto:dag.odenhall at gmail.com>> wrote:
I'd love to have a proper parser and source-location-aware AST for sake of editor/IDE tools, so +1 from me. If you don't end up doing this after all, I'd still like to see your parser in a separate package, although I understand if you don't feel like maintaining two parsers especially given the tedious process for verifying they work similarly. I guess it could still be useful in the same way we find haskell-src-exts useful despite some incompatibilities with GHC.

On Thu, Mar 14, 2013 at 3:53 PM, Duncan Coutts <duncan.coutts at googlemail.com<mailto:duncan.coutts at googlemail.com>> wrote:
Hi folks,

I want to give you advance notice that I would like to make Cabal depend
on parsec. The implication is that GHC would therefore depend on parsec
and thus it would become a core package, rather than just a HP package.
So this would affect both GHC and the HP, though I hope not too much.

The rationale is that Cabal needs to parse things, like .cabal files and
currently we do not have a decent parser in the core libraries. By
decent I mean one that can produce error messages with source locations
and that doesn't have unpredictable memory use. The only parser in the
core libraries at the moment is Text.ParserCombinators.ReadP from the
base package and that fails my "decent" criteria on both counts. Its
idea of an error message is (), and on some largish .cabal files we take
100s of MB to parse (I realise that the ReadP in the base package is a
cutdown version so I don't mean to malign all ReadP-style libs out
there).

Partly due to the performance problem, the terrible .cabal file error
messages, and partly because Doaitse Swierstra keeps asking me if .cabal
files have a grammar, I've been writing a new .cabal parser. It uses an
alex lexer and a parsec parser. It's fast and the error messages are
pretty good. I have reverse engineered a grammar that closely matches
the existing parser and .cabal files in the wild, though I'm not sure
Doaitse will be satisfied with the approach I've taken to handling
layout.

Why did I choose parsec? Practicality dictates that I can only use
things in the core libraries, and the nearest thing we have to that is
the parser lib that is in the HP. I tried to use happy but I could not
construct a grammar/lexer combo to handle the layout (also, happy is not
exactly known for its great error messages).

I've been doing regression testing against hackage and I'm satisfied
that the new parser matches close enough. I've uncovered all kinds of
horrors with .cabal files in the wild relying on quirks of the old
parser. I've made adjustments for most of them but I will be breaking a
half dozen old packages (most of those don't actually build correctly
because though their syntax errors are not picked up by the parser, they
do cause failure eventually).

So far I've just done the outline parser, not the individual field
parsers. I'll be doing those next and then integrate. So this change is
still a bit of a ways off, but I thought it'd be useful to warn people
now.

Duncan

_______________________________________________
cabal-devel mailing list
cabal-devel at haskell.org<mailto:cabal-devel at haskell.org>
http://www.haskell.org/mailman/listinfo/cabal-devel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/cabal-devel/attachments/20130318/2cd32f1b/attachment-0001.htm>