haddock 2 and GHC

[Claus Reinke]

>> 1 updating ghc-paths and notifying existing clients
>> 2 making (some) ghc api clients less dependent on a single ghc    version
> 
> dynamic linking will solve (1), but at this stage I don't think we have 
> time to get dynamic linking fully working and in the binary distributions 
> for 6.10.1.  We might have it working in a build-from-source form, though.

http://hackage.haskell.org/trac/ghc/wiki/Status/Releases

"Things we plan to do for sure" needs an update, then? Just to
clarify that the 6.10 September RC will be for 6.10.1, and that
shared libs are still certain for 6.10.x, just not 6.10.1?
 
> As I've said before regarding (2), it's feasible to make the .hi format 
> stable across minor releases of GHC, but not major releases.  Making an 
> extensible .hi format seems completely unrealistic - it's not just the 
> format that changes, but the semantics.  "Losing inforamtion" might be 
> completely disastrous if you actually *needed* that information.  Not to 
> mention the fact that trying to do both forwards and backwards 
> compatibility in anything but a trivial way gives you a quadratic-sized 
> testing surface, which is something we really have to worry about.

Yes, I noticed that you stopped reading my messages about this
a while ago;-) else you'd have noticed that I no longer talk about
keeping the .hi file stable, nor about splitting it into a stable and a
version-specific section. And I specifically restricted my wishlist
to ghc api clients who do not need that extra information. 

I don't know to what extent what I'm asking for is possible (well, 
it is obviously possible, but whether it is practical is another question).
And you've convinced me that there are situations that wouldn't
be practical to cover.

But saying that the incompatibilities are difficult to handle at the
source only re-emphasizes my point: I would like to avoid every 
ghc api user and every ghc api client user having to worry about 
this versionitis, which gives a rather larger testing surface than the 
one you're worried about.

------------ the suggestion

What I've been suggesting in more recent messages is for each
ghc version to be able to show its own .hi format in a not version-
specific form, and for each ghc version to be able to read such
a not version-specific form, similar to Malcolm's recent suggestion.

As I see it, there's a range of levels at which interface files can
be represented:

1 complete haskell source(.hs): all information, but unprocessed
2 subset haskell source (.hs-boot): only interface-relevant info,
    but unprocessed
3 internal interface format (.hi): only interface-relevant info,
    processed and ready to use

Since typical ghc-based haskell environments don't have 
library sources (it would be great if Cabal optionally installed 
package sources as well, btw, just as it does documentation), 
we need something that can be generated from .hi files. 

Would it be possible to extract Haskell source files (somewhere
between levels 1 and 2 above) from which the given .hi files could 
have been generated, a kind of of reverse-engineered 
"Haskell source level interface format" (.hsi)? Any GHC version
reading such a .hsi file could then do its own processing to generate
its own .hi file format for that.

--------------- example scenario

The most naive implementation of this suggestion would be for
each ghc to be able to show and read .hi-file info in Haskell
source form. Concretely:

the issue again

- ghc api client GAC (think Haddock/HaRe/tag file generator/..), 
    built with ghc 6.10
- haskell sources HS, buildable against ghc 6.10, but currently
    using ghc 6.11 (6.10 has been replaced and is no longer installed)
=> GAC can't process HS

current workarounds

- rebuild GAC with ghc 6.11, which might not be possible
    if the ghc api details have changed (which is usually the
    reason for changing ghc versions), and leads to one version
    of each GAC per ghc installation
- rebuild the libraries that HS depends on with ghc 6.10, which 
    depends on keeping all the old ghc versions around as long 
    as there are any ghc api clients that were built with any of 
    them; not to mention that the library code may not be available
    for either 6.10 or 6.11, or may not be trivial to build (think
    binary releases and build environments), and that there are 
    now two (or more) versions of HS+build-depends

suggestions

- we've already established that we're in trouble if
    - GAC actually wants to run the code
    - GAC needs to refer to any other details that
        have changed between 6.10 and 6.11
    - HS or its imports don't build with GAC's ghc

- but if that isn't the case:
    - ask ghc 6.11 to explain its .hi files in Haskell terms
        ghc6.11 --show-iface-haskell libdir/*.hi -hsiodir hsi

    - direct ghc-api 6.10 to consult hsi files for missing libs
        GAC +RTS -hsidir hsi

Again, this will not help 6.10 GACs to process sources that 
depend on features new in 6.11. But I don't see why it wouldn't
allow 6.10 GACs to process sources that haven't changed, just
happen to be hosted over 6.11 now. And I don't see why I should
need more than one version of any (non-rts) ghc api client, either: 
the latest version should be able to process sources hosted in older
ghc versions.

Take this "typical Haskell project";-)

    -- | 'HelloWorld' module
    module HelloWorld where
    -- | 'main' function
    main = putStrLn "Hello World!"

Any haddock or ghc version should be able to process that,
whether haddock is a ghc api client or not. And that should still
be possible if we import a package that could be built with any
ghc version.

Claus