extending GHC plugins with Hooks

Tue Aug 27 18:18:28 CEST 2013

I think there are a couple of things that are worth exposing.  Looking
at Luite's patch, for example, there are a few that a basic high-level
platform wouldn't solve.  A few things an API might reasonably want to
customise:

  - Parse custom flags (when wrapping GHC)

  - Hook into GHC's compilation manager or use one-shot mode and use
Shake as a more flexible compilation manager

  - Foreign function calling conventions (e.g., for GHCJS)

  - Compilation ways (can't think of a good example here)

  - The location of built-in / hardcoded identifiers (i.e., the PrelNames stuff)

  - How modules are found (Finder) and where they are stored (e.g.,
network-backed store, in-memory only)

  - Wrap/Adapt various stages of the
renamer/typechecker/optimiser/codegen.  Mostly for IDE tools
(including tools to examine compiler transformations), but custom
backends also need to replace the codegen bits

  - Run custom stages in DriverPipeline or adapt existing ones
(including not running them)

  - Both Lambdachine and GHCJS implement a custom code generator.  I
think GHCJS is translating from STG.  Lambdachine used to start from
CorePrep, but due to the weird "unarise" stuff I had to switch to
using STG which isn't too bad as it also seems more stable. A custom
code generator requires disabling the "Asm" pass and all subsequent
passes from DriverPipeline

  - I'm not so sure about syntactic extensions.  They might be better
implemented using haskell-src-exts and then translated into plain
Haskell

If we can get these things integrated nicely then it would be
worthwhile to wait another year.

On 27 August 2013 09:17, Simon Peyton-Jones <simonpj at microsoft.com> wrote:
> I’m sympathetic to supporting GHCJS.  But, as described in my email, there
> seems to be several different proposals for extending plugins swirling
> around, and I’m not sure how they relate to each other.
>
>
>
> My suggestion (for you guys to lead J):
>
> ·         Identify the minimum necessary for GHCJS in 7.8
>
> ·         Debate the remaining proposals under less time pressure
>
>
>
> Simon
>
>
>
> From: Carter Schonwald [mailto:carter.schonwald at gmail.com]
> Sent: 23 August 2013 19:36
> To: Daniel F
> Cc: Simon Peyton-Jones; ghc-devs
> Subject: Re: extending GHC plugins with Hooks
>
>
>
> indeed, ghcjs is one of the things i'm most looking forward to!
>
> That the hooks machinery needed to nicely support GHCJS will enable many
> other use cases is also something i'm excited about too... would be a shame
> for neither to work with 7.8 ....
>
>
>
> -Carter
>
>
>
> On Fri, Aug 23, 2013 at 2:09 PM, Daniel F <difrumin at gmail.com> wrote:
>
> Hello
>
>
>
> On Thu, Aug 22, 2013 at 8:13 PM, Simon Peyton-Jones <simonpj at microsoft.com>
> wrote:
>
> Luite, Edsko, Thomas, Nicolas
>
>
>
> You have all variously proposed improvements to the GHC API and/or the
> plug-in mechanism.  I have been so swamped in the last few months that I
> have not had a chance to look carefully at your proposals, nor how they
> relate to each other.
>
>
>
> We are now only three weeks away from wanting to do a feature freeze on GHC
> 7.8, and there are a lot of other things that we want to complete
>
>                 http://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.8
>
> (Mostly they have gestating for some time.)
>
>
>
> So I’m hoping you’ll be ok with not putting these plugin-related changes
> into 7.8.  I have the sense that they’d benefit from more discussion among
> the folk interested in plugins.  Perhaps some of the ideas could be combined
> nicely; I don’t know.  And the people who are going to write plugins are
> also probably up for building HEAD anyhow.
>
>
>
> (Exception: Luite, I think you have some fairly narrow, specific changes
> that would help GHCJS, and I’m probably fine with those if you care to send
> patches.)
>
>
>
> Does this mean that GHCJS patches will be accepted, but it has to use some
> other mechanism (not Hooks)?
>
> It would be really nice to see ghcjs support in the upcoming GHC.
>
>
>
>
>
>
> Please say if you think there’s a really strong reason for putting stuff in
> the 7.8.
>
>
>
> Thanks
>
>
>
> Simon
>
>
>
> From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Luite
> Stegeman
> Sent: 21 August 2013 03:51
> To: ghc-devs
> Subject: extending GHC plugins with Hooks
>
>
>
> hi all,
>
>
>
> Sorry for taking so long to get back with this. I'm proposing a somewhat
> general way for adding 'hooks' to the GHC API, where users can override
> parts of the default compiling pipeline.
>
>
>
> Hooks are simply functions or actions that replace existing compiler
> functionality. This means that usually only one application can use a
> specific hook at a time.
>
>
>
> The obvious data structure to store the hooks is DynFlags. Unfortunately
> defining hooks in DynFlags directly would give birth to the mother of all
> import cycles, and it would also break the split-dll scheme on Windows. So
> here's the idea:
>
>
>
> - Define each hook in the module where it's exported
>
> - For each hook make a 'phantom' DataType and an instance for the Hook type
> familiy
>
> - Add a TypeRep based map in DynFlags [0]
>
> - For each hooked function, check for existence of a hook in DynFlags,
> otherwise run the default. Example:
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L83
>
>
>
> Now this approach does have some disadvantages:
>
> - No clear integration with existing plugins (I've tried adding an
> onLoadPlugin field to Plugin, where the Plugin could update DynFlags when
> it's loaded, but it was a bit messy, and plugins would not be loaded in time
> for some hooks, particularly what Edsko needs)
>
> - More of GHC depends on type families
>
> - Decentralized hooks definitions feel a bit messy
>
>
>
> So I'm open to suggestions for improvements (or replacements) of this
> scheme. I have some time the coming weeks to clean up or change the patch.
>
>
>
> We've been testing some hooks with GHCJS for a while, and so far they seem
> to provide what we need (but I'm going to doublecheck the coming weeks that
> we don't have missing functionality):
>
>
>
> - Customizations for linking JavaScript code with our own library locations
> [1]
>
> - Hooking into the DriverPipeline so we can use the compilation manager [2]
>
> - Desugaring customizations to remove some C-isms from the FFI code [3]
>
> - Typechecking foreign import javascript imports [4]
>
> - Override the built-in GHC.Prim so we can customize primop types [5]
>
>
>
> I think it's easy to add those for Edsko and Thomas as well.
>
>
>
> luite
>
>
>
> [0]
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L239
>
> [1]
> https://github.com/ghcjs/ghcjs/blob/master/src/Compiler/GhcjsHooks.hs#L44
>
> [2]
> https://github.com/ghcjs/ghcjs/blob/master/src/Compiler/GhcjsHooks.hs#L192
>
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L335
>
> [3] https://github.com/ghcjs/ghcjs/blob/master/src/Gen2/Foreign.hs#L67
>
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L83
>
> [4] https://github.com/ghcjs/ghcjs/blob/master/src/Gen2/Foreign.hs#L68
>
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L1374
>
> [5]
> https://github.com/ghcjs/ghcjs/blob/master/src/Compiler/GhcjsHooks.hs#L191
>
> https://github.com/ghcjs/ghcjs-build/blob/master/refs/patches/ghc-ghcjs.patch#L239
>
>
>
> (sorry the linked patch also contains other modifications required for
> GHCJS, that i'm going to submit separately)
>
>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
>
>
> --
> Sincerely yours,
> -- Daniil Frumin
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>