Making GHCi awesomer?

Christopher Done chrisdone at gmail.com
Sat Oct 18 15:48:48 UTC 2014


Good evening,

So I’ve been working on Haskell user-facing tooling in general for
some years. By that I mean the level of Emacs talking with Haskell
tools.

I wrote the interactive-haskell-mode (most functionality exists
in this file
<https://github.com/haskell/haskell-mode/blob/master/haskell-process.el#L1>
).
which launches a GHCi process in a pipe and tries very earnestly to
handle input/output with the process reasonably.

For Emacs fanciers: Written in Elisp, there’s a nice command queue
that you put commands onto, they will all be run on a FIFO one-by-one
order, and eventually you’ll get a result back. Initially it was just
me using it, but with the help of Herbert Riedel it’s now a mode on
equal footing with the venerable inferior-haskell-mode all ye Emacs
users know and love. It’s part of haskell-mode and can be enabled by
enabling the interactive-haskell-mode minor mode.

For years I’ve been using GHCi as a base and it’s been very reliable
for almost every project I’ve done (the only exceptions are things
like SDL and OpenGL, which are well known to be difficult to load in
GHCi, at least on Linux). I think we’ve built up
a good set of functionality
<https://github.com/haskell/haskell-mode/wiki/Haskell-Interactive-Mode>
purely based on asking GHCi things and getting it to do things.

I literally use GHCi for everything. For type-checking, type info, I
even send “:!cabal build” to it. Everything goes through it. I love my
GHCi.

Now, I’m sort of at the end of the line of where I can take GHCi. Here
are the problems as I see them today:

   1. There is no programmatic means of communicating with the
   process. I can’t send a command and get a result cleanly, I have to
   regex match on the prompt, and that is only so reliable. At the
   moment we solve this by using \4 (aka ‘END OF TRANSMISSION’). Also
   messages (warnings, errors, etc.) need to be parsed which is also
   icky, especially in the REPL when e.g. a defaulted Integer warning
   will mix with the output. Don’t get me started on handling
   multi-line prompts! Hehe.
   2. GHCi, as a REPL, does not distinguish between stdout, stderr and
   the result of your evaluation. This can be problematic for making a
   smooth REPL UI, your results can often (with threading) be
   interspersed in unkind ways. I cannot mitigate this with any kind
   of GHCi trickery.
   3. It forgets information when you reload. (I know this is intentional.)
   4. Not enough information is exposed to the user. (Is there ever? ;)
   5. There is a time-to-market overhead of contributing to GHCi — if I
   want a cool feature, I can write it on a locally compiled version
   of GHC. But for the work projects I have, I’m restricted to given
   GHC versions, as are other people. They have to wait to get the
   good features.
   6. This is just a personal point — I’ve like to talk to GHCi over a
   socket, so that I can run it on a remote machine. Those familiar
   with Common Lisp will be reminded of SLIME and Swank.

Examples for point 4 are:

   - Type of sub-expressions.
   - Go to definition of thing at point (includes local scope).
   - Local-scope completion.
   - A hoogle-like query (as seen in Idris recently).
   - Documentation lookup.
   - Suggest imports for symbols.
   - Show core for the current module.
   - Show CMM for the current module, ASM, etc. SLIME can do this.
   - Expand the template-haskell at point.
   - The :i command is amazingly useful, but programmatic access would be
   even better.¹
   - Case split anyone?
   - Etc.

¹I’ve integrated with it in Emacs so that I can C-c C-i any identifier
and it’ll popup a buffer with the :i result and then within that
buffer I can drill down further with C-c C-i again. It makes for
very natural exploration of a type.

You’ve seen some of these features in GHC Mod, in hdevtools, in the FP
Haskell
Center, maybe some are in Yi, possibly also in Leksah (?).

So in light of point (5), I thought: I’ve used the GHC API before, it
can do interactive evaluation, why not write a project like
“ghc-server” which encodes all these above ideas as a “drop-in”
replacement for GHCi? After all I could work on my own without anybody
getting my way over architecture decisions, etc.

And that’s what I did. It’s
here <https://github.com/chrisdone/ghc-server>. Surprisingly, it kind of
works. You run it in your directoy like you would do “cabal repl”
and it sets up all the extensions and package dependencies and starts
accepting connections. It will compile across three major GHC
versions. Hurray! Rub our hands together and call it done, right?
Sadly not, the trouble is twofold:

   1. The first problem with this is that every three projects will
   segfault or panic when trying to load in a project that GHCi will
   load in happily. The reasons are mysterious to me and I’ve already
   lugged over the GHC API to get to this point, so that kind of thing
   happening means that I have to fall back to my old GHCi-based
   setup, and is disappointing. People have similar complaints of GHC
   Mod & co. “Getting it to work” is a deterrant.
   2. While this would be super beneficial for me, and has been a good
   learning experience for “what works and what doesn’t”, we end up
   with yet another alternative tool, that only a few people are
   using.
   3. There are just certain behaviours and fixes here and there that
   GHCi does that take time to reproduce.

So let’s go back to the GHCi question: is there still a development
overhead for adding features to GHCi? Yes, new ideas need acceptance
and people have to wait (potentially a year) for a new feature that
they could be using right now.

An alternative method is to do what Herbert did which is to release a
“ghci-ng” <http://hackage.haskell.org/package/ghci-ng> which sports
new shiny features that people (with the right GHC version) will be
able to compile and use as a drop-in for GHCi. It’s the same codebase,
but with more stuff! An example is the “:complete” command, this lets
IDE implementers do completion at least at the REPL level. Remember
the list of features earlier? Why are they not in GHCi?

So, of course, this got me thinking that I could instead make
ghc-server be based off of GHCi’s actual codebase. I could rebase upon
the latest GHC release and maintain 2-3 GHC versions backwards. That’s
certainly doable, it would essentially give me “GHCi++”. Good for me,
I just piggy back on the GHCi goodness and then use the GHC API for
additional things as I’m doing now.

But is there a way I can get any of this into the official repo? For
example, could I hack on this (perhaps with Herbert) as “ghci-ng”,
provide an alternative JSON communication layer (e.g. via some
—use-json flag) and and socket listener (—listen-on ), a way
to distinguish stdout/stderr (possibly by forking a process, unsure at
this stage), and then any of the above features (point 4) listed. I
make sure that I’m rebasing upon HEAD, as if to say ghci-ng is a kind
of submodule, and then when release time comes we merge back in any
new stuff since the last release. Early adopters can use
ghci-ng, and everyone benefits from official GHC releases.

The only snag there is that, personally speaking, it would be better
if ghci-ng would compile on older GHC versions. So if GHC 7.10 is the
latest release, it would still be nice (and it *seems* pretty
feasible) that GHC 7.8 users could still cabal install it without
issue. People shouldn’t have to wait if they don’t have to.

Well, that’s everything. Thoughts?

Ciao!
​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20141018/d6b9a555/attachment-0001.html>


More information about the ghc-devs mailing list