[Haskell-cafe] hs-plugins and memory leaks

[Don Stewart]

qdunkan:
> However, there are some questions I've had about it for a long time.
> The 'yi' paper mentions both 'yi' and 'lambdabot' as users of
> hs-plugins.  However, both those projects have long since abandoned
> it.  I can't find any documentation on why, or even any documentation
> at all for Yi wrt its dynamic code execution system, but from looking
> at the source it looks like it uses hint for dynamic code execution
> and dyre for configuration.  Dyre in turn uses serialization to pass
> the old state to the reconfigured app.  So we have retreated from the
> idea of hotswapping the application state.

Once active development of hs-plugins stopped, along with the
portability issues, it behooved projects like e.g. xmonad or yi, to aim
for simpler reconfiguration strategies, other than native code hot
loading.

> I can think of one possible reason, and that's a memory leak.  In
> ghc/rts/Linker.c:unloadObj there's a commented out line '//
> stgFree(oc->image);'.  In a test program I wrote that behaves like
> 'plugs', every executed line increases the size of the program by
> 12-16k.  I have to remove the resolveObjs call from plugs for it to
> work, but once I do it displays the same leak.
  
> So my questions are:
> 
> Why did lambdabot and yi abandon plugins?

Because it was unmaintained for around 5 years, and was fundamentally
less portable than simpler state serialization solutions that offered
some of the same benefits as full code hot swapping.

> Is unloadObj a guaranteed memory leak?  As far as I can tell, it's
> never called within ghc itself.  If the choices are between a memory
> leak no matter how you use it and dangerous but correct if you use it
> right, shouldn't we at least have the latter available as an option?
> E.g. a reallyUnloadObj function that also frees the image.

GHC never unloads object code, so yes, it will "leak" old code.

> Long shot, but are there any more principled ways to guarantee no
> pointers to a chunk of code exist?  The only thing I can think of is
> to have the state be totally strict and consist only of types from the
> static core.  Would it be possible to hand responsibility for the
> memory off to the garbage collector?

It's really hard.
  
-- Don