[Haskell-cafe] real-time audio processing [Was: can Haskell do everyting as we want?]

[Stephen Sinclair]

On Aug 3, 8:31 pm, Jeremy Shaw <jer... at n-heptane.com> wrote:
>  The only area I have had any trouble with Haskell is doing realtime
> music synthesis. And only because the garbage collector is not
> realtime friendly. That is not unfixable though. However, I am
> thinking that the best way to do realtime synthesis with Haskell is to
> use it to create a DSL that uses LLVM to create code at runtime so
> that the realtime code is outside the scope of the normal RTS and
> garbage collector.

I'm also very interested in this topic---how to apply a general-
purpose functional language to real-time needs, even if it is in a
domain-specific way.

Indeed, it mostly comes down to memory management and the fact that
functional concepts like closures and laziness require a lot of
dynamic allocation and garbage collection.  Certainly, one solution is
to provide a real-time-friendly memory manager.

However, it's interesting to notice that, as proven by FAUST [1], a
huge amount of DSP algorithms can be expressed functionally in a real-
time-compatible way by describing them as static diagrams of connected
blocks.  These can be efficiently compiled to imperative code with no
dynamic memory allocation required.

So yes, if such a language were available as an embedded DSL in
Haskell (one effort can be found here [2]), it could be generated at
run-time using LLVM and called out to.  Alternatively, it would be
very cool if it were possible to generate code statically at compile
time, just like in FAUST.  I can imagine this being very useful,
especially if it could be generalized to operate on datatypes other
than floating points, and had easy access to data structures provided
by non-RT portions of the code.  If there are mutability requirements,
it could be made to run in the ST monad with a pre-allocated
workspace.

Of course _modifying_ such structures at run-time is always a dynamic
thing by definition, although there is the possibility of dynamically
generating a replacement block diagram while an one existing one runs,
and using an atomic pointer swap to switch them without causing
interruptions.

In any case, as far as I know the only thing in the way is that it's
impossible to tell GHC to compile a section of code in such as way as
to guarantee avoidance of memory management.  Compilers and runtime-
systems always seem to be either RT- or non-RT-friendly, but never
seem to support the idea of code *portions* that have differing
requirements.  My point is, RT code _can_ be expressed functionally,
even if the RT-ness imposes certain expressivity restrictions.  It
would be very cool to be able to mark sections of code as following
such a "sub-language" and be guaranteed that the compiler will
generate GC-free code for a particular function, if possible, or
otherwise fail.

Even if such a sub-language were no more expressive than C, it would
be nice to be able to write it in Haskell instead of dropping down to
C, so that data can be easily shared with non-RT parts, and Haskell's
type checker could be exploited.  It seems strange to me that with
technology like Haskell and GHC we still depend on using C to express
these last remaining droplets of real-time determinism requirements---
strange, since I think of higher-level languages like Haskell to be
supersets of the capabilities of C---and annoying, because it means
having to deal with the complexities of language interoperability,
just for a few low-level components of an application.

I realize some of this has probably been discussed in conjunction with
FRP.  I'm aware of one paper on RT-FRP that talks about requiring an
RT-friendly sub-language [3], but I don't pretend to follow it
completely.  I'm almost sure it also has something to do with arrows,
but I have very little idea what they are, since I'm still just
"getting" monads at this point in my personal Haskell understanding.
I hope someone more knowledgeable about these things on this list
might be able to comment on their relation to real-time determinism.

[1] http://faust.grame.fr/
[2] http://claudiusmaximus.goto10.org/cm/2009-12-04_heist_dataflow_algebra.html
[3] http://www.haskell.org/frp/rt-frp.pdf