From kiwamu at debian.or.jp  Wed Feb 19 05:39:00 2014
From: kiwamu at debian.or.jp (Kiwamu Okabe)
Date: Wed, 19 Feb 2014 14:39:00 +0900
Subject: [jhc] Rewrite NetBSD kernel driver using Ajhc Haskell compiler
Message-ID: <CAEvX6dki6=N5D70pucRqsbVWhW8c9C9qbPKefYc1Um7UNiseqg@mail.gmail.com>

Hi all,

NetBSD kernel's HD Audio sound driver has interrupt handler.
The interrupt handler of the demo is re-written by Haskell language
using Ajhc Haskell Compiler http://ajhc.metasepi.org/.

Demo movie could be found at following link.

http://www.youtube.com/watch?v=XEYcR5RG5cA

At the demo, run following operations.

First, set breakpoint at the interrupt of finding headphone,
and see Haskell function names on backtrace.

Second, set breakpoint s_alloc() function, that allocate area
in Haskell heap. Make sure of calling the function while anytime
running kernel.

Nevertheless, playing wav file does not break up.

The source code is found at https://github.com/metasepi/netbsd-arafura-s1.
And a picture of the architecture is also at
http://www.flickr.com/photos/masterq/12616929055/.

Thank's,
-- 
Kiwamu Okabe

From kiwamu at debian.or.jp  Wed Feb 19 15:08:13 2014
From: kiwamu at debian.or.jp (Kiwamu Okabe)
Date: Thu, 20 Feb 2014 00:08:13 +0900
Subject: [jhc] [Haskell-cafe] Rewrite NetBSD kernel driver using Ajhc
	Haskell compiler
In-Reply-To: <5304C658.7050507@update.uu.se>
References: <CAEvX6dki6=N5D70pucRqsbVWhW8c9C9qbPKefYc1Um7UNiseqg@mail.gmail.com>
 <2562D045-28FB-4555-84F8-3326AFB0839F@gmail.com>
 <5304C658.7050507@update.uu.se>
Message-ID: <CAEvX6dn4wGbH3s7gbrcxyQ7xye+54=B3zvEwgP3MxDL6tqki5w@mail.gmail.com>

Hi Johnny,

On Wed, Feb 19, 2014 at 11:57 PM, Johnny Billquist <bqt at update.uu.se> wrote:
> Maybe someone with more insight could explain to an idiot like me how
> Haskell garbage collection is handled when running in the kernel?

First, s_alloc() is Ajhc Haskell compiler's allocator,
and gc_perform_gc() is GC runner.

https://github.com/ajhc/ajhc/blob/6c053b940812689fe22d2dbe2b7ad835dab8a3c7/rts/rts/gc_jgc.c#L560
https://github.com/ajhc/ajhc/blob/6c053b940812689fe22d2dbe2b7ad835dab8a3c7/rts/rts/gc_jgc.c#L119

The s_alloc() try to find free area. But if not, it runs GC.

https://github.com/ajhc/ajhc/blob/6c053b940812689fe22d2dbe2b7ad835dab8a3c7/rts/rts/gc_jgc.c#L435

By the way, where from s_alloc() get a memory chunk?
Now using kern_malloc(9).

https://github.com/metasepi/netbsd-arafura-s1/blob/arafura-s1/metasepi/sys/hsdummy/__dummy4jhc.c#L21

Regards,
-- 
Kiwamu Okabe

From kiwamu at debian.or.jp  Wed Feb 19 15:30:59 2014
From: kiwamu at debian.or.jp (Kiwamu Okabe)
Date: Thu, 20 Feb 2014 00:30:59 +0900
Subject: [jhc] [Haskell-cafe] Rewrite NetBSD kernel driver using Ajhc
	Haskell compiler
In-Reply-To: <CAEvX6dn4wGbH3s7gbrcxyQ7xye+54=B3zvEwgP3MxDL6tqki5w@mail.gmail.com>
References: <CAEvX6dki6=N5D70pucRqsbVWhW8c9C9qbPKefYc1Um7UNiseqg@mail.gmail.com>
 <2562D045-28FB-4555-84F8-3326AFB0839F@gmail.com>
 <5304C658.7050507@update.uu.se>
 <CAEvX6dn4wGbH3s7gbrcxyQ7xye+54=B3zvEwgP3MxDL6tqki5w@mail.gmail.com>
Message-ID: <CAEvX6dkYpcr1Janqwt6L+hqszjEE3b4hkiHP0y8wTrgQerKznw@mail.gmail.com>

Hi Johnny, I had forgotten important point.

> On Wed, Feb 19, 2014 at 11:57 PM, Johnny Billquist <bqt at update.uu.se> wrote:
>> Maybe someone with more insight could explain to an idiot like me how
>> Haskell garbage collection is handled when running in the kernel?

Running a simple logic in the kernel, it doesn't call GC.

http://www.slideshare.net/master_q/20140117-11th-wocs2/19

Ajhc Haskell compiler creates program binary that has "context local heap".
A lot of binaries have only one global heap.
If C language code calls Haskell code, Ajhc runtime create new arena for GC.
Sometime Haskell code runs allocator (s_alloc()),
then the runtime allocates Heap area.
Haskell context gets the end of life, the arena and heap are recovered
by the runtime.

In the kernel, the recover is called before the GC.
But I think my custom kernel is ready to call GC in interrupt handler.

Thank's,
-- 
Kiwamu Okabe

From kiwamu at debian.or.jp  Thu Feb 20 04:34:08 2014
From: kiwamu at debian.or.jp (Kiwamu Okabe)
Date: Thu, 20 Feb 2014 13:34:08 +0900
Subject: [jhc] [Haskell-cafe] Rewrite NetBSD kernel driver using Ajhc
	Haskell compiler
In-Reply-To: <CAD=7U2A7DnRHEO-rHrGT0XkOXk+J7wAFLQu-0ewNTUju5Beo9A@mail.gmail.com>
References: <CAEvX6dki6=N5D70pucRqsbVWhW8c9C9qbPKefYc1Um7UNiseqg@mail.gmail.com>
 <2562D045-28FB-4555-84F8-3326AFB0839F@gmail.com>
 <5304C658.7050507@update.uu.se>
 <CAEvX6dn4wGbH3s7gbrcxyQ7xye+54=B3zvEwgP3MxDL6tqki5w@mail.gmail.com>
 <CAEvX6dkYpcr1Janqwt6L+hqszjEE3b4hkiHP0y8wTrgQerKznw@mail.gmail.com>
 <CAD=7U2A7DnRHEO-rHrGT0XkOXk+J7wAFLQu-0ewNTUju5Beo9A@mail.gmail.com>
Message-ID: <CAEvX6dmP1iC+-wOXd9Tmu3YcukkNbRygeRURioFnvC59cSPb=g@mail.gmail.com>

Hi Mike,

On Thu, Feb 20, 2014 at 8:34 AM, Mike Meyer <mwm at mired.org> wrote:
>> Running a simple logic in the kernel, it doesn't call GC.
>
> Having looked over the source, I couldn't get much of a feel for it -
> it mostly seemed to be FFI & type declarations. Which does make sense
> since the goal is to provide strong type checking in the kernel. Maybe
> I'm looking in the wrong place in that rather large repository?

You are right. It's just "exercise".
I would like to prove that can write code with functional language in
interrupt context.

> My question is how much does coding to meet the requirements for a
> device driver - not doing GC being one of them - warp the resulting
> Haskell code? Is it still pretty much idiomatic Haskell, or would it
> be easier for a Haskell programmer to figure out the C it replaced?
> Most of the time I've seen people try and get a modern language to
> meet such requirements (me included), you might as well have stuck
> with C as far as code improvement goes.

This exercise has logic code at following.

https://github.com/metasepi/netbsd-arafura-s1/blob/arafura-s1/metasepi/sys/hssrc/Dev/Pci/Auich.hs
https://github.com/metasepi/netbsd-arafura-s1/blob/arafura-s1/metasepi/sys/hssrc/Dev/Pci/Auich/Intr.hs
https://github.com/metasepi/netbsd-arafura-s1/blob/arafura-s1/metasepi/sys/hssrc/Dev/Pci/Hdaudio/Hdaudio.hs

I re-write these from NetBSD's implementation, for a month.
But now can get more speed with our new tool,
convert C language API definition into Haskell API definition
semi-automatically.

https://github.com/ajhc/struct2hs

Following is usage of the tool.

$ struct2hs $HOME/src/netbsd-arafura-s1/obj/tooldir/bin/i486--netbsdelf-gcc
"-Di386 -I$HOME/src/netbsd-arafura-s1/sys/arch/i386/compile/obj/GENERIC_HS
... sys/dev/pci/hdaudio/hdaudio.c | tail -15

newtype {-# CTYPE "struct pdevinit" #-} Pdevinit = Pdevinit ()
foreign import primitive "const.sizeof(struct pdevinit)"
  sizeOf_Pdevinit :: Int
foreign import primitive "const.offsetof(struct pdevinit, pdev_attach)"
  offsetOf_Pdevinit_pdev_attach :: Int
p_Pdevinit_pdev_attach :: Ptr Pdevinit -> IO (Ptr (Ptr (FunPtr (Int -> IO ()))))
p_Pdevinit_pdev_attach p = return $ plusPtr p $ offsetOf_Pdevinit_pdev_attach
foreign import ccall "dynamic" call_Pdevinit_pdev_attach ::
  FunPtr (Int -> IO ()) -> Int -> IO ()
foreign import primitive "const.offsetof(struct pdevinit, pdev_count)"
  offsetOf_Pdevinit_pdev_count :: Int
p_Pdevinit_pdev_count :: Ptr Pdevinit -> IO (Ptr Int)
p_Pdevinit_pdev_count p = return $ plusPtr p $ offsetOf_Pdevinit_pdev_count

> One of the advantages C has in the kernel - the BSD kernels, anyway,
> reading Linux kernel makes me queasy - is that it isn't that different
> from C in userland. And the kernel groups have been working on
> reducing over the differences.

I hope so...
But, I think Ajhc is not good for real kernel products,
after this hard challenge.
ATS language is much betther than Ajhc.
Hongwei, as ATS author, says following. I perfectly agree his opinion.

http://metasepi.org/posts/2013-12-24-jats-ug.html#%E4%B8%80%E9%80%9A%E3%81%AE%E3%83%A1%E3%83%BC%E3%83%AB

Regards,
-- 
Kiwamu Okabe

From kiwamu at debian.or.jp  Thu Feb 20 09:01:08 2014
From: kiwamu at debian.or.jp (Kiwamu Okabe)
Date: Thu, 20 Feb 2014 18:01:08 +0900
Subject: [jhc] [Haskell-cafe] Rewrite NetBSD kernel driver using Ajhc
	Haskell compiler
In-Reply-To: <5305BEB3.4090106@update.uu.se>
References: <CAEvX6dki6=N5D70pucRqsbVWhW8c9C9qbPKefYc1Um7UNiseqg@mail.gmail.com>
 <2562D045-28FB-4555-84F8-3326AFB0839F@gmail.com>
 <5304C658.7050507@update.uu.se>
 <CAEvX6dn4wGbH3s7gbrcxyQ7xye+54=B3zvEwgP3MxDL6tqki5w@mail.gmail.com>
 <CAEvX6dkYpcr1Janqwt6L+hqszjEE3b4hkiHP0y8wTrgQerKznw@mail.gmail.com>
 <5305BEB3.4090106@update.uu.se>
Message-ID: <CAEvX6dmJD4YbwMi1Mzj1JFiNq3zuksk93tf+ERD2nqxXucrYhw@mail.gmail.com>

Hi Johnny,

On Thu, Feb 20, 2014 at 5:37 PM, Johnny Billquist <bqt at update.uu.se> wrote:
> Are you saying that you essentially avoid GC by creating a large fresh heap
> every time you call something written in Haskell, and then delete the heap
> when the Haskell function returns? And that the current piece of code
> running in Haskell is short enough that GC never is done?

No, does not delete. Pool it.
See following code compiled by Ajhc.
Ajhc compile Haskell code into C code.

https://gist.github.com/master-q/9109334

Please note auichIntr() function that is entry point C => Haskell.

https://gist.github.com/master-q/9109334#file-hsmain-c-L2051

int
auichIntr(HsPtr x272)
{
        arena_t arena;
        gc_t gc;
        gc = NULL;
        arena = NULL;
        jhc_alloc_init(&gc,&arena);
        jhc_hs_init(gc,arena);
        int x273 = ((int)fFE$__CCall_auichIntr(gc,arena,(uintptr_t)x272));
        jhc_alloc_fini(gc,arena);
        return x273;
}

The code post-calls jhc_alloc_fini() that pool Haskell heap (named
megablock) into free_megablocks.

https://github.com/ajhc/ajhc/blob/d93468e34f4514209048d4a92b1549e079ccd3fb/rts/rts/gc_jgc.c#L251

void
jhc_alloc_fini(gc_t gc,arena_t arena) {
-- snip --
        SLIST_FOREACH(pg, &arena->monolithic_blocks, link) {
                SLIST_INSERT_HEAD(&free_monolithic_blocks, pg, link);
        }
        SLIST_FOREACH(mb, &arena->megablocks, next) {
                SLIST_INSERT_HEAD(&free_megablocks, mb, next);
        }
        if(arena->current_megablock) {
                SLIST_INSERT_HEAD(&free_megablocks,
arena->current_megablock, next);
        }

Also s_alloc(), Haskell heap allocator, try to get Haskell heap from the pool.
If not found, it posix_memalign new megablock.

https://github.com/ajhc/ajhc/blob/d93468e34f4514209048d4a92b1549e079ccd3fb/rts/rts/gc_jgc.c#L392

struct s_megablock *
s_new_megablock(arena_t arena)
{
        jhc_rts_lock();
        struct s_megablock *mb = SLIST_FIRST(&free_megablocks);
        if (mb) {
                SLIST_REMOVE(&free_megablocks, mb, s_megablock, next);
        } else {
                mb = malloc(sizeof(*mb));
#ifdef _JHC_JGC_LIMITED_NUM_MEGABLOCK
                static int count = 0;
                if (count >= _JHC_JGC_LIMITED_NUM_MEGABLOCK) {
                        abort();
                }
                mb->base = aligned_megablock + (MEGABLOCK_SIZE) * count;
                count++;
#else
                mb->base = jhc_aligned_alloc(MEGABLOCK_SIZE);
#endif

Regards,
-- 
Kiwamu Okabe