From donn at avvanta.com  Tue Nov  4 23:13:56 2008
From: donn at avvanta.com (Donn Cave)
Date: Tue Nov  4 23:08:49 2008
Subject: [nhc-users] fps and Foreign(Obj/Ptr)
Message-ID: <20081105041356.06BDF276CB6@mail.avvanta.com>

I've run into a problem where it looks like my application crashes with
"Error: allocation limit (1024) exceeded for Foreign(Obj/Ptr)" because
it's using too many ByteStrings.  Does that seem plausible?

Can I simply increase the size of that foreign[] array, by several orders
of magnitude, to get around this?

thanks!

	Donn Cave, donn@avvanta.com
From Malcolm.Wallace at cs.york.ac.uk  Wed Nov  5 06:19:29 2008
From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace)
Date: Wed Nov  5 06:18:55 2008
Subject: [nhc-users] fps and Foreign(Obj/Ptr)
In-Reply-To: <20081105041356.06BDF276CB6@mail.avvanta.com>
References: <20081105041356.06BDF276CB6@mail.avvanta.com>
Message-ID: <20081105111929.6200d196.Malcolm.Wallace@cs.york.ac.uk>

"Donn Cave" <donn@avvanta.com> wrote:

> I've run into a problem where it looks like my application crashes
> with "Error: allocation limit (1024) exceeded for Foreign(Obj/Ptr)"
> because it's using too many ByteStrings.  Does that seem plausible?

Yes, it seems plausible.  nhc98's runtime system has a constant-size
table for storing ForeignPtrs.

> Can I simply increase the size of that foreign[] array, by several
> orders of magnitude, to get around this?

Yes, but you will need to recompile nhc98's RTS.  (Not hard.) In
src/runtime/Kernel/cdata.c, just change the #define'd constant
MAX_FOREIGNOBJ, and then (at the toplevel)
    make runtime
    make install

*However*, you should also be aware that ByteStrings have been optimised
for use in ghc, not nhc98.  In fact, it turns out that using ByteString
is orders of magnitude slower than using plain String, in nhc98.  I
can't remember the exact factor, but it was something of the order of
60x slower.

Regards,
    Malcolm
From dons at galois.com  Wed Nov  5 14:18:03 2008
From: dons at galois.com (Don Stewart)
Date: Wed Nov  5 14:13:00 2008
Subject: [nhc-users] fps and Foreign(Obj/Ptr)
In-Reply-To: <20081105111929.6200d196.Malcolm.Wallace@cs.york.ac.uk>
References: <20081105041356.06BDF276CB6@mail.avvanta.com>
	<20081105111929.6200d196.Malcolm.Wallace@cs.york.ac.uk>
Message-ID: <20081105191803.GA390@scytale.galois.com>

Malcolm.Wallace:
> "Donn Cave" <donn@avvanta.com> wrote:
> 
> > I've run into a problem where it looks like my application crashes
> > with "Error: allocation limit (1024) exceeded for Foreign(Obj/Ptr)"
> > because it's using too many ByteStrings.  Does that seem plausible?
> 
> Yes, it seems plausible.  nhc98's runtime system has a constant-size
> table for storing ForeignPtrs.
> 
> > Can I simply increase the size of that foreign[] array, by several
> > orders of magnitude, to get around this?
> 
> Yes, but you will need to recompile nhc98's RTS.  (Not hard.) In
> src/runtime/Kernel/cdata.c, just change the #define'd constant
> MAX_FOREIGNOBJ, and then (at the toplevel)
>     make runtime
>     make install
> 
> *However*, you should also be aware that ByteStrings have been optimised
> for use in ghc, not nhc98.  In fact, it turns out that using ByteString
> is orders of magnitude slower than using plain String, in nhc98.  I
> can't remember the exact factor, but it was something of the order of
> 60x slower.

For one benchmark.

I'd imagine there's others where space is a premium, where it would be
no contest to use a packed representation.

-- Don
From donn at avvanta.com  Wed Nov  5 15:23:50 2008
From: donn at avvanta.com (Donn Cave)
Date: Wed Nov  5 15:18:45 2008
Subject: [nhc-users] fps and Foreign(Obj/Ptr)
Message-ID: <20081105202350.2ACEE93C52@mail.avvanta.com>

Quoth Don Stewart <dons@galois.com>:

[... re ByteString performance ]

| I'd imagine there's others where space is a premium, where it would be
| no contest to use a packed representation.

Right, that was the initial attraction.  I'm trying to put something
together to exercise a little IMAP parser, and that means chunks of
data that are potentially large enough that people worry about whether
the size value could be larger than 32 bits.  So it makes sense to
use ByteStrings from the start - starting with socket input.

At the other end, selected bits of data go to the C++ UI, and that's
the other place where String isn't much use.  That's where I was running
out of foreign objects, and the larger foreign[] does help, thanks!

Now I see I have a leak, though - the foreign objects are piling up
where I was hoping they'd be gc'd, so I have more work to do.  Maybe
complicated by the fact that the Haskell producer and C++ consumer
of these data are in separate threads, but I don't think that's my
immediate problem.  I bet if all Haskell objects were allocated in
a fixed 1024 element list, people would notice space leaks much more
quickly and reliably!

	Donn Cave, donn@avvanta.com