pinned byte arrays

[Duncan Coutts]

Pinning the arrays gives the GC much less flexibility. Especially if
your objects are small. It means that the GC can't move things to
compact the heap and you'll end up with lots of holes in between heap
objects of things that were collected but the space couldn't be re-used
because other objects didn't happen to fit in it.

You end up with a very fragmented heap. You're essentially completely
preventing the compacting GC from doing any compacting. Why can't it
compact? Because you've told it that it isn't allowed to move anything!

Honestly, I don't see why you think pinning all these little arrays
should make things any better. It just means they can't be moved around
it doesn't mean that magically they never have to be considered or freed
by the GC.

You only want to pin if you must do for some external reason or if the
object is very big and therefore takes a significant amount of time to
move.

Duncan

On Wed, 2006-10-18 at 19:50 +0400, Bulat Ziganshin wrote:
> Hello glasgow-haskell-users,
> 
> i have program that holds in memory a lot of strings (filenames) and
> use my own packed-string library to represent these string. this
> library uses newByteArray# to allocate strings. in my benchmark run
> (with 300.000 filenames in memory) program prints the following
> statistics:
> 
>  37,502,956 bytes maximum residency (16 sample(s))
> 
>        2928 collections in generation 0 (  6.90s)
>          16 collections in generation 1 ( 12.20s)
> 
>          65 Mb total memory in use
> 
> most of this memory occupied by filenames. note that 28 mb of memory
> is used just for GC procedure (i use compacting GC)
> 
> then, i thought that using pinned byte arrays should significantly
> improve memory usage because these arrays can't be moved around and
> therefore, i thought, these arrays will not be involved in GC. amount
> of data which should be compacted by GC will decrease and amount of
> memory wasted by GC should also decrease
> 
> so i replaced in my packed str lib newByteArray# with
> newPinnedByteArray# and seen the following:
> 
>  53,629,344 bytes maximum residency (14 sample(s))
> 
>        2904 collections in generation 0 (  6.97s)
>          14 collections in generation 1 ( 13.16s)
> 
>         100 Mb total memory in use
> 
> why? why memory usage was increased? why another 47 megs is just
> wasted out? how GC works with pinned byte arrays? and why GC times was
> almost the same - i thought that GC for pinned data should be very
> different than GC for unpinned byte arrays
>