From simonmarhaskell at gmail.com  Fri Jun  1 04:39:45 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Fri Jun  1 04:35:41 2007
Subject: ghc-6.6.1 for FreeBSD/amd64 binary distribution
In-Reply-To: <A3E0E6D9-1150-4923-9E77-67C82A4469BD@comcast.net>
References: <E51A7023-ECD3-4099-BCAB-146A81F8BCA6@comcast.net>
	<465DD91E.80602@gmail.com>
	<A3E0E6D9-1150-4923-9E77-67C82A4469BD@comcast.net>
Message-ID: <465FDB51.5030205@gmail.com>

Gregory Wright wrote:

>> The x86-64 (or amd64 if you like) Linux port doesn't do relocation of 
>> data references outside 2Gb.  It is the subject of this bug:
>>
>>   http://hackage.haskell.org/trac/ghc/ticket/781
>>
>> the underlying problem is that the relocatable reference is only 32 
>> bits, because we're working in the small memory model, but the address 
>> of the symbol might be outside the current 2Gb slice, because it's in 
>> a shared library somewhere.  The system linker solves this by 
>> relocating the data itself from the shared library into the main 
>> program's 2Gb slice (I think), but we can't do this in GHCi.
>>
> 
> Hmm.  I would have thought that the absolute addresses of static data in 
> the shared library would have been
> resolved by the run time loader and written into the global offset 
> table.  It seems that all of the
> shared libraries on FreeBSD/amd64 are loaded at addresses above 2 GB, 
> e.g., above 0x800000000.
> Perhaps rtld allocates space below 2GB and fills in adjusts the GOT to 
> point to the data in lower memory.
> (There's not very much of it).  I'll have to think about this a while.

The linker allocates space for the data in the .data or .bss of the binary. 
e.g. this is what I get when I refer to environ from a C function:

0000000000500aa8  w    O .bss   0000000000000008              environ@@GLIBC_2.2.5

so at runtime, the linker must use this as the location for environ, and links 
the references from libc to here.  In order to do this, the data must have been 
annotated with a .size directive in the shared library, so the linker knows how 
much space to allocate for it.

In fact it does this for all static data references from the main program; this 
is the dreaded R_COPY relocation type.

I don't really know if its possible to check whether a particular reference is 
to code or data in GHC's linker - how did you do this?  Currently the linker 
just assumes they're all code, and allocates a jump table in the low 2Gb and 
re-route jumps through it (see x86_64_high_symbol() in rts/Linker.c).

Cheers,
	Simon

From topekarem at gmail.com  Fri Jun  1 10:00:51 2007
From: topekarem at gmail.com (TOPE KAREM)
Date: Fri Jun  1 09:56:31 2007
Subject: 2-days old in Haskell
Message-ID: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>

I am very new to Haskell, and I am using this webpage as a learning source:
http://www.haskell.org/haskellwiki/Haskell_in_5_steps#Install_Haskell

I downloaded and instal GHC and it works as was said.

I tried to write my first Haskell program:

prelude> "Hello World!"
"Hello World, World!"

it works as said.

Then: (This is the problem)

I would like to create a source code and compile it using GHC compiler. I
opened a notepad and type this program:

main = putStrLn "Hello, Word!"

I saved this file as hello.hs in the same directory GHCi 6.6.1 was
installed.

When I tried to compile it as instucted on the webpage ( $ ghc  -o hello
hello.hs), I got this error message.

<interactive>: 1:0: parse error on input '$'

please what am I doing wrong.

Thank you and sorry for your time.

Tope
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20070601/5956b340/attachment.htm
From cmb21 at kent.ac.uk  Fri Jun  1 10:05:42 2007
From: cmb21 at kent.ac.uk (C.M.Brown)
Date: Fri Jun  1 10:01:48 2007
Subject: 2-days old in Haskell
In-Reply-To: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
References: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
Message-ID: <Pine.GSO.4.58.0706011504010.24471@myrtle.ukc.ac.uk>

Hi Tope,

> When I tried to compile it as instucted on the webpage ( $ ghc  -o hello
> hello.hs), I got this error message.

are you actually typing in that '$' into the command prompt? try leaving
out the '$' and just type "ghc -o hello hello.hs".

Chris.

From ketil at ii.uib.no  Fri Jun  1 10:07:37 2007
From: ketil at ii.uib.no (Ketil Malde)
Date: Fri Jun  1 10:03:16 2007
Subject: 2-days old in Haskell
In-Reply-To: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
References: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
Message-ID: <1180706857.22433.68.camel@nmd9999>

On Fri, 2007-06-01 at 16:00 +0200, TOPE KAREM wrote:

> I opened a notepad and type this program:
> 
> main = putStrLn "Hello, Word!"
> 
> I saved this file as hello.hs in the same directory GHCi 6.6.1 was
> installed.

You can store it anywhere, as long as GHC is in your search path.

> When I tried to compile it as instucted on the webpage ( $ ghc  -o
> hello hello.hs), I got this error message.
> 
> <interactive>: 1:0: parse error on input '$' 
> 
> please what am I doing wrong.

Several things :-)

You don't usually compile from within GHCi, but rather from a system
command prompt.  And: the dollar sign is the typical Unix command
prompt, you're not supposed to type it in.

Generally, you're better off using the make option to ghc, type "ghc
--make hello.hs" instead.

-k

From stefanor at cox.net  Fri Jun  1 10:09:40 2007
From: stefanor at cox.net (Stefan O'Rear)
Date: Fri Jun  1 10:05:21 2007
Subject: 2-days old in Haskell
In-Reply-To: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
References: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
Message-ID: <20070601140940.GB3135@localhost.localdomain>

On Fri, Jun 01, 2007 at 04:00:51PM +0200, TOPE KAREM wrote:
> I am very new to Haskell, and I am using this webpage as a learning source:
> http://www.haskell.org/haskellwiki/Haskell_in_5_steps#Install_Haskell
> 
> I downloaded and instal GHC and it works as was said.
> 
> I tried to write my first Haskell program:
> 
> prelude> "Hello World!"
> "Hello World, World!"
> 
> it works as said.
> 
> Then: (This is the problem)
> 
> I would like to create a source code and compile it using GHC compiler. I
> opened a notepad and type this program:
> 
> main = putStrLn "Hello, Word!"
> 
> I saved this file as hello.hs in the same directory GHCi 6.6.1 was
> installed.
> 
> When I tried to compile it as instucted on the webpage ( $ ghc  -o hello
> hello.hs), I got this error message.
> 
> <interactive>: 1:0: parse error on input '$'
> 
> please what am I doing wrong.
> 
> Thank you and sorry for your time.

Three things:

1. The $ is just a sample prompt, a convention used to indicate what
   you should typed.

2. The command is for a shell (CMD or COMMAND), not GHCi.

3. The command is wrong!  for Windows it should be "ghc -o hello.exe
   hello.hs".  Assuming you aren't knowledgable yet on the differences
   between Windows and Unix programming environments, it might be a good
   idea to pick another tutorial.

Stefan
From lekktu at gmail.com  Fri Jun  1 11:51:27 2007
From: lekktu at gmail.com (Juanma Barranquero)
Date: Fri Jun  1 11:47:08 2007
Subject: 6.6.1 for windows
In-Reply-To: <20070512122732.GA15469@matrix.chaos.earth.li>
References: <463E43F6.6020601@iee.org>
	<20070512122732.GA15469@matrix.chaos.earth.li>
Message-ID: <f7ccd24b0706010851l30dc07e1ma18765b1c9e25deb@mail.gmail.com>

Hi.

> You've probably already noticed, but in case not, there are now two
> installers available for Windows:
>
> http://www.haskell.org/ghc/download_ghc_661.html#windows

I installed the "old style Windows Installer, prepared by Sigbjorn
Finne". ghc --version says:

The Glorious Glasgow Haskell Compilation System, version 6.6.1.20070503

Isn't that an snapshot version number?

             Juanma
From duncan.coutts at worc.ox.ac.uk  Fri Jun  1 16:20:01 2007
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Fri Jun  1 16:15:00 2007
Subject: debugging why we end up calling the wrapper rather than the worker
Message-ID: <1180729201.16937.16.camel@localhost>

Hia all,

I'm trying to figure out why this piece of code does not optimise the
way I expect. It's binary serialisation again. Yes, again.

The crux is the write function

write :: Int -> (Ptr Word8 -> IO ()) -> Put ()
write !n body = Put $ \c buf@(Buffer fp o u l) ->
  if n <= l
    then write' c fp o u l
    else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0

  where {-# NOINLINE write' #-}
        write' c !fp !o !u !l =
          -- warning: this is a tad hardcore
          B.inlinePerformIO
            (withForeignPtr fp
              (\p -> body $! (p `plusPtr` (o+u))))
          `seq` c () (Buffer fp o (u+n) (l-n))

In the if condition, the first path is the fast path. We do want this
write' join point, we want to make a call to write' at runtime rather
than inlining it into both the fast and slow paths. This is why we tag
it with NOINLINE. Simon kindly recently fixed a bug where this NOINLINE
was not remembered in the .hi files, so it now really does not inline
write' into both branches.

However, looking at the core/stg we can see that we're always calling
the write' wrapper function that takes boxed arguments and calls the
wrapper. It seems to me that the calls in both branches ought to be
calls directly to the worker rather than the wrapper. We do have all the
unboxed arguments available but they get boxed up to make the call to
the write' wrapper. This isn't going to do good things for performance.

Here's the STG code:

So the important question is whether we have unboxed versions of all the
args available. First of all lets look at the wrapper and see what args
it unboxes when it calls the worker ($wwrite'_s17v in this case).

write'_s17W = sat-only \r [w1_s17P w2_s17C w3_s17G w4_s17J w5_s17M]
  case w2_s17C of w6_s18S {
    GHC.ForeignPtr.ForeignPtr ww6_s17Q ww7_s17R ->
      case w3_s17G of w7_s18T {
        GHC.Base.I# ww8_s17S ->
          case w4_s17J of w8_s18U {
            GHC.Base.I# ww9_s17T ->
              case w5_s17M of w9_s18V {
                GHC.Base.I# ww10_s17U ->
                  $wwrite'_s17v
                  w1_s17P
                  ww6_s17Q
                  ww7_s17R
                  ww8_s17S
                  ww9_s17T
                  ww10_s17U;
        };
      };
    };
  };

So out of the args [w1_s17P w2_s17C w3_s17G w4_s17J w5_s17M] all but the
first (which is a continuation function) get unboxed and the bits passed
on to $wwrite'_s17v. So we should expect that if we have all those
components available that we could make a direct call to the worker
$wwrite'_s17v rather than going via the wrapper write'_s17W.

So lets look at the STG code that we get from this bit of source:

  if n <= l
    then write' c fp o u l
    else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0

(The STG code here is from a use when n was 3):

We want to see if all the components that the worker $wwrite'_s17v needs
are available:

case <=# [3 ww5_s17X] of wild1_s18W {
  GHC.Base.False ->
  case Put.newBuffer w_s17Z n1_r16x of sat_s18d {
    __DEFAULT ->
      let { sat_s189 = NO_CCS GHC.Base.I#! [ww4_s187]; } in
      let { sat_s186 = NO_CCS GHC.Base.I#! [ww3_s184]; } in
      let { sat_s183 = NO_CCS GHC.ForeignPtr.ForeignPtr! [ww1_s180 ww2_s181]; } in
      let { sat_s18b = \u [] Put.flushOld w_s17Z n1_r16x sat_s183 sat_s186 sat_s189;
      } in  write'_s17W sat_s18b sat_s18d lvl_r16z lvl_r16z lvl_r16z;
      };
  GHC.Base.True ->
    let { sat_s18l = NO_CCS GHC.Base.I#! [ww5_s17X]; } in
    let { sat_s18j = NO_CCS GHC.Base.I#! [ww4_s187]; } in
    let { sat_s18h = NO_CCS GHC.Base.I#! [ww3_s184]; } in
    let { sat_s18f = NO_CCS GHC.ForeignPtr.ForeignPtr! [ww1_s180 ww2_s181];
    } in  write'_s17W w_s17Z sat_s18f sat_s18h sat_s18j sat_s18l;
};

In the False case it looks like there is an excuse for calling the
wrapper, because we're calling the wrapper for both Put.flushOld and
Put.newBuffer. For Put.flushOld it's just to construct the continuation
which is a boxed arg anyway, so that's not stopping us calling the
worker. The Put.newBuffer returns a ForeignPtr and the write' worker
takes the components of the ForeignPtr unboxed. However we are doing a
case analysis on the result of Put.newBuffer so we could easily extract
the components and pass them on to the write' worker.

In the True case there is no excuse at all as far as I can figure out.
We are explicitly boxing up exactly the arguments that the wrapper
unboxes. So we really should be able to call the worker directly with
the appropriate unboxed values as args.

So what is going on here? Does NOINLINE prevent calling the worker or
something?

>From a quick experiment it would appear so:

{-# OPTIONS_GHC -fbang-patterns #-}
module Foo (foo, bar) where

foo :: Int -> Int
foo n = bar n

bar :: Int -> Int
bar !n = bar (n+1)

As is, bar will not be inlined because it's recursive and looking at the
STG code we see that foo makes a call to bar's worker:

Foo.$wbar = \r [ww_sc8] case +# [ww_sc8 1] of sat_sca { __DEFAULT -> Foo.$wbar sat_sca; };
Foo.bar = \r [w_scd] case w_scd of w1_scl { GHC.Base.I# ww_scg -> Foo.$wbar ww_scg; };
Foo.foo = \r [eta_sck] Foo.bar eta_sck;

however if we add in what you'd think is a redundant pragma
{-# NOINLINE bar #-}

and look at the stg code again:

Foo.bar = \r [w_sc9] case w_sc9 of w1_sco { GHC.Base.I# ww_scc -> Foo.$wbar ww_scc; };
Foo.$wbar =
    \r [ww_scf]
        case +# [ww_scf 1] of sat_sch {
          __DEFAULT ->
              let { sat_scj = NO_CCS GHC.Base.I#! [sat_sch]; } in  Foo.bar sat_scj;
        };
Foo.foo = \r [eta_scn] Foo.bar eta_scn;

then we see that foo is now calling bar's wrapper, and what's worse, bar
is calling it's wrapper in the recursive call! Oh noes!

So it seems to me that NOINLINE should prevent inlining but not prevent
calling the worker rather than the wrapper. I don't fully understand how
NOINLINE interacts with the worker/wrapper transform (or I wouldn't have
been surprised by this behaviour). I'm guessing that it works by doing
the worker/wrapper split and then trying to inline the wrapper into as
many call sites as possible. If this is indeed how it works then it'd
explain why attaching NOINLINE to the function causes the observed
behaviour since looking at the .hi file we see that the NOINLINE is
attached to the wrapper function and not the worker.

So perhaps the solution is to attach the NOINLINE to the worker rather
than the wrapper when doing the worker/wrapper split. Would that work or
cause other problems?

Seems otherwise I'm stuck. I thought I could use NOINLINE to control the
creation of join points like in my original example.

Duncan

From duncan.coutts at worc.ox.ac.uk  Fri Jun  1 18:20:17 2007
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Fri Jun  1 18:15:10 2007
Subject: too much let-floating
Message-ID: <1180736418.6210.30.camel@localhost>

Hia folks,

More performance bugs or misunderstandings. Yes, binary serialisation
again.

Consider this example from an instance for the Binary serialisation
class. We get lots and lots of code that looks like this:

data Foo a = Foo Int | Bar ...

instance Binary Foo where
  put (Foo a) = putTag 0 >> put a
  put (Bar ...) = putTag 1 ...

Let's expand this one step so we can see what we want to happen:

put (Foo a) = word8 (fromIntegral 0 :: Word8) >> word64le a

so we want to combine the bounds checks in word8 and word64le so that we
do just one check here rather than two.

Lets expand it another step:

put (Foo a) = write 1 (pokeWord8 (fromIntegral 0 :: Word8))
    `thenPut` write 8 (pokeWord64le (fromIntegral a :: Word64))

Now we have a rule:
"write/write" forall n m a b.
    write n a `thenPut` write m b
  = write (n+m) (\p -> a p >> b (p `plusPtr` n))

so obviously we'd like to apply this rule here to common-up the bounds
check that write n performs.

Normally this rule works fine, but in this specific example it does not.
Can you spot why?

Here's a simple case that works:

foo :: Word8 -> Word8 -> Put ()
foo a b = word8 a >> word8 b

and the one that doesn't

bar :: Word8 -> Put ()
bar a = word8 0 >> put a

What's the difference? Why should one work and the other not?

Of course I gave it away in the email subject: let floating.

In the second example, the 'bar' function, word8 0 does not depend on
the arguments of the function so it gets floated out:

lvl_s198 = write 1 (pokeWord8 0)

bar n =
  thenPut
    lvl_s198
    (write 8 (pokeWord64le n))

and now the rule does not match this because 'write' isn't there
directly in the expression.

This is a bit disappointing of course, so how do we fix it. There are
two possibilities as far as I can see. Either don't let float it, or
have the rule matcher look through the indirection.

The rule matcher is already looks through lets, but that's a slightly
different issue. That's for a situation like:

bar n =
  thenPut
    (let lvl_s198 = pokeWord8 0
      in write 1 lvl_s198)
    (write 8 (pokeWord64le n))

where we've got an actual let expression whose body would match the
rule's pattern.

It would not always be a good idea to "look up" let floated vars when
doing rule matching since it could lead to duplication. Though in this
example the let-floated expression is used only once, so that is not a
problem.

So one possibility might be to look up variables for the purpose of rule
matching if they are only used once.

Another possibility might be to not let-float it in the first place.
Usually let-floating things like this is a good idea, so how can we spot
that we might want to keep it in applicative form? The fact that it's
mentioned in the pattern of a rule is the strongest hint. As I
understand it, being mentioned in the head of a rule is already taken
into account when doing let-floating. However in this case the head of
the let-floated expression is not the head of the rule, but a sub
expression:

    write n a `thenPut` write m b
  = write (n+m) (\p -> a p >> b (p `plusPtr` n))

'thenPut' is the head of this rule's pattern. But we'd rather not float
'write' out either. So perhaps a similar heuristic should be applied to
all the free variables mentioned in the pattern.

Another approach might be for the library author to declare that some
function is cheap, or is otherwise important to not float out, but
should be kept in applicative form to aid rule matching.

We have similar problems with list comprehensions and enumFromTo:

[ ... | i <- [0..n], j <- [0..m] ]

because in the second generator [0..m] does not depend on the first
generator, it gets floated out and shared. Of course with fusion we can
make the [0..m] extremely cheap, but not if it has been floated out. If
it gets floated out and shared in memory between each iteration of the
first generator, then we really do have to traverse the list data
structure each iteration, rather than just incrementing an unboxed
number. So this is another example where we'd like to say that a
function, enumFromTo in this case, is extremely cheap and should not be
floated out, even if it looses sharing.

Actually, my example is simpler because it does not involve any loss of
sharing.

Duncan

From monique.louise at gmail.com  Sat Jun  2 17:02:14 2007
From: monique.louise at gmail.com (Monique Monteiro)
Date: Sat Jun  2 16:57:50 2007
Subject: GHC Extensibility
In-Reply-To: <DCA83D5F3861C941808187D4D387875011E0061604@EA-EXMSG-C312.europe.corp.microsoft.com>
References: <d3c68c840705300745p374e0f88g901be21d5ff9641f@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E00614E7@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840705300958g445e98a1occ83e4a6ef4a4704@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E0061604@EA-EXMSG-C312.europe.corp.microsoft.com>
Message-ID: <d3c68c840706021402m1a37e86djf3e70ec5f2b60faf@mail.gmail.com>

Hi all,

  I'm able to compile the .NET code generator with GHC without
compiler errors, but now I have the linkage error below.  (In fact
there are other similar messages, this is only an example).  This is
strange because the *.o/*.hi files were all generated without error
messages.  Has anyone any idea about how to solve it?

(...)
stage1/ilGen/ILGen.o(.rodata+0x50):fake: undefined reference to `mtlzm1zi0zi1_C
ntrolziMonadziStateziLazzy_zdf14_closure'
stage1/ilGen/ILGen.o(.rodata+0x68):fake: undefined reference to `mtlzm1zi0zi1_C
ntrolziMonadziStateziLazzy_zdf14_closure'
stage1/ilGen/ILGen.o(.rodata+0x74):fake: undefined reference to `mtlzm1zi0zi1_C
ntrolziMonadziStateziLazzy_zdf14_closure'
stage1/ilGen/ILGen.o(.rodata+0x88):fake: undefined reference to `mtlzm1zi0zi1_C
ntrolziMonadziStateziLazzy_zdf14_closure'
stage1/ilGen/ILGen.o(.rodata+0xb8):fake: more undefined references to `mtlzm1zi
zi1_ControlziMonadziStateziLazzy_zdf14_closure' follow
stage1/ilGen/ILGen.o(.rodata+0x32bc):fake: undefined reference to `mtlzm1zi0zi1
ControlziMonadziStateziLazzy_evalState_closure'
stage1/ilGen/ILGen.o(.rodata+0x32d0):fake: undefined reference to `mtlzm1zi0zi1
ControlziMonadziStateziLazzy_zdf14_closure'
collect2: ld returned 1 exit status
<<ghc: 19046880 bytes, 6 GCs, 176570/181504 avg/max bytes residency (2 samples)
 16M in use, 0.03 INIT (0.00 elapsed), 0.06 MUT (5.61 elapsed), 0.02 GC (0.03 e
apsed) :ghc>>
make: *** [stage1/ghc] Error 1

Monique
From eivuokko at gmail.com  Sat Jun  2 18:37:53 2007
From: eivuokko at gmail.com (Esa Ilari Vuokko)
Date: Sat Jun  2 18:33:28 2007
Subject: GHC Extensibility
In-Reply-To: <d3c68c840706021402m1a37e86djf3e70ec5f2b60faf@mail.gmail.com>
References: <d3c68c840705300745p374e0f88g901be21d5ff9641f@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E00614E7@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840705300958g445e98a1occ83e4a6ef4a4704@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E0061604@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840706021402m1a37e86djf3e70ec5f2b60faf@mail.gmail.com>
Message-ID: <cd4c8b540706021537t538b853t59b1e895e9ad276e@mail.gmail.com>

Hi,

On 6/3/07, Monique Monteiro <monique.louise@gmail.com> wrote:
> Hi all,
>
>   I'm able to compile the .NET code generator with GHC without
> compiler errors, but now I have the linkage error below.  (In fact
> there are other similar messages, this is only an example).  This is
> strange because the *.o/*.hi files were all generated without error
> messages.  Has anyone any idea about how to solve it?
>
> (...)
> stage1/ilGen/ILGen.o(.rodata+0x50):fake: undefined reference to `mtlzm1zi0zi1_C
> ntrolziMonadziStateziLazzy_zdf14_closure'

It seems to me, that the code in ILGen is using Control.Monad.State
(that nowdays defaults to .Lazy) from package mtl.

For stage1, this can be solved by adding "-package mtl" to ghc command
used to linking, for next stages, you'd need to add mtl to bootstrap cycle,
I think.  I have no idea how that is done.  Borrowing code from mtl might be
easier.

HTH,
Esa
From monique.louise at gmail.com  Sat Jun  2 22:31:08 2007
From: monique.louise at gmail.com (Monique Monteiro)
Date: Sat Jun  2 22:26:42 2007
Subject: GHC Extensibility
In-Reply-To: <cd4c8b540706021537t538b853t59b1e895e9ad276e@mail.gmail.com>
References: <d3c68c840705300745p374e0f88g901be21d5ff9641f@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E00614E7@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840705300958g445e98a1occ83e4a6ef4a4704@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E0061604@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840706021402m1a37e86djf3e70ec5f2b60faf@mail.gmail.com>
	<cd4c8b540706021537t538b853t59b1e895e9ad276e@mail.gmail.com>
Message-ID: <d3c68c840706021931pfa88da3q7537b985f5f4534b@mail.gmail.com>

On 6/2/07, Esa Ilari Vuokko <eivuokko@gmail.com> wrote:
> For stage1, this can be solved by adding "-package mtl" to ghc command
> used to linking, for next stages, you'd need to add mtl to bootstrap cycle,
> I think.  I have no idea how that is done.  Borrowing code from mtl might be
> easier.

GHC gives "unknown package: mtl" when I add this option.  Should I
download it separately?

-- 
__________________________________________________________
Monique Monteiro, MSc
MCP .NET Framework 2.0 / SCJP / IBM OOAD
Project Manager
Recife Microsoft Innovation Center
+55 81 34198137
http://www.cin.ufpe.br/~mlbm
http://thespoke.net/blogs/moniquelouise/default.aspx
monique@qualiti.com.br
MSN: monique_louise@msn.com
From eivuokko at gmail.com  Sun Jun  3 05:25:00 2007
From: eivuokko at gmail.com (Esa Ilari Vuokko)
Date: Sun Jun  3 05:20:34 2007
Subject: GHC Extensibility
In-Reply-To: <d3c68c840706021931pfa88da3q7537b985f5f4534b@mail.gmail.com>
References: <d3c68c840705300745p374e0f88g901be21d5ff9641f@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E00614E7@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840705300958g445e98a1occ83e4a6ef4a4704@mail.gmail.com>
	<DCA83D5F3861C941808187D4D387875011E0061604@EA-EXMSG-C312.europe.corp.microsoft.com>
	<d3c68c840706021402m1a37e86djf3e70ec5f2b60faf@mail.gmail.com>
	<cd4c8b540706021537t538b853t59b1e895e9ad276e@mail.gmail.com>
	<d3c68c840706021931pfa88da3q7537b985f5f4534b@mail.gmail.com>
Message-ID: <cd4c8b540706030225h3adba163hb7d99cf64cd7500b@mail.gmail.com>

On 6/3/07, Monique Monteiro <monique.louise@gmail.com> wrote:
> On 6/2/07, Esa Ilari Vuokko <eivuokko@gmail.com> wrote:
> > For stage1, this can be solved by adding "-package mtl" to ghc command
> > used to linking, for next stages, you'd need to add mtl to bootstrap cycle,
> > I think.  I have no idea how that is done.  Borrowing code from mtl might be
> > easier.
>
> GHC gives "unknown package: mtl" when I add this option.  Should I
> download it separately?

Maybe it needs version as well, mtl-1.0.1.  You should have the package already
as you managed to compile the haskell files.

Best regards,
Esa

PS. The symbol names are partly z-encoded (you can find details in
ghc sources) and partly contain compiler's invented bits.

mtlzm1zi0zi1_ControlziMonadziStateziLazzy_zdf14_closure becomes

mtl       => mtl    (ie package name)
zm       => -       (dash to separate package name and version)
1zi0zi1 => 1.0.1 (version)
_          => Some convention to put _ there.
ControlziMonadziStateziLazzy
            => Control.Monad.State.Lazy (module)
_zdf14_closure
            => closure number 14 (number is rather arbitary)
From simonpj at microsoft.com  Mon Jun  4 07:31:34 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Mon Jun  4 07:27:06 2007
Subject: too much let-floating
In-Reply-To: <1180736418.6210.30.camel@localhost>
References: <1180736418.6210.30.camel@localhost>
Message-ID: <DCA83D5F3861C941808187D4D387875011E0328789@EA-EXMSG-C312.europe.corp.microsoft.com>

| This is a bit disappointing of course, so how do we fix it. There are
| two possibilities as far as I can see. Either don't let float it, or
| have the rule matcher look through the indirection.

This is a tricky one.  One possibility would be to postpone full laziness until later in the optimisation pipeline.  But then some useful sharing isn't as easily accessible.  For example, if you look in DynFlags, where the main pass structure is defined, you'll see:

        -- Don't inline anything till full laziness has bitten
        -- In particular, inlining wrappers inhibits floating
        -- e.g. ...(case f x of ...)...
        --  ==> ...(case (case x of I# x# -> fw x#) of ...)...
        --  ==> ...(case x of I# x# -> case fw x# of ...)...
        -- and now the redex (f x) isn't floatable any more
        -- Similarly, don't apply any rules until after full
        -- laziness.  Notably, list fusion can prevent floating.

Making the rule matcher look through lets could be possible.  Maybe even on a rule-by-rule basis.  But consider
        f x = let xs = map expensive [1..x]
                in (sum xs, prod xs)
If you are prepared to duplicate xs, you can fuse with both 'sum' and 'product'.  But duplicating 'xs' duplicates an arbitrarily large computation. Are you sure you want that?  The beauty of the "only match when the subexpression is literally there" idea is that you *know* it's the unique occurrence.

Matching on a 'let' that's outside a lambda is potentially even more extreme; the example above was only 2-way sharing.

I think you might be on solider ground if you declare that some functions are "cheap enough to duplicate"; in this case your 'write' function.  But even then you need to take care:
        let x = cheap_fun a b in \y. ....(foo x)....
Now a rule (foo (cheap_fun p q)) might perhaps match.  But what if 'a' or 'b' was expensive?!  Then we'd prefer that the defn of x looked like
        let a = <a-code>; b = <b-code>; x = cheap_fun a b
        in \y. ...
But now it's less easy to match rules like (foo (cheap_fun (g x) (h y)))

Nevertheless, my nose tells me that promising that a function is cheap may be the most robust way forward.

Simon

an/listinfo/glasgow-haskell-users
From simonpj at microsoft.com  Mon Jun  4 07:44:25 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Mon Jun  4 07:39:58 2007
Subject: debugging why we end up calling the wrapper rather than the
 worker
In-Reply-To: <1180729201.16937.16.camel@localhost>
References: <1180729201.16937.16.camel@localhost>
Message-ID: <DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>

| So it seems to me that NOINLINE should prevent inlining but not prevent
| calling the worker rather than the wrapper. I don't fully understand how
| NOINLINE interacts with the worker/wrapper transform (or I wouldn't have
| been surprised by this behaviour). I'm guessing that it works by doing
| the worker/wrapper split and then trying to inline the wrapper into as
| many call sites as possible. If this is indeed how it works then it'd
| explain why attaching NOINLINE to the function causes the observed
| behaviour since looking at the .hi file we see that the NOINLINE is
| attached to the wrapper function and not the worker.

Exactly.  And indeed, it doesn't really make sense to do a w/w split on a NOINLINE thing, if this is what happens.

| So perhaps the solution is to attach the NOINLINE to the worker rather
| than the wrapper when doing the worker/wrapper split. Would that work or
| cause other problems?

That is easily changed.  But consider that you may have put that NOININE pragma there to stop the thing inlining so that a RULE would fire.  We presumably do not want to uncritically make a NOINLINE thing into an INLINE thing, just because it's strict; that would nullify the carefully set pragmas to make sure the rules worked.

I suggest you say NOINLINE [0]; that prevents inlining until the final phases of compilation, which is probably what you want.  See if that works.

Meanwhile, I should probably do no w/w for NOINLINE things.  I'll postpone until this thread settles.

Simon

From duncan.coutts at worc.ox.ac.uk  Mon Jun  4 08:39:09 2007
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Mon Jun  4 08:33:53 2007
Subject: debugging why we end up calling the wrapper rather than the
	worker
In-Reply-To: <DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
References: <1180729201.16937.16.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
Message-ID: <1180960749.22405.21.camel@localhost>

On Mon, 2007-06-04 at 12:44 +0100, Simon Peyton-Jones wrote:
> | So it seems to me that NOINLINE should prevent inlining but not prevent
> | calling the worker rather than the wrapper. I don't fully understand how
> | NOINLINE interacts with the worker/wrapper transform (or I wouldn't have
> | been surprised by this behaviour). I'm guessing that it works by doing
> | the worker/wrapper split and then trying to inline the wrapper into as
> | many call sites as possible. If this is indeed how it works then it'd
> | explain why attaching NOINLINE to the function causes the observed
> | behaviour since looking at the .hi file we see that the NOINLINE is
> | attached to the wrapper function and not the worker.
> 
> Exactly.  And indeed, it doesn't really make sense to do a w/w split
> on a NOINLINE thing, if this is what happens.
> 
> | So perhaps the solution is to attach the NOINLINE to the worker
> rather
> | than the wrapper when doing the worker/wrapper split. Would that
> work or
> | cause other problems?
> 
> That is easily changed.  But consider that you may have put that
> NOININE pragma there to stop the thing inlining so that a RULE would
> fire.  We presumably do not want to uncritically make a NOINLINE thing
> into an INLINE thing, just because it's strict; that would nullify the
> carefully set pragmas to make sure the rules worked.

Ah, yes of course.

> I suggest you say NOINLINE [0]; that prevents inlining until the final
> phases of compilation, which is probably what you want.  See if that
> works.

But that allows it to be inlined in phase 0, and that's exactly what I
don't want. I really do not want this function inlined, I want it to be
a join point.

So in this example, I'm not trying to do rule matching, I just want to
create a join point.

Duncan

From duncan.coutts at worc.ox.ac.uk  Mon Jun  4 08:54:10 2007
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Mon Jun  4 08:48:54 2007
Subject: too much let-floating
In-Reply-To: <DCA83D5F3861C941808187D4D387875011E0328789@EA-EXMSG-C312.europe.corp.microsoft.com>
References: <1180736418.6210.30.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328789@EA-EXMSG-C312.europe.corp.microsoft.com>
Message-ID: <1180961650.22405.36.camel@localhost>

On Mon, 2007-06-04 at 12:31 +0100, Simon Peyton-Jones wrote:
> | This is a bit disappointing of course, so how do we fix it. There are
> | two possibilities as far as I can see. Either don't let float it, or
> | have the rule matcher look through the indirection.
> 
> This is a tricky one.  One possibility would be to postpone full
> laziness until later in the optimisation pipeline.  But then some
> useful sharing isn't as easily accessible.  For example, if you look
> in DynFlags, where the main pass structure is defined, you'll see:
> 
>         -- Don't inline anything till full laziness has bitten
>         -- In particular, inlining wrappers inhibits floating
>         -- e.g. ...(case f x of ...)...
>         --  ==> ...(case (case x of I# x# -> fw x#) of ...)...
>         --  ==> ...(case x of I# x# -> case fw x# of ...)...
>         -- and now the redex (f x) isn't floatable any more
>         -- Similarly, don't apply any rules until after full
>         -- laziness.  Notably, list fusion can prevent floating.
> 
> Making the rule matcher look through lets could be possible.  Maybe
> even on a rule-by-rule basis.  But consider
>         f x = let xs = map expensive [1..x]
>                 in (sum xs, prod xs)
> If you are prepared to duplicate xs, you can fuse with both 'sum' and
> 'product'.  But duplicating 'xs' duplicates an arbitrarily large
> computation. Are you sure you want that?  The beauty of the "only
> match when the subexpression is literally there" idea is that you
> *know* it's the unique occurrence.

No, I don't want to duplicate. But in my example the let var was only
used once, so there was no sharing problem. So in this 

> Matching on a 'let' that's outside a lambda is potentially even more
> extreme; the example above was only 2-way sharing.

So instead of having the rule matcher do some kind of on the fly let
inlining, it should not have been floated out in the first place. So if
the let was indeed originally outside the lambda then inlining it adds
more allocation, but if we choose not to let-float it then we're not
making the program worse, we're just missing an opportunity to reduce
allocation. But if we can identify a good reason not to let float it
(like it appearing in a rule) then we can avoid thinking about
duplicating let bound things.

> I think you might be on solider ground if you declare that some
> functions are "cheap enough to duplicate"; in this case your 'write'
> function.  But even then you need to take care:
>         let x = cheap_fun a b in \y. ....(foo x)....
> Now a rule (foo (cheap_fun p q)) might perhaps match.  But what if 'a'
> or 'b' was expensive?!  Then we'd prefer that the defn of x looked
> like
>         let a = <a-code>; b = <b-code>; x = cheap_fun a b
>         in \y. ...
> But now it's less easy to match rules like (foo (cheap_fun (g x) (h y)))
> 
> Nevertheless, my nose tells me that promising that a function is cheap
> may be the most robust way forward.


Yes, but I'm not sure this case is really one where we're dealing with
duplicating a cheap thing. We're just not let-floating something that
could be let-floated.

In my original example, once we get to a later phase of compilation, the
let bound thing does get inlined again. Once the bind and write
functions get inlined, lots of case expressions appear and then it
becomes obvious that it'd be beneficial to inline, and ghc does so. And
as I say, it was only used in one place. So the frustrating thing is
that it is let-floated *only* for the one phase where I need it not to
be floated so I can rule match on it :-).

So how about the idea of taking the rule pattern into account when
deciding to let-float? In this case it should be clear that the benefits
of let floating are pretty marginal, so I'd guess it just needs a little
extra nudge to decide that this case isn't beneficial to float out. That
is, my guess is that we'd not loose many beneficial let-floatings by
taking the presence of a rule into account. But that's only a guess.

Duncan

From simonpj at microsoft.com  Mon Jun  4 09:01:35 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Mon Jun  4 08:57:06 2007
Subject: debugging why we end up calling the wrapper rather than the
	worker
In-Reply-To: <1180960749.22405.21.camel@localhost>
References: <1180729201.16937.16.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
	<1180960749.22405.21.camel@localhost>
Message-ID: <DCA83D5F3861C941808187D4D387875011E0328823@EA-EXMSG-C312.europe.corp.microsoft.com>

| But that allows it to be inlined in phase 0, and that's exactly what I
| don't want. I really do not want this function inlined, I want it to be
| a join point.

Remind me why you really don't want it inlined, ever?  Even if it's small etc.

S
From topekarem at gmail.com  Mon Jun  4 09:22:52 2007
From: topekarem at gmail.com (TOPE KAREM)
Date: Mon Jun  4 09:18:22 2007
Subject: 2-days old in Haskell
In-Reply-To: <20070601140940.GB3135@localhost.localdomain>
References: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
	<20070601140940.GB3135@localhost.localdomain>
Message-ID: <3717bcc50706040622p1597539ewf57d6f3757721079@mail.gmail.com>

Thanks to all ( Stefan, Ketil and Brown). It works both on Linux and
Windows.

Tope



On 6/2/07, Stefan O'Rear <stefanor@cox.net> wrote:
>
> On Fri, Jun 01, 2007 at 04:00:51PM +0200, TOPE KAREM wrote:
> > I am very new to Haskell, and I am using this webpage as a learning
> source:
> > http://www.haskell.org/haskellwiki/Haskell_in_5_steps#Install_Haskell
> >
> > I downloaded and instal GHC and it works as was said.
> >
> > I tried to write my first Haskell program:
> >
> > prelude> "Hello World!"
> > "Hello World, World!"
> >
> > it works as said.
> >
> > Then: (This is the problem)
> >
> > I would like to create a source code and compile it using GHC compiler.
> I
> > opened a notepad and type this program:
> >
> > main = putStrLn "Hello, Word!"
> >
> > I saved this file as hello.hs in the same directory GHCi 6.6.1 was
> > installed.
> >
> > When I tried to compile it as instucted on the webpage ( $ ghc  -o hello
> > hello.hs), I got this error message.
> >
> > <interactive>: 1:0: parse error on input '$'
> >
> > please what am I doing wrong.
> >
> > Thank you and sorry for your time.
>
> Three things:
>
> 1. The $ is just a sample prompt, a convention used to indicate what
>   you should typed.
>
> 2. The command is for a shell (CMD or COMMAND), not GHCi.
>
> 3. The command is wrong!  for Windows it should be "ghc -o hello.exe
>   hello.hs".  Assuming you aren't knowledgable yet on the differences
>   between Windows and Unix programming environments, it might be a good
>   idea to pick another tutorial.
>
> Stefan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20070604/0d8bc292/attachment.htm
From duncan.coutts at worc.ox.ac.uk  Mon Jun  4 09:50:12 2007
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Mon Jun  4 09:44:56 2007
Subject: debugging why we end up calling the wrapper rather than the
	worker
In-Reply-To: <DCA83D5F3861C941808187D4D387875011E0328823@EA-EXMSG-C312.europe.corp.microsoft.com>
References: <1180729201.16937.16.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
	<1180960749.22405.21.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328823@EA-EXMSG-C312.europe.corp.microsoft.com>
Message-ID: <1180965012.22405.45.camel@localhost>

On Mon, 2007-06-04 at 14:01 +0100, Simon Peyton-Jones wrote:
> | But that allows it to be inlined in phase 0, and that's exactly what I
> | don't want. I really do not want this function inlined, I want it to be
> | a join point.
> 
> Remind me why you really don't want it inlined, ever?  Even if it's small etc.

I'm partitioning a fast path and a slow path. I don't want an extra copy
of the code hanging around just because of the slow path. So I want both
the fast and slow paths to share a single copy of the object code for
doing the writing to memory. Then the fast path is just a jump to this
code, and the slow path calls a bunch of other out of line functions to
fix things up before jumping.

So there is no advantage to inlining here, except call overhead, but
that should be low too. In fact if it's not impossible to imaging that
the jump to the function could be combined with the conditional test &
jump, rather than it being a conditional test & jump followed by an
unconditional jump in the fast path.

The code was:

write :: Int -> (Ptr Word8 -> IO ()) -> Put ()
write !n body = Put $ \c buf@(Buffer fp o u l) ->
  if n <= l
    then write' c fp o u l  --fast path
    else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0
 
  where {- NOINLINE write' -}
        write' c !fp !o !u !l =
          -- warning: this is a tad hardcore
          B.inlinePerformIO
            (withForeignPtr fp
              (\p -> body $! (p `plusPtr` (o+u))))
          `seq` c () (Buffer fp o (u+n) (l-n))

where 'body' is an IO function that writes half a dozen bytes into a
memory block.

Duncan

From sedillard at ucdavis.edu  Mon Jun  4 16:28:36 2007
From: sedillard at ucdavis.edu (Scott Dillard)
Date: Mon Jun  4 16:24:05 2007
Subject: specialization using RULES
Message-ID: <d9b657cd0706041328y3b7dcee4v602d192f0f4c56c3@mail.gmail.com>

Hello,

This is somewhat related to this thread:

http://www.haskell.org/pipermail/glasgow-haskell-users/2007-May/012646.html

In that email I asked about the performance of gaussian elimination on
small matrices represented by unboxed arrays, because I noticed that
unsafeRead was unexpectedly slower than readArray (the cause of this
is still unknown.) Mirko Rahn replied with an indexless algorithm
where unboxed arrays are replaced by lists-of-lists.

http://www.haskell.org/pipermail/glasgow-haskell-users/2007-May/012648.html

Besides being infinitely more elegant, I noticed this was considerably
faster than my array-based functions, perhaps because of
inlining/deforestation/list fusion or some other crazy GHC
optimization. So I re-wrote my little affine geometry library
representing vectors as lists and matrices as list-of-lists.

One thing I wanted this library to do was to enforce dimensionality of
the vectors at the type level, so you could not add a two-vector to a
three-vector, even though both functions are just "zipWith (+)".
After trying to tune my library I came across the SPECIALIZATION/RULES
pagmas in the GHC manual, and thought that this phantom dimensionality
type would be great for specializing the vector functions, and indeed
it speeds things up quite a bit. For instance, if I define

> zipWithV :: (Dim d) => (s->s->s) -> Vec d s -> Vec d s -> Vec d s
> zipWithV f (Vec u) (Vec v) = Vec( zipWith f u v )

then I can specialize

> zipWithV3 :: (s->s->s) -> Vec Three s -> Vec Three s -> Vec Three s
> zipWithV3 f (Vec [x,y,z]) (Vec [i,j,k]) = Vec [ f x i, f y j, f k z ]
> {-# RULES "zipWith3" zipWithV = zipWithV3 #-}

and it makes a nice improvement to the performance of this function.
But here comes my problem: Ideally, I should only have to specialize
zipWith, map and foldr like this, because everything you could want to
do with a vector can be implemented with these functions. (Well,
everything I want to do, anyway.) But unfortunately, if I define,
say...

> instance (Num s) => Num (Vec d s ) where ... (-) = zipWithV (-)
> sqrNorm v = sumV ( mapV sqr v ) where sqr x = x*x

And I use this as

> distance u v = sqrt ( sqrNorm (u-v) )
> doSomethingWith (distance (u :: Vec Three Double) v)

The rules do not fire. They only seem to fire if the specialized
function is called directly, such as

> doSomethingWith ( zipWith (-) (u :: Vec Three Double) v )

I surely do not want to have to specialize all of my vector functions.
Just the building blocks.  Can anyone shed some light on my situation?
Under what conditions do the rules fire? As I understand it, they fire
whenever the types are the same. As far as I can tell, this is the
case. How can I change things to make the rules fire more often?

Thanks,
Scott
From john at repetae.net  Mon Jun  4 20:36:35 2007
From: john at repetae.net (John Meacham)
Date: Mon Jun  4 20:32:07 2007
Subject: OT: shell prompt
In-Reply-To: <1180706857.22433.68.camel@nmd9999>
References: <3717bcc50706010700i622ff942pb57fe2e5ff5b7a77@mail.gmail.com>
	<1180706857.22433.68.camel@nmd9999>
Message-ID: <20070605003635.GB13997@momenergy.repetae.net>

On Fri, Jun 01, 2007 at 04:07:37PM +0200, Ketil Malde wrote:
> You don't usually compile from within GHCi, but rather from a system
> command prompt.  And: the dollar sign is the typical Unix command
> prompt, you're not supposed to type it in.

Incidentally, I find it useful when giving examples of unix command
lines to use a semicolon ';' as the prompt, as it has no effect in
bourne derived shells so you can always cut-n-paste the whole line into
your shell and be good to go.

        John

-- 
John Meacham - ?repetae.net?john?
From simonpj at microsoft.com  Tue Jun  5 03:49:39 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Tue Jun  5 03:45:14 2007
Subject: specialization using RULES
In-Reply-To: <d9b657cd0706041328y3b7dcee4v602d192f0f4c56c3@mail.gmail.com>
References: <d9b657cd0706041328y3b7dcee4v602d192f0f4c56c3@mail.gmail.com>
Message-ID: <DCA83D5F3861C941808187D4D387875011E0328AA7@EA-EXMSG-C312.europe.corp.microsoft.com>


| The rules do not fire. They only seem to fire if the specialized
| function is called directly, such as
|
| > doSomethingWith ( zipWith (-) (u :: Vec Three Double) v )

That's probably because to fire
        distance must be inlined
        but sumV and mapV must not
which is obviously a bit delicate.

To do that you need fairly fine grain control over exactly when inlining takes place. That's what the [n] part of an INLINE pragma is for.  Look in GHC.Base for examples.  Don and Duncan are experts.

I wish there was a more robust way to do this.

Simon

From simonpj at microsoft.com  Tue Jun  5 04:14:06 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Tue Jun  5 04:09:34 2007
Subject: too much let-floating
In-Reply-To: <1180961650.22405.36.camel@localhost>
References: <1180736418.6210.30.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328789@EA-EXMSG-C312.europe.corp.microsoft.com>
	<1180961650.22405.36.camel@localhost>
Message-ID: <DCA83D5F3861C941808187D4D387875011E0328AD4@EA-EXMSG-C312.europe.corp.microsoft.com>

| No, I don't want to duplicate. But in my example the let var was only
| used once, so there was no sharing problem.

Not so in general -- floating outside a lambda that is called many times can dramatically increase sharing.

You're right that all you want is to *forgo* an optimisation; but I want to avoid people complaining about lost optimisations!

| In my original example, once we get to a later phase of compilation, the
| let bound thing does get inlined again. Once the bind and write
| functions get inlined, lots of case expressions appear and then it
| becomes obvious that it'd be beneficial to inline, and ghc does so. And

Hmm.  Indeed, looking at it:

        lvl_s198 = write 1 (pokeWord8 0)

I believe that there are no redexes there, correct?  So we are gaining no sharing of work. GHC is simply avoiding an allocation, by floating to the top level, rather than actually avoiding work.

Is that always so in the cases you are bothered about?  That is, the annoying floating is saving allocation but not work?  And GHC can see that?  If so, perhaps we can make the first run of FloatOut not do allocation-saving. There's a second run later I think.

You could have a go at this if you like.  Look at SetLevels.lhs line 400.  I think you might try something like replacing 'True' with:
        floatConsts env || not (exprIsCheap expr)

The 'floatConsts' flag is False for the first run of FloatOut but True for the second.

| So how about the idea of taking the rule pattern into account when
| deciding to let-float

That could be the next thing to try. I used to look at LHSs etc, but decided that it was not robust enough.

Simon
From igloo at earth.li  Tue Jun  5 11:58:54 2007
From: igloo at earth.li (Ian Lynagh)
Date: Tue Jun  5 11:54:22 2007
Subject: ghc-6.6.1 for FreeBSD/amd64 binary distribution
In-Reply-To: <A3E0E6D9-1150-4923-9E77-67C82A4469BD@comcast.net>
References: <E51A7023-ECD3-4099-BCAB-146A81F8BCA6@comcast.net>
	<465DD91E.80602@gmail.com>
	<A3E0E6D9-1150-4923-9E77-67C82A4469BD@comcast.net>
Message-ID: <20070605155854.GA23148@matrix.chaos.earth.li>


Hi Gregory,

On Wed, May 30, 2007 at 05:56:31PM -0400, Gregory Wright wrote:
> 
> On May 30, 2007, at 4:05 PM, Simon Marlow wrote:
> 
> >Gregory Wright wrote:
> >
> >>I have put a binary distribution of ghc-6.6.1 for FreeBSD/amd64
> >>at
> >>http://www.haskell.org/ghc/dist/6.6.1/ghc-6.6.1-x86_64-unknown- 
> >>freebsd.tar.bz2

Great, thanks!

> >yay!  Ian will supply a link from the download page in due course,  
> >I'm sure.

Now done.

> I still owe you some source patches to make the compiler build out of  
> the
> box.  If 6.6.1 is the end of the line, would you prefer that these be  
> against HEAD?

Yup, HEAD is best.

> darcs send
> route (now possible, since darcs builds on x86_64)?

darcs send is fine.


Thanks
Ian

From wferi at niif.hu  Tue Jun  5 16:13:41 2007
From: wferi at niif.hu (Wagner Ferenc)
Date: Tue Jun  5 16:09:07 2007
Subject: GHC 6.6.1 on Debian Etch?
Message-ID: <877iqiyxai.fsf@szonett.ki.iif.hu>

Hi,

what's the best way to install GHC 6.6.1 on a Debian Etch system?
Basically: are there installable packages available somewhere, or
should I recompile the Sid packages, or create a stub package from a
binary .tar.gz bundle, or some other option I didn't think of?
-- 
Thanks,
Feri.
From iavor.diatchki at gmail.com  Tue Jun  5 18:05:30 2007
From: iavor.diatchki at gmail.com (Iavor Diatchki)
Date: Tue Jun  5 18:00:54 2007
Subject: Prelude not in haskell98?
Message-ID: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>

Hello,

I am using GHC 6.6 and I am trying to build a library using Cabal.
The library is written in Haskell'98 so I made the Cabal file depend
only on the package "haskell98".  Unfortunately building the library
fails with the following error:

MyModule.hs:1:0:
    Failed to load interface for `Prelude':
      it is a member of package base, which is hidden

The module "MyModule" does not have any imports (except for the
implicitly imported Prelude).
If I add an extra dependency on package "base", then the library compiles.

All this seems to indicate that the Prelude is not a part of the
"haskell98" package.  Is this the case, and if so, is this
intentional?  It would be nice if we could create Cabal packages that
explicitly indicate that the library depends only on Haskell 98
libraries.

-Iavor
From stefanor at cox.net  Tue Jun  5 18:13:21 2007
From: stefanor at cox.net (Stefan O'Rear)
Date: Tue Jun  5 18:08:47 2007
Subject: GHC 6.6.1 on Debian Etch?
In-Reply-To: <877iqiyxai.fsf@szonett.ki.iif.hu>
References: <877iqiyxai.fsf@szonett.ki.iif.hu>
Message-ID: <20070605221321.GA3166@localhost.localdomain>

On Tue, Jun 05, 2007 at 10:13:41PM +0200, Wagner Ferenc wrote:
> Hi,
> 
> what's the best way to install GHC 6.6.1 on a Debian Etch system?
> Basically: are there installable packages available somewhere, or
> should I recompile the Sid packages, or create a stub package from a
> binary .tar.gz bundle, or some other option I didn't think of?

Yes.

Install the binary .tar.gz bundle *without creating a stub package*

It installs in /usr/local, so dpkg must not know about it.

Stefan
From ndmitchell at gmail.com  Tue Jun  5 18:23:57 2007
From: ndmitchell at gmail.com (Neil Mitchell)
Date: Tue Jun  5 18:19:21 2007
Subject: Prelude not in haskell98?
In-Reply-To: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
Message-ID: <404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>

Hi Iavor,

> All this seems to indicate that the Prelude is not a part of the
> "haskell98" package.  Is this the case, and if so, is this
> intentional?  It would be nice if we could create Cabal packages that
> explicitly indicate that the library depends only on Haskell 98
> libraries.

Its true, and its intentional. The trend nowadays is to make programs
depend only on base, not only on haskell98 - i.e. import
System.Environment (and others) instead of System. hakell98 will be
around forever, but base is the new "standard libraries" set.

Thanks

Neil
From igloo at earth.li  Tue Jun  5 19:39:20 2007
From: igloo at earth.li (Ian Lynagh)
Date: Tue Jun  5 19:34:45 2007
Subject: Prelude not in haskell98?
In-Reply-To: <404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>
Message-ID: <20070605233920.GA32312@matrix.chaos.earth.li>

On Tue, Jun 05, 2007 at 11:23:57PM +0100, Neil Mitchell wrote:
> 
> >All this seems to indicate that the Prelude is not a part of the
> >"haskell98" package.  Is this the case, and if so, is this
> >intentional?  It would be nice if we could create Cabal packages that
> >explicitly indicate that the library depends only on Haskell 98
> >libraries.
> 
> Its true, and its intentional. The trend nowadays is to make programs
> depend only on base, not only on haskell98 - i.e. import
> System.Environment (and others) instead of System. hakell98 will be
> around forever, but base is the new "standard libraries" set.

Right, the problem is that if Prelude was in haskell98 then it wouldn't
be possible to have a program that /didn't/ depend on haskell98 (short
of -fno-implicit-prelude extensions, or having a Prelude in both (which
would mean you couldn't depend on both base and haskell98)).


Thanks
Ian

From stefanor at cox.net  Tue Jun  5 19:50:11 2007
From: stefanor at cox.net (Stefan O'Rear)
Date: Tue Jun  5 19:45:39 2007
Subject: Prelude not in haskell98?
In-Reply-To: <20070605233920.GA32312@matrix.chaos.earth.li>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>
	<20070605233920.GA32312@matrix.chaos.earth.li>
Message-ID: <20070605235011.GA5480@localhost.localdomain>

On Wed, Jun 06, 2007 at 12:39:20AM +0100, Ian Lynagh wrote:
> On Tue, Jun 05, 2007 at 11:23:57PM +0100, Neil Mitchell wrote:
> > 
> > >All this seems to indicate that the Prelude is not a part of the
> > >"haskell98" package.  Is this the case, and if so, is this
> > >intentional?  It would be nice if we could create Cabal packages that
> > >explicitly indicate that the library depends only on Haskell 98
> > >libraries.
> > 
> > Its true, and its intentional. The trend nowadays is to make programs
> > depend only on base, not only on haskell98 - i.e. import
> > System.Environment (and others) instead of System. hakell98 will be
> > around forever, but base is the new "standard libraries" set.
> 
> Right, the problem is that if Prelude was in haskell98 then it wouldn't
> be possible to have a program that /didn't/ depend on haskell98 (short
> of -fno-implicit-prelude extensions, or having a Prelude in both (which
> would mean you couldn't depend on both base and haskell98)).

There's also a pragmatic reason.  Currently it is impossible to have a
cycle of module imports cross package boundaries; so if the Prelude was
in haskell98, then (since the Prelude depends on non-haskell98 base
modules for its implementation) *Nothing* in base could use the prelude.

Stefan
From isaacdupree at charter.net  Tue Jun  5 20:05:38 2007
From: isaacdupree at charter.net (Isaac Dupree)
Date: Tue Jun  5 19:59:31 2007
Subject: Prelude not in haskell98?
In-Reply-To: <20070605235011.GA5480@localhost.localdomain>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>	<20070605233920.GA32312@matrix.chaos.earth.li>
	<20070605235011.GA5480@localhost.localdomain>
Message-ID: <4665FA52.10707@charter.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stefan O'Rear wrote:
>> Right, the problem is that if Prelude was in haskell98 then it wouldn't
>> be possible to have a program that /didn't/ depend on haskell98 (short
>> of -fno-implicit-prelude extensions, or having a Prelude in both (which
>> would mean you couldn't depend on both base and haskell98)).
> 
> There's also a pragmatic reason.  Currently it is impossible to have a
> cycle of module imports cross package boundaries; so if the Prelude was
> in haskell98, then (since the Prelude depends on non-haskell98 base
> modules for its implementation) *Nothing* in base could use the prelude.

(Assuming haskell98 depends on base currently)  It seems we would want
haskell98 to re-export base's Prelude.  It's just that that's not
implemented in released-ghc package management code.

Isaac

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGZfpSHgcxvIWYTTURAhPvAJwIe+CTJnOBpOLQohbK+kC8Rg2XawCeK+E0
H8vIR1fCNZE9C2/dMLXLvQw=
=aMyC
-----END PGP SIGNATURE-----
From simonmarhaskell at gmail.com  Wed Jun  6 03:53:47 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Wed Jun  6 03:49:16 2007
Subject: Prelude not in haskell98?
In-Reply-To: <4665FA52.10707@charter.net>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>	<20070605233920.GA32312@matrix.chaos.earth.li>	<20070605235011.GA5480@localhost.localdomain>
	<4665FA52.10707@charter.net>
Message-ID: <4666680B.2000208@gmail.com>

Isaac Dupree wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Stefan O'Rear wrote:
>>> Right, the problem is that if Prelude was in haskell98 then it wouldn't
>>> be possible to have a program that /didn't/ depend on haskell98 (short
>>> of -fno-implicit-prelude extensions, or having a Prelude in both (which
>>> would mean you couldn't depend on both base and haskell98)).
>> There's also a pragmatic reason.  Currently it is impossible to have a
>> cycle of module imports cross package boundaries; so if the Prelude was
>> in haskell98, then (since the Prelude depends on non-haskell98 base
>> modules for its implementation) *Nothing* in base could use the prelude.
> 
> (Assuming haskell98 depends on base currently)  It seems we would want
> haskell98 to re-export base's Prelude.  It's just that that's not
> implemented in released-ghc package management code.

Yes, packages that re-export modules would be a useful feature.  Then we could 
have a version of the base package that only exported the compiler-independent 
modules, for example.

Cheers,
	Simon
From wferi at niif.hu  Wed Jun  6 04:17:27 2007
From: wferi at niif.hu (Wagner Ferenc)
Date: Wed Jun  6 04:12:52 2007
Subject: GHC 6.6.1 on Debian Etch?
In-Reply-To: <20070605221321.GA3166@localhost.localdomain> (Stefan O'Rear's
	message of "Tue, 5 Jun 2007 15:13:21 -0700")
References: <877iqiyxai.fsf@szonett.ki.iif.hu>
	<20070605221321.GA3166@localhost.localdomain>
Message-ID: <87abvdlco8.fsf@szonett.ki.iif.hu>

Stefan O'Rear <stefanor@cox.net> writes:

> On Tue, Jun 05, 2007 at 10:13:41PM +0200, Wagner Ferenc wrote:
>
>> what's the best way to install GHC 6.6.1 on a Debian Etch system?
>> Basically: are there installable packages available somewhere, or
>> should I recompile the Sid packages, or create a stub package from a
>> binary .tar.gz bundle, or some other option I didn't think of?
>
> Yes.
>
> Install the binary .tar.gz bundle *without creating a stub package*
>
> It installs in /usr/local, so dpkg must not know about it.

Hmm, you have got a point, but then I can't uninstall/upgrade later
easily, I'm afraid.
-- 
Thanks,
Feri.
From iavor.diatchki at gmail.com  Wed Jun  6 12:19:10 2007
From: iavor.diatchki at gmail.com (Iavor Diatchki)
Date: Wed Jun  6 12:14:33 2007
Subject: Prelude not in haskell98?
In-Reply-To: <4666680B.2000208@gmail.com>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>
	<20070605233920.GA32312@matrix.chaos.earth.li>
	<20070605235011.GA5480@localhost.localdomain>
	<4665FA52.10707@charter.net> <4666680B.2000208@gmail.com>
Message-ID: <5ab17e790706060919v33e4e37et86c2adeb476f82ea@mail.gmail.com>

Hi,
Just curious, what external modules does the Prelude depend on?
The only unusual feature that I can think of in the GHC Prelude is the
fact that the partial functions throw exceptions, and so perhaps the
Prelude implementation uses some extra modules for that?
-Iavor


On 6/6/07, Simon Marlow <simonmarhaskell@gmail.com> wrote:
> Isaac Dupree wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Stefan O'Rear wrote:
> >>> Right, the problem is that if Prelude was in haskell98 then it wouldn't
> >>> be possible to have a program that /didn't/ depend on haskell98 (short
> >>> of -fno-implicit-prelude extensions, or having a Prelude in both (which
> >>> would mean you couldn't depend on both base and haskell98)).
> >> There's also a pragmatic reason.  Currently it is impossible to have a
> >> cycle of module imports cross package boundaries; so if the Prelude was
> >> in haskell98, then (since the Prelude depends on non-haskell98 base
> >> modules for its implementation) *Nothing* in base could use the prelude.
> >
> > (Assuming haskell98 depends on base currently)  It seems we would want
> > haskell98 to re-export base's Prelude.  It's just that that's not
> > implemented in released-ghc package management code.
>
> Yes, packages that re-export modules would be a useful feature.  Then we could
> have a version of the base package that only exported the compiler-independent
> modules, for example.
>
> Cheers,
>         Simon
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
From gwright at comcast.net  Thu Jun  7 11:56:35 2007
From: gwright at comcast.net (Gregory Wright)
Date: Thu Jun  7 11:51:57 2007
Subject: Failure building HEAD
Message-ID: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>


Hi,

Today I was building HEAD using macports and had the error:

configure: Using compiler: ../../compiler/ghc-inplace
configure: Compiler flavor: GHC
configure: Compiler version: 6.7.20070607
configure: Using package tool: ../../utils/ghc-pkg/ghc-pkg-inplace
checking for gcc... gcc
checking for C compiler default output file name... configure: error:  
C compiler cannot create executables
See `config.log' for more details.

(this was in libraries/base).  The config.log showed the problem:

configure:1680: checking for C compiler default output file name
configure:1683: gcc -I/opt/local/include --configure-option=-O2 -I/ 
opt/local/include --configure-option=-I/opt/local/include -L/opt/ 
local/lib --configure-option=-L/opt/local/lib conftest.c  >&5
cc1: error: unrecognized command line option "-fconfigure-option=-O2"
cc1: error: unrecognized command line option "-fconfigure-option=-I/ 
opt/local/include"cc1: error: unrecognized command line option "- 
fconfigure-option=-L/opt/local/lib"configure:1686: $? = 1


It looks like cabal is messed up, passing "--configure-option=" when  
it should be
stripping it out.

Is this a problem with the HEAD branch or do I need a development  
version
of cabal from the darcs repository to build the latest ghc?

Best Wishes,
Greg

From mdanish at andrew.cmu.edu  Thu Jun  7 17:33:30 2007
From: mdanish at andrew.cmu.edu (Matthew Danish)
Date: Thu Jun  7 17:28:49 2007
Subject: Seemingly subtle change causes large performance variation
Message-ID: <20070607213330.GA2031@mapcar.org>

Hello,

I've been playing with the INTEST problem on SPOJ which demonstrates
the ability to write a program which processes large quantities of
input data.  http://www.spoj.pl/problems/INTEST/

I came across some curious behavior while cleaning up the program.
The original program, which runs fast (enough), is:

module Main(main) where
import Control.Monad
import Data.Maybe
import qualified Data.ByteString.Char8 as B

divisibleBy :: Int -> Int -> Bool
a `divisibleBy` n = a `rem` n == 0

main :: IO ()
main = do
    [n,k] <- (map int . B.split ' ') `fmap` B.getLine :: IO [Int]

    let
        doLine :: Int -> Int -> IO Int
        doLine r _ = B.getLine >>= testDiv r
        testDiv r l
         | int l `divisibleBy` k = return (r + 1)
         | otherwise             = return r

    foldM doLine 0 [1..n] >>= print

    where
        int :: B.ByteString -> Int
        int = fst . fromJust . B.readInt 




But when I make a slight modification, the program chews up a ton more memory
and takes more time:

module Main(main) where
import Control.Monad
import Data.Maybe
import qualified Data.ByteString.Char8 as B

divisibleBy :: Int -> Int -> Bool
a `divisibleBy` n = a `rem` n == 0

main :: IO ()
main = do
    [n,k] <- (map int . B.split ' ') `fmap` B.getLine :: IO [Int]

    let
        doLine :: Int -> Int -> IO Int
        doLine r _ = B.getLine >>= return . testDiv r
        -- 'return' moved here      ^^
        testDiv r l
         | int l `divisibleBy` k = r + 1
         | otherwise             = r

    foldM doLine 0 [1..n] >>= print

    where
        int :: B.ByteString -> Int
        int = fst . fromJust . B.readInt 




This program will generate sample data:

import System.Random
import System.Environment
import Control.Monad

main = do
    [n] <- map read `fmap` getArgs :: IO [Int]
    k   <- randomRIO (1, 100)
    putStrLn $ unwords [show n, show k]
    replicateM_ n $ randomRIO (1, 10^9) >>= print


Note that the same behavior occurs even if I manually inline the local
function and try: return (if .. then .. else).

Some sample runs:

$ ghc/compiler/ghc-inplace 
ghc-6.7.20070601: no input files

$ ghc/compiler/ghc-inplace --make -O2 intest.hs
[1 of 1] Compiling Main             ( intest.hs, intest.o )
Linking intest ...
$ ghc/compiler/ghc-inplace --make -O2 intest_slow.hs
[1 of 1] Compiling Main             ( intest_slow.hs, intest_slow.o )
Linking intest_slow ...

$ time ./intest +RTS -tstderr -RTS < test1
./intest +RTS -tstderr
8830
<<ghc: 134876896 bytes, 248 GCs, 28672/28672 avg/max bytes residency
(1 samples), 2M in use, 0.00 INIT (0.00 elapsed), 0.12 MUT (0.12
elapsed), 0.00 GC (0.00 elapsed) :ghc>>

real    0m0.129s
user    0m0.124s
sys     0m0.006s

$ time ./intest_slow +RTS -tstderr -RTS < test1
./intest_slow +RTS -tstderr
8830
<<ghc: 144278584 bytes, 269 GCs, 7030784/21843968 avg/max bytes
residency (6 samples), 38M in use, 0.00 INIT (0.00 elapsed), 0.13 MUT
(0.14 elapsed), 0.10 GC (0.15 elapsed) :ghc>>

real    0m0.296s
user    0m0.238s
sys     0m0.058s

-- 
-- Matthew Danish -- user: mrd domain: cmu.edu
-- OpenPGP public key: C24B6010 on keyring.debian.org
From simonpj at microsoft.com  Fri Jun  8 01:51:54 2007
From: simonpj at microsoft.com (Simon Peyton-Jones)
Date: Fri Jun  8 01:47:14 2007
Subject: debugging why we end up calling the wrapper rather than the
 worker
In-Reply-To: <DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
References: <1180729201.16937.16.camel@localhost>
	<DCA83D5F3861C941808187D4D387875011E0328798@EA-EXMSG-C312.europe.corp.microsoft.com>
Message-ID: <DCA83D5F3861C941808187D4D387875011E03E6723@EA-EXMSG-C312.europe.corp.microsoft.com>


| Meanwhile, I should probably do no w/w for NOINLINE things.

I've now done this in the HEAD: no worker/wrapper split for NOINLINE things.
Yell if anything breaks, but I don't see that anything bad should happen.

Simon

From wferi at niif.hu  Fri Jun  8 03:51:12 2007
From: wferi at niif.hu (Ferenc Wagner)
Date: Fri Jun  8 03:46:30 2007
Subject: GHC 6.6.1 on Debian Etch?
In-Reply-To: <87abvdlco8.fsf@szonett.ki.iif.hu> (Wagner Ferenc's message of
	"Wed, 06 Jun 2007 10:17:27 +0200")
References: <877iqiyxai.fsf@szonett.ki.iif.hu>
	<20070605221321.GA3166@localhost.localdomain>
	<87abvdlco8.fsf@szonett.ki.iif.hu>
Message-ID: <877iqelw9b.fsf@tac.ki.iif.hu>

Wagner Ferenc <wferi@niif.hu> writes:

> Stefan O'Rear <stefanor@cox.net> writes:
>
>> On Tue, Jun 05, 2007 at 10:13:41PM +0200, Wagner Ferenc wrote:
>>
>>> what's the best way to install GHC 6.6.1 on a Debian Etch system?
>>> Basically: are there installable packages available somewhere, or
>>> should I recompile the Sid packages, or create a stub package from a
>>> binary .tar.gz bundle, or some other option I didn't think of?
>>
>> Yes.
>>
>> Install the binary .tar.gz bundle *without creating a stub package*
>>
>> It installs in /usr/local, so dpkg must not know about it.
>
> Hmm, you have got a point, but then I can't uninstall/upgrade later
> easily, I'm afraid.

I poked a little further down this road.  For creating a package, it
would be useful if I could ./configure --prefix=/usr and later
DESTDIR=/tmp/stage make install (or similar, you got the idea) so that
I can gather the would-be-installed files into a package for later
installation.  If no pathes are wired into the binaries during make
install, then setting prefix to /tmp/stage/usr would do the trick, but
it's not the case, I guess.  Something similar must be necessary for
Igloo to create the Debian Sid packages, so I find it somewhat
surprising that it's not present in Makefile.in (or Makefile-bin.in?)
On the other hand, he must be working with the source distribution,
which may have this infrastructure...  Anyway, it would be simple to
add this by augmenting the install commands, but before I dive into
this, I wanted to ask for your opinions.
-- 
Thanks,
Feri.
From stefan at cs.uu.nl  Fri Jun  8 04:07:13 2007
From: stefan at cs.uu.nl (Stefan Holdermans)
Date: Fri Jun  8 04:02:32 2007
Subject: Indirection vs. in-place update of closures
Message-ID: <5990B443-3664-464B-90F4-1B1B31D25621@cs.uu.nl>

Hi,

Is there any data available on how often, in GHC-generated code,  
closures are being updated in-place as opposed to and in comparison  
with being replaced by indirections to their results?

How is it decided whether an in-place or an indirecting update is  
performed?

Thanks,

   Stefan
From simonmarhaskell at gmail.com  Fri Jun  8 04:42:15 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Fri Jun  8 04:37:38 2007
Subject: Prelude not in haskell98?
In-Reply-To: <5ab17e790706060919v33e4e37et86c2adeb476f82ea@mail.gmail.com>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>	
	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>	
	<20070605233920.GA32312@matrix.chaos.earth.li>	
	<20070605235011.GA5480@localhost.localdomain>	
	<4665FA52.10707@charter.net> <4666680B.2000208@gmail.com>
	<5ab17e790706060919v33e4e37et86c2adeb476f82ea@mail.gmail.com>
Message-ID: <46691667.5000105@gmail.com>

Iavor Diatchki wrote:
> Hi,
> Just curious, what external modules does the Prelude depend on?
> The only unusual feature that I can think of in the GHC Prelude is the
> fact that the partial functions throw exceptions, and so perhaps the
> Prelude implementation uses some extra modules for that?

I'm not really sure what you mean by "external modules".  In GHC the Prelude is 
comprised almost entirely of re-exports from other modules.  You can follow the 
source code links from the Haddock docs to take a look:

http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html

One big dependency is that the Prelude exports some I/O functionality, so it 
pulls in the entire I/O library.

Cheers,
	Simon


> -Iavor
> 
> 
> On 6/6/07, Simon Marlow <simonmarhaskell@gmail.com> wrote:
>> Isaac Dupree wrote:
>> > -----BEGIN PGP SIGNED MESSAGE-----
>> > Hash: SHA1
>> >
>> > Stefan O'Rear wrote:
>> >>> Right, the problem is that if Prelude was in haskell98 then it 
>> wouldn't
>> >>> be possible to have a program that /didn't/ depend on haskell98 
>> (short
>> >>> of -fno-implicit-prelude extensions, or having a Prelude in both 
>> (which
>> >>> would mean you couldn't depend on both base and haskell98)).
>> >> There's also a pragmatic reason.  Currently it is impossible to have a
>> >> cycle of module imports cross package boundaries; so if the Prelude 
>> was
>> >> in haskell98, then (since the Prelude depends on non-haskell98 base
>> >> modules for its implementation) *Nothing* in base could use the 
>> prelude.
>> >
>> > (Assuming haskell98 depends on base currently)  It seems we would want
>> > haskell98 to re-export base's Prelude.  It's just that that's not
>> > implemented in released-ghc package management code.
>>
>> Yes, packages that re-export modules would be a useful feature.  Then 
>> we could
>> have a version of the base package that only exported the 
>> compiler-independent
>> modules, for example.
>>
>> Cheers,
>>         Simon
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> Glasgow-haskell-users@haskell.org
>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>
> 

From simonmarhaskell at gmail.com  Fri Jun  8 04:45:18 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Fri Jun  8 04:40:38 2007
Subject: Indirection vs. in-place update of closures
In-Reply-To: <5990B443-3664-464B-90F4-1B1B31D25621@cs.uu.nl>
References: <5990B443-3664-464B-90F4-1B1B31D25621@cs.uu.nl>
Message-ID: <4669171E.6040901@gmail.com>

Stefan Holdermans wrote:

> Is there any data available on how often, in GHC-generated code, 
> closures are being updated in-place as opposed to and in comparison with 
> being replaced by indirections to their results?
> 
> How is it decided whether an in-place or an indirecting update is 
> performed?

The anser to these questions is easy.  Closures are never updated in place :-)

GHC used to have selective update-in-place in the past (pre-4.00), but it had 
highly complex interactions with various other aspects of the execution model, 
so we removed it, or rather, didn't re-implement it in the big runtime rewrite 
of 4.00.

Cheers,
	Simon
From simonmarhaskell at gmail.com  Fri Jun  8 05:02:48 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Fri Jun  8 04:58:07 2007
Subject: "GHC as a library" stalls when nobody is consuming it's output
In-Reply-To: <1180277964.4229.6.camel@localhost.localdomain>
References: <1180277964.4229.6.camel@localhost.localdomain>
Message-ID: <46691B38.2000708@gmail.com>

Mads Lindstr?m wrote:

> While trying to implement a GUI for GHCi, I have run into an annoying
> concurrency problems. I think "GHC as a library" is at fault, as it
> stalls (maybe some deadlock) when nobody is consuming it's output.

Thanks Mads, I took a look but didn't get to the bottom of the problem yet.  It 
behaves differently with the current development GHC, at least.  For now I've 
created a ticket for it:

http://hackage.haskell.org/trac/ghc/ticket/1419

and I'll get to it before the 6.8 release.

Cheers,
	Simon
From Malcolm.Wallace at cs.york.ac.uk  Fri Jun  8 06:50:47 2007
From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace)
Date: Fri Jun  8 06:48:37 2007
Subject: Prelude not in haskell98?
In-Reply-To: <46691667.5000105@gmail.com>
References: <5ab17e790706051505o189b0e8cpcdb2ff350299f9d7@mail.gmail.com>
	<404396ef0706051523t1775de0j90dc766912cfc741@mail.gmail.com>
	<20070605233920.GA32312@matrix.chaos.earth.li>
	<20070605235011.GA5480@localhost.localdomain>
	<4665FA52.10707@charter.net> <4666680B.2000208@gmail.com>
	<5ab17e790706060919v33e4e37et86c2adeb476f82ea@mail.gmail.com>
	<46691667.5000105@gmail.com>
Message-ID: <20070608115047.41e5ed93.Malcolm.Wallace@cs.york.ac.uk>

> Iavor Diatchki wrote:
> > Just curious, what external modules does the Prelude depend on?
> 
> I'm not really sure what you mean by "external modules".  In GHC the
> Prelude is  comprised almost entirely of re-exports from other
> modules.

Just to note that, although ghc's inversion of the expected dependency
relationship is mildly confusing the first time you come across it, that
design decision is not the only one possible.  Other compilers, such as
nhc98, treat the haskell98 Prelude more like it is specified in the
Report, as a stand-alone module that depends on nothing else.  In such a
compiler, the 'base' package does not (and cannot) contain the Prelude,
which necessarily lives at a lower more basic level in the library
hierarchy.

Regards,
    Malcolm
From wferi at niif.hu  Fri Jun  8 09:30:03 2007
From: wferi at niif.hu (Ferenc Wagner)
Date: Fri Jun  8 09:25:21 2007
Subject: GHC 6.6.1 on Debian Etch?
In-Reply-To: <877iqelw9b.fsf@tac.ki.iif.hu> (Ferenc Wagner's message of "Fri, 
	08 Jun 2007 09:51:12 +0200")
References: <877iqiyxai.fsf@szonett.ki.iif.hu>
	<20070605221321.GA3166@localhost.localdomain>
	<87abvdlco8.fsf@szonett.ki.iif.hu> <877iqelw9b.fsf@tac.ki.iif.hu>
Message-ID: <87vedyk204.fsf@tac.ki.iif.hu>

In the end I created a new ghc6 package from the binary bundle by
slight modifications and shameless theft from Ian Lynagh control
file.  It seems to work together with libreadline4 from Sarge.  I'm
willing to share it with anybody interested.
-- 
Regards,
Feri.
From igloo at earth.li  Fri Jun  8 11:24:14 2007
From: igloo at earth.li (Ian Lynagh)
Date: Fri Jun  8 11:19:31 2007
Subject: Failure building HEAD
In-Reply-To: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>
References: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>
Message-ID: <20070608152414.GA11873@matrix.chaos.earth.li>

On Thu, Jun 07, 2007 at 11:56:35AM -0400, Gregory Wright wrote:
> 
> Is this a problem with the HEAD branch or do I need a development  
> version
> of cabal from the darcs repository to build the latest ghc?

I'm not sure exactly what you mean. You need the latest darcs Cabal in
libraries/Cabal in your GHC tree, and if this is a tree that you have
previously built then you may also need to first remove
libraries/stamp/bootstrapping.Cabal. However, you don't need the latest
Cabal registered with the GHC that you are building /with/.


Thanks
Ian

From gwright at comcast.net  Fri Jun  8 11:35:04 2007
From: gwright at comcast.net (Gregory Wright)
Date: Fri Jun  8 11:30:20 2007
Subject: Failure building HEAD
In-Reply-To: <20070608152414.GA11873@matrix.chaos.earth.li>
References: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>
	<20070608152414.GA11873@matrix.chaos.earth.li>
Message-ID: <E72B9E3D-3C92-4BFB-99DC-681D1F50CBF4@comcast.net>


Hi Ian,

On Jun 8, 2007, at 11:24 AM, Ian Lynagh wrote:

> On Thu, Jun 07, 2007 at 11:56:35AM -0400, Gregory Wright wrote:
>>
>> Is this a problem with the HEAD branch or do I need a development
>> version
>> of cabal from the darcs repository to build the latest ghc?
>
> I'm not sure exactly what you mean. You need the latest darcs Cabal in
> libraries/Cabal in your GHC tree, and if this is a tree that you have
> previously built then you may also need to first remove
> libraries/stamp/bootstrapping.Cabal. However, you don't need the  
> latest
> Cabal registered with the GHC that you are building /with/.
>

I started by deleting my old directory and doing a fresh get from the  
darcs
repository so there was nothing left over from a previous build.  It  
looks as
if something is broken then.  For example, "--configure-option=-O2"  
is showing up in the
command line of gcc when one of the test programs is built by the  
configure
script in libraries/base. I assume that this should just have been "- 
O2".
The configure script fails because gcc barfs on the unrecognized option.

So no new dependencies have been introduced in the HEAD build as I
understand it.  Anyone else have problems building HEAD in the last  
couple
of days.  I know it's not just me; it was reported to me by another  
person
using macports to build HEAD.

Best Wishes,
Greg

From igloo at earth.li  Fri Jun  8 12:03:16 2007
From: igloo at earth.li (Ian Lynagh)
Date: Fri Jun  8 11:58:37 2007
Subject: Failure building HEAD
In-Reply-To: <E72B9E3D-3C92-4BFB-99DC-681D1F50CBF4@comcast.net>
References: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>
	<20070608152414.GA11873@matrix.chaos.earth.li>
	<E72B9E3D-3C92-4BFB-99DC-681D1F50CBF4@comcast.net>
Message-ID: <20070608160316.GA12030@matrix.chaos.earth.li>


Hi Gregory,

On Fri, Jun 08, 2007 at 11:35:04AM -0400, Gregory Wright wrote:
> 
> I started by deleting my old directory and doing a fresh get from the  
> darcs
> repository so there was nothing left over from a previous build.  It  
> looks as
> if something is broken then.  For example, "--configure-option=-O2"  
> is showing up in the
> command line of gcc when one of the test programs is built by the  
> configure
> script in libraries/base. I assume that this should just have been "- 
> O2".
> The configure script fails because gcc barfs on the unrecognized option.

Hmm, what is the complete command (i.e. with all arguments) that starts
"cd base && setup/Setup configure"?

Do you have a mk/build.mk?


Thanks
Ian

From gwright at comcast.net  Fri Jun  8 13:41:32 2007
From: gwright at comcast.net (Gregory Wright)
Date: Fri Jun  8 13:36:48 2007
Subject: Failure building HEAD
In-Reply-To: <20070608160316.GA12030@matrix.chaos.earth.li>
References: <1D01E206-B041-44A1-ADF5-D20220C7A4EA@comcast.net>
	<20070608152414.GA11873@matrix.chaos.earth.li>
	<E72B9E3D-3C92-4BFB-99DC-681D1F50CBF4@comcast.net>
	<20070608160316.GA12030@matrix.chaos.earth.li>
Message-ID: <5C1A849F-09D2-4354-992C-75E2E91A964F@comcast.net>


Hi Ian,

On Jun 8, 2007, at 12:03 PM, Ian Lynagh wrote:

>
> Hmm, what is the complete command (i.e. with all arguments) that  
> starts
> "cd base && setup/Setup configure"?
>


make -C libraries all
rm -f -f stamp/configure.library.*.base base/unbuildable
( cd base && setup/Setup configure \
             --enable-library-profiling --enable-split-objs \
            --prefix=/opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/ 
destroot//opt/local \
            --with-compiler=../../compiler/ghc-inplace \
            --with-hc-pkg=../../utils/ghc-pkg/ghc-pkg-inplace \
            --with-hsc2hs=../../utils/hsc2hs/hsc2hs-inplace \
            --with-ld=/usr/bin/ld \
            --datasubdir=ghc \
            --haddock-args="--use-contents=../index.html \
                            --use-index=../doc-index.html" \
            --configure-option='--prefix=/opt/local' --configure- 
option='--prefix=/opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/ 
destroot//opt/local' --configure-option='--mandir=/opt/local/var/db/ 
dports/build/_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/ 
work/destroot//opt/local/share/man/' --configure-option='--with- 
readline-includes=/opt/local/include' --configure-option='--with- 
readline-libraries=/opt/local/lib' --configure-option='--with-gmp- 
includes=/opt/local/include' --configure-option='--with-gmp- 
libraries=/opt/local/lib' --configure-option='--disable-openal' -- 
configure-option='--disable-alut' --configure-option='CFLAGS=-I/opt/ 
local/include --configure-option=-O2' --configure-option='CPPFLAGS=-I/ 
opt/local/include --configure-option=-I/opt/local/include' -- 
configure-option='LDFLAGS=-L/opt/local/lib --configure-option=-L/opt/ 
local/lib' \
            --configure-option=--with-cc=gcc ) \
       && touch stamp/configure.library.build-profiling- 
splitting.base || touch base/unbuildable
Setup: Warning: Unknown field 'nhc98-options'
configure: Reading installed packages...
Configuring base-2.1...
configure: Using install prefix: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local
configure: Binaries installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/bin
configure: Libraries installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/lib/base-2.1/ghc-6.7.20070608
configure: Private binaries installed in: /opt/local/var/db/dports/ 
build/_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/ 
destroot/opt/local/libexec
configure: Data files installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/share/ghc
configure: Using compiler: ../../compiler/ghc-inplace
configure: Compiler flavor: GHC
configure: Compiler version: 6.7.20070608
configure: Using package tool: ../../utils/ghc-pkg/ghc-pkg-inplace
configure: Using ar found on system at: /usr/bin/ar
configure: Using haddock found on system at: /opt/local/bin/haddock
configure: Using ld given by user at: /usr/bin/ld
configure: No pfesetup found
configure: Using ranlib found on system at: /usr/bin/ranlib
configure: Using runghc found on system at: /opt/local/bin/runghc
configure: Using runhugs found on system at: /opt/local/bin/runhugs
configure: Using tar found on system at: /usr/bin/tar
configure: Using happy: /opt/local/bin/happy
configure: Using alex: /opt/local/bin/alex
configure: Using hsc2hs: ../../utils/hsc2hs/hsc2hs-inplace
configure: Using c2hs: /opt/local/bin/c2hs
configure: Using cpphs: /opt/local/bin/cpphs
configure: No greencard found
configure: Reading installed packages...
Configuring base-2.1...
configure: Dependency rts-any: using rts-1.0
configure: Using install prefix: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local
configure: Binaries installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/bin
configure: Libraries installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/lib/base-2.1/ghc-6.7.20070608
configure: Private binaries installed in: /opt/local/var/db/dports/ 
build/_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/ 
destroot/opt/local/libexec
configure: Data files installed in: /opt/local/var/db/dports/build/ 
_Users_gwright_src_macports-trunk_dports_lang_ghc-devel/work/destroot/ 
opt/local/share/ghc
configure: Using compiler: ../../compiler/ghc-inplace
configure: Compiler flavor: GHC
configure: Compiler version: 6.7.20070608
configure: Using package tool: ../../utils/ghc-pkg/ghc-pkg-inplace
checking for gcc... gcc
checking for C compiler default output file name... configure: error:  
C compiler cannot create executables
See `config.log' for more details.
configure: Using ar found on system at: /usr/bin/ar
configure: Using haddock found on system at: /opt/local/bin/haddock
configure: Using ld given by user at: /usr/bin/ld
configure: No pfesetup found
configure: Using ranlib found on system at: /usr/bin/ranlib
configure: Using runghc found on system at: /opt/local/bin/runghc
configure: Using runhugs found on system at: /opt/local/bin/runhugs
configure: Using tar found on system at: /usr/bin/tar
configure: Using happy: /opt/local/bin/happy
configure: Using alex: /opt/local/bin/alex
configure: Using hsc2hs: ../../utils/hsc2hs/hsc2hs-inplace
configure: Using c2hs: /opt/local/bin/c2hs
configure: Using cpphs: /opt/local/bin/cpphs
configure: No greencard found
ifBuildable/ifBuildable base setup/Setup build \
                                 --ghc-option=-O --ghc-option=-Rghc- 
timing --ghc-option=-fgenerics
ifBuildable: base is unbuildable
make[1]: *** [build.library.base] Error 1
make: *** [stage1] Error 2
Error: Target com.apple.build returned: shell command "env  
DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib make all" returned error 2

> Do you have a mk/build.mk?
>
>

Here is mk/build.mk.  Some of these may be obsolete; I think
this is mostly unchanged from 6.5 days when more hints were needed
to link libraries not in the usual locations.

#
# Local configuration overrides for DarwinPorts
#
SRC_CC_OPTS += -I/opt/local/include
SRC_HC_OPTS += -I/opt/local/include -I/usr/include -L/opt/local/lib - 
L/usr/lib
EXTRA_HSC2HS_OPTS += -I/opt/local/include
EXTRA_LD_OPTS += -L/opt/local/lib
EXTRA_LD_OPTS += -L/usr/lib
~



Best Wishes,
Greg

From mechvel at botik.ru  Fri Jun  8 14:14:07 2007
From: mechvel at botik.ru (Serge D. Mechveliani)
Date: Fri Jun  8 14:09:28 2007
Subject: > 2^32 b, 6.6 for Itanium
Message-ID: <20070608181407.GA19616@scico.botik.ru>

Dear GHC developers,

I have the two questions.


1. How  ghc-6.6  runs on 64-bit machines having more that  2^32  
   byte memory ?
   Can it run into "Segmentation fault" ? 


2. Can you prepare a reliable  ghc-6.6 binary  for  Itanium a64 Linux
   ?  
Or maybe, someone can contribute ?

One of my colleagues says the following.

-------------------------------------
He has taken such a binary from 
> (http://www.haskell.org/ghc/distribution_packages.html#debian),
>
somehow installed it and tried to make from source. It reports

>/tmp/ghc12430_0/ghc12430_0.s: Assembler messages:
>
>/tmp/ghc12430_0/ghc12430_0.s:51:0:
>     Error: Operand 1 of `mov' should be a general register
-------------------------------------

(though, I have not looked so far, in how all this was done). 


Regards,

-----------------
Serge Mechveliani
mechvel@botik.ru


From mechvel at botik.ru  Mon Jun 11 14:48:16 2007
From: mechvel at botik.ru (Serge D. Mechveliani)
Date: Mon Jun 11 14:43:29 2007
Subject: unneeded laziness case
Message-ID: <20070611184816.GA25561@scico.botik.ru>

People,

A couple of times I suffered of unneeded strictness in Haskell.
And now unneeded laziness bites.
Consider the following program:

-------------------------------------------------------
main = 
  let mK = <formed in a certain complex way> 
                                            :: Map.Map Partition Integer 
      row      = [0, 1, 0 ... 0] 
      (mK', p) = decompose mK row
      infoK'   = sum $ map sum $ mapmap (\ n -> rem n 2) $ Map.elems mK'
  in
  putStr (shows                p 
                -- seq infoK' 
          "\n")
-------------------------------------------------------

In  ghc-6.6,  it needs at least  100 Mb  of heap to perform.
If I set   seq infoK'   between `shows' and `p', then it suffices  
                                                                14 Mb.
If I set there   seq mK',  then again it needs  100 Mb.
7 times as large.

mK  is a triangular  n by n  matrix,  n  is about  900
    (so, there are about 1 million of elements in  mK). 
mK  has small integers in it.
p   is found actually by solving the linear system  X x mK = row.  

Now: how to program real problems in Haskell ?
Should I try to set  seq  everywhere and see whether it helps ?

Also note:  seq mK'  was not sufficient. I do not know, probably, as  
mK' has structure,  seq  does not evaluate all its levels.
Therefore the programmer needs to invent a function that gathers much
of information of mK' and puts it into one Integer
(infoK' = sum ... mK'). Then,  seq infoK'  forces most of evaluation 
of  mK'. 
Is this a generic style to program real problems ? 
To write all this strange code instead of plain 

   let (_, p) = decompose mK row  in  putStr (shows p "\n")
?

The story was as follows.
My friend asked me to decompose certain large symmetric polynomial
into elementary symmetric ones. This was needed for a real mathematical
problem. I applied my DoCon program, which is written in Haskell.
1000 Mb RAM was not enough. I started to investigate, because I 
suspected that this case needs several times smaller memory. I found
a fragment of the type shown above. And I never use `seq'. 
I have put a bit smaller example to make it 300 Mb sufficient.   
The effect was discovered by occasion. I have forgotten that `decompose'
returns a pair with a large matrix as fst and wrote

   let  p = decompose mK row  in  putStr $ show p

It has computed in a relatively small memory, and fast, but before the  
p  value, it printed several pages of the matrix value.    
I said "Oh! we need to improve this, we only need p, the second part":

  let  (_, p) = decompose mK row  in  putStr $ show p

And this takes  7 times  more of memory and 2 times more of time!
Then, I understood, what is happening and verified the idea by 
introducing  seq.

And now, I wonder ...

Thank you in advance for your help and comments. 

Regards,

-----------------
Serge Mechveliani
mechvel@botik.ru
From Malcolm.Wallace at cs.york.ac.uk  Tue Jun 12 06:39:30 2007
From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace)
Date: Tue Jun 12 06:38:36 2007
Subject: Seemingly subtle change causes large performance variation
In-Reply-To: <20070607213330.GA2031@mapcar.org>
References: <20070607213330.GA2031@mapcar.org>
Message-ID: <20070612113930.0dd2c5e6.Malcolm.Wallace@cs.york.ac.uk>

Matthew Danish <mdanish@andrew.cmu.edu> wrote:

> I've been playing with the INTEST problem on SPOJ which demonstrates
> the ability to write a program which processes large quantities of
> input data.  http://www.spoj.pl/problems/INTEST/

I don't know if anyone replied to this already, so here is my attempt to
explain the space leak.

>         doLine :: Int -> Int -> IO Int
>         doLine r _ = B.getLine >>= testDiv r
>         testDiv r l
>          | int l `divisibleBy` k = return (r + 1)
>          | otherwise             = return r
> 
> But when I make a slight modification, the program chews up a ton more
> memory and takes more time:
> 
>         doLine :: Int -> Int -> IO Int
>         doLine r _ = B.getLine >>= return . testDiv r
>         -- 'return' moved here      ^^
>         testDiv r l
>          | int l `divisibleBy` k = r + 1
>          | otherwise             = r

In the first version, the strictness of the IO monad ensures that
'testDiv' must be evaluated, at least sufficiently to find the 'return'
monadic action inside it.  In particular, this forces the evaluation of
the guard, and therefore the call of `divisibleBy`.

Whereas in the latter version, the 'return' is wrapped _outside_ the
call to 'testDiv', so the monadic action is found immediately, whilst
the value being returned in in the monad is still lazily calculated.
Thus, a collection of 'testDiv r' applications builds up until the Int
values are actually used, at which point they are reduced.

Regards,
    Malcolm
From clemens at endorphin.org  Tue Jun 12 09:24:07 2007
From: clemens at endorphin.org (Clemens Fruhwirth)
Date: Tue Jun 12 09:28:36 2007
Subject: Locating shared libraries
Message-ID: <87y7ip71c8.wl%clemens@endorphin.org>

Hi,

I'm hacking on shared library support for GHC and it's coming along quite nicely.
http://hpaste.org/192

My initial hacks are available from:

http://clemens.endorphin.org/patches/ghc-20070605-initial-shared-libs.patch
(works only with x86-64 atm, on i386 the NCG dies in the register
allocator when compiling cmm files RTS)

http://clemens.endorphin.org/patches/cabal-20070528-initial-shared-library.patch

libtool usually takes care of creating shared libraries under *nix
system. libtool solves a few minor problems associated with:

1) creating shared libraries
2) linking programs that depend on shared libraries
3) running programs that depend on shared libraries

libtool is tailored to C compilers and the general opinion from #ghc
towards libtool seems to be: "hands off". From the list above, I will
try to sketch solutions without libtool.

1) creating shared libraries:

At the moment, my second patch teaches Cabal how to build shared
libraries. Basically, this is:
  * add -fPIC to the compiler invocation (and -optc-fPIC for c-sources),
  * invoke "ld -shared -Bsymbolic -o foo.so obj1.o obj2.o ...". 

ATM, ld is not invoked with the inter-library dependencies for the
shared library being built. This is not problematic as the final
executable will include all dependencies due to the ghc package
dependency tracking. But DT_NEEDED on ELF influences the sequence in
which shared library initializers are run. I have not yet investigated
if this leads to any problems.

To solve this little shortcoming, the ld-invocation could be delegated
to GHC. "ghc -o libHSfoo.so Foo1.o Foo2.o". We already have a similar
facility for DLLs (see MkDLL in DriverPipeline.hs). This could be
abstracted into MkShared, and platform specific knowledge could be
encapsulated in GHC. The benefit would be that we could easily access
the package information and we could create shared libraries that
contain proper DT_NEEDED sections.

2) Linking programs

Linking should work out of the box:

"ghc -dynamic -o HelloWorld HelloWorld.o" creates dynamically linked
executable.

3) Running programs

This a typical problem:
./HelloWorld 
./HelloWorld: error while loading shared libraries: libHShaskell98-1.0_dyn.so: cannot open shared object file: No such file or directory

There are several ways to add search paths for dynamic linking: either
we do it temporarily or we encode the search paths into the
executables. On ELF platforms, this works by adding -rpath to the
linker flags. This adds two new entries in the .dynamic section
(DT_RPATH, DT_RUNPATH) both responsible for signalling additional
search paths to the dynamic linker, ld.so. According to Simon Marlow,
Windows has similar mechanism via manifest files.

Let's see how libtool handles this situation. libtool differentiates
between installed and uninstalled libraries. When linking against
installed libraries not in the standard search path, libtool uses
-rpath to add these search paths to the created executable. When
linking against uninstalled libraries, libtool still uses -rpath but
pointing to the directory the uninstalled library is going to be
installed in. libtool derives this information from the .la
files+Makefiles.

In any case, libtool creates a wrapper in the build directory that
takes care of executing the program linked against uninstalled shared
libraries. There are two strategies for accomplishing this:
 * add the paths of the uninstalled shared libraries to LD_LIBRARY_PATH
 * relink the executable with additional -rpath's
libtool chooses the second strategy.

How do we translate these solutions to GHC? The first question is
whether we expect 

ghc -dynamic -package uninstalled-package -o Hello Hello.o
./Hello

to work or whether we require manual intervention in these cases. If
we expect this to work without intervention, we have the same options
as libtool:

* create a wrapper that takes care of locating the uninstalled shared
  libraries and sets LD_LIBRARY_PATH. 

* create a binary with rpath of the uninstalled libraries, and create
  an additional executable for deployment without these rpaths.

In any case we have to modify the installer scripts to either know
about where to locate the real binary, either ask ghc where to find
the real binary, or either delegate the installation to ghc. The last
option is basically the libtool way, "libtool --mode=install ..."

When we decide to create a deployable executable at the "-o" spot, we
need to 

* modify the invocation to manually pick up the libraries by modifying
  LD_LIBRARY_PATH.. this is pretty unpractical.

* delegate invocation to ghc. Maybe "ghc --execute HelloWorld". 
  libtool has a similar mechanism for executing 3rd party programs in
  the "dynamic environment" of the compiler executable. For instance,
  "gdb HelloWorld" would fails for libtool as HelloWorld is a wrapper,
  but "libtool --mode=execute gdb HelloWorld" works, as libtool
  rewrites to HelloWorld to .libs/lt-HelloWorld.

And now something completely different: Create a custom ELF program
interpreter for Haskell programs. Using INTERP in the ELF program
header, loads up this interpreter and delegates control to it. Usually
this is /lib/ld-linux.so.2, the dynamic linker, but we can replace that.

Haskell has its own idea of libraries/packages. We have package.conf
which gives us the location of the installed libraries. This is ok for
static linking, as at link time ghc is running and knows how to invoke
gcc with the correct paths. It does not matter, if package.conf is
updated afterwards as the statically linked programs contain a copy of
the library anyway. For dynamic linking this phase is delayed and when
we encode rpath such as "/usr/lib/network-2.0/ghc-6.6/", we can not
update to network-2.1 without breaking this executables.

A custom programming loading stub could access the global and local
package.conf and extract the library path for the dependencies and execve 

/lib64/ld-linux.so.2 --library-path=<paths of the dependencies> HelloWorld <args>

This certainly gives us more flexibility than encoding all these
rpaths statically into HelloWorld. To solve the inplace execution
directly from the build directory, we might create
.HelloWorld.package.conf in case a non-standard package.conf is used
(non-standard=different from global and local) and have the stub
loader to check for this file.

I agree that the last scheme sounds a bit wild, but I argue that
that's what ELF designers had in mind when they specified the INTERP
header. Of course, this is only a solution for ELF platforms.

Opinions :) ?
--
Fruhwirth Clemens - http://clemens.endorphin.org 


From mechvel at botik.ru  Tue Jun 12 10:57:19 2007
From: mechvel at botik.ru (Serge D. Mechveliani)
Date: Tue Jun 12 10:52:28 2007
Subject: unneeded laziness example
Message-ID: <20070612145719.GA26020@scico.botik.ru>

People,

I wrote recently 

> And now unneeded laziness bites.
> [..]
> In  ghc-6.6,  it needs at least  100 Mb  of heap to perform.
> If I set   seq infoK'   between `shows' and `p', then it suffices 14 Mb.
> [..]
> Now: how to program real problems in Haskell ?
> Should I try to set  seq  everywhere and see whether it helps ?


I am sorry. This was false alarm.
I continued the investigation and have found that 

  Haskell and GHC are NOT guilty in this example

(uh! how good).

This is the specifics of the method.  
Probably, one needs to think of  seq  only if one applies such a method with 
expenses depending heavily on the computation order.

An interesting thing, though. The simplified and contrived example is

---------------------------------------------------
main = putStr $ test 18

test :: Integer -> String
test    w       =  shows (f tM) "\n"
  where
  f           = Prelude.minimum . map Prelude.minimum
  mM          = makeMatrix w
  strictnessM = Prelude.minimum $ map Prelude.minimum mM

  tM = -- seq strictnessM $
       transpose mM
---------------------------------------------------

Here  mM  is  n by n  matrix of small integers,  n = 385  for  w = 18.

The program prints  f $ transpose mM.  

makeMatrix  computes the rows of  mM  in a complex way, finding  m(i,j)
uses the table of m(i',j') for other pairs (i',j'). And the time and 
space costs depend heavily on the order in which  m(i,j)  are computed.
Applying `transpose' changes (due to Haskell laziness) this order into 
more expensive one: 5 times more of heap needed. 
And adding  `seq ...'  before `transpose' returns it to initial order.
So, everything is natural.

Regards,

-----------------
Serge Mechveliani
mechvel@botik.ru
From bos at serpentine.com  Tue Jun 12 11:28:08 2007
From: bos at serpentine.com (Bryan O'Sullivan)
Date: Tue Jun 12 11:23:12 2007
Subject: Locating shared libraries
In-Reply-To: <87y7ip71c8.wl%clemens@endorphin.org>
References: <87y7ip71c8.wl%clemens@endorphin.org>
Message-ID: <466EBB88.7030706@serpentine.com>

Clemens Fruhwirth wrote:

> At the moment, my second patch teaches Cabal how to build shared
> libraries. Basically, this is:
>   * add -fPIC to the compiler invocation (and -optc-fPIC for c-sources),
>   * invoke "ld -shared -Bsymbolic -o foo.so obj1.o obj2.o ...". 
> 
> ATM, ld is not invoked with the inter-library dependencies for the
> shared library being built. This is not problematic as the final
> executable will include all dependencies due to the ghc package
> dependency tracking. But DT_NEEDED on ELF influences the sequence in
> which shared library initializers are run. I have not yet investigated
> if this leads to any problems.
> 
> To solve this little shortcoming, the ld-invocation could be delegated
> to GHC. "ghc -o libHSfoo.so Foo1.o Foo2.o".

You'd presumably want this to be "ghc -shared", yes?

It's definitely preferable to cook this knowledge into ghc, rather than 
to have the smarts in Cabal.  The advantage of keeping the knowledge in 
GHC is that it's more decoupled, and it mirrors the behaviour people 
expect from other compilers (this is what gcc does, for example).

> 3) Running programs

> Let's see how libtool handles this situation.

I would recommend against following libtool's lead in this area. 
Libtool's fondness for cooking RPATH into binaries makes it very 
difficult to deal with, because it's quite common for those binaries to 
get installed and distributed, RPATH and all.  RPATH should only be used 
by a user who knows they have a large-calibre weapon pointed at their foot.

> How do we translate these solutions to GHC? The first question is
> whether we expect 
> 
> ghc -dynamic -package uninstalled-package -o Hello Hello.o
> ./Hello
> 
> to work or whether we require manual intervention in these cases.

Manual intervention is definitely the right thing.  I could perhaps see 
Cabal's build command having a --in-place option that spits out a little 
shell script that augments LD_LIBRARY_PATH appropriately, but anything 
more will lead to trouble.

> I agree that the last scheme sounds a bit wild, but I argue that
> that's what ELF designers had in mind when they specified the INTERP
> header.

Let's not go there, please :-)  Such a move would be a big maintenance 
problem in its own right, and would make a lot of extra work for people 
packaging GHC for different distributions as they would need to cook up 
hacks for e.g. local SELinux policies regarding special ELF attributes 
and memory protections.

	<b
From rich.neswold at gmail.com  Tue Jun 12 12:26:52 2007
From: rich.neswold at gmail.com (Rich Neswold)
Date: Tue Jun 12 12:21:55 2007
Subject: Locating shared libraries
In-Reply-To: <466EBB88.7030706@serpentine.com>
References: <87y7ip71c8.wl%clemens@endorphin.org>
	<466EBB88.7030706@serpentine.com>
Message-ID: <14cf844b0706120926s38694705gee3d7b7b170fb9a1@mail.gmail.com>

On 6/12/07, Bryan O'Sullivan <bos@serpentine.com> wrote:
>
> > Let's see how libtool handles this situation.
>
> I would recommend against following libtool's lead in this area.
> Libtool's fondness for cooking RPATH into binaries makes it very
> difficult to deal with, because it's quite common for those binaries to
> get installed and distributed, RPATH and all.  RPATH should only be used
> by a user who knows they have a large-calibre weapon pointed at their
> foot.
>

The NetBSD Project has a different opinion and strongly encourages the use
of RPATHs. Letting the user alter the lookup path for installed binaries is
seen as a security problem. Since every NetBSD system has its libraries in
one directory and third-party libraries are in /usr/pkg/lib, there's no
problem in hardcoding the paths.
Just another data point...

-- 
Rich

JID: rich@neswold.homeunix.net
AIM: rnezzy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20070612/fe8a8cdf/attachment.htm
From bos at serpentine.com  Tue Jun 12 13:52:29 2007
From: bos at serpentine.com (Bryan O'Sullivan)
Date: Tue Jun 12 13:47:59 2007
Subject: Locating shared libraries
In-Reply-To: <14cf844b0706120926s38694705gee3d7b7b170fb9a1@mail.gmail.com>
References: <87y7ip71c8.wl%clemens@endorphin.org>	
	<466EBB88.7030706@serpentine.com>
	<14cf844b0706120926s38694705gee3d7b7b170fb9a1@mail.gmail.com>
Message-ID: <466EDD5D.8030306@serpentine.com>

Rich Neswold wrote:

> The NetBSD Project has a different opinion and strongly encourages the 
> use of RPATHs.

NetBSD is of course entitled to its opinions.  Indeed, this reinforces 
my point that use of RPATH should not be cooked into portable tools, 
because it's a policy that varies across ELF-using platforms.

	<b
From igloo at earth.li  Tue Jun 12 20:50:01 2007
From: igloo at earth.li (Ian Lynagh)
Date: Tue Jun 12 20:45:03 2007
Subject: > 2^32 b, 6.6 for Itanium
In-Reply-To: <20070608181407.GA19616@scico.botik.ru>
References: <20070608181407.GA19616@scico.botik.ru>
Message-ID: <20070613005001.GA20571@matrix.chaos.earth.li>


Hi Serge,

On Fri, Jun 08, 2007 at 10:14:07PM +0400, Serge D. Mechveliani wrote:
> 
> 1. How  ghc-6.6  runs on 64-bit machines having more that  2^32  
>    byte memory ?
>    Can it run into "Segmentation fault" ? 

It's a bug if it does.

> 2. Can you prepare a reliable  ghc-6.6 binary  for  Itanium a64 Linux
>    ?  
> Or maybe, someone can contribute ?
> 
> One of my colleagues says the following.
> 
> -------------------------------------
> He has taken such a binary from 
> > (http://www.haskell.org/ghc/distribution_packages.html#debian),
> >
> somehow installed it and tried to make from source. It reports
> 
> >/tmp/ghc12430_0/ghc12430_0.s: Assembler messages:
> >
> >/tmp/ghc12430_0/ghc12430_0.s:51:0:
> >     Error: Operand 1 of `mov' should be a general register
> -------------------------------------

If you just try to do a source build without any mk/build.mk then it
will try to do a registerised build, which is unlikely to work.

If you put

    GhcUnregisterised=YES
    GhcWithNativeCodeGen=NO
    GhcWithInterpreter=NO
    SplitObjs=NO
    GhcNotThreaded=YES

in mk/build.mk before building then it should work.


Thanks
Ian

From dm.maillists at gmail.com  Tue Jun 12 21:55:47 2007
From: dm.maillists at gmail.com (Daniel McAllansmith)
Date: Tue Jun 12 21:50:56 2007
Subject: GHC 6.6 panics when compiling HSet from collections package
Message-ID: <200706131355.48615.dm.maillists@gmail.com>

Hi.

I just got the collections package from 
http://darcs.haskell.org/packages/collections-ghc6.6

When trying to build it with GHC 6.6 on an amd64 linux machine using Cabal I 
got the following:

[12 of 57] Compiling Data.Tree.AVL.IntMap.Internals.HSet ( 
Data.Tree.AVL.IntMap/Data/Tree/AVL/IntMap/Internals/HSet.hs, 
dist/build/Data/Tree/AVL/IntMap/Internals/HSet.o )
ghc-6.6: panic! (the 'impossible' happened)
  (GHC version 6.6 for x86_64-unknown-linux):
        cgPanic
    tpl{v s2zg} [lid]
    static binds for:
    collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.intersectionMaybeH{v 
rji} [gid]
    collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.$Lr2jfforkL{v r2jf} 
[gid]
    collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.$Lr2jhforkR{v r2jh} 
[gid]
    local binds for:
    SRT label 
collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.intersectionWithH'{v 
rjg}_srt



Anyone seen this before?  A real GHC bug, a problem with my GHC installation, 
or a problem with the collections package?


Cheers
Daniel
From ahey at iee.org  Wed Jun 13 04:05:52 2007
From: ahey at iee.org (Adrian Hey)
Date: Wed Jun 13 04:15:01 2007
Subject: GHC 6.6 panics when compiling HSet from collections package
In-Reply-To: <200706131355.48615.dm.maillists@gmail.com>
References: <200706131355.48615.dm.maillists@gmail.com>
Message-ID: <466FA560.1050103@iee.org>

Daniel McAllansmith wrote:
> Hi.
> 
> I just got the collections package from 
> http://darcs.haskell.org/packages/collections-ghc6.6
> 
> When trying to build it with GHC 6.6 on an amd64 linux machine using Cabal I 
> got the following:
> 
> [12 of 57] Compiling Data.Tree.AVL.IntMap.Internals.HSet ( 
> Data.Tree.AVL.IntMap/Data/Tree/AVL/IntMap/Internals/HSet.hs, 
> dist/build/Data/Tree/AVL/IntMap/Internals/HSet.o )
> ghc-6.6: panic! (the 'impossible' happened)
>   (GHC version 6.6 for x86_64-unknown-linux):
>         cgPanic
>     tpl{v s2zg} [lid]
>     static binds for:
>     collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.intersectionMaybeH{v 
> rji} [gid]
>     collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.$Lr2jfforkL{v r2jf} 
> [gid]
>     collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.$Lr2jhforkR{v r2jh} 
> [gid]
>     local binds for:
>     SRT label 
> collections-0.3:Data.Tree.AVL.IntMap.Internals.HSet.intersectionWithH'{v 
> rjg}_srt
> 
> 
> 
> Anyone seen this before?  A real GHC bug, a problem with my GHC installation, 
> or a problem with the collections package?

It's a known bug in in ghc 6.6, you need to upgrade to ghc 6.6.1.

BTW, beware of using some of the stuff that I've written for this :-)

The Data.Tree.AVL part (including Data.Map.AVL and Data.Set.AVL)
should be fairly safe as it's been heavily tested.

But the Data.Trie.General part is still under active development,
volatile, unfinished and completely untested.

Also, don't use the Data.Tree.AVL.IntMap stuff either if you can
avoid it. I believe it works fine, but I've decided it would be
best to obsolete this and subsume it within Data.Trie.General
as Data.Trie.General.IntGT

Regards
--
Adrian Hey



From srdjan.stipic at gmail.com  Wed Jun 13 05:47:23 2007
From: srdjan.stipic at gmail.com (=?UTF-8?Q?Sr=C4=91an_Stipi=C4=87?=)
Date: Wed Jun 13 05:42:24 2007
Subject: Running multithreaded GHC on Itanium
Message-ID: <721db22f0706130247t7a7fac02h214b1335694d6200@mail.gmail.com>

I am currently trying to modify GHC 6.6.1 to run on Itanium in
multithreaded mode (+RTS -N).
Is it possible to do those modifications without generating back-end
for Itanium?
(If I am correct GHC on Itanium uses gcc to generate the code)

Thanks
Srdjan
From clemens at endorphin.org  Wed Jun 13 09:00:04 2007
From: clemens at endorphin.org (Clemens Fruhwirth)
Date: Wed Jun 13 09:04:36 2007
Subject: Locating shared libraries
In-Reply-To: <466EBB88.7030706@serpentine.com>
References: <87y7ip71c8.wl%clemens@endorphin.org>
	<466EBB88.7030706@serpentine.com>
Message-ID: <87wsy86mcr.wl%clemens@endorphin.org>

At Tue, 12 Jun 2007 08:28:08 -0700,
Bryan O'Sullivan <bos@serpentine.com> wrote:
> 
> Clemens Fruhwirth wrote:
> 
> > To solve this little shortcoming, the ld-invocation could be delegated
> > to GHC. "ghc -o libHSfoo.so Foo1.o Foo2.o".
> 
> You'd presumably want this to be "ghc -shared", yes?
> 
> It's definitely preferable to cook this knowledge into ghc, rather than 
> to have the smarts in Cabal.  The advantage of keeping the knowledge in 
> GHC is that it's more decoupled, and it mirrors the behaviour people 
> expect from other compilers (this is what gcc does, for example).

Full ack.

> > 3) Running programs
> 
> > Let's see how libtool handles this situation.
> 
> I would recommend against following libtool's lead in this area. 
> Libtool's fondness for cooking RPATH into binaries makes it very 
> difficult to deal with, because it's quite common for those binaries to 
> get installed and distributed, RPATH and all.  RPATH should only be used 
> by a user who knows they have a large-calibre weapon pointed at their foot.

Did I understand that correctly that you don't want to see binaries
with rpath's pointing to install directories such as /usr/lib/gcc-6.6?
So, this forces us to use a wrapper in all cases.

> > I agree that the last scheme sounds a bit wild, but I argue that
> > that's what ELF designers had in mind when they specified the INTERP
> > header.
> 
> Let's not go there, please :-)  Such a move would be a big maintenance 
> problem in its own right, and would make a lot of extra work for people 
> packaging GHC for different distributions as they would need to cook up 
> hacks for e.g. local SELinux policies regarding special ELF attributes 
> and memory protections.

Running i386 binaries on x86-64 platform basically does the same
thing: switch the ELF program interpreter (ld-linux-x86_64.so.2 to
ld-linux.so.2), so if these projects can't handle the full glory of
the ELF specification then it's probably not our problem. Yes, I agree
that this might not be the most trouble-free solution, but certainly
it's the most flexible one.

But, let's consider another approach before going in that direction:
Push the responsiblity for maintaining dynamic library information to
"ghc-pkg register". The custom ELF interpreter proposed above would
basically cook information from package.conf into stuff like "ld.so
--library-path <..>".  We can play the game differently and prepare the
information of package.conf when running ghc-pkg register, instead of
delaying this to program startup.

On way of digesting this information would be to collect all dynamic
libraries in a single directory; either something haskell specific
/usr/lib/ghc-6.6/dynlibs or even /usr/lib. If we start to create links
from /usr/lib/libHSbase.so -> /usr/lib/ghc-6.6/libHSbase.so, we might
even drop the wrapper.

Other variants is to have 
   ghc-pkg register <info-for-network-package>
generate little stubs like
   export LD_LIBRARY_PATH=/usr/lib/network-2.0/ghc-6.6/:$LD_LIBRARY_PATH
in /usr/lib/ghc-6.6/package-scripts/. Every deployed wrapper could
then have the form

#!/bin/sh
source /usr/lib/ghc-6.6/package-scripts/*
$0.real-binary "$*"
-- 
Fruhwirth Clemens - http://clemens.endorphin.org 


From simonmarhaskell at gmail.com  Wed Jun 13 10:03:46 2007
From: simonmarhaskell at gmail.com (Simon Marlow)
Date: Wed Jun 13 09:58:51 2007
Subject: Locating shared libraries
In-Reply-To: <87y7ip71c8.wl%clemens@endorphin.org>
References: <87y7ip71c8.wl%clemens@endorphin.org>
Message-ID: <466FF942.1090406@gmail.com>

Thanks for the analysis, Clemens.  Here's what I think we should do, firstly for 
Unix:

   - GHC distributions will install the shared libraries in standard
     locations. (OS packagers have the option of using non-std locations
     together with whatever mechnism is appropriate to get the non-std locations
     to be registered in ld.so.conf).

   - an installed GHC will just link binaries as normal; the shared libs are
     in the standard locations, so the binary will work.

   - we add an option to GHC, say -hardwire-lib-paths, that tells it to use
     -rpath (or equivalent) when linking,

   - an uninstalled GHC uses -hardwire-lib-paths by default.  If you want to
     generate a deployable executable using an uninstalled GHC, you must
     turn off this behaviour using -no-hardwire-lib-paths.  We have some
     flexibility here: we could reverse this default, but I think I'd often find
     myself generating binaries that crash because I forgot -hardwire-lib-paths.

   - a GHC installed in a non-standard location (e.g. your home directory) will
     also use -hardwire-lib-paths by default.  I think we want this, otherwise
     we'll get several bug reports per day about missing shared libraries.

   - GHC may warn if you link a binary to any shared libraries that are not in
     standard locations and you didn't use -hardwire-lib-paths.  I'm not
     sure if this is possible in general, but it would be nice.

   - binaries that come with a GHC distribution will all be non-hardwired
     executables with wrapper scripts that set LD_LIBRARY_PATH.  This is so
     that we can choose where to install the bindist at install-time.
     Many of these executables already have wrapper scripts anyway, this isn't a
     big deal.

For Windows:

   - In a GHC distribution, ghc.exe is in the same directory as the library DLLs,
     so by default it will link to them (the binary's directory is searched for
     DLLs first on Windows).  We still have a relocatable GHC installation tree.

   - GHC uses -hardwire-lib-paths by default, implemented by embedding manifests
     into binaries it creates.

   - we provide a way to generate a deployable binary by collecting all the DLLs
     it refers to in a bundle.

Cheers,
	Simon

Clemens Fruhwirth wrote:

> I'm hacking on shared library support for GHC and it's coming along quite nicely.
> http://hpaste.org/192
> 
> My initial hacks are available from:
> 
> http://clemens.endorphin.org/patches/ghc-20070605-initial-shared-libs.patch
> (works only with x86-64 atm, on i386 the NCG dies in the register
> allocator when compiling cmm files RTS)
> 
> http://clemens.endorphin.org/patches/cabal-20070528-initial-shared-library.patch
> 
> libtool usually takes care of creating shared libraries under *nix
> system. libtool solves a few minor problems associated with:
> 
> 1) creating shared libraries
> 2) linking programs that depend on shared libraries
> 3) running programs that depend on shared libraries
> 
> libtool is tailored to C compilers and the general opinion from #ghc
> towards libtool seems to be: "hands off". From the list above, I will
> try to sketch solutions without libtool.
> 
> 1) creating shared libraries:
> 
> At the moment, my second patch teaches Cabal how to build shared
> libraries. Basically, this is:
>   * add -fPIC to the compiler invocation (and -optc-fPIC for c-sources),
>   * invoke "ld -shared -Bsymbolic -o foo.so obj1.o obj2.o ...". 
> 
> ATM, ld is not invoked with the inter-library dependencies for the
> shared library being built. This is not problematic as the final
> executable will include all dependencies due to the ghc package
> dependency tracking. But DT_NEEDED on ELF influences the sequence in
> which shared library initializers are run. I have not yet investigated
> if this leads to any problems.
> 
> To solve this little shortcoming, the ld-invocation could be delegated
> to GHC. "ghc -o libHSfoo.so Foo1.o Foo2.o". We already have a similar
> facility for DLLs (see MkDLL in DriverPipeline.hs). This could be
> abstracted into MkShared, and platform specific knowledge could be
> encapsulated in GHC. The benefit would be that we could easily access
> the package information and we could create shared libraries that
> contain proper DT_NEEDED sections.
> 
> 2) Linking programs
> 
> Linking should work out of the box:
> 
> "ghc -dynamic -o HelloWorld HelloWorld.o" creates dynamically linked
> executable.
> 
> 3) Running programs
> 
> This a typical problem:
> ./HelloWorld 
> ./HelloWorld: error while loading shared libraries: libHShaskell98-1.0_dyn.so: cannot open shared object file: No such file or directory
> 
> There are several ways to add search paths for dynamic linking: either
> we do it temporarily or we encode the search paths into the
> executables. On ELF platforms, this works by adding -rpath to the
> linker flags. This adds two new entries in the .dynamic section
> (DT_RPATH, DT_RUNPATH) both responsible for signalling additional
> search paths to the dynamic linker, ld.so. According to Simon Marlow,
> Windows has similar mechanism via manifest files.
> 
> Let's see how libtool handles this situation. libtool differentiates
> between installed and uninstalled libraries. When linking against
> installed libraries not in the standard search path, libtool uses
> -rpath to add these search paths to the created executable. When
> linking against uninstalled libraries, libtool still uses -rpath but
> pointing to the directory the uninstalled library is going to be
> installed in. libtool derives this information from the .la
> files+Makefiles.
> 
> In any case, libtool creates a wrapper in the build directory that
> takes care of executing the program linked against uninstalled shared
> libraries. There are two strategies for accomplishing this:
>  * add the paths of the uninstalled shared libraries to LD_LIBRARY_PATH
>  * relink the executable with additional -rpath's
> libtool chooses the second strategy.
> 
> How do we translate these solutions to GHC? The first question is
> whether we expect 
> 
> ghc -dynamic -package uninstalled-package -o Hello Hello.o
> ./Hello
> 
> to work or whether we require manual intervention in these cases. If
> we expect this to work without intervention, we have the same options
> as libtool:
> 
> * create a wrapper that takes care of locating the uninstalled shared
>   libraries and sets LD_LIBRARY_PATH. 
> 
> * create a binary with rpath of the uninstalled libraries, and create
>   an additional executable for deployment without these rpaths.
> 
> In any case we have to modify the installer scripts to either know
> about where to locate the real binary, either ask ghc where to find
> the real binary, or either delegate the installation to ghc. The last
> option is basically the libtool way, "libtool --mode=install ..."
> 
> When we decide to create a deployable executable at the "-o" spot, we
> need to 
> 
> * modify the invocation to manually pick up the libraries by modifying
>   LD_LIBRARY_PATH.. this is pretty unpractical.
> 
> * delegate invocation to ghc. Maybe "ghc --execute HelloWorld". 
>   libtool has a similar mechanism for executing 3rd party programs in
>   the "dynamic environment" of the compiler executable. For instance,
>   "gdb HelloWorld" would fails for libtool as HelloWorld is a wrapper,
>   but "libtool --mode=execute gdb HelloWorld" works, as libtool
>   rewrites to HelloWorld to .libs/lt-HelloWorld.
> 
> And now something completely different: Create a custom ELF program
> interpreter for Haskell programs. Using INTERP in the ELF program
> header, loads up this interpreter and delegates control to it. Usually
> this is /lib/ld-linux.so.2, the dynamic linker, but we can replace that.
> 
> Haskell has its own idea of libraries/packages. We have package.conf
> which gives us the location of the installed libraries. This is ok for
> static linking, as at link time ghc is running and knows how to invoke
> gcc with the correct paths. It does not matter, if package.conf is
> updated afterwards as the statically linked programs contain a copy of
> the library anyway. For dynamic linking this phase is delayed and when
> we encode rpath such as "/usr/lib/network-2.0/ghc-6.6/", we can not
> update to network-2.1 without breaking this executables.
> 
> A custom programming loading stub could access the global and local
> package.conf and extract the library path for the dependencies and execve 
> 
> /lib64/ld-linux.so.2 --library-path=<paths of the dependencies> HelloWorld <args>
> 
> This certainly gives us more flexibility than encoding all these
> rpaths statically into HelloWorld. To solve the inplace execution
> directly from the build directory, we might create
> .HelloWorld.package.conf in case a non-standard package.conf is used
> (non-standard=different from global and local) and have the stub
> loader to check for this file.
> 
> I agree that the last scheme sounds a bit wild, but I argue that
> that's what ELF designers had in mind when they specified the INTERP
> header. Of course, this is only a solution for ELF platforms.
> 
> Opinions :) ?
> --
> Fruhwirth Clemens - http://clemens.endorphin.org 
> 
> 
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 

From stefanor at cox.net  Wed Jun 13 10:06:48 2007
From: stefanor at cox.net (Stefan O'Rear)
Date: Wed Jun 13 10:01:49 2007
Subject: Locating shared libraries
In-Reply-To: <87wsy86mcr.wl%clemens@endorphin.org>
References: <87y7ip71c8.wl%clemens@endorphin.org>
	<466EBB88.7030706@serpentine.com>
	<87wsy86mcr.wl%clemens@endorphin.org>
Message-ID: <20070613140648.GC2906@localhost.localdomain>

On Wed, Jun 13, 2007 at 03:00:04PM +0200, Clemens Fruhwirth wrote:
> > > 3) Running programs
> > 
> > > Let's see how libtool handles this situation.
> > 
> > I would recommend against following libtool's lead in this area. 
> > Libtool's fondness for cooking RPATH into binaries makes it very 
> > difficult to deal with, because it's quite common for those binaries to 
> > get installed and distributed, RPATH and all.  RPATH should only be used 
> > by a user who knows they have a large-calibre weapon pointed at their foot.
> 
> Did I understand that correctly that you don't want to see binaries
> with rpath's pointing to install directories such as /usr/lib/gcc-6.6?
> So, this forces us to use a wrapper in all cases.

Please think seriously about mangling the names of Haskell libraries to
include version information and dropping them in $PREFIX/lib with every
other language's libraries.  Haskell is not special, and users expect
libraries to be in /usr/lib.  No wrapper needed, no RPATH needed, as far
as I can see no fanciness at all.

Stefan
From rich.neswold at gmail.com  Wed Jun 13 11:05:11 2007
From: rich.neswold at gmail.com (Rich Neswold)
Date: Wed Jun 13 11:00:15 2007
Subject: Locating shared libraries
In-Reply-To: <20070613140648.GC2906@localhost.localdomain>
References: <87y7ip71c8.wl%clemens@endorphin.org>
	<466EBB88.7030706@serpentine.com>
	<87wsy86mcr.wl%clemens@endorphin.org>
	<20070613140648.GC2906@localhost.localdomain>
Message-ID: <14cf844b0706130805s6caa4c09sd8228a4fae08b96a@mail.gmail.com>

On 6/13/07, Stefan O'Rear <stefanor@cox.net> wrote:
>
> > Did I understand that correctly that y