From simonpj at microsoft.com Tue Sep 1 17:56:07 2009 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Tue Sep 1 17:35:12 2009 Subject: bug in language definition (strictness) In-Reply-To: <20090830003358.GM8113@sliver.repetae.net> References: <4A7ACE92.4020503@cs.nott.ac.uk> <3F6B656D-6F4A-4F4D-B6DB-57C9E2843302@kent.ac.uk> <4A7AD3C9.4090302@gmail.com> <8116107F-2190-4D9B-AB3C-C3BBC1BAE29F@gmail.com> <4A7AE20F.308@gmail.com> <48A5A7F0-0E65-4642-8249-DB78828C373D@gmail.com> <4A7BE00B.8070008@gmail.com> <94B35AB2-E875-4C4F-A644-503B1960EA16@cs.york.ac.uk> <7b977d860908070417q740faab6lb46f7372914fbd1c@mail.gmail.com> <4A8946D5.5020807@gmail.com> <20090830003358.GM8113@sliver.repetae.net> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C35FFE593B6C@EA-EXMSG-C334.europe.corp.microsoft.com> | This suggests a natural implementation (and specification) for pseq, | | pseq :: a -> b -> b | pseq x y = x `seq` lazy y | | where lazy :: a -> a is a compiler provided function equivalent to 'id' | except that it is considered lazy in its argument by the strictness | analyzer. Exactly so! Here is the definition of `pseq` in GHC.Conc: Conc.lhs:368:pseq :: a -> b -> b Conc.lhs:369:pseq x y = x `seq` lazy y And 'lazy' is documented in the user manual: http://www.haskell.org/ghc/docs/latest/html/libraries/ghc-prim/GHC-Prim.html#v%3Alazy Simon From uzytkownik2 at gmail.com Sat Sep 5 16:47:52 2009 From: uzytkownik2 at gmail.com (Maciej Piechotka) Date: Sat Sep 5 16:27:07 2009 Subject: Standarize GHC.Prim.Any In-Reply-To: <1250791182.2598.11.camel@notebook> References: <1250491274.27620.71.camel@notebook> <4A89ED3A.1010507@isaac.cedarswampstudios.org> <1250791182.2598.11.camel@notebook> Message-ID: <1252183672.10681.3.camel@kirk> On Thu, 2009-08-20 at 19:59 +0200, Maciej Piechotka wrote: > Sorry for delay in responding. > > > > Any from GHC.Prim makes unsafeCoerce much useful and safe. Can it be > > > included in Unsafe.Coerce module? > > > > > > Pros: > > > - unsafeCoerce is much more useful with Any (it's a safe placeholder for > > > any value and therefore can be passed simply in/out FFI). > > > > Yes it's a safe placeholder, but so is an existential, I believe... > > data ContainsAny = forall a. ContainsAny a > > to put in the container: > > "ContainsAny x" > > to remove from the container: > > "case ... of ContainsAny x -> unsafeCoerce x" > > Albeit, there might be a slight performance penalty for the extra box, > > which can't be removed in all haskell implementations. > > > > Also, what do you mean about "FFI"? I don't think you can pass the > > "Any" type to foreign functions through the FFI... > > > > Also, can/do all compilers that implement unsafeCoerce implement a safe Any? > > Hugs can do it with just "data Any = Ignored" I believe, not sure about > > nhc, yhc or jhc... > > > > -Isaac > > Well. May be I have one specific problem which seems to not be possible > to be implemented in portable way (I'm not sure if it is possible > outside ghc). Sorry for mentioning FFI without further explanation. > > The problem arise if one would like to implement GClosure from glib. > Once it was implemented in gtk2hs in C using RTS API. I also have my own > implementation in Haskell which is doing something (removed IO monad for > clarity): > applyValue :: Any -> Value -> Any > applyValue f v = > case funsamentalTypeOf v of > Type1 -> (unsafeCoerce f :: HaskellType1 -> Any) $ get v > ... > > Such trick is (looks for me?) safe as a -> b -> ... -> d can be > represented by Any. However I cannot safely cast to function a -> Any. > > To/from FFI it is passed in Ptr (StablePtr Any). > > Regards > PS. I assume that it is not possible as it was done in importable was in > gtk2hs. With any known from the beginning number of parameters function and GADT one can write: data Closure where Closure0 :: IO a Closure1 :: a -> IO b Closure2 :: a -> b -> IO c -- ... However it poses upper build-time limit. Regards -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://www.haskell.org/pipermail/haskell-prime/attachments/20090905/2fd0fd1d/attachment.bin From uzytkownik2 at gmail.com Sat Sep 5 17:06:47 2009 From: uzytkownik2 at gmail.com (Maciej Piechotka) Date: Sat Sep 5 16:52:45 2009 Subject: Allowing safe cast by default Message-ID: <1252184807.10681.22.camel@kirk> As far as I underspend all checks of Data.Typeable.cast can be done by compiler in compile-time. While it impose partially what one may consider syntax sugar I believe that there is no restriction that would not allowed compiler to behave like that: cast :: forall a b. a -> Maybe b -- Pseudo-haskell cast x = staticIf a == b then Just x else Nothing [1] Also I may not grasp every element of Haskell but what is the reason of not implementing Typeable by default? Regards [1] You may say that I need unsafeCoerce but 1. I'd prefere to write let x = fromMaybe (error "Wrong types") $ cast x' :: a instead of let x = unsafeCoerce x' :: a in case when I (not compiler) know that x and x' have the same type (a) as it makes the crash big and in Haskell instead of possible some problems with GC or something which I'll find in totally unrelated piece of program. 2. I may want different actions -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://www.haskell.org/pipermail/haskell-prime/attachments/20090905/24a4d115/attachment-0001.bin From ml at isaac.cedarswampstudios.org Sat Sep 5 23:35:29 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Sat Sep 5 23:14:43 2009 Subject: Standarize GHC.Prim.Any In-Reply-To: <1252183672.10681.3.camel@kirk> References: <1250491274.27620.71.camel@notebook> <4A89ED3A.1010507@isaac.cedarswampstudios.org> <1250791182.2598.11.camel@notebook> <1252183672.10681.3.camel@kirk> Message-ID: <4AA32E01.3020404@isaac.cedarswampstudios.org> Maciej Piechotka wrote: > On Thu, 2009-08-20 at 19:59 +0200, Maciej Piechotka wrote: >> Sorry for delay in responding. >> >>>> Any from GHC.Prim makes unsafeCoerce much useful and safe. Can it be >>>> included in Unsafe.Coerce module? >>>> >>>> Pros: >>>> - unsafeCoerce is much more useful with Any (it's a safe placeholder for >>>> any value and therefore can be passed simply in/out FFI). >>> Yes it's a safe placeholder, but so is an existential, I believe... >>> data ContainsAny = forall a. ContainsAny a >>> to put in the container: >>> "ContainsAny x" >>> to remove from the container: >>> "case ... of ContainsAny x -> unsafeCoerce x" >>> Albeit, there might be a slight performance penalty for the extra box, >>> which can't be removed in all haskell implementations. >>> >>> Also, what do you mean about "FFI"? I don't think you can pass the >>> "Any" type to foreign functions through the FFI... >>> >>> Also, can/do all compilers that implement unsafeCoerce implement a safe Any? >>> Hugs can do it with just "data Any = Ignored" I believe, not sure about >>> nhc, yhc or jhc... >>> >>> -Isaac >> Well. May be I have one specific problem which seems to not be possible >> to be implemented in portable way (I'm not sure if it is possible >> outside ghc). Sorry for mentioning FFI without further explanation. >> >> The problem arise if one would like to implement GClosure from glib. >> Once it was implemented in gtk2hs in C using RTS API. I also have my own >> implementation in Haskell which is doing something (removed IO monad for >> clarity): >> applyValue :: Any -> Value -> Any >> applyValue f v = >> case funsamentalTypeOf v of >> Type1 -> (unsafeCoerce f :: HaskellType1 -> Any) $ get v >> ... >> >> Such trick is (looks for me?) safe as a -> b -> ... -> d can be >> represented by Any. However I cannot safely cast to function a -> Any. >> >> To/from FFI it is passed in Ptr (StablePtr Any). >> >> Regards >> PS. I assume that it is not possible as it was done in importable was in >> gtk2hs. > > With any known from the beginning number of parameters function and GADT > one can write: > > data Closure where > Closure0 :: IO a > Closure1 :: a -> IO b > Closure2 :: a -> b -> IO c > -- ... That isn't GADT syntax... results of "Constructor :: ..." must be "Closure" in this case. What are you trying to do here? An alternative for "Ptr (StablePtr Any)"? Is "Ptr (StablePtr Dynamic)" sufficient, where Dynamic from Data.Dynamic is represented in an implementation-specific way equivalent to data Dynamic = forall a. Typeable a => DynamicConstr a a.k.a. data Dynamic where DynamicConstr :: Typeable a => a -> Dynamic ? The main limitation would be if you want to include a type that's not in Typeable. Is that common, to have types that aren't in Typeable? It would be more annoying but you could add extra concrete-type constructors to a data, for every possibility, like DynamicConstr3 :: (Int, Bool) -> MyDynamic if for some reason (Int, Bool) weren't in Typeable... Would be silly to do, however. Just put things in Typeable? Or, if you know it'll be type-correct, just use ContainsAny as above, with "Ptr (StablePtr ContainsAny)"? It might not be absolutely the most optimized, but portable standard mechanisms were never the place for that kind of hacking. And it'll be good enough. Right? -Isaac From uzytkownik2 at gmail.com Sun Sep 6 07:05:20 2009 From: uzytkownik2 at gmail.com (Maciej Piechotka) Date: Sun Sep 6 06:44:45 2009 Subject: Standarize GHC.Prim.Any In-Reply-To: <4AA32E01.3020404@isaac.cedarswampstudios.org> References: <1250491274.27620.71.camel@notebook> <4A89ED3A.1010507@isaac.cedarswampstudios.org> <1250791182.2598.11.camel@notebook> <1252183672.10681.3.camel@kirk> <4AA32E01.3020404@isaac.cedarswampstudios.org> Message-ID: <1252235120.3647.5.camel@kirk> On Sat, 2009-09-05 at 23:35 -0400, Isaac Dupree wrote: > Maciej Piechotka wrote: > > On Thu, 2009-08-20 at 19:59 +0200, Maciej Piechotka wrote: > >> Sorry for delay in responding. > >> > >>>> Any from GHC.Prim makes unsafeCoerce much useful and safe. Can it be > >>>> included in Unsafe.Coerce module? > >>>> > >>>> Pros: > >>>> - unsafeCoerce is much more useful with Any (it's a safe placeholder for > >>>> any value and therefore can be passed simply in/out FFI). > >>> Yes it's a safe placeholder, but so is an existential, I believe... > >>> data ContainsAny = forall a. ContainsAny a > >>> to put in the container: > >>> "ContainsAny x" > >>> to remove from the container: > >>> "case ... of ContainsAny x -> unsafeCoerce x" > >>> Albeit, there might be a slight performance penalty for the extra box, > >>> which can't be removed in all haskell implementations. > >>> > >>> Also, what do you mean about "FFI"? I don't think you can pass the > >>> "Any" type to foreign functions through the FFI... > >>> > >>> Also, can/do all compilers that implement unsafeCoerce implement a safe Any? > >>> Hugs can do it with just "data Any = Ignored" I believe, not sure about > >>> nhc, yhc or jhc... > >>> > >>> -Isaac > >> Well. May be I have one specific problem which seems to not be possible > >> to be implemented in portable way (I'm not sure if it is possible > >> outside ghc). Sorry for mentioning FFI without further explanation. > >> > >> The problem arise if one would like to implement GClosure from glib. > >> Once it was implemented in gtk2hs in C using RTS API. I also have my own > >> implementation in Haskell which is doing something (removed IO monad for > >> clarity): > >> applyValue :: Any -> Value -> Any > >> applyValue f v = > >> case funsamentalTypeOf v of > >> Type1 -> (unsafeCoerce f :: HaskellType1 -> Any) $ get v > >> ... > >> > >> Such trick is (looks for me?) safe as a -> b -> ... -> d can be > >> represented by Any. However I cannot safely cast to function a -> Any. > >> > >> To/from FFI it is passed in Ptr (StablePtr Any). > >> > >> Regards > >> PS. I assume that it is not possible as it was done in importable was in > >> gtk2hs. > > > > With any known from the beginning number of parameters function and GADT > > one can write: > > > > data Closure where > > Closure0 :: IO a > > Closure1 :: a -> IO b > > Closure2 :: a -> b -> IO c > > -- ... > > That isn't GADT syntax... results of "Constructor :: ..." must be > "Closure" in this case. Sorry. To much delete: data Closure where Closure0 :: IO a -> Closure Closure1 :: (a -> IO b) -> Closure Closure2 :: (a -> b -> IO c) -> Closure ... > What are you trying to do here? An alternative > for "Ptr (StablePtr Any)"? Is "Ptr (StablePtr Dynamic)" sufficient, > where Dynamic from Data.Dynamic is represented in an > implementation-specific way equivalent to > data Dynamic = forall a. Typeable a => DynamicConstr a > a.k.a. > data Dynamic where DynamicConstr :: Typeable a => a -> Dynamic > ? > The main limitation would be if you want to include a type that's not in > Typeable. Is that common, to have types that aren't in Typeable? > Point taken. Regards -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://www.haskell.org/pipermail/haskell-prime/attachments/20090906/c02e3f53/attachment.bin From fw at deneb.enyo.de Wed Sep 16 16:17:08 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed Sep 16 15:55:48 2009 Subject: Unsafe hGetContents Message-ID: <877hvylk57.fsf@mid.deneb.enyo.de> Are there any plans to get rid of hGetContents and the semi-closed handle state for Haskell Prime? (I call hGetContents unsafe because it adds side effects to pattern matching, stricly speaking invalidating most of the transformations which are expected to be valid in a pure language.) From dons at galois.com Wed Sep 16 16:20:49 2009 From: dons at galois.com (Don Stewart) Date: Wed Sep 16 16:01:45 2009 Subject: Unsafe hGetContents In-Reply-To: <877hvylk57.fsf@mid.deneb.enyo.de> References: <877hvylk57.fsf@mid.deneb.enyo.de> Message-ID: <20090916202049.GE19891@whirlpool.galois.com> fw: > Are there any plans to get rid of hGetContents and the semi-closed > handle state for Haskell Prime? > > (I call hGetContents unsafe because it adds side effects to pattern > matching, stricly speaking invalidating most of the transformations > which are expected to be valid in a pure language.) Isn't this a broader complaint about lazy IO in general? -- Don From fw at deneb.enyo.de Wed Sep 16 16:31:29 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed Sep 16 16:10:07 2009 Subject: Unsafe hGetContents In-Reply-To: <20090916202049.GE19891@whirlpool.galois.com> (Don Stewart's message of "Wed, 16 Sep 2009 13:20:49 -0700") References: <877hvylk57.fsf@mid.deneb.enyo.de> <20090916202049.GE19891@whirlpool.galois.com> Message-ID: <87k4zyk4wu.fsf@mid.deneb.enyo.de> * Don Stewart: > fw: >> Are there any plans to get rid of hGetContents and the semi-closed >> handle state for Haskell Prime? >> >> (I call hGetContents unsafe because it adds side effects to pattern >> matching, stricly speaking invalidating most of the transformations >> which are expected to be valid in a pure language.) > > Isn't this a broader complaint about lazy IO in general? Yes, sort of. But doesn't lazy input derive its justification from being present in the prelude? From nicolas.pouillard at gmail.com Thu Sep 17 08:58:59 2009 From: nicolas.pouillard at gmail.com (Nicolas Pouillard) Date: Thu Sep 17 08:41:54 2009 Subject: Unsafe hGetContents In-Reply-To: <877hvylk57.fsf@mid.deneb.enyo.de> References: <877hvylk57.fsf@mid.deneb.enyo.de> Message-ID: <1253192223-sup-3700@peray> Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009: > Are there any plans to get rid of hGetContents and the semi-closed > handle state for Haskell Prime? > > (I call hGetContents unsafe because it adds side effects to pattern > matching, stricly speaking invalidating most of the transformations > which are expected to be valid in a pure language.) Would you consider something like [1] as an acceptable replacement? [1]: http://hackage.haskell.org/package/safe-lazy-io -- Nicolas Pouillard http://nicolaspouillard.fr From marlowsd at gmail.com Mon Sep 21 05:41:38 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Sep 21 05:20:09 2009 Subject: Unsafe hGetContents In-Reply-To: <877hvylk57.fsf@mid.deneb.enyo.de> References: <877hvylk57.fsf@mid.deneb.enyo.de> Message-ID: <4AB74A52.8090405@gmail.com> On 16/09/2009 21:17, Florian Weimer wrote: > Are there any plans to get rid of hGetContents and the semi-closed > handle state for Haskell Prime? > > (I call hGetContents unsafe because it adds side effects to pattern > matching, stricly speaking invalidating most of the transformations > which are expected to be valid in a pure language.) There is no current proposal for this, no. Feel free to start one; information about the process for Haskell Prime proposals is here http://hackage.haskell.org/trac/haskell-prime/wiki/Process Cheers, Simon From marlowsd at gmail.com Mon Sep 21 05:52:41 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Sep 21 05:31:10 2009 Subject: Unsafe hGetContents In-Reply-To: <1253192223-sup-3700@peray> References: <877hvylk57.fsf@mid.deneb.enyo.de> <1253192223-sup-3700@peray> Message-ID: <4AB74CE9.8090901@gmail.com> On 17/09/2009 13:58, Nicolas Pouillard wrote: > Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009: >> Are there any plans to get rid of hGetContents and the semi-closed >> handle state for Haskell Prime? >> >> (I call hGetContents unsafe because it adds side effects to pattern >> matching, stricly speaking invalidating most of the transformations >> which are expected to be valid in a pure language.) > > Would you consider something like [1] as an acceptable replacement? > > [1]: http://hackage.haskell.org/package/safe-lazy-io I rater like this as a workaround for the most common practical problems with lazy I/O, those of resource control. It doesn't address the deeper concern that lazy I/O requires a particular evaluation order and is therefore a bit warty as a language feature - implementing lazy I/O properly in GHC's parallel mutator was somewhat tricky. I'm not of the opinion that we should throw out lazy I/O, but it's still a problematic area in Haskell. Cheers, Simon From nicolas.pouillard at gmail.com Mon Sep 21 07:57:11 2009 From: nicolas.pouillard at gmail.com (Nicolas Pouillard) Date: Mon Sep 21 07:34:00 2009 Subject: Unsafe hGetContents In-Reply-To: <4AB74A52.8090405@gmail.com> References: <877hvylk57.fsf@mid.deneb.enyo.de> <4AB74A52.8090405@gmail.com> Message-ID: <1253534185-sup-9217@peray> Excerpts from Simon Marlow's message of Mon Sep 21 11:41:38 +0200 2009: > On 16/09/2009 21:17, Florian Weimer wrote: > > Are there any plans to get rid of hGetContents and the semi-closed > > handle state for Haskell Prime? > > > > (I call hGetContents unsafe because it adds side effects to pattern > > matching, stricly speaking invalidating most of the transformations > > which are expected to be valid in a pure language.) > > There is no current proposal for this, no. Feel free to start one; > information about the process for Haskell Prime proposals is here > > http://hackage.haskell.org/trac/haskell-prime/wiki/Process An alternate proposition (instead of removing it) would to to move it to System.IO.Unsafe. -- Nicolas Pouillard http://nicolaspouillard.fr From nicolas.pouillard at gmail.com Mon Sep 21 08:07:56 2009 From: nicolas.pouillard at gmail.com (Nicolas Pouillard) Date: Mon Sep 21 07:44:43 2009 Subject: Unsafe hGetContents In-Reply-To: <4AB74CE9.8090901@gmail.com> References: <877hvylk57.fsf@mid.deneb.enyo.de> <1253192223-sup-3700@peray> <4AB74CE9.8090901@gmail.com> Message-ID: <1253534246-sup-868@peray> Excerpts from Simon Marlow's message of Mon Sep 21 11:52:41 +0200 2009: > On 17/09/2009 13:58, Nicolas Pouillard wrote: > > Excerpts from Florian Weimer's message of Wed Sep 16 22:17:08 +0200 2009: > >> Are there any plans to get rid of hGetContents and the semi-closed > >> handle state for Haskell Prime? > >> > >> (I call hGetContents unsafe because it adds side effects to pattern > >> matching, stricly speaking invalidating most of the transformations > >> which are expected to be valid in a pure language.) > > > > Would you consider something like [1] as an acceptable replacement? > > > > [1]: http://hackage.haskell.org/package/safe-lazy-io > > I rater like this as a workaround for the most common practical problems > with lazy I/O, those of resource control. > It doesn't address the deeper > concern that lazy I/O requires a particular evaluation order and is > therefore a bit warty as a language feature When using safe-lazy-io we no longer rely (or a lot less) on the evaluation order (assuming you mean the order of side-effects). Since the way of combining the different inputs is statically chosen by user. > - implementing lazy I/O > properly in GHC's parallel mutator was somewhat tricky. I'm not of the > opinion that we should throw out lazy I/O, but it's still a problematic > area in Haskell. Maybe the 'unsafeGetContents' feature required by a safe-lazy-io would be less problematic, in particular it does not have to ignore exceptions. Best regards, -- Nicolas Pouillard http://nicolaspouillard.fr From thomas.dubuisson at gmail.com Thu Sep 24 17:02:04 2009 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Thu Sep 24 16:40:16 2009 Subject: Strongly Specify Alignment for FFI Allocation Message-ID: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> Aside from section 5.7 (storable) and comments on 'alignPtr', the only mention of alignment in the FFI addendum is on mallocBytes/allocaBytes: "The block of memory is sufficiently aligned for any of the basic foreign types (see Section 3.2) that fits into a memory block of the allocated size" It would be beneficial if this wording was applied to all allocation routines - such as mallocForeignPtrBytes, mallocForeignPtrArray, etc. For the curious, this proposal was born from the real-world issue of pulling Word32's from a ByteString in an efficient but portable manner (binary is portable but inefficient, a straight forward unsafePerformIO/peek is efficient but need alignment). If no glaring issue comes up then I'll formalize / make a ticket, Thomas From dons at galois.com Thu Sep 24 18:13:58 2009 From: dons at galois.com (Don Stewart) Date: Thu Sep 24 17:54:24 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> Message-ID: <20090924221358.GG26703@whirlpool.galois.com> thomas.dubuisson: > Aside from section 5.7 (storable) and comments on 'alignPtr', the only > mention of alignment in the FFI addendum is on > mallocBytes/allocaBytes: > > "The block of memory is sufficiently aligned for any of the basic > foreign types (see Section 3.2) that fits into a memory block of the > allocated size" > > It would be beneficial if this wording was applied to all allocation > routines - such as mallocForeignPtrBytes, mallocForeignPtrArray, etc. > For the curious, this proposal was born from the real-world issue of > pulling Word32's from a ByteString in an efficient but portable manner > (binary is portable but inefficient, a straight forward > unsafePerformIO/peek is efficient but need alignment). As a side issue, the get/put primitives on Data.Binary should be efficient (though they're about twice as fast when specialized to a strict bytestring... stay tuned for a package in this area). -- Don From chak at cse.unsw.edu.au Thu Sep 24 22:08:21 2009 From: chak at cse.unsw.edu.au (Manuel M T Chakravarty) Date: Thu Sep 24 21:46:44 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> Message-ID: Thomas DuBuisson: > Aside from section 5.7 (storable) and comments on 'alignPtr', the only > mention of alignment in the FFI addendum is on > mallocBytes/allocaBytes: > > "The block of memory is sufficiently aligned for any of the basic > foreign types (see Section 3.2) that fits into a memory block of the > allocated size" > > It would be beneficial if this wording was applied to all allocation > routines - such as mallocForeignPtrBytes, mallocForeignPtrArray, etc. > For the curious, this proposal was born from the real-world issue of > pulling Word32's from a ByteString in an efficient but portable manner > (binary is portable but inefficient, a straight forward > unsafePerformIO/peek is efficient but need alignment). I agree that we should be more precise here. > If no glaring issue comes up then I'll formalize / make a ticket, Can you please summarise the exact additions that you would like to see as a follow-up email? I will collect all changes that we want to make to the existing FFI Addendum before it goes into the 2009 issue of Haskell'. Cheers, Manuel From thomas.dubuisson at gmail.com Thu Sep 24 23:12:41 2009 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Thu Sep 24 22:50:53 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> Message-ID: <4c44d90b0909242012u610c98a7ya1fafb1f4725d474@mail.gmail.com> Manual, The verbiage here is borrowed from other parts of the report under the belief it helps readability. Option 1: A global note applying to all functions that allocate memory. Proposed wording for section 5.0: "All successful results of allocation routines will be sufficiently aligned for any of the basic foreign types (see Section 3.2) to fit into a memory block of the allocated size. Array allocation routines will result in the same alignment as the member elements." I believe this to be quite clear but if people here are unsatisfied then a list of example functions could be added at the risk of being tacky. Or we could avoid specifying functions by mentioning categories of functions at the risk of muddying the waters: "This alignment guarantee applies equally to computation-bound allocations, ForeignPtr references, and Ptr references." Option 2: Individual notes for all allocation routines Add to both mallocForeignPtr and mallocForeignPtrBytes: "The same alignment constraint as for mallocBytes holds." Add to both mallocForeginPtrArray and mallocForeignPtrArray0: "The resulting pointer will be suitably aligned to hold values of type b." Many other functions aren't so deficient as to require extra words - they contain references to functions that already have alignment comments, "These functions behave like x, y, z". Still, this hazy area is why I prefer option 1. If people actually prefer option 2 then I'll do a more through vetting of the function descriptions and likely come up with a larger list of alterations. Cheers, Thomas On Thu, Sep 24, 2009 at 7:08 PM, Manuel M T Chakravarty wrote: > Thomas DuBuisson: >> >> Aside from section 5.7 (storable) and comments on 'alignPtr', the only >> mention of alignment in the FFI addendum is on >> mallocBytes/allocaBytes: >> >> "The block of memory is sufficiently aligned for any of the basic >> foreign types (see Section 3.2) that fits into a memory block of the >> allocated size" >> >> It would be beneficial if this wording was applied to all allocation >> routines - such as mallocForeignPtrBytes, mallocForeignPtrArray, etc. >> For the curious, this proposal was born from the real-world issue of >> pulling Word32's from a ByteString in an efficient but portable manner >> (binary is portable but inefficient, a straight forward >> unsafePerformIO/peek is efficient but need alignment). > > I agree that we should be more precise here. > >> If no glaring issue comes up then I'll formalize / make a ticket, > > Can you please summarise the exact additions that you would like to see as a > follow-up email? ?I will collect all changes that we want to make to the > existing FFI Addendum before it goes into the 2009 issue of Haskell'. > > Cheers, > Manuel > > > > From duncan.coutts at worc.ox.ac.uk Fri Sep 25 07:54:35 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Sep 25 08:05:47 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <20090924221358.GG26703@whirlpool.galois.com> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> <20090924221358.GG26703@whirlpool.galois.com> Message-ID: <1253879675.28115.214.camel@localhost> On Thu, 2009-09-24 at 23:13 +0100, Don Stewart wrote: > > It would be beneficial if this wording was applied to all allocation > > routines - such as mallocForeignPtrBytes, mallocForeignPtrArray, etc. > > For the curious, this proposal was born from the real-world issue of > > pulling Word32's from a ByteString in an efficient but portable manner > > (binary is portable but inefficient, a straight forward > > unsafePerformIO/peek is efficient but need alignment). > > As a side issue, the get/put primitives on Data.Binary should be > efficient (though they're about twice as fast when specialized to a > strict bytestring... stay tuned for a package in this area). They are efficient within the constraint of doing byte reads and reconstructing a multi-byte word using bit twiddling. eg: getWord16be :: Get Word16 getWord16be = do s <- readN 2 id return $! (fromIntegral (s `B.index` 0) `shiftl_w16` 8) .|. (fromIntegral (s `B.index` 1)) Where as reading an aligned word directly is rather faster. The problem is that the binary API cannot guarantee alignment so we have to be pessimistic. We could do better on machines that are tolerant of misaligned memory accesses such as x86. We'd need to use cpp to switch between two implementations depending on if the arch supports misaligned memory access and if it's big or little endian. #ifdef ARCH_ALLOWS_MISALIGNED_MEMORY_ACCESS #ifdef ARCH_LITTLE_ENDIAN getWord32le = getWord32host #else getWord32le = ... #endif etc Note also that currently the host order binary ops are not documented as requiring alignment, but they do. They will fail eg on sparc or ppc for misaligned access. Duncan From allbery at ece.cmu.edu Fri Sep 25 23:20:01 2009 From: allbery at ece.cmu.edu (Brandon S. Allbery KF8NH) Date: Fri Sep 25 22:58:21 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <1253879675.28115.214.camel@localhost> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> <20090924221358.GG26703@whirlpool.galois.com> <1253879675.28115.214.camel@localhost> Message-ID: On Sep 25, 2009, at 07:54 , Duncan Coutts wrote: > pessimistic. We could do better on machines that are tolerant of > misaligned memory accesses such as x86. We'd need to use cpp to switch Hm. I thought x86 could be tolerant (depending on a cpu configuration bit) but the result was so slow that it wasn't worth it? -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part Url : http://www.haskell.org/pipermail/haskell-prime/attachments/20090925/acad1a8a/PGP.bin From duncan.coutts at googlemail.com Mon Sep 28 09:09:41 2009 From: duncan.coutts at googlemail.com (Duncan Coutts) Date: Mon Sep 28 09:45:43 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> <20090924221358.GG26703@whirlpool.galois.com> <1253879675.28115.214.camel@localhost> Message-ID: <1254143381.4588.454.camel@localhost> On Sat, 2009-09-26 at 04:20 +0100, Brandon S. Allbery KF8NH wrote: > On Sep 25, 2009, at 07:54 , Duncan Coutts wrote: > > pessimistic. We could do better on machines that are tolerant of > > misaligned memory accesses such as x86. We'd need to use cpp to switch > > > Hm. I thought x86 could be tolerant (depending on a cpu configuration > bit) but the result was so slow that it wasn't worth it? It's slow and you would not want to do it much, however I think it's still comparable in speed to doing a series of byte reads/writes and using bit twiddling to convert to/from the larger word type. It's probably also faster to do an unaliged operation sometimes than to do an alignment test each time and call a special unaliged version. Duncan From john at repetae.net Tue Sep 29 18:45:52 2009 From: john at repetae.net (John Meacham) Date: Tue Sep 29 18:23:52 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <4c44d90b0909242012u610c98a7ya1fafb1f4725d474@mail.gmail.com> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> <4c44d90b0909242012u610c98a7ya1fafb1f4725d474@mail.gmail.com> Message-ID: <20090929224552.GA4734@sliver.repetae.net> What about something like > alloca $ \ptr -> ... where ptr :: Ptr Word8. wouldn't ptr only need byte alignment? if the implementation allocates such values on the stack then it may be hard to guarentee otherwise. So, I think the extra verbiage should only apply to routines that allocate based on a size in bytes, rather than when allocating a specific type like with the plain alloca and malloc routines. John -- John Meacham - ?repetae.net?john? - http://notanumber.net/ From thomas.dubuisson at gmail.com Tue Sep 29 21:52:17 2009 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Tue Sep 29 21:30:13 2009 Subject: Strongly Specify Alignment for FFI Allocation In-Reply-To: <20090929224552.GA4734@sliver.repetae.net> References: <4c44d90b0909241402t2915416ala9f94827c451260e@mail.gmail.com> <4c44d90b0909242012u610c98a7ya1fafb1f4725d474@mail.gmail.com> <20090929224552.GA4734@sliver.repetae.net> Message-ID: <4c44d90b0909291852p78f8546bqe5b9286c6c6e348d@mail.gmail.com> John Meacham wrote: > What about something like > >> alloca $ \ptr -> ... where ptr :: Ptr Word8. > > wouldn't ptr only need byte alignment? if the implementation allocates > such values on the stack then it may be hard to guarentee otherwise. > > So, I think the extra verbiage should only apply to routines that > allocate based on a size in bytes, rather than when allocating a > specific type like with the plain alloca and malloc routines. I had a knee-jerk reaction against this comment, but the more I reflect the more I agree it's probably the right way to go. Thomas