From simonpj at microsoft.com Tue Jun 24 12:10:32 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Tue Jun 24 12:02:10 2008 Subject: [Hs-Generics] RE: Traversible Functor Data, or: X marks the spot (Re: Haddock version during build) In-Reply-To: <010101c8d613$ead35420$c91c7ad5@cr3lt> References: <638ABD0A29C8884A91BC5FB5C349B1C32AE5945270@EA-EXMSG-C334.europe.corp.microsoft.com><4847CFC3.3060401@gmail.com><20080613153421.GA23920@matrix.chaos.earth.li><404396ef0806131001r2aac149ctb0c7684ad193da02@mail.gmail.com><404396ef0806140223u72def380s4f3db107f880cb2@mail <00da01c8d3ed$9f5be9d0$d63b8351@cr3lt> <010101c8d613$ead35420$c91c7ad5@cr3lt> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE70CCA49@EA-EXMSG-C334.europe.corp.microsoft.com> | I was surprised to see Data instances for (a->b) and IO a, | since for such non-algebraic types, there isn't anything to | gfoldl or gmap over. And those instances do indeed seem | to offer very little functionality (not to mention the runtime | errors..). I'm surprised too. I thought Data meant, well, data. Does anyone out there *want* functions and IO in Data? | ps. What is the right list for this topic? Possibly generics@haskell.org Simon From claus.reinke at talk21.com Tue Jun 24 14:45:18 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Tue Jun 24 14:36:57 2008 Subject: [Hs-Generics] Traversable Functor Data,or: X marks the spot Message-ID: <015401c8d62a$746efb60$c91c7ad5@cr3lt> Dear Generics;-) this is a repost from cvs-libraries of an experiment which you might find of interest, and on which I'd welcome feedback. One impact being that SYB can implement your GMap test (on which it currently defaults), among other things (see message 1 below, the technique probably applies to Uniplate as well?). One issue being that Data instances for non-algebraic types that default to doing nothing render this technique unsafe (see message 2 below). Btw, the announcement of the unified generics library project was so long ago that I had forgotten about it and about this list (thanks to Simon PJ for reminding me). My interest in this is partially from my past with HaRe, partially from wanting generic traversal support over GHC AST types, so I'm more interested in generic traversal/analysis style libraries. Claus --------- message 1 The issue at hand: can we use Data/Typeable to do what Functor and Traversible do so easily, namely type-changing maps? Or should there be support for deriving all of Data/Typeable/Functor/Traversible over GHC's AST? A drastically simplified example from David's Haddock 2 code (Haddock.Interface.Rename) needs to do a kind of mapM renameDoc :: Monad m => HsDoc Name -> m (HsDoc DocName) which is straightforward with Traversible, but that is not derivable yet (David has been working on that, though), while the usual basis of SYB, Data/Typeable, is derivable, but all SYB transformations are based on gfoldl, the type of which does not permit type-changing maps: gfoldl :: (Data a) =>(forall a1 b. (Data a1) => c (a1 -> b) -> a1 -> c b) -> (forall g. g -> c g) -> a-> c a One could probably go from heterogeneous Data types to a homogeneously typed tree representation, do the map there, then transform back, but that would require additional overhead and a type coercion for the backtransform. Also, it seems a pity that the derived code for gfoldl can handle such maps - it is just gfoldl's type that gets in the way (and trying to generalize that type is a whole different kind of headache..). While boilerplate scrappers around the globe eagerly await the release of "Generalizing the type of gfold, or: to braindamage and beyond" (author: unknown; release date: unknown; release: unlikely;-) I thought I'd have a go at the smaller problem of finding a way to bypass gfoldl's type systematically in order to use its derivable code for things like fmap and traverse. This message summarizes how far I've got so far, and asks for comments (does this do what it should? is it safe? assuming it can be cleaned up to do what it should in a safe way, does this mean that deriving Data/ Typeable for GHCs AST types will be sufficient, or should we still have Traversible as well? etc.). First, here is an implementation of renameDoc in terms of gfoldl and unsafeCoerce (slightly cleaned up version of what I sent in the other thread earlier): data Name = Name String deriving (Show,Data,Typeable) data DocName = DocName String deriving (Show,Data,Typeable) renameDoc :: Monad m => HsDoc Name -> m (HsDoc DocName) renameDoc (DocIdentifier ids) = mapM (\(Name n)->return (DocName n)) ids >>= return . DocIdentifier renameDoc hsDoc = n2d (gfoldl k return hsDoc) where k c x = ap c (mkM (d2n . renameDoc) x) n2d :: Monad m => m (HsDoc Name) -> m (HsDoc DocName) n2d = unsafeCoerce d2n :: Monad m => m (HsDoc DocName) -> m (HsDoc Name) d2n = unsafeCoerce 'DocIdentifier :: [id] -> HsDoc id' is the only constructor in HsDoc that involves the parameter type 'id', so renameDoc either does the parameter type conversion or -for other constructors- recurses into the subexpressions, using gfoldl to build a monadic map. The important insight here is that gfoldl's code can handle the task, we just pretend that our map is type-preserving to conform to gfoldl's restrictive type, by coercing the result types (inside gfoldl, we pretend that renameDoc returns a (HsDoc Name), which outside gfoldl, we coerce back to (HsDoc DocName)). Assuming that noone looks at the return types inside gfoldl (at least, not in a way that would be affected by this change in type - SYB does support adhoc overloading, after all, as in mkM here), this seems to work: testDoc = DocAppend (DocParagraph (DocAppend (DocIdentifier [Name "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [Name "don't",Name "go",Name "anywhere"])) *Main> renameDoc testDoc DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [DocName "don't",DocName "go",DocName "anywhere"])) But can we generalize this, and what about those coercions? Can we -ideally- define something like fmap and traverse in terms of gfoldl, and hide the uglyness in their implementations? Well, I'll spare you (and me;-) the details of my struggle with the type system, and just show the results, with some comments. First, the simpler fmap: -- "X marks the spots";-) X should be private data X = X deriving (Data,Typeable) fmap' :: (Data (f X)) => (a -> b) -> f a -> f b fmap' f x = markTheSpots (rec (wrap f)) x where markTheSpots :: (f X -> f X) -> (f a -> f b) markTheSpots f = unsafeCoerce . f . unsafeCoerce rec :: (Data a) => (X -> X) -> a -> a rec f x = (gmapT (rec f) `extT` f) x wrap :: (a -> b) -> (X -> X) wrap f = unsafeCoerce . f . unsafeCoerce Surprisingly simple for something that seemed impossible at first, isn't it?-) Since we're already committed (for this experiment, at least) to some type fiddling, we can make more constructive use of unsafeCoerce, for two purposes: 1. We wrap the function parameter f, to make it look like a type-preserving function, on some private type X. 2. We mark the occurrences of the type constructor f's parameter type, by coercing 'f a' to 'f X' and 'f X' to 'f b'. Then, we simply use SYB to apply f, when its type matches the parameter, or to recurse into the subexpressions using gmapT, otherwise. If X is private, f will only be applied to the "functor parameter" _positions_ in 'f a', not to other "functor parameter" _type_ occurrences in 'f a': *Main> fmap' not $ (True,True) (True,False) *Main> fmap' not $ [True,True] [False,False] *Main> fmap' (\(Name s)->(DocName s)) testDoc DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [DocName "don't",DocName "go",DocName "anywhere"])) Note how we use one kind of recursion where a manual implementation of fmap would use two: handling subexpressions (which we'd usually do by pattern matching and expression construction) and functor type recursion. Ecouraged by this success, we are ready to tackle the slightly more tricky traverse, using the same techniques: mark the spot, wrap the worker, one kind of recursion. Only this time, we need to take care of the applicative plumbing as well, so it's gfoldl instead of gmapT, and some more complex types. We need the usual SYB type extension support, but for Applicative, not Monad (SYB was defined before Applicative existed, it seems..): -- type extension over Applicative f mkF :: forall f a b . (Applicative f,Typeable a,Typeable b) => (b -> f b) -> a -> f a mkF f x = case gcast (F f) of { Just (F f) -> f x; Nothing -> pure x } extF :: forall t f a b . (Typeable a,Typeable b) => (a -> f a) -> (b -> f b) -> (a -> f a) (f `extF` fspec) x = case gcast (F fspec) of { Just (F fspec) -> fspec x; Nothing -> f x } newtype F f x = F { unF :: x -> f x } And here we go: traverse' :: forall f t a b . (Applicative f,Typeable1 f, Typeable1 t,Data (t X), Typeable a) => (a -> f b) -> t a -> f (t b) traverse' f x = markTheSpots (rec (wrap f)) x where markTheSpots :: forall a b . (t X -> f (t X)) -> (t a -> f (t b)) markTheSpots f = unsafeCoerce . f . unsafeCoerce wrap :: forall a b . (a -> f b) -> (X -> f X) wrap f = unsafeCoerce . f . unsafeCoerce rec :: forall x . Data x => (X -> f X) -> x -> f x rec f x = (gfoldl (k f) z `extF` f) x k :: forall a b . Data a => (X -> f X) -> f (a -> b) -> a -> f b k f c x = c <*> (mkF (rec f :: Data a => a -> f a) `extF` f) x z c = pure c This does seem to do the right thing, so I don't seem to be completely on the wrong track: *Main> traverse' (pure . not) (True,True) (True,False) *Main> traverse' (pure . not) [True,True] [False,False] *Main> traverse' print testDoc Name "well-typed" Name "don't" Name "go" Name "anywhere" DocAppend (DocParagraph (DocAppend (DocIdentifier [()]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [(),(),()])) *Main> traverse' (pure) testDoc DocAppend (DocParagraph (DocAppend (DocIdentifier [Name "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [Name "don't",Name "go",Name "anywhere"])) *Main> traverse' (pure . (\(Name s)->DocName s)) testDoc DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier [DocName "don't",DocName "go",DocName "anywhere"])) but I'm too battered from trying to coerce gfoldl to analyze this properly at the moment, so I'm sending this in the hope of (a) not having to look at gfoldl's type for a while !-) and (b) getting some feedback (is this useful? does the extra overhead matter?..), caveats (cyclic programs, nested traversals, escaping Xs, ..?), etc. Over to you, Claus PS. the "X marks the spot" trick reminds me of the popular medicine topic: delivery system plus targeted activation (SYB plus unsafeCoerce). ------------- message 2 > fmap' :: (Data (f X)) => (a -> b) -> f a -> f b > fmap' f x = markTheSpots (rec (wrap f)) x > where > markTheSpots :: (f X -> f X) -> (f a -> f b) > markTheSpots f = unsafeCoerce . f . unsafeCoerce > rec :: (Data a) => (X -> X) -> a -> a > rec f x = (gmapT (rec f) `extT` f) x > wrap :: (a -> b) -> (X -> X) > wrap f = unsafeCoerce . f . unsafeCoerce .. > 1. We wrap the function parameter f, to make it look > like a type-preserving function, on some private type X. > > 2. We mark the occurrences of the type constructor f's > parameter type, by coercing 'f a' to 'f X' and 'f X' to > 'f b'. > > Then, we simply use SYB to apply f, when its type matches > the parameter, or to recurse into the subexpressions using > gmapT, otherwise. If X is private, f will only be applied > to the "functor parameter" _positions_ in 'f a', not to other > "functor parameter" _type_ occurrences in 'f a' ..but f might not be applied at all, which leads to the first issue with this technique: I was surprised to see Data instances for (a->b) and IO a, since for such non-algebraic types, there isn't anything to gfoldl or gmap over. And those instances do indeed seem to offer very little functionality (not to mention the runtime errors..). For fmap'/traverse', this means that f will not be applied, and so it is not a good idea to coerce the types as if f had been applied (because the hidden parameter could be exposed after traversal, with changed type and unchanged representation!).. So we need to restrict the types of fmap'/traverse'. Which leads to two questions: - what is the rationale for having these non-functional Data instances? If one is to have 'instance Data (a->b)', is there a way to make it more functional? - how can we capture algebraic types in a type (class)? I thought that Data would do just that, being designed to gfoldl over concrete data constructors, but apparently not. And I don't really want to have a separate list of all the types for which Data works, or of all the types for which Data doesn't quite work. Claus ps. What is the right list for this topic? From mrchebas at gmail.com Fri Jun 27 06:04:52 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Fri Jun 27 05:56:15 2008 Subject: [Hs-Generics] Traversable Functor Data,or: X marks the spot In-Reply-To: <015401c8d62a$746efb60$c91c7ad5@cr3lt> References: <015401c8d62a$746efb60$c91c7ad5@cr3lt> Message-ID: <4b39c80a0806270304r656266f4v17e4b4fea82518d@mail.gmail.com> Hi Claus! We have a deadline coming soon, so this is only a brief comment about your GMap implementation. I played a bit with your fmap' function to see whether I could find an obvious problem with the implementation. For instance, I used this datatype: data Tricky a = Tricky a Char deriving (Data,Typeable) and tried to transform a value of type "Tricky Char" to "Tricky Bool". I wanted to see whether the second argument of Tricky would be (incorrectly) transformed to Bool. But it turned out that fmap' behaved as expected. So I think that SYB can pass the GMap test. On the other hand, I think that using unsafeCoerce as a way to define generic functions is inelegant and probably bad practice due to possible runtime failures. However, I must admit that I was surprised when I saw your trick :). Probably we'll get back to you around next week. Meanwhile, you may find useful to look at the paper we wrote on comparing generic programming libraries[1]. In particular, you can look at caveats that apply to SYB. Cheers, Alexey [1] http://www.cs.uu.nl/wiki/bin/view/Alexey/ComparingLibrariesForGenericProgrammingInHaskell On Tue, Jun 24, 2008 at 8:45 PM, Claus Reinke wrote: > Dear Generics;-) > > this is a repost from cvs-libraries of an experiment which you might find > of interest, and on which I'd welcome feedback. One impact being that SYB > can implement your GMap test (on which it currently defaults), among other > things (see message 1 below, the technique probably applies to Uniplate as > well?). One issue being that Data instances for non-algebraic types that > default to > doing nothing render this technique unsafe (see message 2 below). > > Btw, the announcement of the unified generics library project was so long > ago that I had forgotten about it and about this list (thanks to Simon PJ > for reminding me). My interest in this is partially from my past with HaRe, > partially from wanting generic traversal support over GHC AST types, so I'm > more interested > in generic traversal/analysis style libraries. > > Claus > > --------- message 1 > > The issue at hand: can we use Data/Typeable to do what Functor and > Traversible do so easily, namely type-changing maps? Or should > there be support for deriving all of Data/Typeable/Functor/Traversible > over GHC's AST? > > A drastically simplified example from David's Haddock 2 code > (Haddock.Interface.Rename) needs to do a kind of mapM > > renameDoc :: Monad m => HsDoc Name -> m (HsDoc DocName) > > which is straightforward with Traversible, but that is not derivable > yet (David has been working on that, though), while the usual basis > of SYB, Data/Typeable, is derivable, but all SYB transformations are based > on gfoldl, the type of which does not permit type-changing maps: > > gfoldl :: (Data a) =>(forall a1 b. (Data a1) => c (a1 -> b) -> a1 > -> c b) > -> (forall g. g -> c g) > -> a-> c a > > One could probably go from heterogeneous Data types to a > homogeneously typed tree representation, do the map there, then transform > back, but that would require additional overhead > and a type coercion for the backtransform. Also, it seems a pity that the > derived code for gfoldl can handle such maps - it is just > gfoldl's type that gets in the way (and trying to generalize that > type is a whole different kind of headache..). > > While boilerplate scrappers around the globe eagerly await the > release of > "Generalizing the type of gfold, or: to braindamage and beyond" > (author: unknown; release date: unknown; release: unlikely;-) > > I thought I'd have a go at the smaller problem of finding a way > to bypass gfoldl's type systematically in order to use its derivable > code for things like fmap and traverse. This message summarizes > how far I've got so far, and asks for comments (does this do > what it should? is it safe? assuming it can be cleaned up to do > what it should in a safe way, does this mean that deriving Data/ > Typeable for GHCs AST types will be sufficient, or should we > still have Traversible as well? etc.). > > First, here is an implementation of renameDoc in terms of > gfoldl and unsafeCoerce (slightly cleaned up version of what > I sent in the other thread earlier): > > data Name = Name String deriving (Show,Data,Typeable) > data DocName = DocName String deriving (Show,Data,Typeable) > > renameDoc :: Monad m => HsDoc Name -> m (HsDoc DocName) > renameDoc (DocIdentifier ids) = mapM (\(Name n)->return (DocName n)) > ids >>= return . DocIdentifier > renameDoc hsDoc = n2d (gfoldl k return hsDoc) > where k c x = ap c (mkM (d2n . renameDoc) x) > > n2d :: Monad m => m (HsDoc Name) -> m (HsDoc DocName) > n2d = unsafeCoerce > d2n :: Monad m => m (HsDoc DocName) -> m (HsDoc Name) > d2n = unsafeCoerce > > 'DocIdentifier :: [id] -> HsDoc id' is the only constructor in HsDoc that > involves the parameter type 'id', so renameDoc either does the > parameter type conversion or -for other constructors- recurses into the > subexpressions, using gfoldl to build a monadic map. > The important insight here is that gfoldl's code can handle the task, > we just pretend that our map is type-preserving to conform to gfoldl's > restrictive type, by coercing the result types (inside gfoldl, we pretend > that renameDoc returns a (HsDoc Name), which outside gfoldl, we coerce back > to (HsDoc DocName)). > > Assuming that noone looks at the return types inside gfoldl (at least, > not in a way that would be affected by this change in type - SYB > does support adhoc overloading, after all, as in mkM here), this > seems to work: > > testDoc = DocAppend (DocParagraph (DocAppend (DocIdentifier [Name > "well-typed"]) DocEmpty)) (DocAppend (DocString "programs") > (DocIdentifier [Name "don't",Name "go",Name "anywhere"])) > > *Main> renameDoc testDoc > DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) > DocEmpty)) > (DocAppend (DocString "programs") (DocIdentifier [DocName > "don't",DocName "go",DocName "anywhere"])) > > But can we generalize this, and what about those coercions? > Can we -ideally- define something like fmap and traverse in terms of > gfoldl, and hide the uglyness in their implementations? > > Well, I'll spare you (and me;-) the details of my struggle with > the type system, and just show the results, with some comments. First, the > simpler fmap: > > -- "X marks the spots";-) X should be private > data X = X deriving (Data,Typeable) > > fmap' :: (Data (f X)) => (a -> b) -> f a -> f b > fmap' f x = markTheSpots (rec (wrap f)) x > where > markTheSpots :: (f X -> f X) -> (f a -> f b) > markTheSpots f = unsafeCoerce . f . unsafeCoerce > rec :: (Data a) => (X -> X) -> a -> a > rec f x = (gmapT (rec f) `extT` f) x > wrap :: (a -> b) -> (X -> X) > wrap f = unsafeCoerce . f . unsafeCoerce > > Surprisingly simple for something that seemed impossible > at first, isn't it?-) Since we're already committed (for this > experiment, at least) to some type fiddling, we can make > more constructive use of unsafeCoerce, for two purposes: > > 1. We wrap the function parameter f, to make it look > like a type-preserving function, on some private type X. > > 2. We mark the occurrences of the type constructor f's parameter type, by > coercing 'f a' to 'f X' and 'f X' to > 'f b'. > > Then, we simply use SYB to apply f, when its type matches > the parameter, or to recurse into the subexpressions using > gmapT, otherwise. If X is private, f will only be applied > to the "functor parameter" _positions_ in 'f a', not to other > "functor parameter" _type_ occurrences in 'f a': > > *Main> fmap' not $ (True,True) > (True,False) > *Main> fmap' not $ [True,True] > [False,False] > > *Main> fmap' (\(Name s)->(DocName s)) testDoc > DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) > DocEmpty)) > (DocAppend (DocString "programs") (DocIdentifier [DocName > "don't",DocName "go",DocName "anywhere"])) > > Note how we use one kind of recursion where a manual implementation of fmap > would use two: handling subexpressions > (which we'd usually do by pattern matching and expression > construction) and functor type recursion. > > Ecouraged by this success, we are ready to tackle the slightly more tricky > traverse, using the same techniques: > mark the spot, wrap the worker, one kind of recursion. > Only this time, we need to take care of the applicative > plumbing as well, so it's gfoldl instead of gmapT, and > some more complex types. > > We need the usual SYB type extension support, but for > Applicative, not Monad (SYB was defined before > Applicative existed, it seems..): > > -- type extension over Applicative f > mkF :: forall f a b . (Applicative f,Typeable a,Typeable b) > => (b -> f b) -> a -> f a > mkF f x = case gcast (F f) of { Just (F f) -> f x; Nothing -> pure x } > > extF :: forall t f a b . (Typeable a,Typeable b) => (a -> f a) -> > (b -> f b) -> (a -> f a) > (f `extF` fspec) x = case gcast (F fspec) of { Just (F fspec) -> fspec x; > Nothing -> f x } > > newtype F f x = F { unF :: x -> f x } > > And here we go: > > traverse' :: forall f t a b . (Applicative f,Typeable1 f, > Typeable1 t,Data (t X), > Typeable a) > => (a -> f b) -> t a -> f (t b) > traverse' f x = markTheSpots (rec (wrap f)) x > where > markTheSpots :: forall a b . (t X -> f (t X)) -> (t a -> f (t b)) > markTheSpots f = unsafeCoerce . f . unsafeCoerce > wrap :: forall a b . (a -> f b) -> (X -> f X) > wrap f = unsafeCoerce . f . unsafeCoerce > > rec :: forall x . Data x => (X -> f X) -> x -> f x > rec f x = (gfoldl (k f) z `extF` f) x k :: forall a b . Data a => > (X -> f X) -> f (a -> b) -> a -> f b > k f c x = c <*> (mkF (rec f :: Data a => a -> f a) `extF` f) x > z c = pure c > > This does seem to do the right thing, so I don't seem to be > completely on the wrong track: > > *Main> traverse' (pure . not) (True,True) > (True,False) > > *Main> traverse' (pure . not) [True,True] > [False,False] > > *Main> traverse' print testDoc > Name "well-typed" > Name "don't" > Name "go" > Name "anywhere" > DocAppend (DocParagraph (DocAppend (DocIdentifier [()]) DocEmpty)) > (DocAppend (DocString "programs") (DocIdentifier [(),(),()])) > > *Main> traverse' (pure) testDoc > DocAppend (DocParagraph (DocAppend (DocIdentifier [Name "well-typed"]) > DocEmpty)) (DocAppend (DocString "programs") (DocIdentifier > [Name "don't",Name "go",Name "anywhere"])) > > *Main> traverse' (pure . (\(Name s)->DocName s)) testDoc > DocAppend (DocParagraph (DocAppend (DocIdentifier [DocName "well-typed"]) > DocEmpty)) > (DocAppend (DocString "programs") (DocIdentifier [DocName > "don't",DocName "go",DocName "anywhere"])) > > but I'm too battered from trying to coerce gfoldl to analyze this > properly at the moment, so I'm sending this in the hope of (a) > not having to look at gfoldl's type for a while !-) and (b) getting > some feedback (is this useful? does the extra overhead matter?..), caveats > (cyclic programs, nested traversals, escaping Xs, ..?), etc. > > Over to you, > Claus > > PS. the "X marks the spot" trick reminds me of the popular > medicine topic: delivery system plus targeted activation > (SYB plus unsafeCoerce). > > ------------- message 2 > > fmap' :: (Data (f X)) => (a -> b) -> f a -> f b >> fmap' f x = markTheSpots (rec (wrap f)) x >> where >> markTheSpots :: (f X -> f X) -> (f a -> f b) >> markTheSpots f = unsafeCoerce . f . unsafeCoerce >> rec :: (Data a) => (X -> X) -> a -> a >> rec f x = (gmapT (rec f) `extT` f) x >> wrap :: (a -> b) -> (X -> X) >> wrap f = unsafeCoerce . f . unsafeCoerce >> > .. > >> 1. We wrap the function parameter f, to make it look >> like a type-preserving function, on some private type X. >> >> 2. We mark the occurrences of the type constructor f's parameter type, >> by coercing 'f a' to 'f X' and 'f X' to >> 'f b'. >> >> Then, we simply use SYB to apply f, when its type matches >> the parameter, or to recurse into the subexpressions using >> gmapT, otherwise. If X is private, f will only be applied >> to the "functor parameter" _positions_ in 'f a', not to other >> "functor parameter" _type_ occurrences in 'f a' >> > > ..but f might not be applied at all, which leads to the first > issue with this technique: > > I was surprised to see Data instances for (a->b) and IO a, since for such > non-algebraic types, there isn't anything to gfoldl or gmap over. And those > instances do indeed seem > to offer very little functionality (not to mention the runtime > errors..). > For fmap'/traverse', this means that f will not be applied, and so it is > not a good idea to coerce the types as if f had been applied (because the > hidden parameter could be exposed after traversal, with changed type and > unchanged > representation!).. So we need to restrict the types of fmap'/traverse'. > Which leads to two questions: > > - what is the rationale for having these non-functional > Data instances? > > If one is to have 'instance Data (a->b)', is there a way > to make it more functional? > > - how can we capture algebraic types in a type (class)? > > I thought that Data would do just that, being designed > to gfoldl over concrete data constructors, but apparently > not. And I don't really want to have a separate list of all the types > for which Data works, or of all the types for which Data doesn't quite > work. > > Claus > > ps. What is the right list for this topic? > > > _______________________________________________ > Generics mailing list > Generics@haskell.org > http://www.haskell.org/mailman/listinfo/generics > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080627/6ac374e6/attachment-0001.htm From claus.reinke at talk21.com Fri Jun 27 16:48:54 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Fri Jun 27 16:40:21 2008 Subject: [Hs-Generics] Traversable Functor Data,or: X marks the spot References: <015401c8d62a$746efb60$c91c7ad5@cr3lt> <4b39c80a0806270304r656266f4v17e4b4fea82518d@mail.gmail.com> Message-ID: <028201c8d897$37dc39a0$4d268351@cr3lt> Hi Alexey, > We have a deadline coming soon, so this is only a brief comment about your > GMap implementation. Thanks, the silence had me worried! I'll wait till next week, then, meanwhile just a few comments on your comments;-) > wanted to see whether the second argument of Tricky would be (incorrectly) > transformed to Bool. But it turned out that fmap' behaved as expected. So I That's the idea - the wrapped function parameter will be delivered everywhere by the SYB framework, but will only be applied to the marked type. The marker type should play no other role. > On the other hand, I think that using unsafeCoerce as a way to define > generic functions is inelegant and probably bad practice due to possible > runtime failures. However, I must admit that I was surprised when I saw your > trick :). Haskell is full of odd corners (eg, the IO model is not backed up by a uniqueness type system, so is really unsafe under the abstraction, and neither the implementation nor the optimizer are verified by the type system; the Typeable stuff only works by convention for the instances; ..). Ideally, the language/type system will be extended to be able to express all the capabilities of gfoldl's code in gfoldl's type. But as long as that isn't the case, unsafeCoerce allows us to extend the type system, just as unsafePerformIO allows us to extend the evaluator. But since we are dealing in extensions, it is up to us to demonstrate that we haven't broken anything (well, me, in this case, but after staring at gfoldl's type and its implication for several days, I felt like calling for a little outside perspective:-). My assumption was roughly that 'everywhere f' would apply 'f' to all occurences typed 'a' provided that 'f' has 'a' generic special case for type 'a'. That assumption is violated by the Data instances for (a->b) and (IO a), so as long as those instances exist, it is easy to come up with counterexamples for fmap'. That is why I was asking for the rationale for those instances - if it was just convenience, they should be dropped, especially since SYB doesn't permit overloading to polymorpic types, so those default instances are not easily bypassed. Once that is sorted, the next issue is whether the private marker type can escape and be observed by other means. fmap' is constructed in such a way that neither 'f' nor the calling context see 'X', so as long as the generic scaffolding provided by 'Data' doesn't mess up, that should be fine. For things like traverse', other operators might be applied to things coerced to 'X', so that needs to be looked into and perhaps improved. > Probably we'll get back to you around next week. Meanwhile, you may find > useful to look at the paper we wrote on comparing generic programming > libraries[1]. In particular, you can look at caveats that apply to SYB. Yep, I guess I'll have to look through some more of the generic papers on my reading heap:-) Btw, the overall idea behind my experiment is whether we can use 'Data/Typeable', which can be derived in GHC, to avoid the need for deriving support for other generic instances (such as Functor, Traversable). Again, a very pragmatic perspective (what can we do with the basis we have now, rather than: what better basis would be like to have?), but closely related to this list's aim of a unified generic programming framework. Btw, expanding SYB's invariant maps to variant ones raises the whole issue of co- vs contra-variance: if one really wants to fmap over types containing the parameter type in function types, one would need to handle both positive and negative occurrences, presumably by a pair of dual functions (a->b, b->a)? Looking forward to more feedback after that deadline. Thanks, Claus From claus.reinke at talk21.com Sun Jun 29 15:23:00 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Sun Jun 29 15:15:51 2008 Subject: [Hs-Generics] Re: Traversible Functor Data, or: X marks the spot (Re: Haddock version during build) References: <00da01c8d3ed$9f5be9d0$d63b8351@cr3lt><010101c8d613$ead35420$c91c7ad5@cr3lt><638ABD0A29C8884A91BC5FB5C349B1C32AE70CCA49@EA-EXMSG-C334.europe.corp.microsoft.com> <20080629164147.GA11809@matrix.chaos.earth.li> Message-ID: <00b701c8da1d$c06ee920$e1117ad5@cr3lt> >> Does anyone out there *want* functions and IO in Data? > > I have a feeling you sometimes need those instances to make the > typechecker happy, even though you aren't actually doing anything > generic on those parts of the types. > > syb-with-class has also defined IO and (->) instances, but no comment as > to why. I found this remark in "Scrap More Boilerplate: .." (section 5.3, non-representable data types): Lastly, it is convenient to give Data instances even for types that are not strictly data types, such as function types or monadic IO types. Otherwise deriving ( Data ) would fail for a data type that had even one constructor with a functional argument type, so the user would instead have to write the Data instance by hand. Instead, we make all such types into vacuous instances of Data. Traversal will safely cease for values of such types. However, values of these types can not be read and shown. So the convenience hypothesis seems correct. The problem with this is that it assumes that the vacuous instances do not involve any types that the generic traversals are operating on. Also, while I could add such a vacuous instance Data (a->b), and then use deriving Data over the remaining types, I cannot remove an existing instance Data (a->b), nor can I easily override its effects. I suggest to separate the vacuous from the proper instances, and to expose only the former via Data.Generics. That way, the convenience is only an import away, but doesn't get in the way of non-standard applications. Claus From claus.reinke at talk21.com Sun Jun 29 18:40:50 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Sun Jun 29 18:32:14 2008 Subject: [Hs-Generics] instance Data (in-)conveniences (Re: Traversible Functor Data, or: X marks the spot) References: <20080629164147.GA11809@matrix.chaos.earth.li> <00b701c8da1d$c06ee920$e1117ad5@cr3lt> <20080629194555.GA26024@matrix.chaos.earth.li> Message-ID: <00eb01c8da39$2ff7a910$e1117ad5@cr3lt> >> I suggest to separate the vacuous from the proper instances, >> and to expose only the former via Data.Generics. That way, >> the convenience is only an import away, but doesn't get in >> the way of non-standard applications. > > The problem with this is that it leads to conflicting instances if you > import both. I was thinking of splitting Data.Generics.Instances, moving some of its Data instances to Data.Generics.DummyInstances, and not importing/exporting the latter from Data.Generics. Importing both Data.Generics and Data.Generics.DummyInstances would give the current situation, no conflicting instances. And those who need application-specific instances instead of the dummies can avoid importing the dummy instances. In particular, the instances instance (Data a, Data b) => Data (a -> b) instance Typeable a => Data (IO a) instance Typeable a => Data (Ptr a) instance Typeable a => Data (StablePtr a) instance Typeable a => Data (IORef a) instance Typeable a => Data (ForeignPtr a) instance (Typeable s, Typeable a) => Data (ST s a) instance Typeable a => Data (TVar a) instance Typeable a => Data (MVar a) instance Typeable a => Data (STM a) instance (Data a, Integral a) => Data (Ratio a) claim to have polymorphic/generic components, but use the defaults gfoldl _ z = z gmapT f x = unID (gfoldl k ID x) where k (ID c) x = ID (c (f x)) meaning that gmapT (=id) will never get access to those component types. Contrast this with the conventional instances for [], Maybe, Either, (,..) in the same module, and especially the abstraction- preserving instance for Array a b. And compare the not very consistent results (sometimes the transformation is applied, sometimes not): > everywhere (mkT ((+1)::Int->Int)) (0,1)::(,) Int Int (1,2) > everywhere (mkT ((+1)::Int->Int)) (0%1)::Ratio Int 0%1 > everywhere (mkT ((+1)::Int->Int)) (return 0::[] Int) [1] > everywhere (mkT ((+1)::Int->Int)) (return 0::IO Int) 0 > everywhere (mkT ((+1)::Int->Int)) (return 0::() -> Int) () 0 > everywhere (mkT ((+1)::Int->Int)) (array (0,0) [(0,0)] :: Array Int Int) array (0,0) [(0,1)] There isn't any straightforward "proper" Data instance for those types, either. But if, say, I'd knew that, for a specific application context, I'd want to apply my transformation (b->b) to the range, and only to the range, of all functions (a->b), via post-composition, how would I even do that, given the existing instance Data (a->b)? I can bypass monomorphic functions using extT, but what about polymorphic functions? > (\(f,g)->(f (),g 0)) $ everywhere (mkT (((+1).)::(()->Int)->(()->Int))) (\()->(0::Int),\_->(0::Int)) (1,0) Is there a way to do this without removing the existing Data instance (which would allow me to define my own)? Claus From simonpj at microsoft.com Mon Jun 30 04:10:16 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Mon Jun 30 04:01:31 2008 Subject: [Hs-Generics] RE: Traversible Functor Data, or: X marks the spot (Re: Haddock version during build) In-Reply-To: <00b701c8da1d$c06ee920$e1117ad5@cr3lt> References: <00da01c8d3ed$9f5be9d0$d63b8351@cr3lt><010101c8d613$ead35420$c91c7ad5@cr3lt><638ABD0A29C8884A91BC5FB5C349B1C32AE70CCA49@EA-EXMSG-C334.europe.corp.microsoft.com> <20080629164147.GA11809@matrix.chaos.earth.li> <00b701c8da1d$c06ee920$e1117ad5@cr3lt> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE72D1B9D@EA-EXMSG-C334.europe.corp.microsoft.com> | I suggest to separate the vacuous from the proper instances, | and to expose only the former via Data.Generics. That way, | the convenience is only an import away, but doesn't get in | the way of non-standard applications. Probably the right thing is to use the libraries modification process to effect the change. (For what it's worth, I don't feel strongly about this.) Simon