From oleg at okmij.org Tue Jul 1 07:46:33 2008 From: oleg at okmij.org (oleg@okmij.org) Date: Tue Jul 1 07:40:15 2008 Subject: [Hs-Generics] Re: gMap in SYB1 In-Reply-To: <015401c8d62a$746efb60$c91c7ad5@cr3lt> Message-ID: <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> Hello! It has occurred to me a year ago that the type-changing gMap is easily possible in SYB. The recent discussion has prompted me to implement that solution. I have committed a patch to http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs with the implementation. I hope this is OK: the previous version of that file was a stub saying that gMap is not supported. The function gMap has the expected type gmap :: (Data a, Data b, Data x, Data y) => (a -> b) -> x -> y There are no unsafe operations used; there is not a single occurrence of insafePerformIO and the ilk. Incidentally, Data.Generics could, in principle at least, be implemented using a safe cast. Of course gmap is not a total operation: one can't really expect (gmap id True) to produce a character. It seems that gunfold is not total anyway. Regarding the instance of Data for functions: SmashA does define (co-variant) traversal `under lambda'. For example, we can (gmap fromEnum) on a function and convert (Bool->Bool) to (Bool->Int). Please see testt3 in http://darcs.haskell.org/generics/comparison/SmashA/Syb4A.hs I do not claim to have any use for that transformation, but it was easy to implement... I will be out of town for two weeks and so unlikely to participate in further discussions until I come back. Cheers, Oleg From claus.reinke at talk21.com Tue Jul 1 18:16:22 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Tue Jul 1 18:07:34 2008 Subject: [Hs-Generics] Re: gMap in SYB1 References: <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> Message-ID: <03a001c8dbc8$192ed1a0$4c177ad5@cr3lt> Hi Oleg! > It has occurred to me a year ago that the type-changing gMap > is easily possible in SYB. The recent discussion has prompted me to > implement that solution. I have committed a patch to > > http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs > > with the implementation. I hope this is OK: the previous version of > that file was a stub saying that gMap is not supported. The function > gMap has the expected type > gmap :: (Data a, Data b, Data x, Data y) => (a -> b) -> x -> y Thanks, looks nice. I suspected there had to be a gfoldl+gunfoldl- based approach, but I couldn't quite wrap my head around it. So simple in hindsight (given the proper auxiliary type constructors), and I might have got there in the end, but it was certainly not "easily possible" for me - so thank you for freeing me from the daily headache of trying yet another approach!-) A slight issue: this version does not distinguish functor parameter from equivalent types: *GMap> gmap not (True,True)::(,) Bool Bool (False,False) Perhaps we can combine our versions to get the best of both? > There are no unsafe operations used; there is not a single occurrence > of insafePerformIO and the ilk. Incidentally, Data.Generics could, in > principle at least, be implemented using a safe cast. Of course gmap > is not a total operation: one can't really expect (gmap id True) to > produce a character. It seems that gunfold is not total anyway. The latter is the reason why this version is safer than mine, I think: 'instance Data (a->b)' implements a fake gfoldl, but turns uses of gunfoldl into runtime errors. Yet another reason why those instances are dubious, but any attempt to fake gunfoldl would have to involve error anyway, it seems (something like 'z (const (error "no such function"))' when grabbing an (a->b) out of thin air). So your version does at least get a proper runtime error in the cases where mine runs right out of type safety.. > Regarding the instance of Data for functions: SmashA does > define (co-variant) traversal `under lambda'. For example, we can > (gmap fromEnum) on a function and convert (Bool->Bool) to (Bool->Int). > Please see testt3 in > > http://darcs.haskell.org/generics/comparison/SmashA/Syb4A.hs > > I do not claim to have any use for that transformation, but it was > easy to implement... I was just surprised that the comparison seemed to make no mention of higher-order (as opposed to higher-kinded) types. Cheers, Claus From claus.reinke at talk21.com Wed Jul 2 06:11:39 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Wed Jul 2 06:03:05 2008 Subject: [Hs-Generics] Re: gMap in SYB1 References: <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> <03a001c8dbc8$192ed1a0$4c177ad5@cr3lt> Message-ID: <006901c8dc2c$0c736520$50447ad5@cr3lt> >> It has occurred to me a year ago that the type-changing gMap >> is easily possible in SYB. The recent discussion has prompted me to >> implement that solution. I have committed a patch to >> >> http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs >> >> gmap :: (Data a, Data b, Data x, Data y) => (a -> b) -> x -> y > > Perhaps we can combine our versions to get the best of both? To wit: data X = X deriving (Data,Typeable) data Y = Y deriving (Data,Typeable) fmap'' :: (Data (f X), Data (f Y)) => (a -> b) -> f a -> f b fmap'' f = markTheSpots (gmap (wrap f)) where wrap :: (a->b) -> (X->Y) wrap = unsafeCoerce markTheSpots :: (f X -> f Y) -> (f a -> f b) markTheSpots = unsafeCoerce so that: *GMap> gmap not (True,True) :: (,) Bool Bool (False,False) *GMap> fmap'' not (True,True) :: (,) Bool Bool (True,False) fmap'' is safer than the original fmap', in that trying to map over functions will run into the gunfold runtime error, eg: *GMap> fmap'' (\True->'c') (id::Bool->Bool) True *** Exception: gunfold I keep worrying that I should do something more to hide X/Y, perhaps similar to the runST trick, but (a) while it was easy to show type-unsafety in fmap' in the presence of non-traversing Data instances, I have yet to find an example exposing X/Y in fmap'', and (b) pretending to be fully polymorphic is not necessarily appropriate in a generic programming context;-) So using specific, but unknown types doesn't feel entirely wrong. gmap doesn't know about X/Y, so can only treat them generically, and the same goes for f, if X/Y aren't exported. Of course, the generic toolbox allows to recover the types, but f never sees X/Y, and gmap doesn't seem to care: *GMap> fmap'' dataTypeOf (True,True) :: (,) Bool DataType (True,DataType {tycon = "Prelude.Bool", datarep = AlgRep [False,True]}) *GMap> fmap'' (\X->Y) (True,True) :: (,) Bool Y :1:21: Couldn't match expected type `X' against inferred type `Bool' In the expression: True In the second argument of `fmap''', namely `(True, True)' In the expression: fmap'' (\ X -> Y) (True, True) :: (,) Bool Y *GMap> fmap'' (fmap'' not) [(True,Just True,True)] [(True,Just True,False)] *GMap> fmap'' (gmap not) [(True,Just True,True)]::[(Bool,Maybe Bool,Bool)] [(False,Just False,False)] Still, the problem with these kinds of proofs is not to show that the cases one has thought of are safe, but to be sure that there are no unsafe cases one hasn't thought of. So, more minds/eyeballs are welcome!-) Claus ps. gmap itself is somewhat tricky to use, depending on type information from the context, and failing with runtime errors if that type information doesn't match. *GMap> gmap not (True,True) :: (Bool,Char) (False,*** Exception: gunfold But then, SYB generally operates at the borders of type safety (what one would expect to be a type error often just leads to unexpected behaviour - there are so few gaps between well-typed SYB programs that the type system, instead of noting that things go wrong, can only cause things to go elsewhere). Are the other generic programming libraries better behaved in this area? From johanj at cs.uu.nl Tue Jul 15 09:47:08 2008 From: johanj at cs.uu.nl (Johan Jeuring) Date: Tue Jul 15 09:37:33 2008 Subject: [Hs-Generics] New paper Message-ID: <17610C2D-F05E-4614-BFAD-B326D5F4D13D@cs.uu.nl> We have just finished a paper in which we use type families and GADTs to do generic programming with fixed points for mutually recursive datatypes: Generic programming with fixed points for mutually recursive datatypes Alexey Rodriguez, Stefan Holdermans, Andres L?h, Johan Jeuring Abstract: Many datatype-generic functions need access to the recursive positions in the structure of the datatype, and therefore adopt a fixed point view on datatypes. Examples include variants of fold that traverse the data following the recursive structure, or the zipper data structure that enables navigation along the recursive positions. However, Hindley-Milner-inspired type systems with algebraic datatypes make it difficult to express fixed points for anything but regular datatypes. Many real-life examples such as abstract syntax trees are in fact systems of mutually recursive datatypes and therefore excluded. Using Haskell's GADTs and type families, we describe a technique that allows a fixed-point view for systems of mutually recursive datatypes. We demonstrate that our approach is widely applicable by giving several examples of generic functions for this view, most prominently the Zipper. See http://people.cs.uu.nl/andres/Rec/ -- Johan From claus.reinke at talk21.com Mon Jul 21 09:15:59 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Mon Jul 21 09:06:13 2008 Subject: [Hs-Generics] Data.Generics with GPS (using Maps to avoid getting lost in Data) Message-ID: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> summary: speed up Syb with Uniplate-inspired techniques == Performance issues in Syb == As it stands, classic Syb, while well-supported in GHC isn't exactly the fastest generic programming option. In fact, it routinely seems to come out last in performance comparisons, sometimes by a substantial factor. That isn't always a problem in practice, because the gains in expressiveness/conciseness/maintainability outweigh the loss in performance, because traversal performance is not the bottleneck, or because practical use often involves hand-tuned traversal schemes instead of the example schemes typically used in benchmarks. Be that as it may, performance is a consideration, negative results have been published, and it seems that performance of experimental code has led to Syb being abandoned in at least one project. While it would be unrealistic to expect Syb traversals to compete with hand-written ones with current compilers, there are several obvious areas where Syb traversal performance could be subjected to improvements: (a) traversing irrelevant parts of the structure (because everything is treated generically) (b) combining results in simple, but inefficient ways ((++) nesting with the structure) (c) repeated runtime type checks to determine which function in a generically extended transformation/query to apply While runtime type checks are inherent in SYB (and alternatives have been proposed that claim to avoid them), all of a/b/c can be addressed to some extent in defining tuned traversals using the basic library. There tends to be a trade-off for a vs c, as avoiding senseless generic traversals of specific substructures implies additional runtime type checks to identify those substructures == Avoiding irrelevant traversals with substructure type maps == This message is concerned with addressing (a), in a generic way that does not require hand-tuning to the types in question, and without adding to the burden of (c). The basic idea comes from the Uniplate library, particularly the version implemented on top of Data (Data.Generics.PlateData). The idea is to compute the set of substructure types for a type under traversal and was described in last year's Uniplate paper ([1], or chapter 3 of Neil's recent thesis [2], section 3.6.2, "Optimising PlateData"), though its generality is somewhat obscured by the claim that: The next optimisation relies on the extra information present in the Uniplate operations - namely the target type. A boilerplate operation walks over a data structure, looking for target values to process. In SYB, the target values may be of any type. For Uniplate the target is a single uniform type. If a value is reached which is not a container for the target type, no further exploration is required of the values children. A Data-based 'contains' operation is defined that returns the list of immediate substructure types, represented by existentially boxed 'undefined's of the appropriate types, for all constructors in a given type. This operation can then be iterated to given all substructure types. One interesting point to note is that -though Syb is data-based- the computation of substructure types is based on reflecting a type-based view back into data. For sum types, every concrete data value only presents a subset of substructure types, but Syb's reflection features make it possible to talk about all possible constructors (*). The performance benefits can be substantial - the draft thesis claims In the benchmarks we improve on SYB by between 30% and 225%, with an average of 145% faster. So it seemed worthwhile to generalise this technique for use in Syb, which turned out to be possible after all, leading to the promised Data.Generics with GPS (Generic Positioning System;-). == Data.Generics.GPS == GPS employs Maps, to avoid getting lost in Data: - for each traversed type, build a Map TypeKey TypeSet, mapping all substructure types of the given type to their substructure types - traversals are short-circuited when the domain types of their queries or transformations cannot be found in the current substructure types - domains of queries and transformations are computed on construction GPS is inspired by Uniplate's PlateData direction finder (contains and DataBox are copied from the Uniplate paper), generalised to tackle SYB's more general queries and transformations (instead of oracles telling whether to stop, follow, or find in a search for type b in type a, we build IntSets of TypeRep keys, both for the domains of traversals and for substructure types; then several short-circuiting decisions can be based on fast intersection tests with the same IntSet). Data.Generics.GPS reexports Data.Generics, modifying everything, everywhere, mkQ, extQ, mkT, extT in such a way that building and extending transformations/queries also computes and records their domains and default transformations/values: data GenericDomainQ r = GenericDomainQ { queryDomain :: TypeSet, defaultValue :: r, genericQuery :: GenericQ r } data GenericDomainT = GenericDomainT { transDomain :: TypeSet, defaultTrans :: GenericT, genericTrans :: GenericT } while traversals accept those refined transformations/queries and add substructure type maps for short-circuiting, eg: everywhere :: forall a . Data a => GenericDomainT -> a -> a everywhere = everywhereWithMap (getSubs subMap) where subMap = fromRoot (undefined::a) Map.empty everywhereWithMap :: forall a . Data a => (forall a . Data a => a -> TypeSet) -> GenericDomainT -> a -> a everywhereWithMap getSubs gdt@(GenericDomainT{transDomain=domain, defaultTrans=dt, genericTrans=t}) x | not $ IS.null $ domain `intersection` getSubs x = t (gmapT (everywhereWithMap getSubs gdt) x) | otherwise = dt x == Performance testing in Paradise == To test the performance, I used the example in which Syb performed worst in the Uniplate paper: computing the bill in the Paradise benchmark. To get easily measurable timings, I added 100000 redundant copies of the departments in genCom. Switching from Data.Generics to Data.Generics.GPS is as easy as changing imports (and possibly types): import qualified Data.Generics as DG import Data.Generics.GPS bill :: Data a => a -> Integer bill = DG.everything (+) (0 `DG.mkQ` billS) where billS (S s) = s bill' :: Data a => a -> Integer bill' = everything (+) (0 `mkQ` billS) where billS (S s) = s Performance improves drastically, as expected (close to hand-tuned versions of everything and everywhere that explicitly skip the Strings in the Paradise types), even though substructure type maps are not yet as widely shared as in PlateData (as trace-instrumented type map construction shows). Unfortunately, a serious performance problem in Uniplate (of which Neil is already aware) completely masks the PlateData optimisations, so it is not yet visible whether the gap between PlateData and Syb has disappeared completely: -- ghc --make -O2 Main.hs; GHC version 6.9.20080514 $ ./Main.exe > dump "Data.Generics increase: 5 secs" 5550200000 "Data.Generics bill: 12 secs" "Data.Generics.GPS increase: 2 secs" 5550200000 "Data.Generics.GPS bill: 3 secs" "Data.Generics.PlateData increase: 1 min, 59 secs" 5550200000 "Data.Generics.PlateData bill: 1 min, 54 secs" == General implications of the techniques used == TypeRep keys combined with IntSets or IntMaps should be of general interest to reader of this list, as they can be used to speed up other generic programming problems as well, including typecase and extensible records libraries. Two examples should make this connection obvious: - generic queries and transformations generalise record selection and update (particularly plain to see in Uniplates universeBi/transformBi) - the core of Smash is to replace Syb's runtime type checks with compile-time type checks, but Smash's static typecase is based on HList's extensible record selection, encoded in type-class programs which current compilers do not yet partially evaluate to entirely static, constant-time selection; so, while Smash conceptually replaces runtime type selection with compile-time type selection, its compile-time type selection is compiled into runtime type selection Now, think about combining conceptually nice type-class-based type-case and record operations with pragmatically efficient IntMap-based runtime representations and selection/modification operations!-) Feedback, comments, bug-reports welcome, as usual. Claus [1] http://www-users.cs.york.ac.uk/~ndm/uniplate/ [2] http://www-users.cs.york.ac.uk/~ndm/thesis/ (*) this wouldn't work so well on non-regular types, because of the potentially infinite set of substructure types -------------- next part -------------- A non-text attachment was scrubbed... Name: GPS.hs Type: application/octet-stream Size: 5885 bytes Desc: not available Url : http://www.haskell.org/pipermail/generics/attachments/20080721/5386d548/GPS.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: Main.hs Type: application/octet-stream Size: 3007 bytes Desc: not available Url : http://www.haskell.org/pipermail/generics/attachments/20080721/5386d548/Main.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: CompanyDatatypes.hs Type: application/octet-stream Size: 1367 bytes Desc: not available Url : http://www.haskell.org/pipermail/generics/attachments/20080721/5386d548/CompanyDatatypes.obj From mrchebas at gmail.com Mon Jul 21 11:02:13 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Mon Jul 21 10:52:19 2008 Subject: [Hs-Generics] Traversable Functor Data,or: X marks the spot In-Reply-To: <028201c8d897$37dc39a0$4d268351@cr3lt> References: <015401c8d62a$746efb60$c91c7ad5@cr3lt> <4b39c80a0806270304r656266f4v17e4b4fea82518d@mail.gmail.com> <028201c8d897$37dc39a0$4d268351@cr3lt> Message-ID: <4b39c80a0807210802s15f1684aq7190221dcc25f880@mail.gmail.com> Hey Claus! Sorry for the delay, just after the deadline I got into two more deadlines so I was very busy with paper writing these last weeks! On Fri, Jun 27, 2008 at 10:48 PM, Claus Reinke wrote: > > > > wanted to see whether the second argument of Tricky would be (incorrectly) >> transformed to Bool. But it turned out that fmap' behaved as expected. So >> I >> > > That's the idea - the wrapped function parameter will be delivered > everywhere by the SYB framework, but will only be applied to the marked > type. The marker type should play no other role. Yes, it looks like sort of defining a local type constant which should not be visible after the transformation. > > > On the other hand, I think that using unsafeCoerce as a way to define >> generic functions is inelegant and probably bad practice due to possible >> runtime failures. However, I must admit that I was surprised when I saw >> your >> trick :). >> > > Haskell is full of odd corners (eg, the IO model is not backed up by a > uniqueness type system, so is really unsafe under the abstraction, > and neither the implementation nor the optimizer are verified by the > type system; the Typeable stuff only works by convention for the > instances; ..). > > Ideally, the language/type system will be extended to be able to > express all the capabilities of gfoldl's code in gfoldl's type. But > as long as that isn't the case, unsafeCoerce allows us to extend > the type system, just as unsafePerformIO allows us to extend > the evaluator. But since we are dealing in extensions, it is up to > us to demonstrate that we haven't broken anything (well, me, > in this case, but after staring at gfoldl's type and its implication > for several days, I felt like calling for a little outside perspective:-). As I said before, I don't consider this an elegant use of unsafeCoerce. The reason is that if you later want to generalize other functions over type constructors (for example, zipWith), you have to again deal with unsafeCoerce. It would already be a slight improvement if this trick can be encapsulated into a reusable combinator. If you want to make SYB support definitions that abstract over type constructors, you may want to have a look at "Scrap your Boilerplate Revolutions". Although, I don't immediately see a way to generalize gfoldl in order to encode the "lifted spine view". > > > My assumption was roughly that 'everywhere f' would apply > 'f' to all occurences typed 'a' provided that 'f' has 'a' generic > special case for type 'a'. That assumption is violated by the > Data instances for (a->b) and (IO a), so as long as those > instances exist, it is easy to come up with counterexamples > for fmap'. That is why I was asking for the rationale for those > instances - if it was just convenience, they should be dropped, > especially since SYB doesn't permit overloading to polymorpic > types, so those default instances are not easily bypassed. Yes, I think I would also prefer those instances to reside in a separate module, so that you can choose not to import them. > > > Once that is sorted, the next issue is whether the private > marker type can escape and be observed by other means. > fmap' is constructed in such a way that neither 'f' nor the > calling context see 'X', so as long as the generic scaffolding > provided by 'Data' doesn't mess up, that should be fine. Probably this is something you have to prove whenever you define a generic function in this style. > > > For things like traverse', other operators might be applied > to things coerced to 'X', so that needs to be looked into > and perhaps improved. > > Probably we'll get back to you around next week. Meanwhile, you may find >> useful to look at the paper we wrote on comparing generic programming >> libraries[1]. In particular, you can look at caveats that apply to SYB. >> > > Yep, I guess I'll have to look through some more of the > generic papers on my reading heap:-) Btw, the overall idea > behind my experiment is whether we can use 'Data/Typeable', > which can be derived in GHC, to avoid the need for > deriving support for other generic instances (such as Functor, > Traversable). Again, a very pragmatic perspective (what > can we do with the basis we have now, rather than: what > better basis would be like to have?), but closely related to > this list's aim of a unified generic programming framework. I think it's good to see how far the limits of SYB and other libraries can be pushed. But I would personally avoid using gmap as you proposed, until there is a cleaner way to do things or the corner cases are well understood. > > > Btw, expanding SYB's invariant maps to variant ones raises the whole issue > of co- vs contra-variance: if one really wants to fmap over types containing > the parameter type in function types, one would need to handle both positive > and negative occurrences, presumably by a pair of dual functions (a->b, > b->a)? That gives rise to another type transforming map, namely bimap. You can look at its Generic Haskell definition at this URL: https://svn.cs.uu.nl:12443/repos/Generic-Haskell/trunk/Generic-Haskell/lib/GH/Prelude.ghs Cheers, Alexey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080721/437902ef/attachment-0001.htm From mrchebas at gmail.com Mon Jul 21 11:54:39 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Mon Jul 21 11:44:46 2008 Subject: [Hs-Generics] Re: gMap in SYB1 In-Reply-To: <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> References: <015401c8d62a$746efb60$c91c7ad5@cr3lt> <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> Message-ID: <4b39c80a0807210854p3f8c6af5oe32ebbca54361934@mail.gmail.com> Hi Oleg! On Tue, Jul 1, 2008 at 1:46 PM, wrote: > > Hello! > > It has occurred to me a year ago that the type-changing gMap > is easily possible in SYB. The recent discussion has prompted me to > implement that solution. I have committed a patch to > > http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs > > with the implementation. I hope this is OK: the previous version of > that file was a stub saying that gMap is not supported. The function > gMap has the expected type > gmap :: (Data a, Data b, Data x, Data y) => (a -> b) -> x -> y I have tried the following code: > data Tricky a = Tricky a Char deriving (Data,Typeable,Show) > mapTricky :: (Data a,Data b) => (a -> b) -> Tricky a -> Tricky b > mapTricky = gmap > tricky1 = Tricky 'a' 'b' > tr_test1 = mapTricky (=='a') tricky1 > tr_test2 = mapTricky (chr . (+1) . ord) tricky1 The expression tr_test1 prints *GMap> tr_test1 Tricky True 'b' as one would expect. However, tr_test2 prints: *GMap> tr_test2 Tricky 'b' 'c' Here the transforming function is applied to *both* the functor argument and the Char value. The argument of gmap is not only applied to the functor argument but also to 'b' in "tricky1". This problem was already pointed out by Claus. It is important to remark that although gmap passes the test in the benchmark, it does not behave like a functorial map. Furthermore, this trick cannot be reused to implement the crush test. Nevertheless, it is surprising (to me at least) that you can implement something close to type changing map, using only SYB's cast operation. I am going to update the "Comparing libraries" paper to refer to your and Claus' variants of gmap. > > > There are no unsafe operations used; there is not a single occurrence > of insafePerformIO and the ilk. Incidentally, Data.Generics could, in > principle at least, be implemented using a safe cast. Of course gmap > is not a total operation: one can't really expect (gmap id True) to > produce a character. It seems that gunfold is not total anyway. Yes. The more flexible type turns what would have been type errors into strange runtime behaviour: *GMap> gmap (id::Maybe () -> Maybe ()) (Just ()) :: Bool *** Exception: SYB1_2/GMap.lhs:(20,26)-(21,45): Non-exhaustive patterns in case *GMap> gmap (id::Maybe () -> Maybe ()) (Nothing::Maybe ()) :: Bool False Cheers, Alexey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080721/4e883a98/attachment.htm From simonpj at microsoft.com Mon Jul 28 12:46:37 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Mon Jul 28 12:36:24 2008 Subject: [Hs-Generics] Owning SYB In-Reply-To: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> Dear Generic folk [I'm spamming libraries@haskell.org too, in case anyone interested in generics is not on generics@haskell.org.] As you know, Claus has offered a somewhat-detailed proposal for changes to the SYB library (below). But I don't think that we have an active maintainer for any of the generic-programming libraries (esp SYB) apart from Uniplate. Then there's the related question of what generic-programming technology to promote for clients of the GHC API. The obvious candidates are Claus himself, or Alexey Rodriguez, or Thomas Schilling; but perhaps there are others too? Maybe no one has stepped forward because you all think that I'm on the job! But I'm not... I'm busy with GHC itself, and would love a maintainer for SYB and associated gubbins. I fear that otherwise we may lose the benefits of Claus's homework. Simon | -----Original Message----- | From: generics-bounces@haskell.org [mailto:generics-bounces@haskell.org] On Behalf Of Claus Reinke | Sent: 21 July 2008 14:16 | To: generics@haskell.org | Subject: [Hs-Generics] Data.Generics with GPS (using Maps to avoid getting lost in Data) | | | summary: speed up Syb with Uniplate-inspired techniques | | == Performance issues in Syb == | | As it stands, classic Syb, while well-supported in GHC isn't exactly the | fastest generic programming option. In fact, it routinely seems to come | out last in performance comparisons, sometimes by a substantial factor. | | That isn't always a problem in practice, because the gains in | expressiveness/conciseness/maintainability outweigh the loss in | performance, because traversal performance is not the bottleneck, or | because practical use often involves hand-tuned traversal schemes | instead of the example schemes typically used in benchmarks. | | Be that as it may, performance is a consideration, negative results have | been published, and it seems that performance of experimental code has | led to Syb being abandoned in at least one project. While it would be | unrealistic to expect Syb traversals to compete with hand-written ones | with current compilers, there are several obvious areas where Syb | traversal performance could be subjected to improvements: | | (a) traversing irrelevant parts of the structure (because everything is | treated generically) | (b) combining results in simple, but inefficient ways ((++) nesting with | the structure) | (c) repeated runtime type checks to determine which function in a | generically extended transformation/query to apply | | While runtime type checks are inherent in SYB (and alternatives have | been proposed that claim to avoid them), all of a/b/c can be addressed | to some extent in defining tuned traversals using the basic library. | There tends to be a trade-off for a vs c, as avoiding senseless generic | traversals of specific substructures implies additional runtime type | checks to identify those substructures [...rest snipped...] From mads_lindstroem at yahoo.dk Mon Jul 28 13:49:51 2008 From: mads_lindstroem at yahoo.dk (Mads =?ISO-8859-1?Q?Lindstr=F8m?=) Date: Mon Jul 28 13:47:49 2008 Subject: [Hs-Generics] Owning SYB In-Reply-To: <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> Message-ID: <1217267391.4486.4.camel@localhost.localdomain> Hi, Simon Peyton-Jones wrote: > Dear Generic folk > > [I'm spamming libraries@haskell.org too, in case anyone interested in > generics is not on generics@haskell.org.] > > As you know, Claus has offered a somewhat-detailed proposal for > changes to the SYB library (below). But I don't think that we have an > active maintainer for any of the generic-programming libraries (esp > SYB) apart from Uniplate. Then there's the related question of what I do not know how much maintaining is going on, but at least the HappS folks have packaged SYB and put it on Hackage: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class-0.4 Greetings, Mads From claus.reinke at talk21.com Mon Jul 28 14:01:37 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Mon Jul 28 13:51:24 2008 Subject: [Hs-Generics] Re: Owning SYB References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> Message-ID: <017701c8f0db$fc721c60$3d058351@cr3lt> >[I'm spamming libraries@haskell.org too, in case anyone interested in generics >is not on generics@haskell.org.] Given the low number of responders on generics@, it may well be easier to continue on libraries@, cc-ing anyone on generics@ who isn't on libraries@. >As you know, Claus has offered a somewhat-detailed proposal for changes to the >SYB library (below). But I don't think that we have an active maintainer for any >of the generic-programming libraries (esp SYB) apart from Uniplate. Then there's >the related question of what generic-programming technology to promote for clients >of the GHC API. Thanks for raising this, Simon. I've actually been holding an email summarizing several issues (not just performance of default traversal schemes) that I'd like to see adressed in Syb (holding because the Syb authors were/are away, and because my performance improvement experiments are currently stuck on a GHC optimization issue). I'll send that email separately now. >The obvious candidates are Claus himself, or Alexey Rodriguez, or Thomas >Schilling; but perhaps there are others too? Maybe no one has stepped forward >because you all think that I'm on the job! But I'm not... I'm busy with GHC itself, >and would love a maintainer for SYB and associated gubbins. I fear that >otherwise we may lose the benefits of Claus's homework. I'm quite willing to continue pursueing the issues I've raised until I can make concrete suggestions for improving Syb, including summing up the code changes I've been adding to my various messages (certainly there should be patches to accompany proposal tickets in the library process, and I should collect all the strands of text into a single document). I have been waiting for the original Syb authors to return from their well-earned summer camps, but there should probably be a Wiki page somewhere specifically for discussing Syb-related issues and solutions (meanwhile, I've started collecting links/info related to GHC Api type traversals here, including the main Syb issues: http://hackage.haskell.org/trac/ghc/wiki/GhcApiAstTraversals , please feel free to copy stuff from there to a Syb-specific page). But I wouldn't want to take on ownership of Syb at this point, for two reasons, both motivational: (a) it helps to have someone else to "blame" when the consequences of gfoldl's type once again hurt my brain;-), (b) it is really frustrating to get so little interest in these issues, well, we haven't even managed to start a proper discussion on any of the lists I've tried, and as long as there is a Syb owner other than myself, at least I won't be talking entirely to myself!-) Claus From claus.reinke at talk21.com Mon Jul 28 14:13:08 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Mon Jul 28 14:02:55 2008 Subject: [Hs-Generics] Syb Renovations? Issues with Data.Generics Message-ID: <018401c8f0dd$98126480$3d058351@cr3lt> Calling all Syb/Data.Generics users!-) I keep running into problems with Data.Generics, mostly because I actually want to use it (no claims that it is the best or final solution, or that other approaches aren't equally in need of support, just that it is the best-supported working approach right now). Some tricky issues are (sometimes against published expectations) solvable, suggesting useful additions to the library, but some seemingly trivial things have me stumped, suggesting (to me, at least;-) a need for improvements either in the library or in its documentation. Part of the reason I'm interested in this now is that Data/Typeable instances seem likely (I hope:-) to be added to the GHC Api, where Thomas Schilling is working on improvements http://hackage.haskell.org/trac/ghc/wiki/GhcApiStatus http://hackage.haskell.org/trac/ghc/wiki/GhcApiAstTraversals also, the old question of porting HaRe to the GHC Api is currently being looked into again, by Chadda? Fouch?, and crucially depends on Syb's generic traversals. As it is still holiday season, it is a bit early for proposal deadlines, but I'd like to start a discussion of Syb/Data.Generics and collect the issues and solutions arising, in the hope of following up with concrete proposals for improvements. To start the discussion, a simple item: 1. inconvenient convenience instances of Data for non-"data" types Data.Generics.Instances defines instances of Data for many types, including some abstract types that don't really fit into the concrete value based model of Data, like 'IO a' and 'a->b'. Those instances give runtime errors for some class methods, and mainly offer faked (no-op) gmap traversals, serving as a convenience/enabler for 'deriving instance Data': http://www.haskell.org/pipermail/generics/2008-June/000346.html A list of the odd instances in Data.Generics.Instances, with examples of their oddities, can be found here: http://www.haskell.org/pipermail/generics/2008-June/000347.html My suggestion is to split this module into two, and stop the implicit import/export of the incomplete instances from Data.Generics. Reactions to this suggestion have been muted so far (Simon PJ was as surprised as I was about the existence of these instances, but has no strong opinion about the issue, Alexey Rodriguez supports the suggestion, Ian Lynagh points out the difficulty of transition), which is one reason why I'll try to move the discussion to libraries@. Pro: - the instances are still available, and only one explicit import away, so 'deriving instance Data' for types containing uninteresting functions is still convenient - the problematic instances are no longer implicitly imported, so applications that don't want these instances can now avoid them completely, or define their own instances - these convenience instances are not just inconvenient for some applications, due to the way intances are handled in Haskell; they actually violate some "natural" invariants like "everything queries every substructure of the specified type", "everywhere applies a transformation at every substructure of matching type" - the situation is similar to Text.Show.Functions, as the convenience instances don't provide the full expected functionality, just barely enough for deriving to get by Cons: - due to the implicit import and use of these instances, there is no obvious transition scheme; it seems that the least painful process would be to make the change without transition/deprecation period and to document the explicit import option [it would be useful to have a way of deprecating instance imports, so that any deriving scheme depending on imports from a deprecated location would trigger a warning, in this case suggesting the new import location] As I said, I'd like to wait until at least the Syb authors are back from holidays before setting any proposal deadlines, but I'd like to invite feedback from Syb users on this and other Syb issues. Here is a preview on other items I'd like to raise later on, please add your own: 2. Data.Generics.Utils Since Data/Typeable are compiler-derivable (in GHC) while other classes like Functor/Traversable/etc are not, it would be useful if generic instances for those other classes could be defined in terms of Data/Typeable. The Uniplate library already does this for its own classes via Data.Generics.PlateData, and it appears that at least Functor is defineable as well (code exists, proof is only informal at this stage, and those invariant violations and runtime errors in the implicitly imported dummy instances from (1) really get in the way): http://www.haskell.org/pipermail/generics/2008-June/000343.html http://www.haskell.org/pipermail/generics/2008-July/000349.html http://www.haskell.org/pipermail/generics/2008-July/000351.html What other classes can be defined in this way? Traversable (traverse) seems very nearly possible, what else? 3. Performance Naive use of Syb traversal schemes can lead to huge performance losses. Experienced users tend to write their own traversal schemes, using Syb's low-level Api directly, but we can take inspiration from some Uniplate/PlateData optimization techniques and generalise them for use with Syb's high-level traversal scheme Api, yielding similar performance gains for everywhere/everything: http://www.haskell.org/pipermail/generics/2008-July/000353.html Another direction that might be worth exploring is to use Maps instead of nested generic extensions to define adhoc-overloaded transformation and queries (I've actually started playing with that, but am currently stuck on GHC ticket #2463). 4. Useability There is probably nothing one can do to make the types of Syb's low-level Api less of a brain hazard, but not all of the stumbling blocks seem to be necessary consequences of the carefully crafted edifice of interactions between nearly polymorphic types, runtime type checks and type reflection. Examples: - there doesn't seem to be a way to get hold of a types' constructors, only of constructor representations, structure scaffolds, and structure generators - the actual domain on which a transformation/query acts is hidden behind the near-polymorphic default type of generic extensions - I can't seem to figure out how to use typeOf1, when the other Syb operations only give me 'forall a . Data a => a'; instead, I seem to be forced to use something like: [ mkTyConApp tyCon (init tyArgs) | not (null tyArgs) ] where (tyCon,tyArgs) = splitTyConApp typeRep - others? What are your personal gripes with Syb/Data/Typeable, and for which of them do you see a chance of addressing them by changing/adding code? Claus From johanj at cs.uu.nl Mon Jul 28 17:37:30 2008 From: johanj at cs.uu.nl (Johan Jeuring) Date: Mon Jul 28 17:27:17 2008 Subject: [Hs-Generics] Re: Owning SYB In-Reply-To: <017701c8f0db$fc721c60$3d058351@cr3lt> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> Message-ID: <4998C44D-7B83-416A-83EA-5BBC3069DEE8@cs.uu.nl> Dear Claus, Simon and others, >> [I'm spamming libraries@haskell.org too, in case anyone interested >> in generics is not on generics@haskell.org.] > > Given the low number of responders on generics@, it may well be easier > to continue on libraries@, cc-ing anyone on generics@ who isn't on > libraries@. I know we have been rather quiet, but please keep mailing to the generics list. As a reaction to Simon's request, we have been discussing (among ourselves) whether or not to take over SYB support in Utrecht. At the moment we are working on a release of a version of EMGM, and a new library base on type families and GADTs (about which we have written a TR available via http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html) . I think the general feeling is that it would be a good thing to also release/maintain SYB, but we want to talk about this a bit more, and right now I am in Birmingham for a conference. I'll be back in Utrecht on Thursday, and we can discuss it then. Hopefully we can respond shortly after Thursday. Kind regards, Johan >> As you know, Claus has offered a somewhat-detailed proposal for >> changes to the SYB library (below). But I don't think that we have >> an active maintainer for any of the generic-programming libraries >> (esp SYB) apart from Uniplate. Then there's the related question >> of what generic-programming technology to promote for clients of >> the GHC API. > > Thanks for raising this, Simon. I've actually been holding an email > summarizing several issues (not just performance of default > traversal schemes) that I'd like to see adressed in Syb (holding > because the Syb authors were/are away, and because my performance > improvement experiments are currently stuck on a > GHC optimization issue). I'll send that email separately now. > >> The obvious candidates are Claus himself, or Alexey Rodriguez, or >> Thomas Schilling; but perhaps there are others too? Maybe no one >> has stepped forward because you all think that I'm on the job! But >> I'm not... I'm busy with GHC itself, and would love a maintainer >> for SYB and associated gubbins. I fear that otherwise we may lose >> the benefits of Claus's homework. > > I'm quite willing to continue pursueing the issues I've raised until > I can > make concrete suggestions for improving Syb, including summing up the > code changes I've been adding to my various messages (certainly there > should be patches to accompany proposal tickets in the library > process, and I should collect all the strands of text into a single > document). > I have been waiting for the original Syb authors to return from > their well-earned summer camps, but there should probably be a Wiki > page somewhere specifically for discussing Syb-related issues and > solutions > (meanwhile, I've started collecting links/info related to GHC Api > type traversals here, including the main Syb issues: > http://hackage.haskell.org/trac/ghc/wiki/GhcApiAstTraversals , please > feel free to copy stuff from there to a Syb-specific page). > > But I wouldn't want to take on ownership of Syb at this point, for > two reasons, both motivational: (a) it helps to have someone else to > "blame" when the consequences of gfoldl's type once again hurt my > brain;-), > (b) it is really frustrating to get so little interest in these > issues, well, we haven't even managed to start a proper discussion > on any of the lists > I've tried, and as long as there is a Syb owner other than myself, > at least I won't be talking entirely to myself!-) > > Claus > > _______________________________________________ > Generics mailing list > Generics@haskell.org > http://www.haskell.org/mailman/listinfo/generics From ndmitchell at gmail.com Mon Jul 28 17:50:10 2008 From: ndmitchell at gmail.com (Neil Mitchell) Date: Mon Jul 28 17:39:53 2008 Subject: [Hs-Generics] Re: Owning SYB In-Reply-To: <017701c8f0db$fc721c60$3d058351@cr3lt> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> Message-ID: <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> Hi >> As you know, Claus has offered a somewhat-detailed proposal for changes to >> the SYB library (below). But I don't think that we have an active >> maintainer for any of the generic-programming libraries (esp SYB) apart from >> Uniplate. Then there's the related question of what generic-programming >> technology to promote for clients of the GHC API. > > Thanks for raising this, Simon. I've actually been holding an email > summarizing several issues (not just performance of default traversal > schemes) that I'd like to see adressed in Syb (holding because the Syb > authors were/are away, and because my performance improvement experiments > are currently stuck on a > GHC optimization issue). I'll send that email separately now. I think SYB would best be maintained by someone who does not already maintain some kind of boilerplate removal library. There are lots of experiments into GADT's and other mechanisms, but SYB1+2 is a very useful design point - and one that should be preserved. I think the maintainer should do three things in addition to general release management: 1) Speed improvements, if possible 2) API tweaks, maybe a few extra functions (universe equivalent would be nice) 3) Make it work with Hugs - I've always been surprised that SYB doesn't work with Hugs, and I don't think its that much work. As a result of these points, I think Claus is probably the perfect person to take over as maintainer. > (a) it helps to have someone else to "blame" > when the consequences of gfoldl's type once again hurt my brain;-), You can still blame those that went before - I don't think an SYB maintainer should be changing the type of gfoldl - its too fundamental. > (b) it is really frustrating to get so little interest in these issues, > well, we haven't even managed to start a proper discussion on any of the > lists I am interested. I have starred your emails, and will respond in the next few days. I've been in a tent without electricity for the last few days! Thanks Neil From oleg at okmij.org Tue Jul 29 03:51:35 2008 From: oleg at okmij.org (oleg@okmij.org) Date: Tue Jul 29 03:44:24 2008 Subject: [Hs-Generics] compositional gMap in SYB1 In-Reply-To: <006901c8dc2c$0c736520$50447ad5@cr3lt> Message-ID: <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> Hello! Claus wrote: > Perhaps we can combine our versions to get the best of both? I have done so. I have committed a new version of /comparison/SYB1_2/GMap.lhs with gmap2 of the type > gmap2 :: forall a b c . (Data a, Data b, > Data (c a), Data (c b), Data (c X)) => > (a -> b) -> c a -> c b The compositionality tests seem to pass. There are no unsafe operations or any unsafe extensions. Only standard Data.Typeable and Generics.Data operations are being used. The code also contains an optimization: if it is determined that a structured value has no components to map, no traversal is performed and the value is returned as it is. That should speed things up a little. The trick is producing something out of nothing. Too bad that the deadline to amend a Haskell workshop paper has passed, and so has the deadline for the generics workshop. Perhaps we should write a separate paper? I'm afraid I would be out of town again for two weeks and so would not be able to follow the discussion closely. Cheers, Oleg From johanj at cs.uu.nl Tue Jul 29 04:01:00 2008 From: johanj at cs.uu.nl (Johan Jeuring) Date: Tue Jul 29 03:50:42 2008 Subject: [Hs-Generics] compositional gMap in SYB1 In-Reply-To: <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> References: <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> Message-ID: > I have done so. I have committed a new version of > /comparison/SYB1_2/GMap.lhs > with gmap2 of the type Interesting. >> gmap2 :: forall a b c . (Data a, Data b, >> Data (c a), Data (c b), Data (c X)) => >> (a -> b) -> c a -> c b > > The compositionality tests seem to pass. There are no unsafe > operations or any unsafe extensions. Only standard Data.Typeable and > Generics.Data operations are being used. The code also contains an > optimization: if it is determined that a structured value has no > components to map, no traversal is performed and the value is returned > as it is. That should speed things up a little. The trick is producing > something out of nothing. > > Too bad that the deadline to amend a Haskell workshop paper > has passed, and so has the deadline for the generics workshop. Perhaps > we should write a separate paper? We're thinking about extending the Haskel Symposium paper to a journal version, in which we would also take efficiency into account (i.e., also try to set up something like a benchmark for generic programming). Would something like that be suitable you think? -- Johan From mrchebas at gmail.com Tue Jul 29 09:50:24 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Tue Jul 29 09:40:05 2008 Subject: [Hs-Generics] Re: compositional gMap in SYB1 In-Reply-To: <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> References: <006901c8dc2c$0c736520$50447ad5@cr3lt> <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> Message-ID: <4b39c80a0807290650p43ece36aub0a32a64333da592@mail.gmail.com> On Tue, Jul 29, 2008 at 9:51 AM, wrote: > > Hello! > > Claus wrote: > > Perhaps we can combine our versions to get the best of both? > > I have done so. I have committed a new version of > /comparison/SYB1_2/GMap.lhs > with gmap2 of the type > > > gmap2 :: forall a b c . (Data a, Data b, > > Data (c a), Data (c b), Data (c X)) => > > (a -> b) -> c a -> c b > > The compositionality tests seem to pass. There are no unsafe > operations or any unsafe extensions. Only standard Data.Typeable and > Generics.Data operations are being used. The code also contains an > optimization: if it is determined that a structured value has no > components to map, no traversal is performed and the value is returned > as it is. That should speed things up a little. The trick is producing > something out of nothing. That's a clever combination of both techniques. You are passing around an explicit type representation at run-time, to ensure that the transforming function applies only to X-positions. Also you don't pretend that there are X-values in there (as Claus' version did) so there is no need for unsafeCoerce. One question: the runtime checks in traverse are for internal consistency only, right?. I could not think of runtime errors arising from gmap2-calls. Furthermore, I think the issue of unfold errors is solved, right? (Although these properties are not known by the type system.) > > > Too bad that the deadline to amend a Haskell workshop paper > has passed, and so has the deadline for the generics workshop. Perhaps > we should write a separate paper? Yes, that's a pity. I will mention this in the presentation though (and update the technical report, which admittedly is rather delayed). > > > I'm afraid I would be out of town again for two weeks and so > would not be able to follow the discussion closely. > > Cheers, Alexey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080729/54f1332d/attachment.htm From mrchebas at gmail.com Tue Jul 29 10:10:00 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Tue Jul 29 09:59:41 2008 Subject: [Hs-Generics] Re: Owning SYB In-Reply-To: <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> Message-ID: <4b39c80a0807290710i486c9d8udb3fcf4c79f39f8e@mail.gmail.com> On Mon, Jul 28, 2008 at 11:50 PM, Neil Mitchell wrote: > Hi > > >> As you know, Claus has offered a somewhat-detailed proposal for changes > to > >> the SYB library (below). But I don't think that we have an active > >> maintainer for any of the generic-programming libraries (esp SYB) apart > from > >> Uniplate. Then there's the related question of what generic-programming > >> technology to promote for clients of the GHC API. > > > > Thanks for raising this, Simon. I've actually been holding an email > > summarizing several issues (not just performance of default traversal > > schemes) that I'd like to see adressed in Syb (holding because the Syb > > authors were/are away, and because my performance improvement experiments > > are currently stuck on a > > GHC optimization issue). I'll send that email separately now. > > I think SYB would best be maintained by someone who does not already > maintain some kind of boilerplate removal library. There are lots of > experiments into GADT's and other mechanisms, but SYB1+2 is a very > useful design point - and one that should be preserved. I agree that SYB1+2 is a useful design point and that improvements should not affect the basic functionality and design. A library with a different gfoldl would very likely change SYB significantly. However, I don't think that there is a problem with a person maintaining several generic programming libraries, as long as the maintainer effects changes/improvements on SYB conservatively. I think the > maintainer should do three things in addition to general release > management: > > 1) Speed improvements, if possible > > 2) API tweaks, maybe a few extra functions (universe equivalent would be > nice) > > 3) Make it work with Hugs - I've always been surprised that SYB > doesn't work with Hugs, and I don't think its that much work. > > As a result of these points, I think Claus is probably the perfect > person to take over as maintainer. > > > (a) it helps to have someone else to "blame" > > when the consequences of gfoldl's type once again hurt my brain;-), > > You can still blame those that went before - I don't think an SYB > maintainer should be changing the type of gfoldl - its too > fundamental. > > > (b) it is really frustrating to get so little interest in these issues, > > well, we haven't even managed to start a proper discussion on any of the > > lists > > I am interested. I have starred your emails, and will respond in the > next few days. I've been in a tent without electricity for the last > few days! I would be fine with Claus as maintainer. As Johan mentioned, there are also people in Utrecht who are interested in contributing with development or even willing to take over maintainership if necessary. Probably it is best to let Johan&co say more on Thursday :). On a personal note, I prefer to not volunteer as a maintainer of SYB since I have other personal priorities at the moment (focus on writing thesis, searching for jobs), which also explains my slow reactions to emails. Cheers, Alexey -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080729/6569f78c/attachment-0001.htm From mrchebas at gmail.com Tue Jul 29 10:35:42 2008 From: mrchebas at gmail.com (Alexey Rodriguez) Date: Tue Jul 29 10:25:26 2008 Subject: [Hs-Generics] Re: gMap in SYB1 In-Reply-To: <006901c8dc2c$0c736520$50447ad5@cr3lt> References: <20080701114633.3E07FAA92@Adric.metnet.fnmoc.navy.mil> <03a001c8dbc8$192ed1a0$4c177ad5@cr3lt> <006901c8dc2c$0c736520$50447ad5@cr3lt> Message-ID: <4b39c80a0807290735r553912d6o299d60038d6ff88a@mail.gmail.com> Hi Claus, On Wed, Jul 2, 2008 at 12:11 PM, Claus Reinke wrote: > > ps. gmap itself is somewhat tricky to use, depending on > type information from the context, and failing with runtime errors if > that type information doesn't match. > > *GMap> gmap not (True,True) :: (Bool,Char) > (False,*** Exception: gunfold > > But then, SYB generally operates at the borders > of type safety (what one would expect to be a > type error often just leads to unexpected behaviour > - there are so few gaps between well-typed SYB > programs that the type system, instead of noting > that things go wrong, can only cause things to go > elsewhere). > > Are the other generic programming libraries better > behaved in this area? A small (and late) reaction to this point. The reason that this particular SYB gmap behaves badly is that deserialization is partial. You will have a runtime error (or exceptional return value) if you deserialize an input as a value of type X while it has been serialized as a value of type Y. Since this version of gmap is based on serialization followed by deserialization, you will obtain such errors when demanding incompatible types (as in your example). I think that Oleg's latest version enforces additional type safety (by means of the gmap2 wrapper) so this problem is no longer present. Other libraries (for example EMGM and RepLib) represent type constructors explicitly, so gmap can be implemented directly without the need of serialization/deserialization. There are no such runtime problems in those approaches. There are other examples of serialization-based generic functions: the implementation of generic transpose by Norell and Jansson [1], and also the generic conversion work by Atanassow and Jeuring[2]. Cheers, Alexey [1] http://citeseer.ist.psu.edu/649955.html [2] http://citeseer.ist.psu.edu/583900.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080729/d3217ac2/attachment.htm From claus.reinke at talk21.com Tue Jul 29 15:25:02 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Tue Jul 29 15:15:00 2008 Subject: [Hs-Generics] Re: compositional gMap in SYB1 References: <20080729075135.D3C1DAA92@Adric.metnet.fnmoc.navy.mil> Message-ID: <027701c8f1b0$d3dad050$71338351@cr3lt> > Hello! Hello again!-) > Claus wrote: >> Perhaps we can combine our versions to get the best of both? > > I have done so. I have committed a new version of > /comparison/SYB1_2/GMap.lhs [for readers of libraries@: that is in the generics code repo at http://darcs.haskell.org/generics/comparison/SYB1_2/GMap.lhs , look for gmap2/gmapt and comments; the topic is defining Functor fmap generically (hence gmap) in terms of Data/Typeable] Ah, of course! Silly me, if I had followed my own choice of metaphor to the end, I would have known that X marks the spot only on a map, not on the real thing (or else everyone could find the treasure..). Your shallow, incrementally built not-quite copy is just another representation of a map, and since both map and X are hidden from users of the function, everyone is happy. That something out of nothing trick is also used in the PlateData optimization and in my adaptation of it. The difference, apart from choice of representation, is mainly in whether to extract substructure placeholders/types on a type basis once per traversal or on a value basis, incrementally. Here is a re-run of the technique, for defining traverse in terms of Data/Typeable. Apart from being useful for Data.Traversable library users, the type of 'traverse id' is similar to the type of transpose, which should interest the generics readers. {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE ExistentialQuantification #-} {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE RankNTypes #-} import Control.Applicative import Data.Generics data X = X deriving (Data,Typeable) traverseData :: forall f a b t . (Applicative f,Typeable1 f,Typeable a,Typeable b, Data (t a),Data (t b),Data (t X)) => (a -> f b) -> t a -> f (t b) traverseData f = traverseWithMap f (Dyn (undefined::t X)) where traverseWithMap :: forall f a b x y . (Applicative f,Typeable1 f,Typeable a,Typeable b,Data x,Data y) => (a -> f b) -> Dyn -> x -> f (y) traverseWithMap f (Dyn t) x | typeOf t==typeOf X = maybe (error "traverseData: non-applicable type marked") ($x) (cast f) | otherwise = rebuildWith (traverseWithMap f) t1 x where t1 = Dyn (fromConstr (toConstr x) `asTypeOf` t) rebuildWith :: forall f x y . (Applicative f,Data x,Data y) => (forall x y . (Data x,Data y) => Dyn -> x -> f y) -> Dyn -> x -> f y rebuildWith f (Dyn t) x = case gunfold (k f) (\g -> UnfldStateF2 (pure g) tkids kids) con of UnfldStateF2 a [] [] -> a where (tcon,tkids) = dynamize t (con,kids) = dynamize x k :: forall a b . (Data a) => (forall x y . (Data x,Data y) => Dyn -> x -> f y) -> UnfldStateF2 f (a->b) -> UnfldStateF2 f b k f (UnfldStateF2 ca (tkid:tkids) ((Dyn kid):kids)) = UnfldStateF2 (ca <*> (f tkid kid)) tkids kids data UnfldStateF2 f a = UnfldStateF2 (f a) [Dyn] [Dyn] data Dyn = forall a. Data a => Dyn a data Kids a = Kids{growUp:: [Dyn]} dynamize :: Data a => a -> (Constr,[Dyn]) dynamize x = (toConstr x, growUp $ gfoldl f (const (Kids [])) x) where f (Kids l) a = Kids (l ++ [Dyn a]) test = do print $ traverseData (Just . not) tuple print $ traverseData (Just . not) list traverseData print tuple >>= print traverseData print list >>= print where tuple = (True,True) list = [True,True] Btw, while the type of 'traverseData id' matches that of transpose, its behavious doesn't quite, depending on example: *Main> :t traverseData id traverseData id :: (Data (t X), Data (t b), Data (t (f b)), Typeable b, Typeable1 f, Applicative f) => t (f b) -> f (t b) *Main> traverseData id [Just x|x<-[1..3]] Just [1,2,3] *Main> traverseData id [[1..3],[4..6]] [[1,4],[1,5],[1,6],[2,4],[2,5],[2,6],[3,4],[3,5],[3,6]] > optimization: if it is determined that a structured value has no > components to map, no traversal is performed and the value is returned > as it is. That should speed things up a little. The trick is producing > something out of nothing. That optimization is another indication of the relation to the substructure types map building, although pleasingly different in form (always nice to learn new tricks;-). > Too bad that the deadline to amend a Haskell workshop paper > has passed, and so has the deadline for the generics workshop. Perhaps > we should write a separate paper? Sounds like a good idea. When you're back. > I'm afraid I would be out of town again for two weeks and so > would not be able to follow the discussion closely. Ah, you're just trying to hide the two-week round trips for lightspeed messages resulting from your space travels!-) Thanks for yet another useful contribution, Claus From claus.reinke at talk21.com Tue Jul 29 15:27:00 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Tue Jul 29 15:16:58 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt> <20080728222302.GA24020@matrix.chaos.earth.li> Message-ID: <027a01c8f1b1$1d1cef00$71338351@cr3lt> >> My suggestion is to split this module into two, and stop the implicit >> import/export of the incomplete instances from Data.Generics. > I don't think that this is a good idea. Could you please elaborate on your reasons? >> Pro: - the instances are still available, and only one explicit import >> away, so 'deriving instance Data' for types containing >> uninteresting functions is still convenient > But suppose you want to make such an instance in a library you write. > Then you will import the instance, which means that it will be visible > to all users of your library too, so they cannot define their own > instance. Nor can they use other libraries that define their own > instance. True. But the current situation is even worse: we either get all instances (good and bad) or none, and to get none, we'd have to jump through hoops (not using Data.Generics, but importing its component modules directly; and anyone trying to mix such code with code importing Data.Generics is going to be in trouble). There might well be a better way of making deriving happy, without such nasty side effects, but it will have to start with not having those faulty instances by default. > I believe instances really do have to be global. Instances are global. The only control we have are over the class we define instances of and the type we define instances for. The instances in question are of a non-local class, for non-local types, so if they are implicitly imported, we're stuck with them. The instances I don't want to be stuck with turn compile-time errors into runtime errors for some methods, and violate nice invariants for other methods. Claus From claus.reinke at talk21.com Tue Jul 29 20:28:33 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Tue Jul 29 20:18:20 2008 Subject: [Hs-Generics] Re: Owning SYB References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> Message-ID: <035901c8f1db$34673510$71338351@cr3lt> Hi Neil, personally, I think that historical preservation of a reference approach is quite a different issue (hosting a copy of the Syb page at haskell.org, and a copy of the library on hackage, would be a start), which might also need a maintainer at some point, but Syb1+2 is part of base right now, and available from Ralf's old pages. So far, it does seem as if most of the items I'm interested in can be done on top of Data.Generics, simply bypassing the higher-level API and providing an alternative, but very close, higher-level API on top of the same low-level API (though I sometimes wonder whether gfoldl's second parameter should be generic rather than polymorphic). But what it comes down to is that I'd like to start with a working, well supported approach (Syb 1+2), and see whether we can close any of the known gaps without starting from scratch with yet-another- generics-library. And if that means morphing what is there into something even more useful, by taking inspirations from Syb 4 or Uniplate or .., I'm all for it, as long as it is a continuous evolution supported by evidence, not a heart-liver-and-lung transplant supported by hope. > 1) Speed improvements, if possible What I'm working on are mostly more convenient access to better performance in the higher-level API (traversal schemes), reducing the need for hand-tuned traversals using the low-level API directly. The part inspired by Uniplate's PlateData seems to be working, the part about replacing nested typecases with Map lookup is currently burried in other effects. > 2) API tweaks, maybe a few extra functions (universe equivalent would be nice) It seems we've got fmap and traverse defineable in terms of Data/ Typeable, so one could derive the latter two, then get the former for free, so to speak. But should these be in some Data.Generics.Utils, or should they move into Data.Traversable, etc (which already has some default functions for defining instances of one class in terms of another)? And if you mean {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE RankNTypes #-} import Data.Generics universe :: forall a . Data a => a -> [a] universe = everything (++) ([] `mkQ` child) where child = return :: a -> [a] then we're probably just talking about better performance from naive definitions (1) again, or is anything else wrong with that definition? > 3) Make it work with Hugs - I've always been surprised that SYB > doesn't work with Hugs, and I don't think its that much work. Hmm, I'm still fond of Hugs, but I haven't used it much recently, so I'd be the wrong person for that job. On casual glance, I can't even think of any Syb-essential language features that aren't supported in Hugs (apart from deriving Data/Typeable), and my old WinHugs doesn't seem to have/support a cpp, which gets in the way of just loading the code - why doesn't Syb work with Hugs? 4) Improve useability Things like 'typeOf1' not working with 'Data a => a', or 'gzipWithT' giving fun type errors unless we eta-expand its first parameter, Syb as a general test-bed for the quality of type error messages, etc. Here, I'm not thinking of immediate cure-alls, but of collecting the various issues, creating tickets and/or a Wiki page and looking for ways out, step by step. One item I haven't mentioned yet: can't we replace the gensym-based TypeRepKeys with something more systematic/standardised? I've wondered about this on previous occasions, to make TypeRepKeys more portable, but it would also be nice just to get rid of that IO tag (reminding us that the keys may change with each program run). Thanks for your confidence, but I'll probably just collect feedback here, contribute my code/docs when I've got everything together (would a separate syb-utils package be preferred, or direct changes to base?) and move on. I look forward to your comments, though, when you get out of that tent!-) Claus From oleg at okmij.org Wed Jul 30 04:14:50 2008 From: oleg at okmij.org (oleg@okmij.org) Date: Wed Jul 30 04:18:07 2008 Subject: [Hs-Generics] Re: compositional gMap in SYB1 In-Reply-To: <027701c8f1b0$d3dad050$71338351@cr3lt> Message-ID: <20080730081450.33566AA88@Adric.metnet.fnmoc.navy.mil> Hello! Alexey wrote: > One question: the runtime checks in traverse are for internal consistency > only, right?. I could not think of runtime errors arising from gmap2-calls. > Furthermore, I think the issue of unfold errors is solved, right? (Although > these properties are not known by the type system.) Yes, this is correct. Although gmap2 could be given a more general type, like (a -> b) -> f x -> f y in which case the run-time errors are significant. Such errors will arise when mapping id on [Int] and expect [Bool] in the result. The mapping will succeed btw on empty lists. Alexey wrote: > I will mention this in the presentation though (and > update the technical report, which admittedly is rather delayed). Thank you! Perhaps the journal version of the comparison paper (which several people suggested) could discuss the issue in some detail (at least from the perspective of performance and static-dynamic trade-off). It seems there might be a possibility of a five-minute presentation about gmap during the `Hot Topics' discussion at the Generic Programming Workshop in Victoria in September -- provided the organizers will schedule the hot topics session and deem gmap hot enough. Should we inquire the workshop organizers? I will be back on Aug 18. I was in Japan and Taiwan in July, and will be in Germany (Hamburg) in August. I guess I get a chance to compare German trains with Japanese ones. Claus wrote: > if I had followed my own choice of metaphor to the end, I would have > known that X marks the spot only on a map, not on the real thing (or > else everyone could find the treasure..). I really like that metaphor! Cheers, Oleg From claus.reinke at talk21.com Wed Jul 30 19:11:28 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Wed Jul 30 19:11:46 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt> <20080728222302.GA24020@matrix.chaos.earth.li> <027a01c8f1b1$1d1cef00$71338351@cr3lt> <20080729195944.GA15169@matrix.chaos.earth.li> Message-ID: <026f01c8f299$9f511b70$36168351@cr3lt> > On Tue, Jul 29, 2008 at 08:27:00PM +0100, Claus Reinke wrote: >> >> My suggestion is to split this module into two, and stop the implicit >> >> import/export of the incomplete instances from Data.Generics. >> >I don't think that this is a good idea. >> Could you please elaborate on your reasons? > That's what the rest of that e-mail was supposed to be. You explained why the change would not give as much flexibility as one might think, or at least not as easily, but you didn't explain why you think it is a bad idea to gain at least the flexibility to choose between instance and no instance for the problematic cases. >> True. But the current situation is even worse: we either get all >> instances (good and bad) or none > I think you should always get all: That would be fine if all instances were completely defined. They aren't. So the partial instances are imposed globally and irrevocably, and instead of compiletime type errors, we get runtime errors. > By the way, is there something somewhere describing the alternate > instance that you want to define? That is the whole point, isn't it? The Data framework isn't designed to cope with things like (a->b) or (IO a), so there are no good instances one could define for these types (if anyone can suggest better instances, please do!-). Hence the incomplete instances mixed in with the standard ones in Data.Generics.Instances. Mostly, I don't want those instances at all (the incomplete ones), so that the typechecker will complain if I try to use Data on something it can't really handle. Scenario 1: We want to use deriving Data on types that have components of types (a->b) or (IO a), but we don't care about those components. This is the case for which the incomplete instances are provided. Scenario 2: Any attempt to use Data (a->b) or Data (IO a) indicates an error. If we want to derive Data for complex structures containing those types, we need to define Data instances for the immediately enclosing structures, or wrap those types in newtypes and define Data instances for those. This is the case for which the incomplete instances get in the way. Scenario 3: We want to use deriving Data on types that have components of types (a->b) or (IO a), we do care about what happens in those components. It is surprisingly tricky to come up with sensible Data instances for these types that do anything more than the current dummies, so this scenario isn't as likely as I thought at first. Scenario 4: We want to handle Data for (a->b) or (IO a) differently, depending on context. Unless we can wrap those types in newtypes, this is very nearly impossible, due to the way instances propagate through projects. The status quo supports only (1), and gives a mixture of runtime errors and wrong results for (2). In particular, the type checker does not help us to find the cases we need to cover to keep our programs from "going wrong". With selective import, we can support (1), or get compiletime errors consistently for (2). We cannot usually support both (1) and (2) in one program, but splitting the Instances module so that we can be more selective in imports seems a worthwhile improvement. Claus PS. The situation is not improved by the current reexports of Data.Generics.Instances from unexpected places. I have a package splitting Data.Generics.Instances into Standard and Dubious, and a Data.Generics.Alt that only reexports Standard instances. But as soon as I use this with, eg, an IntMap, I get duplicate instance errors. A quick grep shows that the following re-export all instances (sometimes deliberately, sometimes accidentally, by importing Data.Generics for other reasons): libraries/array/Data/Array.hs -- libraries/base/Data/Generics/Instances.hs libraries/base/Data/Generics.hs libraries/bytestring/Data/ByteString/Internal.hs libraries/bytestring/Data/ByteString/Lazy/Internal.hs libraries/containers/Data/IntMap.hs libraries/containers/Data/IntSet.hs libraries/containers/Data/Map.hs libraries/containers/Data/Set.hs libraries/containers/Data/Tree.hs libraries/haskell-src/Language/Haskell/Syntax.hs libraries/network/Network/URI.hs libraries/packedstring/Data/PackedString.hs libraries/template-haskell/Language/Haskell/TH/Quote.hs libraries/template-haskell/Language/Haskell/TH/Syntax.hs As far as I can see, none of these depends on the incomplete instances, so these instances get re-exported by accident. If Data.Generics.Instances was split into Data.Generics.Instances.Standard and Data.Generics.Instances.Dubious, and if Data.Generics.Alt only reexported the former, those modules could be more selective in their imports and the leaking of instances could be avoided. From claus.reinke at talk21.com Thu Jul 31 06:36:02 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Thu Jul 31 06:36:09 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt> <20080728222302.GA24020@matrix.chaos.earth.li> <027a01c8f1b1$1d1cef00$71338351@cr3lt> <20080729195944.GA15169@matrix.chaos.earth.li> <026f01c8f299$9f511b70$36168351@cr3lt> <20080731010538.GA20317@matrix.chaos.earth.li> Message-ID: <006101c8f2f9$3c39ab50$0b1b7ad5@cr3lt> >> That is the whole point, isn't it? The Data framework isn't designed >> to cope with things like (a->b) or (IO a), so there are no good instances >> one could define for these types > > OK, I think I've missed your point then. I don't seem to have explained it well - I wouldn't expect so much opposition otherwise!-) Perhaps, some concrete code examples will help (see below). > I don't see a benefit to moving the instances to their own module, which > outweighs the downsides, in my opinion. To recap: I'm suggesting to - split the existing Data.Generics.Instances into Data.Generics.Instances.Standard Data.Generics.Instances.Dubious - provide Data.Generics.Alt, which is Data.Generics without Data.Generics.Instances.Dubious > How do they "get in the way"? Do you mean the typechecker doesn't tell > you which instances you need to define by hand, because deriving worked? Okay, I've cobbled together a package with my various code fragments, for discussion purposes only: http://www.cs.kent.ac.uk/~cr3/tmp/syb/syb-utils-0.0.2008.7.30.tar.gz If you install that, and then try examples/Examples.hs, once as it is and once with -DALT, you will directly see the difference between the status quo and my suggested alternative: the former gives a mixture of happily working code, runtime errors and silently wrong results, the latter gives compiletime type errors for those examples that would otherwise go haywire by defaulting to use non-standard instances (tested with ghci 6.9.20080514, code & output below *). Does that help? Claus * you have to try the two alternatives in different ghc invocations, because of a long-standing ghc session bug that accumulates instances over all modules seen. -------------------------------------------- example code {-# LANGUAGE CPP #-} -- {-# OPTIONS_GHC -DALT #-} import Data.Generics.Utils #ifdef ALT import Data.Generics.Alt -- compiletime type errors #else import Data.Generics -- runtime errors, wrong results #endif import qualified Control.Exception as CE(catch) -------------------------------- examples test = do putStrLn "-- traverseData examples" print $ traverseData (Just . not) tuple print $ traverseData (Just . not) list traverseData print tuple >>= print traverseData print list >>= print print $ traverseData id [ Just x | x <- [1..3::Integer] ] print $ traverseData id [ [1..3], [4..6::Integer] ] putStrLn "-- fmapData examples" print $ fmapData not tuple print $ fmapData not list putStrLn "-- fmapData (a->b) (IO a) examples" safely (print $ map (($True) . fmapData not) ([]::[Bool->Bool])) safely (mapM (fmapData not) ([]::[IO Bool]) >>= print) safely (print $ map (($True) . fmapData not) ([const True]::[Bool->Bool])) safely (mapM (fmapData not) ([return True]::[IO Bool]) >>= print) putStrLn "-- everywhere over inconsistent instances examples" print $ everywhere (mkT inc) (return 0 :: Maybe Integer) print $ everywhere (mkT inc) (return 0 :: [] Integer) print =<< everywhere (mkT inc) (return 0 :: IO Integer) print $ everywhere (mkT inc) (return 0 :: (->) () Integer) () where inc = (+1) :: Integer -> Integer tuple = (True,True) list = [True,True] safely m = CE.catch m (putStrLn . ("exception: "++) . show) -------------------------------------------- example output $ ghc -e test Examples.hs -- traverseData examples Just (True,False) Just [False,False] True (True,()) True True [(),()] Just [1,2,3] [[1,4],[1,5],[1,6],[2,4],[2,5],[2,6],[3,4],[3,5],[3,6]] -- fmapData examples (True,False) [False,False] -- fmapData (a->b) (IO a) examples [] [] exception: gunfold exception: gunfold -- everywhere over inconsistent instances examples Just 1 [1] 0 0 $ ghc -DALT -e test Examples.hs Examples.hs:31:33: No instances for (Data (Bool -> Bool), Data (Bool -> Data.Generics.Utils.X)) arising from a use of `fmapData' at Examples.hs:31:33-44 Possible fix: add an instance declaration for (Data (Bool -> Bool), Data (Bool -> Data.Generics.Utils.X)) In the second argument of `(.)', namely `fmapData not' In the first argument of `map', namely `(($ True) . fmapData not)' In the second argument of `($)', namely `map (($ True) . fmapData not) ([] :: [Bool -> Bool])' Examples.hs:32:16: No instances for (Data (IO Bool), Data (IO Data.Generics.Utils.X)) arising from a use of `fmapData' at Examples.hs:32:16-27 Possible fix: add an instance declaration for (Data (IO Bool), Data (IO Data.Generics.Utils.X)) In the first argument of `mapM', namely `(fmapData not)' In the first argument of `(>>=)', namely `mapM (fmapData not) ([] :: [IO Bool])' In the first argument of `safely', namely `(mapM (fmapData not) ([] :: [IO Bool]) >>= print)' Examples.hs:39:12: No instance for (Data (IO Integer)) arising from a use of `everywhere' at Examples.hs:39:12-59 Possible fix: add an instance declaration for (Data (IO Integer)) In the second argument of `(=<<)', namely `everywhere (mkT inc) (return 0 :: IO Integer)' In a stmt of a 'do' expression: print =<< everywhere (mkT inc) (return 0 :: IO Integer) In the expression: do putStrLn "-- traverseData examples" print $ traverseData (Just . not) tuple print $ traverseData (Just . not) list traverseData print tuple >>= print .... Examples.hs:40:12: No instance for (Data (() -> Integer)) arising from a use of `everywhere' at Examples.hs:40:12-64 Possible fix: add an instance declaration for (Data (() -> Integer)) In the second argument of `($)', namely `everywhere (mkT inc) (return 0 :: (->) () Integer) ()' In the expression: print $ everywhere (mkT inc) (return 0 :: (->) () Integer) () In the expression: do putStrLn "-- traverseData examples" print $ traverseData (Just . not) tuple print $ traverseData (Just . not) list traverseData print tuple >>= print .... From isaacdupree at charter.net Thu Jul 31 11:07:15 2008 From: isaacdupree at charter.net (Isaac Dupree) Date: Thu Jul 31 11:07:23 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics In-Reply-To: <20080731010538.GA20317@matrix.chaos.earth.li> References: <018401c8f0dd$98126480$3d058351@cr3lt> <20080728222302.GA24020@matrix.chaos.earth.li> <027a01c8f1b1$1d1cef00$71338351@cr3lt> <20080729195944.GA15169@matrix.chaos.earth.li> <026f01c8f299$9f511b70$36168351@cr3lt> <20080731010538.GA20317@matrix.chaos.earth.li> Message-ID: <4891D523.4080805@charter.net> Ian Lynagh wrote: > If so, the first "pro" isn't a pro at all. Debatably it's a con - it > would make it less convenient to derive instance Data for types > containing IO etc. maybe in those cases of "convenience" we could use a tool like Data.Derive, and make some name to derive with it like "DubiousData". Then the dubious Data instances wouldn't have to be defined/imported. I wonder if it would be possible to have Derive have a "NotSoDubiousData" that somehow figured out whether dubious types were involved and rejected them.. (Admittedly, last I tried, using Derive or DrIFT for a project that wasn't using it already, was much harder than the built-in deriving mechanism) here's an example (which don't know enough to say whether it supports any particular position) : say we have a type data Something a = Something a (IO a) If we want instance (Data a) => Data (Something a) ignoring the IO member but traversing the "a" member, then if we use the normal deriving where we have dubious instances, then (Something (a->b)) is going to be in Data even if it shouldn't be, and it will be easy not to notice. Given how popular polymorphism is in Haskell, I guess this is fairly likely to happen? -Isaac From claus.reinke at talk21.com Thu Jul 31 11:25:03 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Thu Jul 31 11:25:09 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt><20080728222302.GA24020@matrix.chaos.earth.li><027a01c8f1b1$1d1cef00$71338351@cr3lt><20080729195944.GA15169@matrix.chaos.earth.li> <026f01c8f299$9f511b70$36168351@cr3lt> Message-ID: <00df01c8f321$9ceff120$0b1b7ad5@cr3lt> > That is the whole point, isn't it? The Data framework isn't designed > to cope with things like (a->b) or (IO a), so there are no good instances > one could define for these types (if anyone can suggest better instances, > please do!-). Hence the incomplete instances mixed in with the standard > ones in Data.Generics.Instances. Actually, one could try to improve at least the gmapT and related methods, since they are meant to map over immediate subterms: gmapT f fun = f . fun -- instead of gmapT f fun = fun gmapT f io = (return . f) =<< io -- instead of gmapT f io = io (we simply eta-expand until we can get hold of the "subterms", then apply f) While this would be more suitable for some applications of Data, such as everywhere (and everything, if we do the same for gmapQ), it still doesn't give consistent or complete instances of Data for functions or IO. But I thought I'd mention it an example of usefully different instances, rather than usefully absent instances for these types. Claus From claus.reinke at talk21.com Thu Jul 31 11:43:01 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Thu Jul 31 11:43:03 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt> <20080728222302.GA24020@matrix.chaos.earth.li> <027a01c8f1b1$1d1cef00$71338351@cr3lt> <20080729195944.GA15169@matrix.chaos.earth.li> <026f01c8f299$9f511b70$36168351@cr3lt> <20080731010538.GA20317@matrix.chaos.earth.li> <4891D523.4080805@charter.net> Message-ID: <00fb01c8f324$1e781ea0$0b1b7ad5@cr3lt> > maybe in those cases of "convenience" we could use a tool like > Data.Derive, and make some name to derive with it like "DubiousData". > Then the dubious Data instances wouldn't have to be defined/imported. > I wonder if it would be possible to have Derive have a "NotSoDubiousData" > that somehow figured out whether dubious types were involved and > rejected them.. generating instances of a different class might lead to ambiguities/ overlaps in use, but I like the idea of making deriving smarter to get rid of dummy instances. Let me see if I got your idea correctly. data Something a = Something a (IO a) Currently, we have two choices to get 'Data Something': - use naive deriving, for which we'd need to supply both 'Data a' and 'Data (IO a)', with unwanted consequences for the latter - write the Data instance for Something by hand, skipping the 'IO a' component If deriving was only a little bit smarter, we could write: data Something a = Something a {-# Data: skip #-} (IO a) deriving (Data,Typeable) and the deriving mechanism could give us a modified Data instance for Something that doesn't need 'Data (IO a)'. No need for instances we can't really write, and a clear hint in the source code that the 'IO a' component is treated specially (skipped rather than traversed)! I like the idea, Claus