From jpm at cs.uu.nl Fri Aug 1 02:36:37 2008 From: jpm at cs.uu.nl (=?ISO-8859-1?Q?Jos=E9_Pedro_Magalh=E3es?=) Date: Fri Aug 1 02:36:33 2008 Subject: [Hs-Generics] Re: Owning SYB In-Reply-To: <52f14b210807312328y39ff1ee3m725f479853d06b28@mail.gmail.com> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> <035901c8f1db$34673510$71338351@cr3lt> <52f14b210807312328y39ff1ee3m725f479853d06b28@mail.gmail.com> Message-ID: <52f14b210807312336t7dd9f8f6p33b6118124b6fa5b@mail.gmail.com> Hello all, As Johan mentioned, here in Utrecht we are working on libraries for generic programming. We want to make it easier for people to use generic libraries, so we are packaging EMGM [1] and a library for generic programming for mutually recursive datatypes [2]. We intend to release these on Hackage soon (Summer vacations are delaying us a bit), along with useful generic applications (a zipper and a generic rewriting framework). Maintaining SYB fits well in this idea, and if no other natural maintainers volunteer, I (with some support from the other people at Utrecht) am happy to take it upon me. I probably won't do heavy development on the library, but including patches, and providing support is fine. We're also planning to maintain EMGM here in Utrecht, although we didn't develop that ourselves. Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so. Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries. Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer. Cheers, Pedro [1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080801/a1fd90f3/attachment.htm From simonpj at microsoft.com Fri Aug 1 06:32:45 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Fri Aug 1 06:32:43 2008 Subject: [Hs-Generics] Re: Owning SYB In-Reply-To: <52f14b210807312336t7dd9f8f6p33b6118124b6fa5b@mail.gmail.com> References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt> <638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com> <017701c8f0db$fc721c60$3d058351@cr3lt> <404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com> <035901c8f1db$34673510$71338351@cr3lt> <52f14b210807312328y39ff1ee3m725f479853d06b28@mail.gmail.com> <52f14b210807312336t7dd9f8f6p33b6118124b6fa5b@mail.gmail.com> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE773E442@EA-EXMSG-C334.europe.corp.microsoft.com> Thank you! This is great news. Having a set of generic libraries with a single owner would be great. (Some comparative discussion would be useful too, to help people select the right package for their needs.) Simon From: generics-bounces@haskell.org [mailto:generics-bounces@haskell.org] On Behalf Of Jos? Pedro Magalh?es Sent: 01 August 2008 07:37 To: generics@haskell.org; libraries@haskell.org Subject: Re: [Hs-Generics] Re: Owning SYB Hello all, As Johan mentioned, here in Utrecht we are working on libraries for generic programming. We want to make it easier for people to use generic libraries, so we are packaging EMGM [1] and a library for generic programming for mutually recursive datatypes [2]. We intend to release these on Hackage soon (Summer vacations are delaying us a bit), along with useful generic applications (a zipper and a generic rewriting framework). Maintaining SYB fits well in this idea, and if no other natural maintainers volunteer, I (with some support from the other people at Utrecht) am happy to take it upon me. I probably won't do heavy development on the library, but including patches, and providing support is fine. We're also planning to maintain EMGM here in Utrecht, although we didn't develop that ourselves. Recently, (at least) Claus and Oleg have been posting interesting suggestions of improvements/modifications to SYB. Those should be further analyzed and discussed, and finally introduced (or not) in the library. The generic map for SYB, for instance, evolved from the "impossible to implement", through the "unsafe implementation", until the latest gmap2 as described by Oleg [3]. If further tests show this function behaves as expected, then it's clearly a good candidate for extending SYB. We should also rethink if other things previously deemed impossible remain so. Maintaining SYB, alongside with the other generic libraries, will require things such as: * Releasing packages in Hackage, properly documented with Haddock; * Updating such packages as necessary for new releases of GHC; * Writing examples of how to use the libraries (from a user perspective); * Writing testsuites, which are important for checking backwards compatibility of any changes; * Having an updated webpage linking to the library sources, documentation, possibly a bug tracker, etc. These are all things we plan to do for the libraries. Additionally, we could think of improving syb-with-class [4] in parallel with regular SYB. This is something to ask to its maintainer. Cheers, Pedro [1] http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html [4] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080801/5fcee4be/attachment-0001.htm From claus.reinke at talk21.com Fri Aug 1 10:02:02 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Fri Aug 1 10:02:09 2008 Subject: [Hs-Generics] Re: Owning SYB References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt><638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com><017701c8f0db$fc721c60$3d058351@cr3lt><404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com><035901c8f1db$34673510$71338351@cr3lt><52f14b210807312328y39ff1ee3m725f479853d06b28@mail.gmail.com> <52f14b210807312336t7dd9f8f6p33b6118124b6fa5b@mail.gmail.com> Message-ID: <02a201c8f3df$2da1e3a0$e6327ad5@cr3lt> Hi Pedro, and thanks for volunteering! I include a summary of where I'm at, for your information (and that of other interested readers;-) > Recently, (at least) Claus and Oleg have been posting interesting > suggestions of improvements/modifications to SYB. Those should be further > analyzed and discussed, and finally introduced (or not) in the library. The > generic map for SYB, for instance, evolved from the "impossible to > implement", through the "unsafe implementation", until the latest gmap2 as > described by Oleg [3]. If further tests show this function behaves as > expected, then it's clearly a good candidate for extending SYB. We should > also rethink if other things previously deemed impossible remain so. Further discussion welcome, of course!-) And it isn't just about getting fmap/traverse/.. from Data/Typeable, but also about reorganizing imports of Data instances, and providing alternatives of everywhere/everything that avoid traversals of irrelevant subterms. A current snapshot of my code can be found here: http://www.cs.kent.ac.uk/~cr3/tmp/syb/syb-utils-0.0.2008.7.31.tar.gz It currently contains: a) a suggested split for the Instances module, with alternatives to Data.Generics that either export only Standard instances, or no instances at all. On top of the existing Syb in base, these are somewhat tricky to use, because of the implicit re-exports of Data.Generics.Instances from other libraries http://www.haskell.org/pipermail/generics/2008-July/000371.html and the long-standing GHC "session accumulates instances" bug http://hackage.haskell.org/trac/ghc/ticket/2182 http://www.haskell.org/ghc/docs/latest/html/users_guide/bugs.html#bugs-ghc relevant modules: Data.Generics.Alt, Data.Generics.NoInstances, Data.Generics.Instances.Standard, Data.Generics.Instances.Dubious, examples\Examples.hs status: I think this change should go into base (perhaps renaming Data.Generics.Alt to Data.Generics.Standard, and deprecating Data.Generics and Data.Generics.Instances), for all the reasons discussed here recently, and because that GHC bug makes it near impossible to provide this change as an addon to base. Existing importers of Data.Generics or Data.Generics.Instances in core and extralibs should be redirected to one of the new modules. b) variants of everywhere/everything/mkQ/.. that retain the type domain of generically extended queries, build substructure type maps of types to be traversed, and use a slight generalisation of the Uniplate PlateData trick to avoid traversals of irrelevant substructures (usually but not always including, and not limited to, Strings) relevant modules: Data.Generics.GPS, examples\GPSbenchmark.hs, examples\CompanyDatatypes.hs status: This could be provided as an addon package. Performance of generic queries and transformation on the usual Paradise benchmark improve as expected; there is some overhead, which is visible in an alternative benchmark where there are no irrelevant substructures. The current code also tries to replace the linear search for applicable specific-type-queries with Map lookup, but here the overhead seems to outweigh the benefits, so I'll probably disable this code in future. One issue that I still need to address are nested types: just as Data.Generics.PlateData, Data.Generics.GPS currently fails for these (no finite representation of substructure types); current plan is to recognize nested types and to fall back to unoptimized traversals. c) generic default instance methods for Control.Monad.Functor and Data.Traversable relevant modules: Data.Generics.Utils, examples\Examples.hs status: under discusssion. could become part of that add-on package, or move into base. d) some Data/Typeable instances and utilities for GHC Api relevant modules: GHC.Syb.Instances, GHC.Syb.Instances0, GHC.Syb.Utils status: needs more testing, will probably be rendered obsolete when GHC Api starts providing those instances itself. > Maintaining SYB, alongside with the other generic libraries, will require > things such as: > * Releasing packages in Hackage, properly documented with Haddock; > * Updating such packages as necessary for new releases of GHC; > * Writing examples of how to use the libraries (from a user perspective); > * Writing testsuites, which are important for checking backwards > compatibility of any changes; > * Having an updated webpage linking to the library sources, documentation, > possibly a bug tracker, etc. > These are all things we plan to do for the libraries. > > Additionally, we could think of improving syb-with-class [4] in parallel > with regular SYB. This is something to ask to its maintainer. > > > Cheers, > Pedro > > [1] > http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA > [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html > [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html > [4] > http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class > -------------------------------------------------------------------------------- > _______________________________________________ > Generics mailing list > Generics@haskell.org > http://www.haskell.org/mailman/listinfo/generics > From claus.reinke at talk21.com Fri Aug 1 10:08:38 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Fri Aug 1 10:08:40 2008 Subject: [Hs-Generics] Re: Owning SYB References: <00ca01c8eb33$ed05d120$75097ad5@cr3lt><638ABD0A29C8884A91BC5FB5C349B1C32AE76B0E25@EA-EXMSG-C334.europe.corp.microsoft.com><017701c8f0db$fc721c60$3d058351@cr3lt><404396ef0807281450h435faeddv310f1ff984a8b0d7@mail.gmail.com><035901c8f1db$34673510$71338351@cr3lt><52f14b210807312328y39ff1ee3m725f479853d06b28@mail.gmail.com><52f14b210807312336t7dd9f8f6p33b6118124b6fa5b@mail.gmail.com> <02a201c8f3df$2da1e3a0$e6327ad5@cr3lt> Message-ID: <02a801c8f3e0$198946f0$e6327ad5@cr3lt> oops, keybinding malfunction - that message went out unfinished. Seems to have most parts, though, so I leave it at that. One other thing I meant to ask was about procedure, given that Syb is currently in base and hence under the library modification process. How is this going to combine with an active maintainer and some parts on hackage? Claus ----- Original Message ----- From: "Claus Reinke" To: "Jos? Pedro Magalh?es" ; ; Sent: Friday, August 01, 2008 3:02 PM Subject: Re: [Hs-Generics] Re: Owning SYB > Hi Pedro, > > and thanks for volunteering! I include a summary of where I'm at, for > your information (and that of other interested readers;-) > >> Recently, (at least) Claus and Oleg have been posting interesting >> suggestions of improvements/modifications to SYB. Those should be further >> analyzed and discussed, and finally introduced (or not) in the library. The >> generic map for SYB, for instance, evolved from the "impossible to >> implement", through the "unsafe implementation", until the latest gmap2 as >> described by Oleg [3]. If further tests show this function behaves as >> expected, then it's clearly a good candidate for extending SYB. We should >> also rethink if other things previously deemed impossible remain so. > > Further discussion welcome, of course!-) And it isn't just about getting > fmap/traverse/.. from Data/Typeable, but also about reorganizing imports > of Data instances, and providing alternatives of everywhere/everything > that avoid traversals of irrelevant subterms. > > A current snapshot of my code can be found here: > > http://www.cs.kent.ac.uk/~cr3/tmp/syb/syb-utils-0.0.2008.7.31.tar.gz > > It currently contains: > > a) a suggested split for the Instances module, with alternatives to Data.Generics that either > export only Standard instances, or no > instances at all. > On top of the existing Syb in base, these are somewhat tricky to use, > because of the implicit re-exports of Data.Generics.Instances from other libraries > > http://www.haskell.org/pipermail/generics/2008-July/000371.html > > and the long-standing GHC "session accumulates instances" bug > > http://hackage.haskell.org/trac/ghc/ticket/2182 > http://www.haskell.org/ghc/docs/latest/html/users_guide/bugs.html#bugs-ghc > > relevant modules: > Data.Generics.Alt, > Data.Generics.NoInstances, > Data.Generics.Instances.Standard, > Data.Generics.Instances.Dubious, > examples\Examples.hs > status: I think this change should go into base (perhaps renaming > Data.Generics.Alt to Data.Generics.Standard, and deprecating > Data.Generics and Data.Generics.Instances), for all the reasons > discussed here recently, and because that GHC bug makes it near > impossible to provide this change as an addon to base. Existing importers of Data.Generics > or Data.Generics.Instances in core > and extralibs should be redirected to one of the new modules. > > b) variants of everywhere/everything/mkQ/.. that retain the type domain > of generically extended queries, build substructure type maps of types > to be traversed, and use a slight generalisation of the Uniplate PlateData > trick to avoid traversals of irrelevant substructures (usually but not always > including, and not limited to, Strings) > > relevant modules: > Data.Generics.GPS, > examples\GPSbenchmark.hs, > examples\CompanyDatatypes.hs > > status: > This could be provided as an addon package. Performance > of generic queries and transformation on the usual Paradise > benchmark improve as expected; there is some overhead, > which is visible in an alternative benchmark where there are no irrelevant substructures. > > The current code also tries to replace the linear search for > applicable specific-type-queries with Map lookup, but here > the overhead seems to outweigh the benefits, so I'll probably > disable this code in future. > > One issue that I still need to address are nested types: just > as Data.Generics.PlateData, Data.Generics.GPS currently > fails for these (no finite representation of substructure types); > current plan is to recognize nested types and to fall back to > unoptimized traversals. > c) generic default instance methods for Control.Monad.Functor > and Data.Traversable > > relevant modules: > Data.Generics.Utils, > examples\Examples.hs > > status: > under discusssion. could become part of that add-on package, > or move into base. > > d) some Data/Typeable instances and utilities for GHC Api > > relevant modules: > GHC.Syb.Instances, > GHC.Syb.Instances0, > GHC.Syb.Utils > > status: > needs more testing, will probably be rendered obsolete when > GHC Api starts providing those instances itself. > >> Maintaining SYB, alongside with the other generic libraries, will require >> things such as: >> * Releasing packages in Hackage, properly documented with Haddock; >> * Updating such packages as necessary for new releases of GHC; >> * Writing examples of how to use the libraries (from a user perspective); >> * Writing testsuites, which are important for checking backwards >> compatibility of any changes; >> * Having an updated webpage linking to the library sources, documentation, >> possibly a bug tracker, etc. >> These are all things we plan to do for the libraries. >> >> Additionally, we could think of improving syb-with-class [4] in parallel >> with regular SYB. This is something to ask to its maintainer. >> >> >> Cheers, >> Pedro >> >> [1] >> http://books.google.com/books?id=OyY3ioMJRAsC&pg=PA199&sig=ACfU3U1nczeRAIjN9mc_vYnL1LnYAs70NA >> [2] http://www.cs.uu.nl/research/techreps/UU-CS-2008-019.html >> [3] http://www.haskell.org/pipermail/generics/2008-July/000362.html >> [4] >> http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class >> > > > -------------------------------------------------------------------------------- > > >> _______________________________________________ >> Generics mailing list >> Generics@haskell.org >> http://www.haskell.org/mailman/listinfo/generics >> > _______________________________________________ > Generics mailing list > Generics@haskell.org > http://www.haskell.org/mailman/listinfo/generics From jpm at cs.uu.nl Mon Aug 4 07:49:26 2008 From: jpm at cs.uu.nl (=?ISO-8859-1?Q?Jos=E9_Pedro_Magalh=E3es?=) Date: Mon Aug 4 07:49:13 2008 Subject: [Hs-Generics] Developing SYB, packaging issues Message-ID: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> Hello all, This message focuses on a problem Claus mentioned before: > One other thing I meant to ask was about procedure, > given that Syb is currently in base and hence under the > library modification process. How is this going to combine > with an active maintainer and some parts on hackage? > > Claus To be able to further develop SYB (see [1] for history), it's probably best to develop it as a separate package, which people can install, upgrade, etc. This would mean the library could be updated independently of GHC, and new GHC releases could then use the most recent version of the package. But how do these things merge? Can/should SYB be moved out of the base package? And, if this happens, can the library being developed as a separate package still use the automatic deriving mechanism? I'm sending this message to libraries@haskell.org too because I guess this problem might have shown up here before. Cheers, Pedro [1] http://search.gmane.org/?query=%22Owning+SYB%22&group=gmane.comp.lang.haskell.libraries&sort=revdate -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080804/f8282e9b/attachment.htm From leather at cs.uu.nl Tue Aug 5 10:46:50 2008 From: leather at cs.uu.nl (Sean Leather) Date: Tue Aug 5 10:46:33 2008 Subject: [Hs-Generics] Developing SYB, packaging issues In-Reply-To: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> References: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> Message-ID: <3c6288ab0808050746p4b5bd85aqe2639318ef3a5585@mail.gmail.com> Hi Pedro, To be able to further develop SYB (see [1] for history), it's probably best > to develop it as a separate package, which people can install, upgrade, etc. > This would mean the library could be updated independently of GHC, and new > GHC releases could then use the most recent version of the package. > Agreed. We talked about this offline, but I wanted to chime in with a +1 here. Just as it would help the GHC developers for a third party to develop and maintain SYB, it would help the developers and users for SYB to be distributed and available as a separate package. > But how do these things merge? Can/should SYB be moved out of the base > package? And, if this happens, can the library being developed as a separate > package still use the automatic deriving mechanism? > I think SYB should be extracted from 'base' into a package. It seems like the only technical thing that might prevent this extraction is the automatic deriving of Typeable and Data. Or does it prevent it? Can anyone clarify this? As a side note, might this work have an impact on the Haskell Platform? Thanks, Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080805/996a868c/attachment.htm From jpm at cs.uu.nl Wed Aug 20 08:23:41 2008 From: jpm at cs.uu.nl (=?ISO-8859-1?Q?Jos=E9_Pedro_Magalh=E3es?=) Date: Wed Aug 20 08:22:39 2008 Subject: [Hs-Generics] Developing SYB, packaging issues In-Reply-To: <20080805174009.GA1677@matrix.chaos.earth.li> References: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> <3c6288ab0808050746p4b5bd85aqe2639318ef3a5585@mail.gmail.com> <20080805174009.GA1677@matrix.chaos.earth.li> Message-ID: <52f14b210808200523h37d5ae7em2cc5bf8a79df2b35@mail.gmail.com> Hello, I understand that with the proposed base package breakup [1], SYB will be moved to a separate package. But I still don't know how this will reflect on the development of the library. In particular: 1) Where is the source code going to be hosted? Here in Utrecht we currently have a repository with several (cabalized) generic programming libraries, SYB included. But maybe SYB will stay in the same repository as GHC? 2) Can development proceed independently of GHC, i.e. can a new version of SYB be released without a new version of GHC? 3) How does the separation affect the automatic instance deriving mechanism? Thanks, Pedro [1] http://hackage.haskell.org/trac/ghc/ticket/1338 On Tue, Aug 5, 2008 at 19:40, Ian Lynagh wrote: > On Tue, Aug 05, 2008 at 04:46:50PM +0200, Sean Leather wrote: > > > > I think SYB should be extracted from 'base' into a package. > > I'll be sending a message about this soon. > > > Thanks > Ian > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080820/7dbc3adf/attachment-0001.htm From claus.reinke at talk21.com Thu Aug 21 04:10:27 2008 From: claus.reinke at talk21.com (Claus Reinke) Date: Thu Aug 21 04:09:31 2008 Subject: [Hs-Generics] Re: Syb Renovations? Issues with Data.Generics References: <018401c8f0dd$98126480$3d058351@cr3lt><20080728222302.GA24020@matrix.chaos.earth.li><027a01c8f1b1$1d1cef00$71338351@cr3lt><20080729195944.GA15169@matrix.chaos.earth.li><026f01c8f299$9f511b70$36168351@cr3lt><20080731010538.GA20317@matrix.chaos.earth.li> <006101c8f2f9$3c39ab50$0b1b7ad5@cr3lt> Message-ID: <009a01c90365$60eb1890$12168351@cr3lt> Some of you already know, but it seems I forgot to mention this here - my code has moved to a darcs repo, with a little bit of documentation and a README summarizing the issues. See my toolbox for more info: http://www.cs.kent.ac.uk/~cr3/toolbox/haskell/#syb-utils Neil: It turned out to be tricky to recognize nested types at the Data/Typeable level, let alone nested types that really have an infinite set of potential substructure types (which are the ones that break the PlateData optimization). Instead, I just count nesting levels (where nesting means that we encounter the top-level type constructor while exploring its substructure types), and set an arbitrary bound beyond which I assume the nesting to be recursive and fall back to the unoptimized case. You might want to apply something similar to PlateData. Cheers, Claus From simonpj at microsoft.com Thu Aug 21 05:16:44 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Thu Aug 21 05:15:38 2008 Subject: [Hs-Generics] Developing SYB, packaging issues In-Reply-To: <52f14b210808200523h37d5ae7em2cc5bf8a79df2b35@mail.gmail.com> References: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> <3c6288ab0808050746p4b5bd85aqe2639318ef3a5585@mail.gmail.com> <20080805174009.GA1677@matrix.chaos.earth.li> <52f14b210808200523h37d5ae7em2cc5bf8a79df2b35@mail.gmail.com> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE853E014@EA-EXMSG-C334.europe.corp.microsoft.com> I understand that with the proposed base package breakup [1], SYB will be moved to a separate package. But I still don't know how this will reflect on the development of the library. In particular: 1) Where is the source code going to be hosted? Here in Utrecht we currently have a repository with several (cabalized) generic programming libraries, SYB included. But maybe SYB will stay in the same repository as GHC? I don't think it matters too much where it's hosted. For us it might be convenient if it was on darcs.haskell.org because it reduces the number of ways in which you can get stuck. But servers are fairly reliable so this probably isn't very important. 2) Can development proceed independently of GHC, i.e. can a new version of SYB be released without a new version of GHC? Yes, I think that independent development is part of the goal. The easiest way to achieve this is for SYB *not* to be a GHC "core package". That is, not needed to build GHC (or GHCi, or the GHC library). Then it's "just a library" like GtK or LibCurl, and you can upgrade it whenever you like. It's more complicated if it's a core package. For example, if the GHC API uses SYB to implement something, then package "ghc-6.9" will depend on package "SYB-2.7", and while you can also have SYB-3.2 installed the ghc-6.9 package will still use the "SYB-2.7". 3) How does the separation affect the automatic instance deriving mechanism? It think it'd make sense for the classes Data and Typable themselves to remain in a "core package", precisely because the deriving mechanism generates code for them. If you change the method signatures, the code has to change, for example. But all the library code layered on top can be in the SYB package. I hope I have this right! Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080821/556a99af/attachment.htm From jpm at cs.uu.nl Thu Aug 21 10:14:25 2008 From: jpm at cs.uu.nl (=?ISO-8859-1?Q?Jos=E9_Pedro_Magalh=E3es?=) Date: Thu Aug 21 10:13:17 2008 Subject: [Hs-Generics] Developing SYB, packaging issues In-Reply-To: <638ABD0A29C8884A91BC5FB5C349B1C32AE853E014@EA-EXMSG-C334.europe.corp.microsoft.com> References: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> <3c6288ab0808050746p4b5bd85aqe2639318ef3a5585@mail.gmail.com> <20080805174009.GA1677@matrix.chaos.earth.li> <52f14b210808200523h37d5ae7em2cc5bf8a79df2b35@mail.gmail.com> <638ABD0A29C8884A91BC5FB5C349B1C32AE853E014@EA-EXMSG-C334.europe.corp.microsoft.com> Message-ID: <52f14b210808210714u49b0b07erddf55abad6de6a9b@mail.gmail.com> Hello, Thanks for your answer. Replying below: On Thu, Aug 21, 2008 at 11:16, Simon Peyton-Jones wrote: > I understand that with the proposed base package breakup [1], SYB will > be moved to a separate package. But I still don't know how this will reflect > on the development of the library. In particular: > > 1) Where is the source code going to be hosted? Here in Utrecht we > currently have a repository with several (cabalized) generic programming > libraries, SYB included. But maybe SYB will stay in the same repository as > GHC? > > I don't think it matters too much where it's hosted. For us it might be > convenient if it was on darcs.haskell.org because it reduces the number of > ways in which you can get stuck. But servers are fairly reliable so this > probably isn't very important. > We might prefer to keep it in an SVN repository where we have other generic libraries, if that is not a big problem. If it is, it can always go to darcs.haskell.org anyway. > > 2) Can development proceed independently of GHC, i.e. can a new version of > SYB be released without a new version of GHC? > > Yes, I think that independent development is part of the goal. The > easiest way to achieve this is for SYB **not** to be a GHC "core package". > That is, not needed to build GHC (or GHCi, or the GHC library). Then it's > "just a library" like GtK or LibCurl, and you can upgrade it whenever you > like. > > It's more complicated if it's a core package. For example, if the GHC API > uses SYB to implement something, then package "ghc-6.9" will depend on > package "SYB-2.7", and while you can also have SYB-3.2 installed the ghc-6.9 > package will still use the "SYB-2.7". > I think indeed having it outside of the core is the best thing. > > 3) How does the separation affect the automatic instance deriving > mechanism? > > It think it'd make sense for the classes Data and Typable themselves to > remain in a "core package", precisely because the deriving mechanism > generates code for them. If you change the method signatures, the code has > to change, for example. But all the library code layered on top can be in > the SYB package. > Ok, that makes sense. Only any changes to methods in Data would need to wait for a new version of GHC. But those should be kept to a minimum, if any at all. It's just a pity that so many methods are inside the Data class (like gmapQ and friends). But then again, there is a reason for them to be there, and it's probably not a good idea to change those anyway. Most development should proceed by adding new things on top of the existing Data class core. What about instances of Data for the base types? Here I see a few possibilities: 1) No types have instances in core. Those could be in the SYB package, or the user could use stand-alone deriving to get them (if that is possible). 2) All types have instances in core, similar to the current Data.Generics.Instances situation. This implies that the situation discussed in [1] (inconvenient Data instances) will remain. 3) Something between the previous two, such as the 'standard' Data instances staying in core, and the others going to the SYB package (where they could be thought over, or separated into another module which is not imported by default). Pedro [1] http://www.haskell.org/pipermail/generics/2008-June/000347.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080821/874fa1b3/attachment.htm From simonpj at microsoft.com Thu Aug 21 12:41:24 2008 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Thu Aug 21 12:40:20 2008 Subject: [Hs-Generics] Developing SYB, packaging issues In-Reply-To: <52f14b210808210714u49b0b07erddf55abad6de6a9b@mail.gmail.com> References: <52f14b210808040449o4189511k1cbe6d430b6fb6da@mail.gmail.com> <3c6288ab0808050746p4b5bd85aqe2639318ef3a5585@mail.gmail.com> <20080805174009.GA1677@matrix.chaos.earth.li> <52f14b210808200523h37d5ae7em2cc5bf8a79df2b35@mail.gmail.com> <638ABD0A29C8884A91BC5FB5C349B1C32AE853E014@EA-EXMSG-C334.europe.corp.microsoft.com> <52f14b210808210714u49b0b07erddf55abad6de6a9b@mail.gmail.com> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C32AE85DFD55@EA-EXMSG-C334.europe.corp.microsoft.com> 1) Where is the source code going to be hosted? Here in Utrecht we currently have a repository with several (cabalized) generic programming libraries, SYB included. But maybe SYB will stay in the same repository as GHC? I don't think it matters too much where it's hosted. For us it might be convenient if it was on darcs.haskell.org because it reduces the number of ways in which you can get stuck. But servers are fairly reliable so this probably isn't very important. We might prefer to keep it in an SVN repository where we have other generic libraries, if that is not a big problem. If it is, it can always go to darcs.haskell.org anyway. I think we would very much prefer NOT to use SVN. We're already using darcs, and will shortly be using Git too. To have SVN too is a situation we'd like to avoid. Darcs or Git please! 3) How does the separation affect the automatic instance deriving mechanism? It think it'd make sense for the classes Data and Typable themselves to remain in a "core package", precisely because the deriving mechanism generates code for them. If you change the method signatures, the code has to change, for example. But all the library code layered on top can be in the SYB package. Ok, that makes sense. Only any changes to methods in Data would need to wait for a new version of GHC. But those should be kept to a minimum, if any at all. It's just a pity that so many methods are inside the Data class (like gmapQ and friends). But then again, there is a reason for them to be there, and it's probably not a good idea to change those anyway. Most development should proceed by adding new things on top of the existing Data class core. What about instances of Data for the base types? Here I see a few possibilities: 1) No types have instances in core. Those could be in the SYB package, or the user could use stand-alone deriving to get them (if that is possible). 2) All types have instances in core, similar to the current Data.Generics.Instances situation. This implies that the situation discussed in [1] (inconvenient Data instances) will remain. 3) Something between the previous two, such as the 'standard' Data instances staying in core, and the others going to the SYB package (where they could be thought over, or separated into another module which is not imported by default). I suspect it'd be better to have all the instances in the SYB package. If there are data types whose instances are sufficiently simple and unlikely to change that they can live in the same module as the Data class itself, then we can do that, but otherwise just put them in the package. S Pedro [1] http://www.haskell.org/pipermail/generics/2008-June/000347.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/generics/attachments/20080821/28d43f96/attachment-0001.htm