From dons at galois.com Sat Aug 1 18:16:26 2009 From: dons at galois.com (Don Stewart) Date: Sat Aug 1 17:59:25 2009 Subject: Deciding on the decision model for adding and excluding packages Message-ID: <20090801221626.GH8940@whirlpool.galois.com> We need to start thinking about how to add or exclude packages from the Haskell Platform. I've written a summary of the discussion so far for evidence-based critieria for deciding on package notability. http://donsbot.wordpress.com/2009/08/01/heuristics-for-blessing-software-packages/ Please read it over and think about what the decision model looks like. So we can feed data in, to make decisions on how to include/exclude packages. -- Don From marlowsd at gmail.com Mon Aug 3 08:31:56 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 3 08:12:51 2009 Subject: Deciding on the decision model for adding and excluding packages In-Reply-To: <20090801221626.GH8940@whirlpool.galois.com> References: <20090801221626.GH8940@whirlpool.galois.com> Message-ID: <4A76D8BC.1040800@gmail.com> On 01/08/2009 23:16, Don Stewart wrote: > We need to start thinking about how to add or exclude packages from the > Haskell Platform. > > I've written a summary of the discussion so far for evidence-based > critieria for deciding on package notability. > > http://donsbot.wordpress.com/2009/08/01/heuristics-for-blessing-software-packages/ > > > Please read it over and think about what the decision model looks like. > So we can feed data in, to make decisions on how to include/exclude packages. From that page, it looks like all the automatic procedures for deciding the top packages result in packages that we don't want in the platform for one reason or another. I'd argue against having a points system: such things also tend to produce unsatisfactory results, so you end up tweaking the weightings to get the results you did want. OTOH, the set of criteria you outlined for evaluating packages are good, but they should be used as a set of guidelines rather than a way to make decisions. In practice we'll probably find that there are criteria not on the list that end up being more important, e.g. utf8-string is immensely popular but is a stopgap measure and we don't really want to endorse its use, and extensible-exceptions is essentially a backwards-compatibility shim, so we don't want it in for similar reasons (OTOH, what about base3?). I suggest the best way forward is to identify needs: we need a package that provides a certain functionality, so look at what is available and go from there. If the available packages aren't suitable for one reason or another, then feed that back to the maintainers and the community. On a concrete note, I think we should seriously consider putting gtk2hs in the platform. As someone pointed out recently (I forget who, sorry!) the point of the platform is to give you the hard-to-install pieces, and gtk2hs is one such piece. Having gtk2hs would be a significant step up in terms of functionality, though. Cheers, Simon From ndmitchell at gmail.com Mon Aug 3 10:10:51 2009 From: ndmitchell at gmail.com (Neil Mitchell) Date: Mon Aug 3 09:51:42 2009 Subject: Deciding on the decision model for adding and excluding packages In-Reply-To: <4A76D8BC.1040800@gmail.com> References: <20090801221626.GH8940@whirlpool.galois.com> <4A76D8BC.1040800@gmail.com> Message-ID: <404396ef0908030710l1ce52b81v1fa644c7b8626cfe@mail.gmail.com> Hi Simon, > I'd argue against having a points system: such things also tend to produce unsatisfactory results, so you end up tweaking > the weightings to get the results you did want. I strongly agree with this (and have argued against Duncan and Don on this before) - 100% coverage is something nice to have if you go about it the right way. If you are required to have 100% coverage (or any other metric) then you cheat. Aim for quality packages, and if someone thinks about test coverage that indicates quality - even if they decide in this particular case coverage is a little meaningless. > On a concrete note, I think we should seriously consider putting gtk2hs in the platform. As someone pointed out recently (I forget > who, sorry!) the point of the platform is to give you the hard-to-install pieces, and gtk2hs is one such piece. Having gtk2hs would > be a significant step up in terms of functionality, though. YES! Gtk2hs is wonderful, and never has an up to date GHC release. I currently don't care too much about the platform (I've got a copy of cabal install and its not really aimed at me anyway) - but if you bundle network, hmatrix and gtk2hs then I can't imagine any Windows user who wouldn't install it. Thanks Neil From ganesh.sittampalam at credit-suisse.com Mon Aug 3 10:20:22 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Mon Aug 3 10:02:05 2009 Subject: Deciding on the decision model for adding and excluding packages In-Reply-To: <20090801221626.GH8940@whirlpool.galois.com> References: <20090801221626.GH8940@whirlpool.galois.com> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F86B@ELON17P32001A.csfb.cs-group.com> Don Stewart wrote: > We need to start thinking about how to add or exclude packages from > the Haskell Platform. > > I've written a summary of the discussion so far for evidence-based > critieria for deciding on package notability. > > http://donsbot.wordpress.com/2009/08/01/heuristics-for-blessing-software -packages/ > > > Please read it over and think about what the decision model looks like. > So we can feed data in, to make decisions on how to include/exclude packages. I suggest making the discussion concrete by opening up for a list of suggestions, and then trying to figure out which of those suggestions should go in. Hopefully we can then back out a set of principles or precedents for the future from the discussion. Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From marlowsd at gmail.com Mon Aug 3 11:45:30 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 3 11:26:24 2009 Subject: PROPOSAL: Add openFileTemplate, openBinaryFileTemplate to System.IO In-Reply-To: <20090725123533.GA12044@matrix.chaos.earth.li> References: <20090724164427.GA13350@matrix.chaos.earth.li> <916b84820907241627o4c7c07f0wc36a9dd006eb28cc@mail.gmail.com> <40a414c20907242228n897b656xbb5f561352d9f0fb@mail.gmail.com> <2518b95d0907242300i4302d939od4bc1671e4fe462f@mail.gmail.com> <40a414c20907242302i49403d50g99588c4a4b041fa@mail.gmail.com> <2518b95d0907242311u1b1a3954uaf29c955ab7f3cb2@mail.gmail.com> <20090725123533.GA12044@matrix.chaos.earth.li> Message-ID: <4A77061A.3010503@gmail.com> On 25/07/2009 13:35, Ian Lynagh wrote: > On Fri, Jul 24, 2009 at 11:11:40PM -0700, Evan Laforge wrote: >> On Fri, Jul 24, 2009 at 11:02 PM, minh thu wrote: >>> 2009/7/25 Evan Laforge: >>>> I've never heard this usage of the word "template" before, so I >>>> guessed it was meant to open some kind of template file. >>>> openWithMode? openWithPermissions? >>> >>> man mktemp >> >> Ohhh, it's for the filename, sorry, all the talk about permissions got >> me on the wrong track. > > Right. Just to clarify for anyone else reading, we currently have: > openTempFile :: FilePath -> String -> IO (FilePath, Handle) > openBinaryTempFile :: FilePath -> String -> IO (FilePath, Handle) > which mask the mode, and we would add > openFileTemplate :: FilePath -> String -> IO (FilePath, Handle) > openBinaryFileTemplate :: FilePath -> String -> IO (FilePath, Handle) > which do not. > > Theese functions are used like this: > openFileTemplate "/foo/bar" "bazXXX.ext" > and open a file called "baz143.ext". "bazXXX.ext" is the template. I'm not too keen on the naming. openFileTemplate differs from openTempFile only in that it uses the default permissions, and yet its name suggests that it does something quite different. The fact that you don't know what the file is going to be called means that the API is only really useful for temporary files, so I have no problem with that being part of the name. Perhaps openTempFileWithDefaultPermissions? Or perhaps we deprecate openTempFile, and make a new openTemporaryFile that takes a Bool argument to say whether to make the file private or not? Cheers, Simon From dons at galois.com Mon Aug 3 12:54:01 2009 From: dons at galois.com (Don Stewart) Date: Mon Aug 3 12:36:58 2009 Subject: Deciding on the decision model for adding and excluding packages In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F86B@ELON17P32001A.csfb.cs-group.com> References: <20090801221626.GH8940@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F86B@ELON17P32001A.csfb.cs-group.com> Message-ID: <20090803165401.GA17991@whirlpool.galois.com> ganesh.sittampalam: > Don Stewart wrote: > > We need to start thinking about how to add or exclude packages from > > the Haskell Platform. > > > > I've written a summary of the discussion so far for evidence-based > > critieria for deciding on package notability. > > > > > http://donsbot.wordpress.com/2009/08/01/heuristics-for-blessing-software > -packages/ > > > > > > Please read it over and think about what the decision model looks > like. > > So we can feed data in, to make decisions on how to include/exclude > packages. > > I suggest making the discussion concrete by opening up for a list of > suggestions, and then trying to figure out which of those suggestions > should go in. Hopefully we can then back out a set of principles or > precedents for the future from the discussion. Package suggestions, you mean? -- Don From ganesh.sittampalam at credit-suisse.com Mon Aug 3 12:56:58 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Mon Aug 3 12:38:17 2009 Subject: Deciding on the decision model for adding and excludingpackages In-Reply-To: <20090803165401.GA17991@whirlpool.galois.com> References: <20090801221626.GH8940@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F86B@ELON17P32001A.csfb.cs-group.com> <20090803165401.GA17991@whirlpool.galois.com> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F86F@ELON17P32001A.csfb.cs-group.com> Don Stewart wrote: > ganesh.sittampalam: >> >> I suggest making the discussion concrete by opening up for a list of >> suggestions, and then trying to figure out which of those suggestions >> should go in. Hopefully we can then back out a set of principles or >> precedents for the future from the discussion. > > Package suggestions, you mean? Yes. Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From dons at galois.com Mon Aug 3 12:58:53 2009 From: dons at galois.com (Don Stewart) Date: Mon Aug 3 12:41:49 2009 Subject: Deciding on the decision model for adding and excludingpackages In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F86F@ELON17P32001A.csfb.cs-group.com> References: <20090801221626.GH8940@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F86B@ELON17P32001A.csfb.cs-group.com> <20090803165401.GA17991@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F86F@ELON17P32001A.csfb.cs-group.com> Message-ID: <20090803165853.GB17991@whirlpool.galois.com> ganesh.sittampalam: > Don Stewart wrote: > > ganesh.sittampalam: > > >> > >> I suggest making the discussion concrete by opening up for a list of > >> suggestions, and then trying to figure out which of those suggestions > >> should go in. Hopefully we can then back out a set of principles or > >> precedents for the future from the discussion. > > > > Package suggestions, you mean? > > Yes. > Ok. I'll put tools in place to collect those. Stay tuned. -- Don From dons at galois.com Mon Aug 3 13:38:11 2009 From: dons at galois.com (Don Stewart) Date: Mon Aug 3 13:21:09 2009 Subject: Deciding on the decision model for adding and excluding packages In-Reply-To: <4A76D8BC.1040800@gmail.com> References: <20090801221626.GH8940@whirlpool.galois.com> <4A76D8BC.1040800@gmail.com> Message-ID: <20090803173811.GE17991@whirlpool.galois.com> marlowsd: > I suggest the best way forward is to identify needs: we need a package > that provides a certain functionality, so look at what is available and > go from there. If the available packages aren't suitable for one reason > or another, then feed that back to the maintainers and the community. I think this is good. We can start by looking at the categories provided by Python: http://docs.python.org/library/index.html And filling in the gaps. Doing this may lead to some concrete proposed packages, which in turn can force the process along. -- Don From lemming at henning-thielemann.de Mon Aug 3 14:25:19 2009 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon Aug 3 14:04:33 2009 Subject: PROPOSAL: Remove Control.OldException In-Reply-To: <5ab17e790907220019l2781997v3e1b8627b4fa0c7c@mail.gmail.com> References: <20090718125113.GA21683@matrix.chaos.earth.li> <20090718165104.GA6997@whirlpool.galois.com> <20090719172930.GA619@matrix.chaos.earth.li> <20090719185630.GA11801@whirlpool.galois.com> <5ab17e790907192327m504521b5h5b0c72ee303036d4@mail.gmail.com> <1248117111.3163.11.camel@localhost> <5ab17e790907220019l2781997v3e1b8627b4fa0c7c@mail.gmail.com> Message-ID: <4A772B8F.2070605@henning-thielemann.de> Iavor Diatchki schrieb: > Hello, > > On Mon, Jul 20, 2009 at 10:11 PM, Duncan > Coutts wrote: >> On Mon, 2009-07-20 at 09:27 +0300, Iavor Diatchki wrote: >>> Hello, >>> Please do not remove OldException. >>> -Iavor >> Can you give any rationale? Are you trying to maintain compatibility >> with base-3 by using CPP + OldException? Is that significantly easier >> than using the extensible-exceptions package? > > 1. I would have to go and change all code that uses exceptions, again. > 2. I am not sure how to write backwards compatible code. CPP + > OldException is not pretty but it works. > 3. The extensible exception package is not Haskell 98 (it uses > existentials), so it really would be nice if _it_ was placed in a > separate package. +1 > On that topic, it would be great if we had a way to > see what are all the extensions used by a program (including those > used by dependent packages). +1 From ross at soi.city.ac.uk Mon Aug 3 15:28:46 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Mon Aug 3 15:09:40 2009 Subject: Proposal #3271: new methods for Data.Sequence In-Reply-To: References: <2518b95d0907191857q29a442b1l86609db8e3e58d17@mail.gmail.com> Message-ID: <20090803192846.GA11752@soi.city.ac.uk> On Mon, Jul 20, 2009 at 02:33:52PM -0400, Louis Wasserman wrote: > Among other things, I manually deforested the stable sorting algorithm, > resulting in a moderate performance gain on simply using Data.List.sortBy. That's not supposed to be necessary, because toList is supposed to fuse with list consumers. Unfortunately it doesn't in cases like this because it doesn't get inlined, even if we add an INLINE pragma, due to GHC bug #2353. If the GHC bug isn't fixed, I think the next best thing would be to force the inlining of toList with a RULES pragma in Data.Foldable. From dons at galois.com Mon Aug 3 19:44:32 2009 From: dons at galois.com (Don Stewart) Date: Mon Aug 3 19:27:29 2009 Subject: Thinking about what's missing in our library coverage Message-ID: <20090803234432.GK17991@whirlpool.galois.com> Following Simon M's advice, I look over the typical "batteries" categories, using Python as input: http://docs.python.org/library/index.html The following things were missing from the current Platform. There are many. How would you identify the top, say, 5 libs to add? -- Don * String support o binary formatting [binary] ? lazy binary parsing/serialising o pcre regexes [pcre-light] [regex-pcre] ? what?s our best regex lib? o unicode text [text] [text-icu] ? packed, unicode text o codecs/encodings ? encodings? * Data types o higher dimensional arrays [hmatrix] o bloomfilter ? bloomfilters o bytestring-tries ? IntMap for ByteStrings o dlist ? difference lists o numbers ? expanded number types * text o attoparsec (simple, bytestring parsing) o polyparse o csv parsing o pandoc ? markdown, reStructuredText, HTML, LaTeX, ConTeXt, Docbook, OpenDocument, ODT, RTF, MediaWiki, groff * math and numerics o blas ? BLAS o cmath ? C math functions o dimensional ? physical dimensions o fftw o mersenne-random ? fast randoms * persistance o anydbm? o sqlite3 o hdbc * compression o bzip2 o zip o tar * file formats o csv o config parser * crypto o hmac, md5, sha, hashing * systems o getopt o logging o termio o editline o mmap * Internet o network-bytestring o ssl o json o feed (rss, atom) o mime o base64 et al o uuencode o cgi o fastcgi o urls o ftp, http, imap, smtp clients o uuid o url parsing o http server o xml-rpc * Multimedia o colour * Internationalization o gettext o locale o i18n * GUIs o gtk2hs * Development o hscolour From igloo at earth.li Mon Aug 3 19:59:01 2009 From: igloo at earth.li (Ian Lynagh) Date: Mon Aug 3 19:39:51 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: <20090803235901.GA2552@matrix.chaos.earth.li> On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart wrote: > > How would you identify the top, say, 5 libs to add? I would not look for libs to add. I would wait for people to come and tell me that they think that particular libs are worthy of addition, and then decide whether or not I agree. Thanks Ian From dons at galois.com Mon Aug 3 19:59:27 2009 From: dons at galois.com (Don Stewart) Date: Mon Aug 3 19:42:20 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803235901.GA2552@matrix.chaos.earth.li> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> Message-ID: <20090803235927.GN17991@whirlpool.galois.com> igloo: > On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart wrote: > > > > How would you identify the top, say, 5 libs to add? > > I would not look for libs to add. I would wait for people to come and > tell me that they think that particular libs are worthy of addition, and > then decide whether or not I agree. That's fine. I'm just trying to get a sense of what people will be proposing, and why. Is this an 'identify the champion' model? We await a champion to propose things? -- Don From felipe.lessa at gmail.com Mon Aug 3 21:42:21 2009 From: felipe.lessa at gmail.com (Felipe Lessa) Date: Mon Aug 3 21:23:15 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: <20090804014221.GA6311@kira.casa> On Mon, Aug 03, 2009 at 04:44:32PM -0700, Don Stewart wrote: > * compression > o bzip2 > o zip > o tar These seem nice. > * systems > o getopt > o editline These, too. > * GUIs > o gtk2hs Now this looks like trouble :). I mean, I would love to know that Gtk2Hs is installed everywhere, but IMHO it would put a great burden to the platform. Would it just bundle the whole Gtk+ inside the instaler? Hmmm... -- Felipe. From bulat.ziganshin at gmail.com Tue Aug 4 02:08:58 2009 From: bulat.ziganshin at gmail.com (Bulat Ziganshin) Date: Tue Aug 4 01:53:44 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804014221.GA6311@kira.casa> References: <20090803234432.GK17991@whirlpool.galois.com> <20090804014221.GA6311@kira.casa> Message-ID: <219470443.20090804100858@gmail.com> Hello Felipe, Tuesday, August 4, 2009, 5:42:21 AM, you wrote: >> o gtk2hs > Now this looks like trouble :). I mean, I would love to know > that Gtk2Hs is installed everywhere, but IMHO it would put a > great burden to the platform. Would it just bundle the whole > Gtk+ inside the instaler? Hmmm... one posible solution to this problem would be to split HP into two editions - basic one should be kept small and simple, even smaller than old GHC distros (i.e. minus opengl-like stuff, with only "core" libraries remaining). while batteries included distro should grow, its main goal - provide one-step installer of whole Haskell world -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com From kr.angelov at gmail.com Tue Aug 4 03:25:06 2009 From: kr.angelov at gmail.com (Krasimir Angelov) Date: Tue Aug 4 03:05:57 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: On 8/4/09, Don Stewart wrote: > * systems > o getopt > o logging > o termio > o editline > o mmap editline is known to have problem with Unicode. Why not Haskeline? Even readline is better than editline. From marlowsd at gmail.com Tue Aug 4 06:29:55 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Tue Aug 4 06:10:49 2009 Subject: PROPOSAL: Remove Control.OldException In-Reply-To: <5ab17e790907220019l2781997v3e1b8627b4fa0c7c@mail.gmail.com> References: <20090718125113.GA21683@matrix.chaos.earth.li> <20090718165104.GA6997@whirlpool.galois.com> <20090719172930.GA619@matrix.chaos.earth.li> <20090719185630.GA11801@whirlpool.galois.com> <5ab17e790907192327m504521b5h5b0c72ee303036d4@mail.gmail.com> <1248117111.3163.11.camel@localhost> <5ab17e790907220019l2781997v3e1b8627b4fa0c7c@mail.gmail.com> Message-ID: <4A780DA3.1050604@gmail.com> On 22/07/2009 08:19, Iavor Diatchki wrote: > Hello, > > On Mon, Jul 20, 2009 at 10:11 PM, Duncan > Coutts wrote: >> On Mon, 2009-07-20 at 09:27 +0300, Iavor Diatchki wrote: >>> Hello, >>> Please do not remove OldException. >>> -Iavor >> >> Can you give any rationale? Are you trying to maintain compatibility >> with base-3 by using CPP + OldException? Is that significantly easier >> than using the extensible-exceptions package? > > 1. I would have to go and change all code that uses exceptions, again. > 2. I am not sure how to write backwards compatible code. CPP + > OldException is not pretty but it works. > 3. The extensible exception package is not Haskell 98 (it uses > existentials), so it really would be nice if _it_ was placed in a > separate package. Not practical, unfortunately. Exceptions are very deeply wired in: they are even dependended on by the Monad IO instance, for example (for "fail", which is well-named), and the Prelude exports catch. Not the extensible version of course, but the Prelude's catch is defined in terms of the extensible catch. > On that topic, it would be great if we had a way to > see what are all the extensions used by a program (including those > used by dependent packages). A prerequisite is to make it an error to use any extensions not declared in the .cabal file. We talked about this a long time ago, but never got around to doing anything about it. Having done that, you can easily see what extensions are being used by a package: look in the .cabal file. Cabal-install could figure out the extensions used by dependencies, although it would always include base which uses a raft of extensions, so I'm not sure how useful that would be. > 4. Ian's proposal did not provide any motivation for this change. Is > the benefit the we clean things up a bit? If so, I think that it is a > bit soon to do it. Ok. We should at least deprecate Control.OldException, so we can remove it in 6.14? Cheers, Simon From marlowsd at gmail.com Tue Aug 4 07:02:38 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Tue Aug 4 06:43:29 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803235901.GA2552@matrix.chaos.earth.li> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> Message-ID: <4A78154E.2010606@gmail.com> On 04/08/2009 00:59, Ian Lynagh wrote: > On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart wrote: >> >> How would you identify the top, say, 5 libs to add? > > I would not look for libs to add. I would wait for people to come and > tell me that they think that particular libs are worthy of addition, and > then decide whether or not I agree. Ok, to kick things off then, I propose the following: Add * binary * getopt * gtk2hs Also * keep an eye on text. We certainly want it, but it's a young package and there's no text I/O yet. * decide which regex package(s) we want * remove html? (we have xhtml) * replace haskell-src with haskell-src-exts * remove packedstring Cheers, Simon From gwern0 at gmail.com Tue Aug 4 07:07:40 2009 From: gwern0 at gmail.com (Gwern Branwen) Date: Tue Aug 4 06:48:39 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78154E.2010606@gmail.com> Message-ID: On Tue, Aug 4, 2009 at 7:02 AM, Simon Marlow wrote: > Add > > ?* binary > ?* getopt > ?* gtk2hs A definite yes to binary and getopt; but gtk2hs? I don't trust its longevity or maintenance, and as other people have pointed out, it will make the platform harder to support. As well, we risk holding back the platform - hasn't gtk2hs lagged GHC releases in the past? (I seem to remember some of the lags being quite lengthy.) > Also > > ?* keep an eye on text. ?We certainly want it, but it's > ? a young package and there's no text I/O yet. > ?* decide which regex package(s) we want > ?* remove html? (we have xhtml) > ?* replace haskell-src with haskell-src-exts > ?* remove packedstring Absolutely. I thought we had already done this - didn't TH's unnecessary use of packedstring get removed a while ago? > Cheers, > ? ? ? ?Simon -- gwern -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: OpenPGP digital signature Url : http://www.haskell.org/pipermail/libraries/attachments/20090804/bce55beb/signature-0001.bin From marlowsd at gmail.com Tue Aug 4 07:26:57 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Tue Aug 4 07:07:48 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: Message-ID: <4A781B01.8080409@gmail.com> On 04/08/2009 12:07, Gwern Branwen wrote: > On Tue, Aug 4, 2009 at 7:02 AM, Simon Marlow wrote: >> Add >> >> * binary >> * getopt >> * gtk2hs > > A definite yes to binary and getopt; but gtk2hs? I don't trust its > longevity or maintenance, and as other people have pointed out, it will > make the platform harder to support. As well, we risk holding back the > platform - hasn't gtk2hs lagged GHC releases in the past? (I seem to > remember some of the lags being quite lengthy.) Adding gtk2hs would be a bold step, no doubt about it. By proposing it I'm hoping to force the issues to the surface: is gtk2hs the GUI lib we want to recommend, or standardise on? If it is, and it has maintenance issues, then those need to be addressed. As far as I'm aware, gtk2hs is the only plausible option for serious GUI development in Haskell at the moment. By bringing gtk2hs into the platform, we would be giving the gtk2hs maintainers a helpful boost; they'd get more testing for one thing. >> Also >> >> * keep an eye on text. We certainly want it, but it's >> a young package and there's no text I/O yet. >> * decide which regex package(s) we want >> * remove html? (we have xhtml) >> * replace haskell-src with haskell-src-exts >> * remove packedstring > > Absolutely. I thought we had already done this - didn't TH's unnecessary > use of packedstring get removed a while ago? Not in the version of TH shipping with GHC 6.10.x, but it will be gone in GHC 6.12. Cheers, Simon From ganesh.sittampalam at credit-suisse.com Tue Aug 4 07:57:12 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Tue Aug 4 07:39:44 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A781B01.8080409@gmail.com> References: <4A781B01.8080409@gmail.com> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> Simon Marlow wrote: > By bringing gtk2hs into the platform, we would be giving the gtk2hs > maintainers a helpful boost; they'd get more testing for one thing. I think that should explicitly not be a reason to bring things into the platform. Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From malcolm.wallace at cs.york.ac.uk Tue Aug 4 08:00:31 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Tue Aug 4 07:41:22 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A781B01.8080409@gmail.com> References: <4A781B01.8080409@gmail.com> Message-ID: > As far as I'm aware, gtk2hs is the only plausible option for serious > GUI development in Haskell at the moment. I have used both wxHaskell and gtk2hs, and they are both perfectly easy to install, and both are entirely adequate for serious GUI development. Just wanted to clear up any fear, uncertainty, or doubt about wx. Regards, Malcolm From Axel.Simon at ens.fr Tue Aug 4 08:21:11 2009 From: Axel.Simon at ens.fr (Axel Simon) Date: Tue Aug 4 08:02:16 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> Message-ID: <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> Hi Simon et al., On Aug 4, 2009, at 13:57, Sittampalam, Ganesh wrote: > Simon Marlow wrote: > >> By bringing gtk2hs into the platform, we would be giving the gtk2hs >> maintainers a helpful boost; they'd get more testing for one thing. > > I think that should explicitly not be a reason to bring things into > the > platform. Bringing Gtk2Hs into the platform is certainly desirable. During the last one or two years, the amount of users has grown steadily which is nice. However, Pete and my time is rather limited and is often being used up by installation issues and questions that could be answered with better documentation. Thus, it would be desirable to bring Gtk2Hs into the platform because it would force us to simplify installation and documentation. For the former part, I wonder if cabalization is important for Gtk2Hs. A cabalized version of Gtk2Hs would allow people to use Cairo and Pango to create PDF documents without the need to install the GUI parts of the library. On the contrary, if Gtk2Hs is shipped with the platform, then all libraries are available anyway and cabalization might not be as important. My question: how important is cabalization for a package that wants to be part of the platform? Axel. From marlowsd at gmail.com Tue Aug 4 08:26:35 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Tue Aug 4 08:07:27 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> Message-ID: <4A7828FB.2040609@gmail.com> On 04/08/2009 12:57, Sittampalam, Ganesh wrote: > Simon Marlow wrote: > >> By bringing gtk2hs into the platform, we would be giving the gtk2hs >> maintainers a helpful boost; they'd get more testing for one thing. > > I think that should explicitly not be a reason to bring things into the > platform. *grin* absolutely :) However, I don't think the platform should just swim around with its mouth open waiting for tasty packages to come along. Sometimes we need to make strategic decisions about what functionality is most important. Cheers, Simon From gale at sefer.org Tue Aug 4 08:29:12 2009 From: gale at sefer.org (Yitzchak Gale) Date: Tue Aug 4 08:10:21 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: <2608b8a80908040529m24617d2cq11c4d2e59837ec32@mail.gmail.com> Don Stewart wrote: > The following things were missing from the current Platform. There are many. > How would you identify the top, say, 5 libs to add? XML support is missing from your list. That is definitely part of every modern "batteries included" platform, and it should be high on our list. We have a number of excellent and mature packages in this category. Regards, Yitz From ganesh.sittampalam at credit-suisse.com Tue Aug 4 08:29:35 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Tue Aug 4 08:10:58 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7828FB.2040609@gmail.com> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <4A7828FB.2040609@gmail.com> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F875@ELON17P32001A.csfb.cs-group.com> Simon Marlow wrote: > On 04/08/2009 12:57, Sittampalam, Ganesh wrote: >> Simon Marlow wrote: >> >>> By bringing gtk2hs into the platform, we would be giving the gtk2hs >>> maintainers a helpful boost; they'd get more testing for one thing. >> >> I think that should explicitly not be a reason to bring things into >> the platform. > > *grin* absolutely :) > > However, I don't think the platform should just swim around with its > mouth open waiting for tasty packages to come along. Sometimes we > need to make strategic decisions about what functionality is most > important. Agreed, and with this in mind perhaps packages should be conditionally accepted, with a defined set of improvements leading to automatic entry. This would allow the platform to drive priorities without dumping things into it half-done. Cheers, Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From igloo at earth.li Tue Aug 4 08:47:06 2009 From: igloo at earth.li (Ian Lynagh) Date: Tue Aug 4 08:27:55 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> Message-ID: <20090804124706.GA9856@matrix.chaos.earth.li> On Tue, Aug 04, 2009 at 02:21:11PM +0200, Axel Simon wrote: > > My question: how important is cabalization for a package that wants to be > part of the platform? I think it's important. It means that: * It can be built and installed in a regular way when creating the binary platform installers, without needing special case code * It can be built and installed in a regular way in Linux distros etc, so it does not place an extra burden on distro maintainers trying to support the platform * The platform can be installed with just cabal-install (although you will also need to install the C libraries etc) * People who want to test newer versions that are intended to come with a future platform release can easily do so Thanks Ian From gale at sefer.org Tue Aug 4 09:32:47 2009 From: gale at sefer.org (Yitzchak Gale) Date: Tue Aug 4 09:13:56 2009 Subject: PROPOSAL: Remove Control.OldException In-Reply-To: <4A780DA3.1050604@gmail.com> References: <20090718125113.GA21683@matrix.chaos.earth.li> <20090718165104.GA6997@whirlpool.galois.com> <20090719172930.GA619@matrix.chaos.earth.li> <20090719185630.GA11801@whirlpool.galois.com> <5ab17e790907192327m504521b5h5b0c72ee303036d4@mail.gmail.com> <1248117111.3163.11.camel@localhost> <5ab17e790907220019l2781997v3e1b8627b4fa0c7c@mail.gmail.com> <4A780DA3.1050604@gmail.com> Message-ID: <2608b8a80908040632l5f616c32uc54c275c1247fe33@mail.gmail.com> Iavor Diatchki wrote: >> ...it would be great if we had a way to >> see what are all the extensions used by a program Yes. Simon Marlow wrote: > A prerequisite is to make it an error to use any extensions not declared in > the .cabal file. We talked about this a long time ago, but never got around > to doing anything about it. So, something like: a ghc option that says "only allow extensions listed in file x", and have cabal build write such a temporary file and then invoke this option? Note: this will cause many packages to break in the first version of cabal for which it is turned on. File this as a ticket for ghc and/or cabal? -Yitz From jeremy.odonoghue at gmail.com Tue Aug 4 10:13:16 2009 From: jeremy.odonoghue at gmail.com (Jeremy O'Donoghue) Date: Tue Aug 4 09:54:05 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <4A781B01.8080409@gmail.com> Message-ID: 2009/8/4 Malcolm Wallace : >> As far as I'm aware, gtk2hs is the only plausible option for serious GUI >> development in Haskell at the moment. > > I have used both wxHaskell and gtk2hs, and they are both perfectly easy to > install, and both are entirely adequate for serious GUI development. ?Just > wanted to clear up any fear, uncertainty, or doubt about wx. Disclaimer: I'm a wxHaskell maintainer. Thanks for the vote, Malcolm. I use wxHaskell in some fairly extensive GUI developments, and it's stable and functional. On a more serious note (and one which applies equally to wxHaskell, BTW), I think the Haskell Platform should consistently use a single license, as to do otherwise heavily complicates matters for the user. I believe that most/all of the libraries in today's platform are BSD (or other very liberal license). Gtk2Hs brings LGPL into the mix, and wxHaskell would, similarly, bring the wxWidgets license (LGPL with binary exception, basically) into the mix. As a Haskell Platform user, I really need the assurance that the licensing situation is straightforward - especially if I'm to promote Haskell at work :-) My vote would be that non-BSD/MIT license automatically excludes a library from inclusion, even though it would exclude my own project. Regards Jeremy From dons at galois.com Tue Aug 4 12:15:22 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 11:58:19 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7828FB.2040609@gmail.com> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <4A7828FB.2040609@gmail.com> Message-ID: <20090804161522.GB22753@whirlpool.galois.com> marlowsd: > On 04/08/2009 12:57, Sittampalam, Ganesh wrote: >> Simon Marlow wrote: >> >>> By bringing gtk2hs into the platform, we would be giving the gtk2hs >>> maintainers a helpful boost; they'd get more testing for one thing. >> >> I think that should explicitly not be a reason to bring things into the >> platform. > > *grin* absolutely :) > > However, I don't think the platform should just swim around with its > mouth open waiting for tasty packages to come along. Sometimes we need > to make strategic decisions about what functionality is most important. I agree with this. There are broader adoption issues we're trying to achieve in this process, after all, and having a better base lib than anyone else helps that. -- Don From jwlato at gmail.com Tue Aug 4 15:42:08 2009 From: jwlato at gmail.com (John Lato) Date: Tue Aug 4 15:22:56 2009 Subject: Thinking about what's missing in our library coverage Message-ID: <9979e72e0908041242q3be70305k3671c1359d9b8a8@mail.gmail.com> > Date: Tue, 4 Aug 2009 15:13:16 +0100 > From: "Jeremy O'Donoghue" > > My vote would be that non-BSD/MIT license automatically excludes a > library from inclusion, even though it would exclude my own project. > +1 John Lato From wren at community.haskell.org Tue Aug 4 16:08:21 2009 From: wren at community.haskell.org (wren ng thornton) Date: Tue Aug 4 15:49:12 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: <4A789535.8020801@community.haskell.org> Don Stewart wrote: > o bytestring-trie ? IntMap for ByteStrings > o dlist ? difference lists Well, if we're looking for a champion, I suggest these two should be added. Both serve demonstrated needs, both are easy to support, and both are reinvented over and over again. Because of that reinvention alone, it'd be nice to canonize a library in order to minimize wasted efforts. (If anyone has complaints about bytestring-trie I'd be more than happy to hear of and address them.) -- Live well, ~wren From dons at galois.com Tue Aug 4 16:08:03 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 15:51:02 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A789535.8020801@community.haskell.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> Message-ID: <20090804200803.GK22887@whirlpool.galois.com> wren: > Don Stewart wrote: >> o bytestring-trie ? IntMap for ByteStrings >> o dlist ? difference lists > > > Well, if we're looking for a champion, I suggest these two should be > added. Both serve demonstrated needs, both are easy to support, and both > are reinvented over and over again. Because of that reinvention alone, > it'd be nice to canonize a library in order to minimize wasted efforts. > > (If anyone has complaints about bytestring-trie I'd be more than happy > to hear of and address them.) Could you do comparative benchmarks for insertion and lookup into * Data.Map String Int * Data.Map ByteString Int * bytestring-trie I don't have a sense for how much better bytestring-trie is. dlist I think is obvious. It should really be part of base (as it is used by Show/Read in ad hoc form). -- Don From wren at community.haskell.org Tue Aug 4 16:13:01 2009 From: wren at community.haskell.org (wren ng thornton) Date: Tue Aug 4 15:53:50 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F875@ELON17P32001A.csfb.cs-group.com> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <4A7828FB.2040609@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F875@ELON17P32001A.csfb.cs-group.com> Message-ID: <4A78964D.4000000@community.haskell.org> Sittampalam, Ganesh wrote: > Simon Marlow wrote: >> Sittampalam, Ganesh wrote: >>> Simon Marlow wrote: >>>> By bringing gtk2hs into the platform, we would be giving the gtk2hs >>>> maintainers a helpful boost; they'd get more testing for one thing. >>> >>> I think that should explicitly not be a reason to bring things into >>> the platform. >> >> *grin* absolutely :) >> >> However, I don't think the platform should just swim around with its >> mouth open waiting for tasty packages to come along. Sometimes we >> need to make strategic decisions about what functionality is most >> important. > > Agreed, and with this in mind perhaps packages should be conditionally > accepted, with a defined set of improvements leading to automatic entry. > This would allow the platform to drive priorities without dumping things > into it half-done. +1. There are many packages which are almost good enough, but have one or two nagging details. I think the process for inclusion of any new package should have this sort of TODO list to ensure that the HP isn't "almost good enough", but actually lives up to its standards. -- Live well, ~wren From wren at community.haskell.org Tue Aug 4 16:24:41 2009 From: wren at community.haskell.org (wren ng thornton) Date: Tue Aug 4 16:06:10 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804200803.GK22887@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> Message-ID: <4A789909.40900@community.haskell.org> Don Stewart wrote: > wren: >> Don Stewart wrote: >>> o bytestring-trie ? IntMap for ByteStrings >>> o dlist ? difference lists >> >> Well, if we're looking for a champion, I suggest these two should be >> added. Both serve demonstrated needs, both are easy to support, and both >> are reinvented over and over again. Because of that reinvention alone, >> it'd be nice to canonize a library in order to minimize wasted efforts. >> >> (If anyone has complaints about bytestring-trie I'd be more than happy >> to hear of and address them.) > > Could you do comparative benchmarks for insertion and lookup into > > * Data.Map String Int > * Data.Map ByteString Int > * bytestring-trie > > I don't have a sense for how much better bytestring-trie is. From Mark Wotton on 2009.03.01 using Microbench hacked to use Integers instead of Ints (to avoid overflow bugs): * Data.List.lookup [(ByteString, Int)]: 160.641ns per iteration / 6225.07 per second. * Data.Map.lookup (Map ByteString Int): 0.881ns per iteration / 1135623.22 per second. * Data.Trie.lookup (Trie Int): 0.243ns per iteration / 4116930.41 per second. I'll try to set up a benchmarking suite to test more recent versions and other functions in the interface. -- Live well, ~wren From dons at galois.com Tue Aug 4 16:25:24 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 16:08:25 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A789909.40900@community.haskell.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> Message-ID: <20090804202524.GL22887@whirlpool.galois.com> wren: > Don Stewart wrote: >> wren: >>> Don Stewart wrote: >>>> o bytestring-trie ? IntMap for ByteStrings >>>> o dlist ? difference lists >>> >>> Well, if we're looking for a champion, I suggest these two should be >>> added. Both serve demonstrated needs, both are easy to support, and >>> both are reinvented over and over again. Because of that reinvention >>> alone, it'd be nice to canonize a library in order to minimize >>> wasted efforts. >>> >>> (If anyone has complaints about bytestring-trie I'd be more than >>> happy to hear of and address them.) >> >> Could you do comparative benchmarks for insertion and lookup into >> >> * Data.Map String Int >> * Data.Map ByteString Int >> * bytestring-trie >> >> I don't have a sense for how much better bytestring-trie is. > > From Mark Wotton on 2009.03.01 using Microbench hacked to use Integers > instead of Ints (to avoid overflow bugs): > > * Data.List.lookup [(ByteString, Int)]: > 160.641ns per iteration / 6225.07 per second. > > * Data.Map.lookup (Map ByteString Int): > 0.881ns per iteration / 1135623.22 per second. > > * Data.Trie.lookup (Trie Int): > 0.243ns per iteration / 4116930.41 per second. > > > I'll try to set up a benchmarking suite to test more recent versions and > other functions in the interface. Thanks! A maintainable testsuite (so we can check this again in the future) will be useful! From dons at galois.com Tue Aug 4 16:38:14 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 16:21:09 2009 Subject: HOWTO: Propose a package be added to the Haskell Platform Message-ID: <20090804203814.GM22887@whirlpool.galois.com> To propose a package be added to the Haskell Platform: http://trac.haskell.org/haskell-platform/wiki/WikiStart * Click on "Propose adding a new package" * Fill out the heuristics/background as best as you can -- Don From qdunkan at gmail.com Tue Aug 4 17:01:53 2009 From: qdunkan at gmail.com (Evan Laforge) Date: Tue Aug 4 16:42:41 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804202524.GL22887@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> <20090804202524.GL22887@whirlpool.galois.com> Message-ID: <2518b95d0908041401y1394ea1bx91673cf66a29fe0d@mail.gmail.com> >> I'll try to set up a benchmarking suite to test more recent versions and >> other functions in the interface. > > Thanks! > > A maintainable testsuite (so we can check this again in the future) will > be useful! I've occasionally wished for a speed and memory test suite for maps. There are a lot of implementations for haskell, with different tradeoffs, and it would be nice to quantify someone's assertion that "X is so much better than Y" or test a new implementation. This is one area where haskell is much more complicated than an imperative language like python, where you just use a built-in hashmap and performance is going to be basically good for just about all uses. As an aside, definitely +1 on including dlist. It's very useful for Writer, so much so I have my own type aliases for it. However, the name is anything but intuitive to someone looking for a list with efficient appends. Given all the above, one nice contribution of HP could be to package together data structures with the promise that their use and performance are mostly orthogonal and they represent the current best practices for a given access pattern, and documentation describing the differences. I.e. we have list -> dlist -> finger tree sequence with roughly increasing capabilities but also increasing constant costs (I'm guessing) and the same sort of story with maps and arrays. Is documentation part of the goal for HP? Is there a place to put it? It would be nice if it could be integrated into the haddock in a discoverable way, i.e. attached to Data or something, or linked off the main TOC. And it would be nice in general if package haddocks could be linked (or maybe they already can?). I'd be willing to make a start on some documentation and some benchmarks for various access patterns with nice graphs and stuff, if there's interest. From malcolm.wallace at cs.york.ac.uk Tue Aug 4 17:04:25 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Tue Aug 4 16:45:16 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78308B.2060601@o-donoghue.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> Message-ID: <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> > As a Haskell Platform user, I really need the assurance that the > licensing situation is straightforward - especially if I'm to promote > Haskell at work :-) > > My vote would be that non-BSD/MIT license automatically excludes a > library from inclusion, even though it would exclude my own project. I wonder if it would be possible to split the Haskell Platform into two parts, platform-BSD and platform-LGPL? The LGPL packages could depend on packages from platform-BSD, but not the other way around. This would guarantee at least one aspect of licence compliance, whilst also making it clear to proprietary users that if they want to avoid the guarantee of freedom, they can simply avoid installing platform- LGPL. Regards, Malcolm From ml at isaac.cedarswampstudios.org Tue Aug 4 17:27:38 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Tue Aug 4 17:08:33 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A789909.40900@community.haskell.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> Message-ID: <4A78A7CA.9060404@isaac.cedarswampstudios.org> >> * Data.Map String Int >> * Data.Map ByteString Int >> * bytestring-trie What about a Unicode/text aware version of ByteString? I mean, I suspect people will want to use this map for strings of characters, and I want to make sure we don't hardcode a type that will cause us trouble in the future (and I have this idea in my head that ByteString is supposed to represent byte-sequences and not character-sequences). Just checking -Isaac From dons at galois.com Tue Aug 4 17:39:07 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 17:22:06 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> Message-ID: <20090804213907.GN22887@whirlpool.galois.com> malcolm.wallace: >> As a Haskell Platform user, I really need the assurance that the >> licensing situation is straightforward - especially if I'm to promote >> Haskell at work :-) >> >> My vote would be that non-BSD/MIT license automatically excludes a >> library from inclusion, even though it would exclude my own project. > > I wonder if it would be possible to split the Haskell Platform into two > parts, platform-BSD and platform-LGPL? The LGPL packages could depend on > packages from platform-BSD, but not the other way around. This would > guarantee at least one aspect of licence compliance, whilst also making > it clear to proprietary users that if they want to avoid the guarantee of > freedom, they can simply avoid installing platform-LGPL. > As an aside: are you aware of the problems using LGPL in the context of GHC statically linking libraries by default (in that they degenerate to GPL, or require significant extra workaround). I'm concerned the burden for satisfying the LGPL while GHC statically links Haskell libraries is too great to impose. -- Don From magnus at therning.org Tue Aug 4 18:02:03 2009 From: magnus at therning.org (Magnus Therning) Date: Tue Aug 4 17:42:55 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804213907.GN22887@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> Message-ID: <4A78AFDB.8030806@therning.org> Don Stewart wrote: > malcolm.wallace: >>> As a Haskell Platform user, I really need the assurance that the >>> licensing situation is straightforward - especially if I'm to promote >>> Haskell at work :-) >>> >>> My vote would be that non-BSD/MIT license automatically excludes a >>> library from inclusion, even though it would exclude my own project. >> I wonder if it would be possible to split the Haskell Platform into two >> parts, platform-BSD and platform-LGPL? The LGPL packages could depend on >> packages from platform-BSD, but not the other way around. This would >> guarantee at least one aspect of licence compliance, whilst also making >> it clear to proprietary users that if they want to avoid the guarantee of >> freedom, they can simply avoid installing platform-LGPL. >> > > As an aside: are you aware of the problems using LGPL in the context of > GHC statically linking libraries by default (in that they degenerate to > GPL, or require significant extra workaround). > > I'm concerned the burden for satisfying the LGPL while GHC statically > links Haskell libraries is too great to impose. AFAIU the burden remains even when GHC supports dynamic linking of Haskell libraries. The goal of introducing dynamic lib support is sharing of code in system memory, the goal _isn't_ to allow upgrading libraries independently of the executables using them. /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus?therning?org Jabber: magnus?therning?org http://therning.org/magnus identi.ca|twitter: magthe -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: OpenPGP digital signature Url : http://www.haskell.org/pipermail/libraries/attachments/20090804/0edec75b/signature.bin From dons at galois.com Tue Aug 4 18:05:23 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 17:48:31 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78AFDB.8030806@therning.org> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> Message-ID: <20090804220523.GA23958@whirlpool.galois.com> magnus: > Don Stewart wrote: >> malcolm.wallace: >>>> As a Haskell Platform user, I really need the assurance that the >>>> licensing situation is straightforward - especially if I'm to promote >>>> Haskell at work :-) >>>> >>>> My vote would be that non-BSD/MIT license automatically excludes a >>>> library from inclusion, even though it would exclude my own project. >>> I wonder if it would be possible to split the Haskell Platform into >>> two parts, platform-BSD and platform-LGPL? The LGPL packages could >>> depend on packages from platform-BSD, but not the other way around. >>> This would guarantee at least one aspect of licence compliance, >>> whilst also making it clear to proprietary users that if they want to >>> avoid the guarantee of freedom, they can simply avoid installing >>> platform-LGPL. >>> >> >> As an aside: are you aware of the problems using LGPL in the context of >> GHC statically linking libraries by default (in that they degenerate to >> GPL, or require significant extra workaround). >> >> I'm concerned the burden for satisfying the LGPL while GHC statically >> links Haskell libraries is too great to impose. > > AFAIU the burden remains even when GHC supports dynamic linking of > Haskell libraries. The goal of introducing dynamic lib support is > sharing of code in system memory, the goal _isn't_ to allow upgrading > libraries independently of the executables using them. > I would appreciate input from the HaXml and HDBC authors (our most popular LGPL-licensed Haskell libraries) about what they feel the licensing issues/constraints should be for the Haskell Platform. Here for example, is how we satisfy LGPL with the C library GMP, which is often statically linked, http://haskell.forkio.com/gmpwindows I've not yet seen anyone publish something on how to satisfy LGPL for Haskell libraries. -- Don From dons at galois.com Tue Aug 4 18:55:41 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 18:38:31 2009 Subject: Adding binary to the Haskell Platform Message-ID: <20090804225541.GF23958@whirlpool.galois.com> Here's a ticket for Simon Marlow's proposal: http://trac.haskell.org/haskell-platform/ticket/86 Let's discuss, then have the steering committee recommend yay/nay. -- Don From dons at galois.com Tue Aug 4 18:59:17 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 18:42:07 2009 Subject: Discussion period Message-ID: <20090804225917.GG23958@whirlpool.galois.com> The Haskell Platform follows the GNOME strategy for releases, with a release cycle and freeze date. We're now in the open package proposal and discussion period, which will close N weeks before the next freeze. I'm not sure what N should be. -- Don From wren at community.haskell.org Tue Aug 4 20:02:25 2009 From: wren at community.haskell.org (wren ng thornton) Date: Tue Aug 4 19:43:15 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78A7CA.9060404@isaac.cedarswampstudios.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> <4A78A7CA.9060404@isaac.cedarswampstudios.org> Message-ID: <4A78CC11.7050808@community.haskell.org> Isaac Dupree wrote: >>> * Data.Map String Int >>> * Data.Map ByteString Int >>> * bytestring-trie > > What about a Unicode/text aware version of ByteString? I mean, I > suspect people will want to use this map for strings of characters, and > I want to make sure we don't hardcode a type that will cause us trouble > in the future (and I have this idea in my head that ByteString is > supposed to represent byte-sequences and not character-sequences). The bytestring-trie package uses ByteStrings as a vector of bytes. That is, there's no built in support for or against textual data (modulo byte==char traditions). Anything that can be rendered into a ByteString can be tried (with performance depending on the suitability of the encoding). I can't really see any way to make use of knowing that the data is textual, though. I've thought about adding a typeclass to automate the encoding/decoding between various "string" types and the ByteStrings used internally. Unfortunately such a class would have a very large number of methods, and is of dubious utility in the big picture. All in all, the problem of rendering an abstract string into a sequence of bytes belongs to another package. -- Live well, ~wren From dons at galois.com Tue Aug 4 20:04:32 2009 From: dons at galois.com (Don Stewart) Date: Tue Aug 4 19:47:24 2009 Subject: Hackage Download Rankings August 2009 Message-ID: <20090805000432.GH23958@whirlpool.galois.com> Total package downloads on Hackage, by August 1 2009, http://www.galois.com/~dons/hackage/august-2009/popularity-august-2009.html All packages by month: http://www.galois.com/~dons/hackage/august-2009/hackage-august-2009.html Raw data: http://www.galois.com/~dons/hackage/august-2009/hackage-downloads-august-2009.csv Enjoy. -- Don P.S. For reference, here's the data from March, http://www.galois.com/~dons/hackage/popularity.html and analysis at the time http://www.galois.com/blog/2009/03/23/one-million-haskell-downloads/ From bos at serpentine.com Tue Aug 4 23:49:57 2009 From: bos at serpentine.com (Bryan O'Sullivan) Date: Tue Aug 4 23:30:45 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78A7CA.9060404@isaac.cedarswampstudios.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> <4A78A7CA.9060404@isaac.cedarswampstudios.org> Message-ID: On Tue, Aug 4, 2009 at 2:27 PM, Isaac Dupree wrote: > * Data.Map String Int >>> * Data.Map ByteString Int >>> * bytestring-trie >>> >> > What about a Unicode/text aware version of ByteString? I assume you're not asking in the context of map-like structures, but in general? If so, look at the text and text-icu packages on Hackage. Regards, Bryan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090804/4c57c617/attachment.html From bos at serpentine.com Tue Aug 4 23:51:38 2009 From: bos at serpentine.com (Bryan O'Sullivan) Date: Tue Aug 4 23:38:32 2009 Subject: Discussion period In-Reply-To: <20090804225917.GG23958@whirlpool.galois.com> References: <20090804225917.GG23958@whirlpool.galois.com> Message-ID: On Tue, Aug 4, 2009 at 3:59 PM, Don Stewart wrote: > The Haskell Platform follows the GNOME strategy for releases, with > a release cycle and freeze date. > > We're now in the open package proposal and discussion period, which > will close N weeks before the next freeze. > I'd give myself six to eight weeks to sort out new packages, their build issues, and their interdependencies prior to making a release, if I were in your shoes. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090804/0634e6e6/attachment-0001.html From ashley at semantic.org Wed Aug 5 00:00:39 2009 From: ashley at semantic.org (Ashley Yakeley) Date: Tue Aug 4 23:41:28 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> Message-ID: <4A7903E7.9090609@semantic.org> Axel Simon wrote: > For the former part, I wonder if cabalization is important for Gtk2Hs. A > cabalized version of Gtk2Hs would allow people to use Cairo and Pango to > create PDF documents without the need to install the GUI parts of the > library. On the contrary, if Gtk2Hs is shipped with the platform, then > all libraries are available anyway and cabalization might not be as > important. I'm a big fan of Gtk2Hs and would really like to see it cabalised (as several packages, most likely), though I can see how that's work. Right now it's a bit annoying to have to go through a separate installation process to install and upgrade it. > My question: how important is cabalization for a package that wants to > be part of the platform? I think it ought to be required. -- Ashley Yakeley From matti.niemenmaa+news at iki.fi Wed Aug 5 01:38:12 2009 From: matti.niemenmaa+news at iki.fi (Matti Niemenmaa) Date: Wed Aug 5 01:19:13 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A789909.40900@community.haskell.org> References: <20090803234432.GK17991@whirlpool.galois.com> <4A789535.8020801@community.haskell.org> <20090804200803.GK22887@whirlpool.galois.com> <4A789909.40900@community.haskell.org> Message-ID: wren ng thornton wrote: > Don Stewart wrote: >> Could you do comparative benchmarks for insertion and lookup into >> >> * Data.Map String Int >> * Data.Map ByteString Int >> * bytestring-trie >> >> I don't have a sense for how much better bytestring-trie is. > I'll try to set up a benchmarking suite to test more recent versions and > other functions in the interface. If you're going to look at Map String as well as Map ByteString, I hope you wouldn't mind tossing the list-tries package (http://hackage.haskell.org/package/list-tries) into the mix. A Patricia trie with Enum keys (since we're dealing with Chars) from Data.ListTrie.Patricia.Map.Enum should beat Data.Map, at least. I've been meaning to benchmark my library myself but I haven't found the time or energy to do so. From kr.angelov at gmail.com Wed Aug 5 03:15:51 2009 From: kr.angelov at gmail.com (Krasimir Angelov) Date: Wed Aug 5 02:56:39 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090804225541.GF23958@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: The binary package is too lazy and in some cases the decoding dies with "stack overflow". At least this was the situation the last time when I tried to use it. Since then I am using hacked version which is stricter. This should be addressed before to add this package to the platform. Another problem that I see is that Int is serialized as 64 bit integer. This makes the serialized data too large. The Int type is also used internally, for example to store the length of a list, the size of a map, etc, so you can't avoid the problem by using custom putInt function. In fact the binary representation is so verbose that sometimes it is more compact to use textual representation. In my hacked version I use serialization for integers which use variable number of bytes. If someone wants to store 64 bit integer then he/she could always use Int64. I know that the binary package could be combined with gzip to make the data more compact but this is unnecessary overhead. Krasimir On 8/5/09, Don Stewart wrote: > Here's a ticket for Simon Marlow's proposal: > > http://trac.haskell.org/haskell-platform/ticket/86 > > Let's discuss, then have the steering committee recommend yay/nay. > > -- Don > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > From malcolm.wallace at cs.york.ac.uk Wed Aug 5 03:22:35 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Wed Aug 5 03:03:23 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804213907.GN22887@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> Message-ID: <2DBF4BE6-1778-498B-AC47-DFAFAE4243D9@cs.york.ac.uk> > As an aside: are you aware of the problems using LGPL in the context > of > GHC statically linking libraries by default (in that they degenerate > to > GPL, or require significant extra workaround). Given that GHC has statically linked *all* the binaries it produces against an LGPL library for at least the last 15 years, I think it is a little bit late for everyone else to start worrying about it! I am aware of the issue (and indeed use the static linking exception in all of my own LGPL-released code) but in any case, dynamic linking (by default) will soon be coming to an optimising compiler near you (or so I hear from the people who are paying for it.) Regards, Malcolm From magnus at therning.org Wed Aug 5 03:40:41 2009 From: magnus at therning.org (Magnus Therning) Date: Wed Aug 5 03:21:28 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <2DBF4BE6-1778-498B-AC47-DFAFAE4243D9@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <2DBF4BE6-1778-498B-AC47-DFAFAE4243D9@cs.york.ac.uk> Message-ID: On Wed, Aug 5, 2009 at 8:22 AM, Malcolm Wallace wrote: >> As an aside: are you aware of the problems using LGPL in the context of >> GHC statically linking libraries by default (in that they degenerate to >> GPL, or require significant extra workaround). > > Given that GHC has statically linked *all* the binaries it produces against > an LGPL library for at least the last 15 years, I think it is a little bit > late for everyone else to start worrying about it! Which library is that? > I am aware of the issue (and indeed use the static linking exception in all > of my own LGPL-released code) but in any case, dynamic linking (by default) > will soon be coming to an optimising compiler near you (or so I hear from > the people who are paying for it.) AFAIU it's important to make the distinction between linking against a C library and a Haskell library. IIRC the former can be modified and replaced without recompiling the whole Haskell program, while the latter can't. /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus?therning?org Jabber: magnus?therning?org http://therning.org/magnus identi.ca|twitter: magthe From malcolm.wallace at cs.york.ac.uk Wed Aug 5 03:44:16 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Wed Aug 5 03:25:04 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090804225541.GF23958@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> > Let's discuss, then have the steering committee recommend yay/nay. We should have _some_ kind of binary library in the Platform, but I don't know whether the proposed library is the right one. In a recent application, I found Data.Binary very slow, both to encode and to decode data. Decoding had stack-overflows, and when I increased the stack, it took about 20mins to read in an 8Mb file. When I then turned on optimisation with -O, the performance improved considerably (down to ~30secs to read the same file). This was using the standard instances of Binary for data structures like Data.Map. 30secs was still too slow, so we ended up needing to write our own improved instance for Data.Map. (Timings were similar with both ghc-6.8.3 and ghc-6.10.3.) Regards, Malcolm From kr.angelov at gmail.com Wed Aug 5 03:51:14 2009 From: kr.angelov at gmail.com (Krasimir Angelov) Date: Wed Aug 5 03:32:03 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> References: <20090804225541.GF23958@whirlpool.galois.com> <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> Message-ID: The stricter version of Data.Binary is here: http://code.haskell.org/gf/src/Data It avoids the stack overflow and *might* be faster. On 8/5/09, Malcolm Wallace wrote: > > Let's discuss, then have the steering committee recommend yay/nay. > > > > We should have _some_ kind of binary library in the Platform, but I don't > know whether the proposed library is the right one. > > In a recent application, I found Data.Binary very slow, both to encode and > to decode data. Decoding had stack-overflows, and when I increased the > stack, it took about 20mins to read in an 8Mb file. When I then turned on > optimisation with -O, the performance improved considerably (down to ~30secs > to read the same file). This was using the standard instances of Binary for > data structures like Data.Map. 30secs was still too slow, so we ended up > needing to write our own improved instance for Data.Map. (Timings were > similar with both ghc-6.8.3 and ghc-6.10.3.) > > Regards, > Malcolm > > > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > From marlowsd at gmail.com Wed Aug 5 04:13:06 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Wed Aug 5 03:53:55 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78AFDB.8030806@therning.org> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> Message-ID: <4A793F12.6060802@gmail.com> On 04/08/2009 23:02, Magnus Therning wrote: > Don Stewart wrote: >> malcolm.wallace: >>>> As a Haskell Platform user, I really need the assurance that the >>>> licensing situation is straightforward - especially if I'm to promote >>>> Haskell at work :-) >>>> >>>> My vote would be that non-BSD/MIT license automatically excludes a >>>> library from inclusion, even though it would exclude my own project. >>> I wonder if it would be possible to split the Haskell Platform into >>> two parts, platform-BSD and platform-LGPL? The LGPL packages could >>> depend on packages from platform-BSD, but not the other way around. >>> This would guarantee at least one aspect of licence compliance, >>> whilst also making it clear to proprietary users that if they want to >>> avoid the guarantee of freedom, they can simply avoid installing >>> platform-LGPL. >>> >> >> As an aside: are you aware of the problems using LGPL in the context of >> GHC statically linking libraries by default (in that they degenerate to >> GPL, or require significant extra workaround). >> >> I'm concerned the burden for satisfying the LGPL while GHC statically >> links Haskell libraries is too great to impose. > > AFAIU the burden remains even when GHC supports dynamic linking of > Haskell libraries. The goal of introducing dynamic lib support is > sharing of code in system memory, the goal _isn't_ to allow upgrading > libraries independently of the executables using them. The latter is a goal, just not one that we'll be achieving right away. And indeed LGPL compliance is one good reason to want it. In order to make a library that supports binary-compatible upgrades, you will have to sacrifice some of the cross-module optimisations that GHC does. We plan to first of all make this possible, and secondly to allow some explicitly-declared cross-module optimisations to take place. In GHC 6.12, each installed package will have a unique Id based on its visible ABI. The hash is derived by GHC from the interface files. Package dependencies use this hash, so when upgrading a package we can detect when the replacement is not binary compatible, and the dependencies will be broken (none of this is actually committed yet, but I posted the first part on the cabal-devel list for review). Once we have this working, we can start building support for compiling packages with a stable/predictable ABI. Cheers, Simon From marlowsd at gmail.com Wed Aug 5 04:14:29 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Wed Aug 5 03:55:20 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <2DBF4BE6-1778-498B-AC47-DFAFAE4243D9@cs.york.ac.uk> Message-ID: <4A793F65.3020009@gmail.com> On 05/08/2009 08:40, Magnus Therning wrote: > On Wed, Aug 5, 2009 at 8:22 AM, Malcolm > Wallace wrote: >>> As an aside: are you aware of the problems using LGPL in the context of >>> GHC statically linking libraries by default (in that they degenerate to >>> GPL, or require significant extra workaround). >> >> Given that GHC has statically linked *all* the binaries it produces against >> an LGPL library for at least the last 15 years, I think it is a little bit >> late for everyone else to start worrying about it! > > Which library is that? GMP, and only on Windows. On Unix systems we normally dynamically link to GMP, if it is availble. Cheers, Simon From malcolm.wallace at cs.york.ac.uk Wed Aug 5 04:16:03 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Wed Aug 5 03:56:51 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804220523.GA23958@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> Message-ID: <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> On 4 Aug 2009, at 23:05, Don Stewart wrote: > > I would appreciate input from the HaXml and HDBC authors (our most > popular LGPL-licensed Haskell libraries) about what they feel the > licensing issues/constraints should be for the Haskell Platform. Licensing clarity is important for users I think. But equally some users may desire to use LGPL libraries too. Hence my suggestion that there be a separate platform of free/LGPL code (and GPL tools), which can depend on the proprietary-friendly BSD-licensed platform, but not the other way round. > I've not yet seen anyone publish something on how to satisfy LGPL > for Haskell libraries. The static-linking exception is the commonest means of working around ghc's technical limitations here. The exception is part of wxHaskell's license (but not Gtk2hs's), and HaXml (+polyparse on which it depends) has the exception too. Regards, Malcolm From marlowsd at gmail.com Wed Aug 5 04:58:36 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Wed Aug 5 04:39:25 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804220523.GA23958@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> Message-ID: <4A7949BC.7010602@gmail.com> On 04/08/2009 23:05, Don Stewart wrote: > magnus: >> Don Stewart wrote: >>> malcolm.wallace: >>>>> As a Haskell Platform user, I really need the assurance that the >>>>> licensing situation is straightforward - especially if I'm to promote >>>>> Haskell at work :-) >>>>> >>>>> My vote would be that non-BSD/MIT license automatically excludes a >>>>> library from inclusion, even though it would exclude my own project. >>>> I wonder if it would be possible to split the Haskell Platform into >>>> two parts, platform-BSD and platform-LGPL? The LGPL packages could >>>> depend on packages from platform-BSD, but not the other way around. >>>> This would guarantee at least one aspect of licence compliance, >>>> whilst also making it clear to proprietary users that if they want to >>>> avoid the guarantee of freedom, they can simply avoid installing >>>> platform-LGPL. >>>> >>> >>> As an aside: are you aware of the problems using LGPL in the context of >>> GHC statically linking libraries by default (in that they degenerate to >>> GPL, or require significant extra workaround). >>> >>> I'm concerned the burden for satisfying the LGPL while GHC statically >>> links Haskell libraries is too great to impose. >> >> AFAIU the burden remains even when GHC supports dynamic linking of >> Haskell libraries. The goal of introducing dynamic lib support is >> sharing of code in system memory, the goal _isn't_ to allow upgrading >> libraries independently of the executables using them. >> > > I would appreciate input from the HaXml and HDBC authors (our most > popular LGPL-licensed Haskell libraries) about what they feel the > licensing issues/constraints should be for the Haskell Platform. > > Here for example, is how we satisfy LGPL with the C library GMP, which > is often statically linked, > > http://haskell.forkio.com/gmpwindows > > I've not yet seen anyone publish something on how to satisfy LGPL > for Haskell libraries. I support the idea of having a separate LGPL platform. Furthermore, we should strive to ensure that the BSD-only part of the platform covers all the core functionality. I don't think that will be too much of a burden, except in the case of a GUI lib where we'll be restricted to GLUT. Cheers, Simon From marlowsd at gmail.com Wed Aug 5 05:05:30 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Wed Aug 5 04:46:23 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> References: <20090804225541.GF23958@whirlpool.galois.com> <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> Message-ID: <4A794B5A.8080905@gmail.com> On 05/08/2009 08:44, Malcolm Wallace wrote: >> Let's discuss, then have the steering committee recommend yay/nay. > > We should have _some_ kind of binary library in the Platform, but I > don't know whether the proposed library is the right one. > > In a recent application, I found Data.Binary very slow, both to encode > and to decode data. Decoding had stack-overflows, and when I increased > the stack, it took about 20mins to read in an 8Mb file. When I then > turned on optimisation with -O, the performance improved considerably > (down to ~30secs to read the same file). This was using the standard > instances of Binary for data structures like Data.Map. 30secs was still > too slow, so we ended up needing to write our own improved instance for > Data.Map. (Timings were similar with both ghc-6.8.3 and ghc-6.10.3.) Ok, the way to proceed is to build a set of benchmarks. I did some rough timings myself recently, and found that binary was roughly comparable to the Binary module in GHC, which as far as I know is fairly fast (though I know there are faster libraries out there). I suspect we're all measuring different things. Would someone like to work on benchmarking binary and identifying the weak points? Cheers, Simon From Christian.Maeder at dfki.de Wed Aug 5 05:12:01 2009 From: Christian.Maeder at dfki.de (Christian Maeder) Date: Wed Aug 5 04:52:52 2009 Subject: Hackage Download Rankings August 2009 In-Reply-To: <20090805000432.GH23958@whirlpool.galois.com> References: <20090805000432.GH23958@whirlpool.galois.com> Message-ID: <4A794CE1.8070600@dfki.de> Don Stewart wrote: > Total package downloads on Hackage, by August 1 2009, > > http://www.galois.com/~dons/hackage/august-2009/popularity-august-2009.html 1 xmonad 39229 2 HTTP 37058 3 zlib 33065 4 Cabal 27314 5 X11 24590 How should I interpret the xmonad count as it relies on X11? X11 is not shipped with recent binary distributions (whereas Cabal is). Cheers Christian From bulat.ziganshin at gmail.com Wed Aug 5 06:48:05 2009 From: bulat.ziganshin at gmail.com (Bulat Ziganshin) Date: Wed Aug 5 06:29:07 2009 Subject: Hackage Download Rankings August 2009 In-Reply-To: <4A794CE1.8070600@dfki.de> References: <20090805000432.GH23958@whirlpool.galois.com> <4A794CE1.8070600@dfki.de> Message-ID: <76304136.20090805144805@gmail.com> Hello Christian, Wednesday, August 5, 2009, 1:12:01 PM, you wrote: > 1 xmonad 39229 > 5 X11 24590 > How should I interpret the xmonad count as it relies on X11? X11 is not > shipped with recent binary distributions (whereas Cabal is). more frequent updates? :) -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com From dbueno at gmail.com Wed Aug 5 08:48:02 2009 From: dbueno at gmail.com (Denis Bueno) Date: Wed Aug 5 08:28:48 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: <6dbd4d000908050548k522c5b81p6433a5eb86295593@mail.gmail.com> On Wed, Aug 5, 2009 at 01:15, Krasimir Angelov wrote: > The binary package is too lazy and in some cases the decoding dies > with "stack overflow". At least this was the situation the last time > when I tried to use it. Since then I am using hacked version which is > stricter. This should be addressed before to add this package to the > platform. Not all encodings are right for everyone. IIRC, many complaints about binary are due to overflows when encoding/decoding large maps or lists. This is because of the [] instance. I, myself, have been bitten by this, but I don't think it's a flaw in the library -- there are plenty of people using the library who haven't complained about the instance. If the instance isn't right for you, write a new one. That's what I did. Denis From felipe.lessa at gmail.com Wed Aug 5 09:23:34 2009 From: felipe.lessa at gmail.com (Felipe Lessa) Date: Wed Aug 5 09:04:24 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <6dbd4d000908050548k522c5b81p6433a5eb86295593@mail.gmail.com> References: <20090804225541.GF23958@whirlpool.galois.com> <6dbd4d000908050548k522c5b81p6433a5eb86295593@mail.gmail.com> Message-ID: <20090805132334.GB16369@kira.casa> On Wed, Aug 05, 2009 at 06:48:02AM -0600, Denis Bueno wrote: > [...] This is because of the [] instance. I, myself, have been > bitten by this, but I don't think it's a flaw in the library -- there > are plenty of people using the library who haven't complained about > the instance. > > If the instance isn't right for you, write a new one. That's what I did. The problem is that people shouldn't be reinventing the wheel everytime. I propose adding something along the lines of this snippet to binary (untested): import Control.Applicative import Control.Monad (replicateM_) -- | Provides a new instance of 'Binary' to lists that 'put's and -- 'get's using chunks instead of forcing the whole spine of -- the list before writing the first byte. This enconding is -- less space-efficient in the disk, though, having an overhead -- of @1 + (length xs `div` 255)@ bytes instead of only four -- bytes (independently of list size), the overhead of the -- default instance. newtype Chunked a = Chunked {unChunked :: [a]} instance Binary (Chunked a) where -- This 'put' should be good enough. put = mapM_ putChunk . chunks 255 . unChunked where chunks 255 [] = [(0,[])] -- not []! chunks _ [] = [] chunks _ xs = let (i,f) = splitAt 255 xs len = length i in (len,i) : chunks len f putChunk (len,xs) = putWord8 len >> mapM_ put xs -- I don't know if this get works nicely, though. get = getWord8 >>= go [] where go acc 0 = return $ concat $ reverse acc go acc len = do xs <- replicateM_ len get len' <- if len == 255 then getWord8 else return 0 go (xs:acc) len' -- Felipe. From Axel.Simon at ens.fr Wed Aug 5 09:37:55 2009 From: Axel.Simon at ens.fr (Axel Simon) Date: Wed Aug 5 09:19:00 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> Message-ID: <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> On Aug 5, 2009, at 10:16, Malcolm Wallace wrote: > On 4 Aug 2009, at 23:05, Don Stewart wrote: >> >> I would appreciate input from the HaXml and HDBC authors (our most >> popular LGPL-licensed Haskell libraries) about what they feel the >> licensing issues/constraints should be for the Haskell Platform. > > Licensing clarity is important for users I think. But equally some > users may desire to use LGPL libraries too. Hence my suggestion > that there be a separate platform of free/LGPL code (and GPL > tools), which can depend on the proprietary-friendly BSD-licensed > platform, but not the other way round. > >> I've not yet seen anyone publish something on how to satisfy LGPL >> for Haskell libraries. > > The static-linking exception is the commonest means of working > around ghc's technical limitations here. The exception is part of > wxHaskell's license (but not Gtk2hs's), and HaXml (+polyparse on > which it depends) has the exception too. I don't think it would be much of a problem to weaken the license of Gtk2Hs to a BSD license. The underlying Gtk+ C library is, of course, LGPL but the C library can be linked in dynamically. Axel. From ndmitchell at gmail.com Wed Aug 5 09:40:54 2009 From: ndmitchell at gmail.com (Neil Mitchell) Date: Wed Aug 5 09:21:44 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805132334.GB16369@kira.casa> References: <20090804225541.GF23958@whirlpool.galois.com> <6dbd4d000908050548k522c5b81p6433a5eb86295593@mail.gmail.com> <20090805132334.GB16369@kira.casa> Message-ID: <404396ef0908050640n7afe2b29la35657e523b0c742@mail.gmail.com> Hi Since we're pointing out flaws in Data.Binary, I might as well add mine too :-) I found that encode/decode of a String was massively slower than serialising a String which I converted to a bytestring on the way in and on the way out. They are exactly equivalent (at the binary representation and the interface level), but String is clearly inefficient. I also noticed that for String one way round was far slower than the other, I think decoding was much slower, which was surprising since encoding should have to do more work (length calls etc) Unfortunatley I got busy with work and never managed to write all the details down, but this might be a good place to start benchmarking. Thanks Neil On Wed, Aug 5, 2009 at 2:23 PM, Felipe Lessa wrote: > On Wed, Aug 05, 2009 at 06:48:02AM -0600, Denis Bueno wrote: >> [...] This is because of the [] instance. ?I, myself, have been >> bitten by this, but I don't think it's a flaw in the library -- there >> are plenty of people using the library who haven't complained about >> the instance. >> >> If the instance isn't right for you, write a new one. ?That's what I did. > > The problem is that people shouldn't be reinventing the wheel > everytime. ?I propose adding something along the lines of this > snippet to binary (untested): > > > import Control.Applicative > import Control.Monad (replicateM_) > > -- | Provides a new instance of 'Binary' to lists that 'put's and > -- ? 'get's using chunks instead of forcing the whole spine of > -- ? the list before writing the first byte. ?This enconding is > -- ? less space-efficient in the disk, though, having an overhead > -- ? of @1 + (length xs `div` 255)@ bytes instead of only four > -- ? bytes (independently of list size), the overhead of the > -- ? default instance. > newtype Chunked a = Chunked {unChunked :: [a]} > > instance Binary (Chunked a) where > ? ?-- This 'put' should be good enough. > ? ?put = mapM_ putChunk . chunks 255 . unChunked > ? ? ? ?where chunks 255 [] = [(0,[])] -- not []! > ? ? ? ? ? ? ?chunks _ ? [] = [] > ? ? ? ? ? ? ?chunks _ ? xs = let (i,f) = splitAt 255 xs > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?len ? = length i > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?in (len,i) : chunks len f > ? ? ? ? ? ? ?putChunk (len,xs) = putWord8 len >> mapM_ put xs > > ? ?-- I don't know if this get works nicely, though. > ? ?get = getWord8 >>= go [] > ? ? ? ?where go acc 0 ? = return $ concat $ reverse acc > ? ? ? ? ? ? ?go acc len = do xs ? <- replicateM_ len get > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?len' <- if len == 255 then getWord8 else return 0 > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?go (xs:acc) len' > > -- > Felipe. > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > From malcolm.wallace at cs.york.ac.uk Wed Aug 5 09:44:26 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Wed Aug 5 09:25:12 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> Message-ID: <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> > I don't think it would be much of a problem to weaken the license of > Gtk2Hs to a BSD license. Don't forget you would need to obtain the consent of all contributors, whose patches are also under the LGPL. Regards, Malcolm From Axel.Simon at ens.fr Wed Aug 5 09:58:42 2009 From: Axel.Simon at ens.fr (Axel Simon) Date: Wed Aug 5 09:39:50 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> Message-ID: On Aug 5, 2009, at 15:44, Malcolm Wallace wrote: >> I don't think it would be much of a problem to weaken the license >> of Gtk2Hs to a BSD license. > > Don't forget you would need to obtain the consent of all > contributors, whose patches are also under the LGPL. True, but if I propose a discussion period on our mailing list during which people can object, then I think that would be sufficient. Axel. From marlowsd at gmail.com Wed Aug 5 10:15:36 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Wed Aug 5 09:56:26 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> Message-ID: <4A799408.6080508@gmail.com> On 05/08/2009 14:58, Axel Simon wrote: > > On Aug 5, 2009, at 15:44, Malcolm Wallace wrote: > >>> I don't think it would be much of a problem to weaken the license of >>> Gtk2Hs to a BSD license. >> >> Don't forget you would need to obtain the consent of all contributors, >> whose patches are also under the LGPL. > > True, but if I propose a discussion period on our mailing list during > which people can object, then I think that would be sufficient. I think strictly speaking you have to get explicit consent, rather than the absence of objection. Cheers, Simon From dons at galois.com Wed Aug 5 10:33:21 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 10:16:20 2009 Subject: Hackage Download Rankings August 2009 In-Reply-To: <4A794CE1.8070600@dfki.de> References: <20090805000432.GH23958@whirlpool.galois.com> <4A794CE1.8070600@dfki.de> Message-ID: <20090805143321.GA27483@whirlpool.galois.com> Christian.Maeder: > Don Stewart wrote: > > Total package downloads on Hackage, by August 1 2009, > > > > http://www.galois.com/~dons/hackage/august-2009/popularity-august-2009.html > > 1 xmonad 39229 > 2 HTTP 37058 > 3 zlib 33065 > 4 Cabal 27314 > 5 X11 24590 > > How should I interpret the xmonad count as it relies on X11? X11 is not > shipped with recent binary distributions (whereas Cabal is). > X11 is shipped in many distros (e.g. Debian). -- Don From dons at galois.com Wed Aug 5 10:38:00 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 10:20:50 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: <20090805143800.GA27517@whirlpool.galois.com> kr.angelov: > The binary package is too lazy and in some cases the decoding dies > with "stack overflow". At least this was the situation the last time > when I tried to use it. Since then I am using hacked version which is > stricter. This should be addressed before to add this package to the > platform. The stack overflow with [a] was fixed a few months ago. I'd be interested to know if there were other issues with instances. > Another problem that I see is that Int is serialized as 64 bit > integer. This makes the serialized data too large. The Int type is > also used internally, for example to store the length of a list, the > size of a map, etc, so you can't avoid the problem by using custom > putInt function. In fact the binary representation is so verbose that > sometimes it is more compact to use textual representation. In my > hacked version I use serialization for integers which use variable > number of bytes. If someone wants to store 64 bit integer then he/she > could always use Int64. I know that the binary package could be > combined with gzip to make the data more compact but this is > unnecessary overhead. For this, we recommend the lazy zlib library to compress as you see fit. -- Don From dons at galois.com Wed Aug 5 10:41:28 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 10:24:25 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> References: <20090804225541.GF23958@whirlpool.galois.com> <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> Message-ID: <20090805144128.GB27517@whirlpool.galois.com> malcolm.wallace: >> Let's discuss, then have the steering committee recommend yay/nay. > > We should have _some_ kind of binary library in the Platform, but I > don't know whether the proposed library is the right one. > > In a recent application, I found Data.Binary very slow, both to encode > and to decode data. Decoding had stack-overflows, and when I increased > the stack, it took about 20mins to read in an 8Mb file. When I then > turned on optimisation with -O, the performance improved considerably > (down to ~30secs to read the same file). This was using the standard > instances of Binary for data structures like Data.Map. 30secs was still > too slow, so we ended up needing to write our own improved instance for > Data.Map. (Timings were similar with both ghc-6.8.3 and ghc-6.10.3.) I don't think I've ever benchmarked without optimizations. Unfortunately, as Data.Map is exported abstractly, we can't use the internal folds as we do for the Data.Sequence instance. Can you submit your improved Map instance? In general, Data.Binary is heavily optimized for the underlying Get/Put monads, but instances for different data types are just straight forward implementations (which seems to be what people complain about the most: very large lists (in the past) and very large Data.Maps). -- Don From ml at isaac.cedarswampstudios.org Wed Aug 5 10:59:35 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Wed Aug 5 10:40:28 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A799408.6080508@gmail.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> <4A799408.6080508@gmail.com> Message-ID: <4A799E57.2090508@isaac.cedarswampstudios.org> Simon Marlow wrote: > On 05/08/2009 14:58, Axel Simon wrote: >> >> On Aug 5, 2009, at 15:44, Malcolm Wallace wrote: >> >>>> I don't think it would be much of a problem to weaken the license of >>>> Gtk2Hs to a BSD license. >>> >>> Don't forget you would need to obtain the consent of all contributors, >>> whose patches are also under the LGPL. >> >> True, but if I propose a discussion period on our mailing list during >> which people can object, then I think that would be sufficient. > > I think strictly speaking you have to get explicit consent, rather than > the absence of objection. which GHC and the other BSD-components don't technically get, but it's strongly implied by submitting a patch. Similar for LGPL+exception (technically a contributor would be allowed to distribute a patch under just LGPL, or just GPL, or even GPL-2-only or GPL-3-only if they were silly). Socially, patches are generally assumed to be the same as the source license... Can we get a list of all the Gtk2Hs code-contributors? (in which if a person only ever submitted less than a dozen lines of significant patches, it's probably not copyright significant) Also, does anyone here have an argument against trying to relicense? (and is it allowed, or is Gtk2Hs a "derivative work" of Gtk+ even when distributed separately?... I think it *ought* to be allowed in this particular circumstance, it wouldn't hardly be against the spirit of LGPL since Gtk2Hs is mainly simply a binding, but I'm not a lawyer :-) -Isaac From dons at galois.com Wed Aug 5 11:06:08 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 10:49:06 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <4A794B5A.8080905@gmail.com> References: <20090804225541.GF23958@whirlpool.galois.com> <7B1B7BD7-5459-4C32-A239-036750A158F5@cs.york.ac.uk> <4A794B5A.8080905@gmail.com> Message-ID: <20090805150608.GA27658@whirlpool.galois.com> marlowsd: > Ok, the way to proceed is to build a set of benchmarks. I did some > rough timings myself recently, and found that binary was roughly > comparable to the Binary module in GHC, which as far as I know is fairly > fast (though I know there are faster libraries out there). > > I suspect we're all measuring different things. Would someone like to > work on benchmarking binary and identifying the weak points? I'll benchmark the default instances, and the underlying primitives. Let's see what that shows up. -- Don From Axel.Simon at ens.fr Wed Aug 5 13:16:34 2009 From: Axel.Simon at ens.fr (Axel Simon) Date: Wed Aug 5 12:57:47 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A799E57.2090508@isaac.cedarswampstudios.org> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> <7FF6731D-30F5-4C53-8228-6E092473411E@cs.york.ac.uk> <4A799408.6080508@gmail.com> <4A799E57.2090508@isaac.cedarswampstudios.org> Message-ID: <744DF6BB-906E-431F-AC56-B79EDD1F42DA@di.ens.fr> On Aug 5, 2009, at 16:59, Isaac Dupree wrote: > Simon Marlow wrote: >> On 05/08/2009 14:58, Axel Simon wrote: >>> >>> On Aug 5, 2009, at 15:44, Malcolm Wallace wrote: >>> >>>>> I don't think it would be much of a problem to weaken the >>>>> license of >>>>> Gtk2Hs to a BSD license. >>>> >>>> Don't forget you would need to obtain the consent of all >>>> contributors, >>>> whose patches are also under the LGPL. >>> >>> True, but if I propose a discussion period on our mailing list >>> during >>> which people can object, then I think that would be sufficient. >> I think strictly speaking you have to get explicit consent, rather >> than the absence of objection. > > which GHC and the other BSD-components don't technically get, but > it's strongly implied by submitting a patch. Similar for LGPL > +exception (technically a contributor would be allowed to > distribute a patch under just LGPL, or just GPL, or even GPL-2-only > or GPL-3-only if they were silly). Socially, patches are generally > assumed to be the same as the source license... > > Can we get a list of all the Gtk2Hs code-contributors? (in which if > a person only ever submitted less than a dozen lines of significant > patches, it's probably not copyright significant) Also, does > anyone here have an argument against trying to relicense? (and is > it allowed, or is Gtk2Hs a "derivative work" of Gtk+ even when > distributed separately?... I think it *ought* to be allowed in this > particular circumstance, it wouldn't hardly be against the spirit > of LGPL since Gtk2Hs is mainly simply a binding, but I'm not a > lawyer :-) When I re-implemented a binding for Gtk+, I chose LGPL because Manuel Chakravaty used LGPL for his Gtk+Hs binding. I did have a few requests to move to a BSD license from people who wanted to ship commercial Haskell applications. We once moved from version 2 of LGPL to version 3 at least for some or our code and we have tried to get consent from the relevant people before which was no big deal. Thus, I don't think it is a big problem unless people are in for an ideological license debate. Axel. From dons at galois.com Wed Aug 5 13:41:25 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 13:24:20 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> Message-ID: <20090805174125.GE27736@whirlpool.galois.com> malcolm.wallace: > On 4 Aug 2009, at 23:05, Don Stewart wrote: >> >> I would appreciate input from the HaXml and HDBC authors (our most >> popular LGPL-licensed Haskell libraries) about what they feel the >> licensing issues/constraints should be for the Haskell Platform. > > Licensing clarity is important for users I think. But equally some > users may desire to use LGPL libraries too. Hence my suggestion that > there be a separate platform of free/LGPL code (and GPL tools), which > can depend on the proprietary-friendly BSD-licensed platform, but not > the other way round. > >> I've not yet seen anyone publish something on how to satisfy LGPL >> for Haskell libraries. > > The static-linking exception is the commonest means of working around > ghc's technical limitations here. The exception is part of wxHaskell's > license (but not Gtk2hs's), and HaXml (+polyparse on which it depends) > has the exception too. Ok. That's good to know. Perhaps in the cabal file for haxml you could modify it to be license: Other than, and state somewhere that it is LGPL with static linking exception. Regarding a separate LGPL "non-free" platform, that will have to happen further down the line. -- Don From alexander.dunlap at gmail.com Wed Aug 5 15:12:48 2009 From: alexander.dunlap at gmail.com (Alexander Dunlap) Date: Wed Aug 5 14:53:51 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090804225541.GF23958@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> On Tue, Aug 4, 2009 at 3:55 PM, Don Stewart wrote: > Here's a ticket for Simon Marlow's proposal: > > ? ?http://trac.haskell.org/haskell-platform/ticket/86 > > Let's discuss, then have the steering committee recommend yay/nay. > > -- Don > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > To add to the laundry list of problems with Data.Binary, I don't like the fact that decode calls error on invalid input. I can't think of any great alternatives (using Maybe as the result type would be too strict, of course, and returning partial results would be difficult with polymorphism), but it seems a bit unclean that decode has to be used with the IO monad to catch the errors. (Of course, the only reason you would have bad input would be if you were using the IO monad, so the practical implications are not great, but still, it would be nice if there was a better way.) Alex From jgoerzen at complete.org Wed Aug 5 16:12:30 2009 From: jgoerzen at complete.org (John Goerzen) Date: Wed Aug 5 15:53:23 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090804220523.GA23958@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> Message-ID: <4A79E7AE.2020004@complete.org> Don Stewart wrote: > I would appreciate input from the HaXml and HDBC authors (our most > popular LGPL-licensed Haskell libraries) about what they feel the > licensing issues/constraints should be for the Haskell Platform. I would be happy to put a static linking exemption into the COPYRIGHT file for all of my LGPL libraries. My intent with using LGPL instead of BSD is not to pollute end users' work or other libraries they use, but rather to keep the LGPL'd library free and open since I am giving out it in a free and open way. That is, I don't want someone to take my LGPL'd library, make it closed source, and sell copies of the library... if they want to sell a different product that happens to use a database and keep it closed source, that's fine with me. Perhaps the community could come up with a standard boilerplate static linking exemption that we could all use? -- John From dons at galois.com Wed Aug 5 16:15:07 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 15:58:15 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A79E7AE.2020004@complete.org> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> Message-ID: <20090805201507.GA28722@whirlpool.galois.com> jgoerzen: > Don Stewart wrote: > > I would appreciate input from the HaXml and HDBC authors (our most > > popular LGPL-licensed Haskell libraries) about what they feel the > > licensing issues/constraints should be for the Haskell Platform. > > I would be happy to put a static linking exemption into the COPYRIGHT > file for all of my LGPL libraries. My intent with using LGPL instead of > BSD is not to pollute end users' work or other libraries they use, but > rather to keep the LGPL'd library free and open since I am giving out it > in a free and open way. That is, I don't want someone to take my LGPL'd > library, make it closed source, and sell copies of the library... if > they want to sell a different product that happens to use a database and > keep it closed source, that's fine with me. > > Perhaps the community could come up with a standard boilerplate static > linking exemption that we could all use? > Great! We might want to borrow ideas from the OCaml license then, http://caml.inria.fr/ocaml/license.en.html The Library is distributed under the terms of the GNU Library General Public License version 2 (included below). As a special exception to the Q Public Licence, ** you may develop application programs, reusable components and other software items that link with the original or modified versions of the Compiler and are not made available to the general public, without any of the additional requirements listed in clause 6c of the Q Public licence. ** From dons at galois.com Wed Aug 5 16:34:43 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 16:17:33 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> Message-ID: <20090805203443.GF28722@whirlpool.galois.com> alexander.dunlap: > To add to the laundry list of problems with Data.Binary, I don't like > the fact that decode calls error on invalid input. I can't think of > any great alternatives (using Maybe as the result type would be too > strict, of course, and returning partial results would be difficult > with polymorphism), but it seems a bit unclean that decode has to be > used with the IO monad to catch the errors. (Of course, the only > reason you would have bad input would be if you were using the IO > monad, so the practical implications are not great, but still, it > would be nice if there was a better way.) That's right. Originally, it used a custom Either type, but it isn't possible to stream decoders that way. I'd consider it an intentional design feature. -- Don From alexander.dunlap at gmail.com Wed Aug 5 16:40:03 2009 From: alexander.dunlap at gmail.com (Alexander Dunlap) Date: Wed Aug 5 16:21:08 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805203443.GF28722@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> Message-ID: <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> On Wed, Aug 5, 2009 at 1:34 PM, Don Stewart wrote: > alexander.dunlap: >> To add to the laundry list of problems with Data.Binary, I don't like >> the fact that decode calls error on invalid input. I can't think of >> any great alternatives (using Maybe as the result type would be too >> strict, of course, and returning partial results would be difficult >> with polymorphism), but it seems a bit unclean that decode has to be >> used with the IO monad to catch the errors. (Of course, the only >> reason you would have bad input would be if you were using the IO >> monad, so the practical implications are not great, but still, it >> would be nice if there was a better way.) > > That's right. Originally, it used a custom Either type, but it isn't > possible to stream decoders that way. > > I'd consider it an intentional design feature. > > -- Don > OK. Would it be worth creating an extensible exception (something like BinaryDecodeError) for this then, instead of using the call to error? That would at least make it less error-prone to catch. Alex From dons at galois.com Wed Aug 5 16:47:07 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 16:29:58 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> Message-ID: <20090805204707.GG28722@whirlpool.galois.com> alexander.dunlap: > On Wed, Aug 5, 2009 at 1:34 PM, Don Stewart wrote: > > alexander.dunlap: > >> To add to the laundry list of problems with Data.Binary, I don't like > >> the fact that decode calls error on invalid input. I can't think of > >> any great alternatives (using Maybe as the result type would be too > >> strict, of course, and returning partial results would be difficult > >> with polymorphism), but it seems a bit unclean that decode has to be > >> used with the IO monad to catch the errors. (Of course, the only > >> reason you would have bad input would be if you were using the IO > >> monad, so the practical implications are not great, but still, it > >> would be nice if there was a better way.) > > > > That's right. Originally, it used a custom Either type, but it isn't > > possible to stream decoders that way. > > > > I'd consider it an intentional design feature. > > > > -- Don > > > > OK. Would it be worth creating an extensible exception (something like > BinaryDecodeError) for this then, instead of using the call to error? > That would at least make it less error-prone to catch. > I think that would be a good idea. Showing how to catch it in the documentation. I'm wary of breaking the 70 packages that use Data.Binary for this, rather, add this as a list of API changes for the next major release. -- Don From igloo at earth.li Wed Aug 5 17:14:28 2009 From: igloo at earth.li (Ian Lynagh) Date: Wed Aug 5 16:55:12 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805204707.GG28722@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> Message-ID: <20090805211428.GA12829@matrix.chaos.earth.li> On Wed, Aug 05, 2009 at 01:47:07PM -0700, Donald Bruce Stewart wrote: > alexander.dunlap: > > > > OK. Would it be worth creating an extensible exception (something like > > BinaryDecodeError) for this then, instead of using the call to error? > > That would at least make it less error-prone to catch. > > I think that would be a good idea. Showing how to catch it in the > documentation. > > I'm wary of breaking the 70 packages that use Data.Binary for this, > rather, add this as a list of API changes for the next major release. If you're planning a change that you are worried would break users of the package, wouldn't it be better to do it before putting it into the platform? Or have I misunderstood what you meant? Thanks Ian From dons at galois.com Wed Aug 5 17:24:09 2009 From: dons at galois.com (Don Stewart) Date: Wed Aug 5 17:06:57 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805211428.GA12829@matrix.chaos.earth.li> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> <20090805211428.GA12829@matrix.chaos.earth.li> Message-ID: <20090805212409.GJ28722@whirlpool.galois.com> igloo: > On Wed, Aug 05, 2009 at 01:47:07PM -0700, Donald Bruce Stewart wrote: > > alexander.dunlap: > > > > > > OK. Would it be worth creating an extensible exception (something like > > > BinaryDecodeError) for this then, instead of using the call to error? > > > That would at least make it less error-prone to catch. > > > > I think that would be a good idea. Showing how to catch it in the > > documentation. > > > > I'm wary of breaking the 70 packages that use Data.Binary for this, > > rather, add this as a list of API changes for the next major release. > > If you're planning a change that you are worried would break users of > the package, wouldn't it be better to do it before putting it into the > platform? > > Or have I misunderstood what you meant? So two things: 1. We can do API-breaking changes in the platform on major releases. 2. When new packages come in, I'm wary of breaking their APIs as established prior to them being added, as then the platform release that includes them is not usable as a platform for those existing packages. Maybe 2 is not a valid concern. Maybe the HP inclusion is the time to break things. But maybe we won't then add popular packages -- because changing their APIs as requested would break too many things :/ This is the tension between "blessed" (small set of perfectionist libs) and "comprehensive" (large set of not-quite-perfect useful libs). And I'm somewhat concerned we're only focusing on the blessedness -- things being perfect -- rather than the comprehensiveness : things being useful and widely used. Do we need a policy decision: * will the Platform Steering Committee make inclusion conditional on API changes? Is that something we want to do? Maybe we just decide on a case-by-case basis. -- Don From igloo at earth.li Wed Aug 5 17:46:18 2009 From: igloo at earth.li (Ian Lynagh) Date: Wed Aug 5 17:27:02 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805212409.GJ28722@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> <20090805211428.GA12829@matrix.chaos.earth.li> <20090805212409.GJ28722@whirlpool.galois.com> Message-ID: <20090805214618.GA13917@matrix.chaos.earth.li> On Wed, Aug 05, 2009 at 02:24:09PM -0700, Donald Bruce Stewart wrote: > igloo: > > On Wed, Aug 05, 2009 at 01:47:07PM -0700, Donald Bruce Stewart wrote: > > > alexander.dunlap: > > > > > > > > OK. Would it be worth creating an extensible exception (something like > > > > BinaryDecodeError) for this then, instead of using the call to error? > > > > That would at least make it less error-prone to catch. > > > > > > I think that would be a good idea. Showing how to catch it in the > > > documentation. > > > > > > I'm wary of breaking the 70 packages that use Data.Binary for this, > > > rather, add this as a list of API changes for the next major release. > > > > If you're planning a change that you are worried would break users of > > the package, wouldn't it be better to do it before putting it into the > > platform? > > > > Or have I misunderstood what you meant? > > So two things: > > 1. We can do API-breaking changes in the platform on major releases. > > 2. When new packages come in, I'm wary of breaking their APIs as > established prior to them being added, as then the platform > release that includes them is not usable as a platform for those > existing packages. I don't understand how adding newapi in HPv2 is worse than adding oldapi in HPv2 replacing it with newapi in HPv3 Either way, when you put newapi in the HP there will be a period where other packages need to catch up with the changes. I'm not opposed to changing APIs in the platform, and if it would take a long time to design, implement and test a planned API change then I wouldn't see a problem with adding an oldapi in the mean time (assuming oldapi is "good enough"). > Maybe 2 is not a valid concern. Maybe the HP inclusion is the time to > break things. But maybe we won't then add popular packages -- because > changing their APIs as requested would break too many things :/ I think we have to be willing to break things, or we will become obsolete. There is a balance between stability and progress, of course. Thanks Ian From wren at community.haskell.org Wed Aug 5 18:32:41 2009 From: wren at community.haskell.org (wren ng thornton) Date: Wed Aug 5 18:14:39 2009 Subject: API changes and HP inclusion [was: Re: Adding binary to the Haskell Platform] In-Reply-To: <20090805214618.GA13917@matrix.chaos.earth.li> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> <20090805211428.GA12829@matrix.chaos.earth.li> <20090805212409.GJ28722@whirlpool.galois.com> <20090805214618.GA13917@matrix.chaos.earth.li> Message-ID: <4A7A0889.2000404@community.haskell.org> Ian Lynagh wrote: > Donald Bruce Stewart wrote: >> So two things: >> >> 1. We can do API-breaking changes in the platform on major releases. >> >> 2. When new packages come in, I'm wary of breaking their APIs as >> established prior to them being added, as then the platform >> release that includes them is not usable as a platform for those >> existing packages. > > I don't understand how > adding newapi in HPv2 > is worse than > adding oldapi in HPv2 > replacing it with newapi in HPv3 > > Either way, when you put newapi in the HP there will be a period where > other packages need to catch up with the changes. The difference is this: If we follow the former (API before HP) then people who need to debug the upgrade process will have to switch back and forth between HP vs raw Cabal, which could be a nightmare. If we follow the latter (HP before API) then developers will only need to toggle between different HP versions, which would be a lot easier to setup in most environments. I think the HP-before-API path is more likely to keep us agile since it makes the API breaking changes easier to fix. However, it does have the downside of including known "broken" packages in the HP, even if only for a little while. The question is, how much do we want the HP to serve as a migration path? If expected breakage is small, then the API changes should probably come before HP inclusion. If the expected breakage is large ---as it will be for any popular package--- then it may be good to have an initial HP which solidifies the "classic" interface so that unknown and one-off projects, which may not be maintained, can still make use of the old API as well as the benefits of the HP. That is, there's some benefit in providing snapshots of the community as it is, rather than the ideal of what we would like. The question is the extent to which this is the HP's job. We do already have Cabal and Hackage which would allow users to install old versions of packages, so the benefits would be in the one-click install as well as capturing the in-time correlations between different packages (which Hackage doesn't currently provide a nice interface for). Providing semi-blessed packages in the HP before changing APIs could help improve buy in, and will reduce the disturbance of adding new packages, but it also increases the instability for end users since there will be more API changes within the HP track. >> Maybe 2 is not a valid concern. Maybe the HP inclusion is the time to >> break things. But maybe we won't then add popular packages -- because >> changing their APIs as requested would break too many things :/ > > I think we have to be willing to break things, or we will become > obsolete. There is a balance between stability and progress, of course. I'd say that stability is actually the balance between legacy and progress. Ironically, Hackage serves both legacy and progress, since it gives access to old versions and GHC is able to switch between different versions of packages. From that perspective, the HP serves as the balance, providing a stable subset of Hackage. This is a rather unique arrangement since most package trees conflate the recording of history and the stable image of it. The question remains, how closely should the fulcrum follow the growing edge? -- Live well, ~wren From johan.tibell at gmail.com Thu Aug 6 02:42:30 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Thu Aug 6 02:23:33 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805204707.GG28722@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> Message-ID: <90889fe70908052342r687bft953071453cd7bc2c@mail.gmail.com> On Wed, Aug 5, 2009 at 10:47 PM, Don Stewart wrote: > alexander.dunlap: >> On Wed, Aug 5, 2009 at 1:34 PM, Don Stewart wrote: >> > alexander.dunlap: >> >> To add to the laundry list of problems with Data.Binary, I don't like >> >> the fact that decode calls error on invalid input. I can't think of >> >> any great alternatives (using Maybe as the result type would be too >> >> strict, of course, and returning partial results would be difficult >> >> with polymorphism), but it seems a bit unclean that decode has to be >> >> used with the IO monad to catch the errors. (Of course, the only >> >> reason you would have bad input would be if you were using the IO >> >> monad, so the practical implications are not great, but still, it >> >> would be nice if there was a better way.) >> > >> > That's right. Originally, it used a custom Either type, but it isn't >> > possible to stream decoders that way. >> > >> > I'd consider it an intentional design feature. >> > >> > -- Don >> > >> >> OK. Would it be worth creating an extensible exception (something like >> BinaryDecodeError) for this then, instead of using the call to error? >> That would at least make it less error-prone to catch. >> > > I think that would be a good idea. Showing how to catch it in the > documentation. > > I'm wary of breaking the 70 packages that use Data.Binary for this, > rather, add this as a list of API changes for the next major release. Would this really break that many libraries? Are there many libraries that catch the exception that can be raised by error? Cheers, Johan From alexander.dunlap at gmail.com Thu Aug 6 02:50:14 2009 From: alexander.dunlap at gmail.com (Alexander Dunlap) Date: Thu Aug 6 02:31:16 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090803234432.GK17991@whirlpool.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> Message-ID: <57526e770908052350v5ba2c3ech473d0173acbeba12@mail.gmail.com> On Mon, Aug 3, 2009 at 4:44 PM, Don Stewart wrote: > > Following Simon M's advice, I look over the typical "batteries" > categories, using Python as input: > > ? ?http://docs.python.org/library/index.html > > The following things were missing from the current Platform. There are many. > How would you identify the top, say, 5 libs to add? > > -- Don > > > ? ?* String support > ? ? ? ? ?o binary formatting [binary] ? lazy binary parsing/serialising > ? ? ? ? ?o pcre regexes [pcre-light] [regex-pcre] ? what?s our best regex lib? > ? ? ? ? ?o unicode text [text] [text-icu] ? packed, unicode text > ? ? ? ? ?o codecs/encodings ? encodings? > ? ?* Data types > ? ? ? ? ?o higher dimensional arrays [hmatrix] > ? ? ? ? ?o bloomfilter ? bloomfilters > ? ? ? ? ?o bytestring-tries ? IntMap for ByteStrings > ? ? ? ? ?o dlist ? difference lists > ? ? ? ? ?o numbers ? expanded number types > ? ?* text > ? ? ? ? ?o attoparsec (simple, bytestring parsing) > ? ? ? ? ?o polyparse > ? ? ? ? ?o csv parsing > ? ? ? ? ?o pandoc ? markdown, reStructuredText, HTML, LaTeX, ConTeXt, Docbook, OpenDocument, ODT, RTF, MediaWiki, groff > ? ?* math and numerics > ? ? ? ? ?o blas ? BLAS > ? ? ? ? ?o cmath ? C math functions > ? ? ? ? ?o dimensional ? physical dimensions > ? ? ? ? ?o fftw > ? ? ? ? ?o mersenne-random ? fast randoms > ? ?* persistance > ? ? ? ? ?o anydbm? > ? ? ? ? ?o sqlite3 > ? ? ? ? ?o hdbc > ? ?* compression > ? ? ? ? ?o bzip2 > ? ? ? ? ?o zip > ? ? ? ? ?o tar > ? ?* file formats > ? ? ? ? ?o csv > ? ? ? ? ?o config parser > ? ?* crypto > ? ? ? ? ?o hmac, md5, sha, hashing > ? ?* systems > ? ? ? ? ?o getopt > ? ? ? ? ?o logging > ? ? ? ? ?o termio > ? ? ? ? ?o editline > ? ? ? ? ?o mmap > ? ?* Internet > ? ? ? ? ?o network-bytestring > ? ? ? ? ?o ssl > ? ? ? ? ?o json > ? ? ? ? ?o feed (rss, atom) > ? ? ? ? ?o mime > ? ? ? ? ?o base64 et al > ? ? ? ? ?o uuencode > ? ? ? ? ?o cgi > ? ? ? ? ?o fastcgi > ? ? ? ? ?o urls > ? ? ? ? ?o ftp, http, imap, smtp clients > ? ? ? ? ?o uuid > ? ? ? ? ?o url parsing > ? ? ? ? ?o http server > ? ? ? ? ?o xml-rpc > ? ?* Multimedia > ? ? ? ? ?o colour > ? ?* Internationalization > ? ? ? ? ?o gettext > ? ? ? ? ?o locale > ? ? ? ? ?o i18n > ? ?* GUIs > ? ? ? ? ?o gtk2hs > ? ?* Development > ? ? ? ? ?o hscolour > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > I would also highly support adding the excellent split library, supporting splitting strings. Split is one of the most-asked for functions in Haskell, and even though it's often easy to use a larger parsing library, the split functions can be very useful, especially for dealing with non-string types. Alex Alex From dons at galois.com Thu Aug 6 02:48:52 2009 From: dons at galois.com (Don Stewart) Date: Thu Aug 6 02:31:38 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <90889fe70908052342r687bft953071453cd7bc2c@mail.gmail.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> <90889fe70908052342r687bft953071453cd7bc2c@mail.gmail.com> Message-ID: <20090806064852.GA30949@whirlpool.galois.com> johan.tibell: > > I'm wary of breaking the 70 packages that use Data.Binary for this, > > rather, add this as a list of API changes for the next major release. > > Would this really break that many libraries? Are there many libraries > that catch the exception that can be raised by error? > Ah! Good question -- maybe we can find out... by building Hackage! From alexander.dunlap at gmail.com Thu Aug 6 03:15:50 2009 From: alexander.dunlap at gmail.com (Alexander Dunlap) Date: Thu Aug 6 02:56:54 2009 Subject: Containers in the Haskell Platform Message-ID: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> Hi all, As we consider what we want to go into the Haskell Platform, I would like to seriously discuss the issue of container polymorphism, especially, but not exclusively, in dealing with strings. We currently have a large number of container types, many of which are going to co-exist long-term, i.e. there is no winner; they are useful in different situations. Here is an incomplete list of what we are dealing with: Fully polymorphic sequence types: [] Data.Sequence Data.Edison sequence types Data.Array.Array & friends Restricted polymorphic sequence types: uvector vector storable-vector Monomorphic sequence types: ByteString (Word8 and Char8) Lazy ByteString (Word8 and Char8) Text Fully polymorphic associative array types: Data.Map Edison associative arrays gmap (?) Less-polymorphic associative array types: bytestring-tries list-tries Data.IntMap I'm sure there are more, of course, but those are the ones that a cursory search reveals. The question, then, is how we ought to deal with all of these types in the Haskell Platform. The current popular libraries deal with the multitude of container types in a very ad-hoc way. For instance, the Parsec parsing library allows you to parse Strings and Strict ByteStrings; it also includes a class so that you can parse other things. Attoparsec allows you to parse only lazy bytestrings. I don't know of any library that parses Text yet. The binary library deals with lazy bytestrings, while binary-strict lets you use strict bytestrings. Libraries like zlib just pick a type (lazy bytestrings) and use that, not accounting for other possible types you may want to use (granted, in zlib's case, the choice of a single data structure, seems fairly reasonable, given potential use cases). The point here is that the same functionality and practically the same code can be split over several different packages based on what data structure you want to use. Worse, there is no standard way of dealing with this separation: Parsec uses a type class, while binary uses two separate packages. And interoperability could potentially become a problem: the split library, for instance, only supports lists, so you have to roll your own solutions for splitting ByteStrings or Text. I think that with the introduction of the Haskell Platform, we ought to take the opportunity to clean up some of this chaos so that we have a clean set of standard libraries for people to build on. The way I see it, we have a few options. We could introduce some type classes to cover the various container operations and ask all Platform packages to only use operations from these classes, where applicable. (The ListLike library would probably be a good place to start.) The issue here is efficiency, massive class dictionaries, and type system issues (we need MPTCs or TFs; also, since not all sequence types are polymorphic, we probably need to have separate map and rigidMap functions like in ListLike, and even that doesn't work for types like uvector that can take some, but not all, concrete element types). We could also implement a standard module-naming scheme: for every package that operates on strings, for instance, we could require the list-based code to go in Foo.Bar.String and the Text code to go in Foo.Bar.Text. Whenever we "blessed" a new string datatype (presumably not very often), all packages would have to add a new module. The issue is code duplication. We could also implement a package naming scheme: have foo-text and foo-string for the two different datatypes. What does everyone think about the need for standardization here and how we ought to do it? I'm sorry that, having less experience with Haskell than many, I can't give as much in the way of concrete proposals, but I think it's important that we hash something out here to get more organization in our Platform. Thanks for your time in reading this admittedly long message, Alex From simonpj at microsoft.com Thu Aug 6 03:22:28 2009 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Thu Aug 6 03:03:13 2009 Subject: Proposal #3271: new methods for Data.Sequence In-Reply-To: <20090803192846.GA11752@soi.city.ac.uk> References: <2518b95d0907191857q29a442b1l86609db8e3e58d17@mail.gmail.com> <20090803192846.GA11752@soi.city.ac.uk> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C357F321EADF@EA-EXMSG-C334.europe.corp.microsoft.com> I will look into this during August. Add yourself to the cc list for #2353 if interested Simon | -----Original Message----- | From: libraries-bounces@haskell.org [mailto:libraries-bounces@haskell.org] On | Behalf Of Ross Paterson | Sent: 03 August 2009 20:29 | To: libraries@haskell.org | Subject: Re: Proposal #3271: new methods for Data.Sequence | | On Mon, Jul 20, 2009 at 02:33:52PM -0400, Louis Wasserman wrote: | > Among other things, I manually deforested the stable sorting algorithm, | > resulting in a moderate performance gain on simply using Data.List.sortBy. | | That's not supposed to be necessary, because toList is supposed to fuse | with list consumers. Unfortunately it doesn't in cases like this because | it doesn't get inlined, even if we add an INLINE pragma, due to GHC bug | #2353. If the GHC bug isn't fixed, I think the next best thing would | be to force the inlining of toList with a RULES pragma in Data.Foldable. | _______________________________________________ | Libraries mailing list | Libraries@haskell.org | http://www.haskell.org/mailman/listinfo/libraries From simonpj at microsoft.com Thu Aug 6 03:22:30 2009 From: simonpj at microsoft.com (Simon Peyton-Jones) Date: Thu Aug 6 03:03:16 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A781B01.8080409@gmail.com> References: <4A781B01.8080409@gmail.com> Message-ID: <638ABD0A29C8884A91BC5FB5C349B1C357F321EAE5@EA-EXMSG-C334.europe.corp.microsoft.com> The Hasekll Platform will be a *lot* more compelling if it has a GUI story. If it's hard for experts to get it built on platform X, what are the poor users supposed to do? The whole point of the HP is to take the pain just once, and let our happy users enjoy the benefits. If it's clear that Gtk2hs is the brand leader, I think Simon is right that we should very seriously consider putting it in the HP. But what about Wx? I'm reluctant to appear to sponsor one or t'other unless there are clear technical or support reasons to do so. Having both would be fine, if their support crews are willing to do the necessary polishing etc. Concerning the "boost" from the HP, inclusion *will* increase the user-base of a package, and that *does* increase the incentive for the library's support crew to roll up their sleeves. And rightly so. That's one of the benefits of the HP! Simon | -----Original Message----- | From: libraries-bounces@haskell.org [mailto:libraries-bounces@haskell.org] On | Behalf Of Simon Marlow | Sent: 04 August 2009 12:27 | To: Gwern Branwen | Cc: libraries@haskell.org | Subject: Re: Thinking about what's missing in our library coverage | | On 04/08/2009 12:07, Gwern Branwen wrote: | > On Tue, Aug 4, 2009 at 7:02 AM, Simon Marlow wrote: | >> Add | >> | >> * binary | >> * getopt | >> * gtk2hs | > | > A definite yes to binary and getopt; but gtk2hs? I don't trust its | > longevity or maintenance, and as other people have pointed out, it will | > make the platform harder to support. As well, we risk holding back the | > platform - hasn't gtk2hs lagged GHC releases in the past? (I seem to | > remember some of the lags being quite lengthy.) | | Adding gtk2hs would be a bold step, no doubt about it. By proposing it | I'm hoping to force the issues to the surface: is gtk2hs the GUI lib we | want to recommend, or standardise on? If it is, and it has maintenance | issues, then those need to be addressed. As far as I'm aware, gtk2hs is | the only plausible option for serious GUI development in Haskell at the | moment. | | By bringing gtk2hs into the platform, we would be giving the gtk2hs | maintainers a helpful boost; they'd get more testing for one thing. | | >> Also | >> | >> * keep an eye on text. We certainly want it, but it's | >> a young package and there's no text I/O yet. | >> * decide which regex package(s) we want | >> * remove html? (we have xhtml) | >> * replace haskell-src with haskell-src-exts | >> * remove packedstring | > | > Absolutely. I thought we had already done this - didn't TH's unnecessary | > use of packedstring get removed a while ago? | | Not in the version of TH shipping with GHC 6.10.x, but it will be gone | in GHC 6.12. | | Cheers, | Simon | | | _______________________________________________ | Libraries mailing list | Libraries@haskell.org | http://www.haskell.org/mailman/listinfo/libraries From jeanphilippe.bernardy at gmail.com Thu Aug 6 05:13:28 2009 From: jeanphilippe.bernardy at gmail.com (Jean-Philippe Bernardy) Date: Thu Aug 6 04:54:12 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805203443.GF28722@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> Message-ID: <953e0d250908060213u1bd9ae71l7b3340118e25a3b8@mail.gmail.com> > That's right. Originally, it used a custom Either type, but it isn't > possible to stream decoders that way. What about this: you could return a pair (Result,Error). By forcing the Result you get an exception if there is an error. You can check the Error always without risk of exception. Cheers, JP. From ganesh.sittampalam at credit-suisse.com Thu Aug 6 05:22:10 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Thu Aug 6 05:03:23 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090804225541.GF23958@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F892@ELON17P32001A.csfb.cs-group.com> Don Stewart wrote: > Here's a ticket for Simon Marlow's proposal: > > http://trac.haskell.org/haskell-platform/ticket/86 > > Let's discuss, then have the steering committee recommend yay/nay. In http://www.haskell.org/pipermail/libraries/2009-July/012139.html, Duncan said: > It is not the intention that the committee members get much more say in > policy decisions than other active contributors to discussion on the > libraries mailing list. We hope therefore that the membership of the > committee does not need to be terribly formal. The only difficulty comes > when consensus cannot be reached on an issue where making some decision > is widely agreed to be better than making no decision. There is no > specified protocol for this situation at this stage. So I'm a bit confused about what the purpose of the steering committee's recommendation would be. Would it amount to a decision about inclusion, or an expression of their opinion in the hope of guiding the community towards a consensus, or something else? The first choice contradicts what Duncan said was the role of the committee, while the second doesn't seem any more worthwhile than individual members expressing their own opinions separately. Cheers, Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From malcolm.wallace at cs.york.ac.uk Thu Aug 6 05:26:49 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Thu Aug 6 05:07:32 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A79E7AE.2020004@complete.org> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> Message-ID: > Perhaps the community could come up with a standard boilerplate static > linking exemption that we could all use? HaXml uses: As a relaxation of clause 6 of the LGPL, the copyright holders of this library give permission to use, copy, link, modify, and distribute, binary-only object-code versions of an executable linked with the original unmodified Library, without requiring the supply of any mechanism to modify or replace the Library and relink (clauses 6a, 6b, 6c, 6d, 6e), provided that all the other terms of clause 6 are complied with. Regards, Malcolm From malcolm.wallace at cs.york.ac.uk Thu Aug 6 05:31:22 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Thu Aug 6 05:12:06 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090805201507.GA28722@whirlpool.galois.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> <20090805201507.GA28722@whirlpool.galois.com> Message-ID: <8261ADB4-B3F4-4C77-B4D5-5AC782F5595B@cs.york.ac.uk> On 5 Aug 2009, at 21:15, Don Stewart wrote: > The Library is distributed under the terms of the GNU Library > General > Public License version 2 (included below). > > As a special exception to the Q Public Licence, ** you may develop > application programs, reusable components and other software > items that > link with the original or modified versions of the Compiler and > are not > made available to the general public, without any of the additional > requirements listed in clause 6c of the Q Public licence. ** That excerpt is highly confusing. Which is it? LGPL or QPL? In fact, the license of the _compiler_ is QPL, and of the _library_ is LGPL. The compiler has an exception to the QPL, and the library has an exception to the LGPL. The LGPL exception is: As a special exception to the GNU Library General Public License, you may link, statically or dynamically, a "work that uses the Library" with a publicly distributed version of the Library to produce an executable file containing portions of the Library, and distribute that executable file under terms of your choice, without any of the additional requirements listed in clause 6 of the GNU Library General Public License. By "a publicly distributed version of the Library", we mean either the unmodified Library as distributed by INRIA, or a modified version of the Library that is distributed under the conditions defined in clause 3 of the GNU Library General Public License. This exception does not however invalidate any other reasons why the executable file might be covered by the GNU Library General Public License. Regards, Malcolm From marlowsd at gmail.com Thu Aug 6 06:28:47 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Thu Aug 6 06:09:32 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> Message-ID: <4A7AB05F.7010809@gmail.com> On 06/08/2009 10:26, Malcolm Wallace wrote: >> Perhaps the community could come up with a standard boilerplate static >> linking exemption that we could all use? > > HaXml uses: > > As a relaxation of clause 6 of the LGPL, the copyright holders of this > library give permission to use, copy, link, modify, and distribute, > binary-only object-code versions of an executable linked with the > original unmodified Library, without requiring the supply of any > mechanism to modify or replace the Library and relink (clauses 6a, > 6b, 6c, 6d, 6e), provided that all the other terms of clause 6 are > complied with. What's the reasoning for dropping 6a-e, but retaining "provided that all the other terms of clause 6 are complied with"? That leaves, amongst other things: ------------ For an executable, the required form of the "work that uses the Library" must include any data and utility programs needed for reproducing the executable from it. However, as a special exception, the materials to be distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. ------------ So I don't think I understand how to comply with this. What "data and utility programs" must I supply with a work that uses the library? And why? The licensee is absolved from having to supply the executable in a way that it can be re-linked with a modified library, so what is the purpose of this part of the license? The other LGPL exception you quoted (from OCaml?) omits the whole of clause 6, which seems a lot simpler. Cheers, Simon From kr.angelov at gmail.com Thu Aug 6 06:49:02 2009 From: kr.angelov at gmail.com (Krasimir Angelov) Date: Thu Aug 6 06:29:45 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090805143800.GA27517@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <20090805143800.GA27517@whirlpool.galois.com> Message-ID: Ok. I tried to replace my hacked version of Data.Binary with the latest version of the library. Indeed the stack overflow error is fixed. I did some further experiments: 1. After upgrading I tried to compare the two implementations. First I was surprised to find that my version of binary is 3-4 times slower than the latest official version. After some experiments I found that the problem was in my version of Data.Binary.Get.getWord8. In the official release it is: getWord8 :: Get Word8 getWord8 = getPtr (sizeOf (undefined :: Word8)) while in my version it was: getWord8 = do S s ss bytes <- get case B.uncons s of Just (w,rest) -> do put $! S rest ss (bytes + 1) return $! w Nothing -> case L.uncons ss of Just (w,rest) -> do put $! mkState rest (bytes + 1) return $! w Nothing -> fail "too few bytes" I don't remember why I had changed it but probably it was an attempt to make it faster since I use getWord8 often. I don't know what had changed but now fist version is much much faster. When I reverted to using the fist version my library become as fast as the official binary library. 2. I tried to revert the implementation of the Get monad from strict to lazy. This made the decoding even faster - from 1.52 sec to 1.08 sec for ~ 3 Mb of data. Good achievement. 3. After the above changes the only differences between my version and the official version are that I have different instance for Int. The impact is that with my instance the output is from 2 to 4 times more compact. As a consequence the decoding is also faster with about 50%. I know that I can use compression to reduce the size of the output but this will make the deserialization only slower, not faster. Why it is so important to store Int as Int64 instead of as variable bytes field? This only adds extra overhead. Regards, Krasimir From ross at soi.city.ac.uk Thu Aug 6 07:26:15 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Thu Aug 6 07:07:01 2009 Subject: Proposal #3271: new methods for Data.Sequence In-Reply-To: <638ABD0A29C8884A91BC5FB5C349B1C357F321EADF@EA-EXMSG-C334.europe.corp.microsoft.com> References: <2518b95d0907191857q29a442b1l86609db8e3e58d17@mail.gmail.com> <20090803192846.GA11752@soi.city.ac.uk> <638ABD0A29C8884A91BC5FB5C349B1C357F321EADF@EA-EXMSG-C334.europe.corp.microsoft.com> Message-ID: <20090806112615.GA5961@soi.city.ac.uk> On Thu, Aug 06, 2009 at 08:22:28AM +0100, Simon Peyton-Jones wrote: > On 03 August 2009 20:29, Ross Paterson wrote: > | On Mon, Jul 20, 2009 at 02:33:52PM -0400, Louis Wasserman wrote: > | > Among other things, I manually deforested the stable sorting algorithm, > | > resulting in a moderate performance gain on simply using Data.List.sortBy. > | > | That's not supposed to be necessary, because toList is supposed to fuse > | with list consumers. Unfortunately it doesn't in cases like this because > | it doesn't get inlined, even if we add an INLINE pragma, due to GHC bug > | #2353. If the GHC bug isn't fixed, I think the next best thing would > | be to force the inlining of toList with a RULES pragma in Data.Foldable. > > I will look into this during August. Add yourself to the cc list for > #2353 if interested. OK, I'll add INLINE toList to Data.Foldable and wait till it starts working. (If it doesn't, I can add RULES to unconditionally force the expansion.) In any case we won't need to manually deforest Data.Sequence.sortBy. The naive version will (soon) be just as fast: sortBy :: (a -> a -> Ordering) -> Seq a -> Seq a sortBy cmp xs = fromList2 (length xs) (Data.List.sortBy cmp (toList xs)) From ross at soi.city.ac.uk Thu Aug 6 07:32:57 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Thu Aug 6 07:13:42 2009 Subject: Proposal #3271: new methods for Data.Sequence In-Reply-To: References: <2518b95d0907191857q29a442b1l86609db8e3e58d17@mail.gmail.com> Message-ID: <20090806113257.GB5961@soi.city.ac.uk> On Mon, Jul 20, 2009 at 02:33:52PM -0400, Louis Wasserman wrote: > Among all the methods I added to Data.Sequence, the most code-intensive > are tails/inits and consDigitToTree/snocDigitToTree. > > The second was more intensively used with some previous zipWith > implementations that have since been replaced, but they add moderate > speed improvements to the significant amount of already-bulky code > for appending two sequences, and I can live with that on my conscience. In the latest version of your patch, consDTree/snocDTree are defined using foldr/foldl, which expands to repeated consTree/snocTree. In that case, it would be simpler to go back to writing out the repeated consTree/snocTree. From malcolm.wallace at cs.york.ac.uk Thu Aug 6 07:45:10 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Thu Aug 6 07:25:53 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7AB05F.7010809@gmail.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> <4A7AB05F.7010809@gmail.com> Message-ID: On 6 Aug 2009, at 11:28, Simon Marlow wrote: > What's the reasoning for dropping 6a-e, but retaining "provided that > all the other terms of clause 6 are complied with"? The specific para you quote starts "For an executable", and we are talking about libraries here, so it is inapplicable. > The other LGPL exception you quoted (from OCaml?) omits the whole of > clause 6, which seems a lot simpler. Clause 6 also includes the following terms, which you did not quote, and which I think are highly pertinent to retain: As an exception to the Sections above, you may also compile or link a "work that uses the Library" with the Library to produce a work containing portions of the Library, and distribute that work under terms of your choice, provided that the terms permit modification of the work for the customer's own use and reverse engineering for debugging such modifications. You must give prominent notice with each copy of the work that the Library is used in it and that the Library and its use are covered by this License. You must supply a copy of this License. If the work during execution displays copyright notices, you must include the copyright notice for the Library among them, as well as a reference directing the user to the copy of this License. Regards, Malcolm From duncan.coutts at worc.ox.ac.uk Thu Aug 6 07:47:29 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 6 07:51:45 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F892@ELON17P32001A.csfb.cs-group.com> References: <20090804225541.GF23958@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F892@ELON17P32001A.csfb.cs-group.com> Message-ID: <1249559249.20327.9371.camel@localhost> On Thu, 2009-08-06 at 10:22 +0100, Sittampalam, Ganesh wrote: > Don Stewart wrote: > > Here's a ticket for Simon Marlow's proposal: > > > > http://trac.haskell.org/haskell-platform/ticket/86 > > > > Let's discuss, then have the steering committee recommend yay/nay. > > In http://www.haskell.org/pipermail/libraries/2009-July/012139.html, > Duncan said: > > > It is not the intention that the committee members get much more say > in > > policy decisions than other active contributors to discussion on the > > libraries mailing list. We hope therefore that the membership of the > > committee does not need to be terribly formal. The only difficulty > comes > > when consensus cannot be reached on an issue where making some > decision > > is widely agreed to be better than making no decision. There is no > > specified protocol for this situation at this stage. > > So I'm a bit confused about what the purpose of the steering committee's > recommendation would be. Would it amount to a decision about inclusion, > or an expression of their opinion in the hope of guiding the community > towards a consensus, or something else? The first choice contradicts > what Duncan said was the role of the committee, while the second doesn't > seem any more worthwhile than individual members expressing their own > opinions separately. Yes, there's clearly some confusion. It's not the job of the steering committee to decide on individual packages going in. We want that to be a community review process. The steering committee can use this or some other package as a concrete example to help the community discuss what the criteria for package inclusion ought to be. Having a specific example may help to clarify the issue (or the details might distract). Duncan From duncan.coutts at worc.ox.ac.uk Thu Aug 6 07:59:15 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 6 07:51:49 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A78154E.2010606@gmail.com> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> Message-ID: <1249559955.20327.9375.camel@localhost> On Tue, 2009-08-04 at 12:02 +0100, Simon Marlow wrote: > On 04/08/2009 00:59, Ian Lynagh wrote: > > On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart wrote: > >> > >> How would you identify the top, say, 5 libs to add? > > > > I would not look for libs to add. I would wait for people to come and > > tell me that they think that particular libs are worthy of addition, and > > then decide whether or not I agree. > > Ok, to kick things off then, I propose the following: > > Add > > * binary > * getopt > * gtk2hs Now that's just crazy-talk! :-) What/where is getopt? It's not on hackage. Elsewhere we've raised our concerns about binary. gtk2hs is of course not cabalised. > Also > > * keep an eye on text. We certainly want it, but it's > a young package and there's no text I/O yet. I'd say go for it. If the current API is good then that's enough. It's not clear that there needs to be separate I/O modules for it. I might suggest hiding all the fusion modules for starter though. > * decide which regex package(s) we want I'd like input from the regex maintainer here. In particular which backend do we want in the platform and can we please avoid having more than one (if we can't choose how do we expect users to choose). > * remove html? (we have xhtml) On the other hand xhtml seems to be going out of fashion. > * replace haskell-src with haskell-src-exts Yes, if the maintainer thinks its ready. > * remove packedstring Yes! And editline. Duncan From duncan.coutts at worc.ox.ac.uk Thu Aug 6 08:02:06 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 6 07:51:50 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> References: <4A781B01.8080409@gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F871@ELON17P32001A.csfb.cs-group.com> <89254074-7E85-411B-B226-D0D47F99B036@di.ens.fr> Message-ID: <1249560126.20327.9377.camel@localhost> On Tue, 2009-08-04 at 14:21 +0200, Axel Simon wrote: > Hi Simon et al., > > On Aug 4, 2009, at 13:57, Sittampalam, Ganesh wrote: > > > Simon Marlow wrote: > > > >> By bringing gtk2hs into the platform, we would be giving the gtk2hs > >> maintainers a helpful boost; they'd get more testing for one thing. > > > > I think that should explicitly not be a reason to bring things into > > the > > platform. > > > Bringing Gtk2Hs into the platform is certainly desirable. During the > last one or two years, the amount of users has grown steadily which > is nice. However, Pete and my time is rather limited and is often > being used up by installation issues and questions that could be > answered with better documentation. Thus, it would be desirable to > bring Gtk2Hs into the platform because it would force us to simplify > installation and documentation. > > For the former part, I wonder if cabalization is important for > Gtk2Hs. A cabalized version of Gtk2Hs would allow people to use Cairo > and Pango to create PDF documents without the need to install the GUI > parts of the library. On the contrary, if Gtk2Hs is shipped with the > platform, then all libraries are available anyway and cabalization > might not be as important. > > My question: how important is cabalization for a package that wants > to be part of the platform? Speaking with my distribution and HP release team hats on I think it is essential. We cannot sensibly write automation tools for packages that are not cabalised. I fully expect it to become a required criteria for package inclusion. Duncan From duncan.coutts at worc.ox.ac.uk Thu Aug 6 08:07:31 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 6 07:51:55 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> Message-ID: <1249560451.20327.9381.camel@localhost> On Wed, 2009-08-05 at 15:37 +0200, Axel Simon wrote: > On Aug 5, 2009, at 10:16, Malcolm Wallace wrote: > > > On 4 Aug 2009, at 23:05, Don Stewart wrote: > >> > >> I would appreciate input from the HaXml and HDBC authors (our most > >> popular LGPL-licensed Haskell libraries) about what they feel the > >> licensing issues/constraints should be for the Haskell Platform. > > > > Licensing clarity is important for users I think. But equally some > > users may desire to use LGPL libraries too. Hence my suggestion > > that there be a separate platform of free/LGPL code (and GPL > > tools), which can depend on the proprietary-friendly BSD-licensed > > platform, but not the other way round. > > > >> I've not yet seen anyone publish something on how to satisfy LGPL > >> for Haskell libraries. > > > > The static-linking exception is the commonest means of working > > around ghc's technical limitations here. The exception is part of > > wxHaskell's license (but not Gtk2hs's), and HaXml (+polyparse on > > which it depends) has the exception too. > > I don't think it would be much of a problem to weaken the license of > Gtk2Hs to a BSD license. The underlying Gtk+ C library is, of course, > LGPL but the C library can be linked in dynamically. I think it'd probably be sufficient to use a static linking exception, like many other libs do. I don't think it's necessary to go all the way for a BSD license (unless it turns out that the community as a whole decides that the whole HP must be BSD and that LGPL with linking exception is not enough). Duncan From marlowsd at gmail.com Thu Aug 6 08:56:59 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Thu Aug 6 08:37:44 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> <4A7AB05F.7010809@gmail.com> Message-ID: <4A7AD31B.9000703@gmail.com> On 06/08/2009 12:45, Malcolm Wallace wrote: > On 6 Aug 2009, at 11:28, Simon Marlow wrote: > >> What's the reasoning for dropping 6a-e, but retaining "provided that >> all the other terms of clause 6 are complied with"? > > The specific para you quote starts "For an executable", and we are > talking about libraries here, so it is inapplicable. No, not at all - the licensee of the library is the person statically linking the library into their executable and distributing it. The license specifically governs their obligations. So what is required in that case? Suppose I want to distribute an executable statically linked with HaXml, what do I have to do to comply with clause 6? >> The other LGPL exception you quoted (from OCaml?) omits the whole of >> clause 6, which seems a lot simpler. > > Clause 6 also includes the following terms, which you did not quote, and > which I think are highly pertinent to retain: > As an exception to the Sections above, you may also compile or link a > "work that uses the Library" with the Library to produce a work > containing portions of the Library, and distribute that work under terms > of your choice, provided that the terms permit modification of the work > for the customer's own use and reverse engineering for debugging such > modifications. You must give prominent notice with each copy of the work > that the Library is used in it and that the Library and its use are > covered by this License. You must supply a copy of this License. If the > work during execution displays copyright notices, you must include the > copyright notice for the Library among them, as well as a reference > directing the user to the copy of this License. Granted, it looks like you do want that bit. Cheers, Simon From malcolm.wallace at cs.york.ac.uk Thu Aug 6 10:18:25 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Thu Aug 6 09:59:08 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7AD31B.9000703@gmail.com> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> <4A7AB05F.7010809@gmail.com> <4A7AD31B.9000703@gmail.com> Message-ID: <81C1DA4E-93F7-4361-B550-5BC16823D260@cs.york.ac.uk> > No, not at all - the licensee of the library is the person > statically linking the library into their executable and > distributing it. The license specifically governs their obligations. Oh, my mistake, yes you are right. However I believe the whole para is just a clarification of 6(a). > Suppose I want to distribute an executable statically linked with > HaXml, what do I have to do to comply with clause 6? My reading of the phrase "any data and utility programs needed for reproducing the executable from it" is that you may not attempt to get round the requirements of the license by providing the proprietary parts of your software in a format that requires the use of special tools (like a linker) that are not otherwise freely available. This might also be intended to exclude techniques like encryption, unless you supply the key etc. So, in the specific case of HaXml, there is no additional burden on you. Regards, Malcolm From ml at isaac.cedarswampstudios.org Thu Aug 6 16:58:49 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Thu Aug 6 16:39:38 2009 Subject: Containers in the Haskell Platform In-Reply-To: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> References: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> Message-ID: <4A7B4409.5000604@isaac.cedarswampstudios.org> Alexander Dunlap wrote: > Hi all, > > As we consider what we want to go into the Haskell Platform, I would > like to seriously discuss the issue of container polymorphism, > ... > Thanks for your time in reading this admittedly long message, Thanks for considering it! I think these are important issues; however, they're difficult ones that have been around for a while, and I don't think we'll be able to solve them quickly just because we're making a Haskell Platform. In some cases issues aren't as bad as they could be. If a library will take Lists of anything, then you can convert your type to a List. If its semantics mean it can only operate on bytes, then it can take lazy-ByteStrings (or Strict if it only operates on small things or if it can't stream data anyway)... Although it may be efficient, bytestrings work badly or not at all for non-byte data. Incidentally, at some point in the future, a decision may depend on what compiler-extensions we're willing to allow into the Platform (I suspect that a nice polymorphic solution [with possibly speed problems] basically requires Type Functions or equivalent..) -Isaac From wasserman.louis at gmail.com Thu Aug 6 17:38:05 2009 From: wasserman.louis at gmail.com (Louis Wasserman) Date: Thu Aug 6 17:19:06 2009 Subject: Proposal #3271: new methods for Data.Sequence Message-ID: Fixed. Is there anything else? Louis Wasserman wasserman.louis@gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090806/d827695e/attachment.html From ml at isaac.cedarswampstudios.org Thu Aug 6 18:16:31 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Thu Aug 6 17:57:20 2009 Subject: licensing [was: Thinking about what's missing in our library coverage] In-Reply-To: <1249560451.20327.9381.camel@localhost> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <93A38983-3E52-42DE-A1E3-79EE887C19BB@cs.york.ac.uk> <1DC647A9-E789-4610-BA42-1C99E98D4E4A@di.ens.fr> <1249560451.20327.9381.camel@localhost> Message-ID: <4A7B563F.3050507@isaac.cedarswampstudios.org> as for LGPL + static linking exception: - I wonder if FSF lawyers have made a recommendation for wording or license for this purpose? (Currently there exists wording from WxWidgets, and some Haskell libraries, and is there anything else in the real world? Are there licenses other than LGPL intended to accomplish this that work better?) - What about LGPL version 3? Can/do we accommodate that? Is it easier or harder to write the exception in v3 (in which the LGPL is written in terms of the GPLv3)? -Isaac From niklas.broberg at gmail.com Thu Aug 6 18:28:35 2009 From: niklas.broberg at gmail.com (Niklas Broberg) Date: Thu Aug 6 18:16:27 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249559955.20327.9375.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> Message-ID: >> ? * replace haskell-src with haskell-src-exts > > Yes, if the maintainer thinks its ready. The maintainer says that the library itself is definitely ready in its current state. :-) The only problem is that it depends on cpphs, which is not in the platform. I personally think that cpphs warrants inclusion in the platform too, but it comes with that same LGPL (+linking exception) "burden" that is already discussed (haskell-src-exts itself uses BSD). I see two possibilities: * Allow LGPL (+linking exception) in the HP and include cpphs. Then haskell-src-exts can replace haskell-src immediately. * I remove the rather small functionality in haskell-src-exts that depends on cpphs (namely deliterating literate source files). Then a slightly stumped haskell-src-exts could replace haskell-src after I release such a stumped version. Ok, I actually see one more possibililty, that Malcolm re-licences cpphs as BSD to have it included. I don't expect that to happen though. :-) Cheers, /Niklas From wren at community.haskell.org Thu Aug 6 23:36:20 2009 From: wren at community.haskell.org (wren ng thornton) Date: Thu Aug 6 23:18:21 2009 Subject: Containers in the Haskell Platform In-Reply-To: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> References: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> Message-ID: <4A7BA134.4080909@community.haskell.org> Alexander Dunlap wrote: > The way I see it, we have a few options. We could introduce some type > classes to cover the various container operations and ask all Platform > packages to only use operations from these classes, where applicable. Depending on what exactly you mean by "applicable", there are worse efficiency problems than large dictionaries. Speaking on behalf of Data.Trie, there are a number of functions that can be offered on tries which give large asymptotic improvements over naive/generic functions on maps. In general the efficiency of tries is comperable with maps, the whole point of using tries is to be able to use these special functions which aren't available to maps. This problem isn't unique to tries, it's the major reason why there are so many container types out there. For instance, the snoc function can be defined for [] but it's not a good idea to use it frequently; snoc on Data.Sequence, however, is perfectly fine to use often. While it'd be nice to have some type classes to make it easier for client code to be polymorphic in the container type, we should be wary of putting too much faith in it. When there are asymptotic or large-constant differences depending on the container, this needs to be made clear to clients and users. This isn't just polymorphism over the representation type, it's polymorphism over *algorithms*. This is a major source of performance problems and confusion in OOP languages, which do provide such generic interfaces. For a language like Haskell the computational complexity of functions belongs to the API and is as important as the type signature and pragmatic properties. One thing that may help though is if we could agree on what functions should belong to such an interface and how they should be named. The bytestring-trie package took after Data.Map and Data.IntMap, though there's been much contention over the names and the order of arguments since then. Having a coherent set of guidelines (specialized to sequence and map structures) would probably help more than technical changes. -- Live well, ~wren From alexander.dunlap at gmail.com Fri Aug 7 01:51:29 2009 From: alexander.dunlap at gmail.com (Alexander Dunlap) Date: Fri Aug 7 01:32:30 2009 Subject: Containers in the Haskell Platform In-Reply-To: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> References: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> Message-ID: <57526e770908062251w4ccb50bcvd94a2a175419855@mail.gmail.com> On Thu, Aug 6, 2009 at 12:15 AM, Alexander Dunlap wrote: > Hi all, > > As we consider what we want to go into the Haskell Platform, I would > like to seriously discuss the issue of container polymorphism, > especially, but not exclusively, in dealing with strings. We currently > have a large number of container types, many of which are going to > co-exist long-term, i.e. there is no winner; they are useful in > different situations. Here is an incomplete list of what we are > dealing with: > > Fully polymorphic sequence types: > [] > Data.Sequence > Data.Edison sequence types > Data.Array.Array & friends > > Restricted polymorphic sequence types: > uvector > vector > storable-vector > > Monomorphic sequence types: > ByteString (Word8 and Char8) > Lazy ByteString (Word8 and Char8) > Text > > Fully polymorphic associative array types: > Data.Map > Edison associative arrays > gmap (?) > > Less-polymorphic associative array types: > bytestring-tries > list-tries > Data.IntMap > > I'm sure there are more, of course, but those are the ones that a > cursory search reveals. > > The question, then, is how we ought to deal with all of these types in > the Haskell Platform. The current popular libraries deal with the > multitude of container types in a very ad-hoc way. For instance, the > Parsec parsing library allows you to parse Strings and Strict > ByteStrings; it also includes a class so that you can parse other > things. Attoparsec allows you to parse only lazy bytestrings. I don't > know of any library that parses Text yet. The binary library deals > with lazy bytestrings, while binary-strict lets you use strict > bytestrings. Libraries like zlib just pick a type (lazy bytestrings) > and use that, not accounting for other possible types you may want to > use (granted, in zlib's case, the choice of a single data structure, > seems fairly reasonable, given potential use cases). > > The point here is that the same functionality and practically the same > code can be split over several different packages based on what data > structure you want to use. Worse, there is no standard way of dealing > with this separation: Parsec uses a type class, while binary uses two > separate packages. And interoperability could potentially become a > problem: the split library, for instance, only supports lists, so you > have to roll your own solutions for splitting ByteStrings or Text. I > think that with the introduction of the Haskell Platform, we ought to > take the opportunity to clean up some of this chaos so that we have a > clean set of standard libraries for people to build on. > > The way I see it, we have a few options. We could introduce some type > classes to cover the various container operations and ask all Platform > packages to only use operations from these classes, where applicable. > (The ListLike library would probably be a good place to start.) The > issue here is efficiency, massive class dictionaries, and type system > issues (we need MPTCs or TFs; also, since not all sequence types are > polymorphic, we probably need to have separate map and rigidMap > functions like in ListLike, and even that doesn't work for types like > uvector that can take some, but not all, concrete element types). > > We could also implement a standard module-naming scheme: for every > package that operates on strings, for instance, we could require the > list-based code to go in Foo.Bar.String and the Text code to go in > Foo.Bar.Text. Whenever we "blessed" a new string datatype (presumably > not very often), all packages would have to add a new module. The > issue is code duplication. > > We could also implement a package naming scheme: have foo-text and > foo-string for the two different datatypes. > > What does everyone think about the need for standardization here and > how we ought to do it? I'm sorry that, having less experience with > Haskell than many, I can't give as much in the way of concrete > proposals, but I think it's important that we hash something out here > to get more organization in our Platform. > > Thanks for your time in reading this admittedly long message, > Alex > Both comments are well-taken; thank you very much for the feedback! I'm starting to understand more about the issue and have a more concrete idea to suggest. Some of the most common libraries for dealing with strings seem to be parsers. Parsing libraries would benefit a lot by becoming polymorphic, as Strings, ByteStrings, Texts, and sequences of other token types all have legitimate reasons to be parsed. Furthermore, parsers seem to be libraries that use a very small number of functions from the string type: I think something along the lines of the following would suffice: class Monoid c => Parseable c where type El c :: * uncons :: c -> Maybe (El c, c) splitAt :: Int -> c -> (c,c) cons :: El c -> c -> c span, break :: (El c -> Bool) -> c -> (c,c) with mempty and mappend used from the Monoid class. I don't think this would have very great efficiency implications, since most of the parsing libraries seem to use those functions anyway. I will try to mock up an example of this and try some tests and benchmarks to see if I can come up with an even more concrete proposal. Any further input is very much appreciated. Thanks! Alex From ndmitchell at gmail.com Fri Aug 7 04:18:07 2009 From: ndmitchell at gmail.com (Neil Mitchell) Date: Fri Aug 7 03:58:48 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> Message-ID: <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> Hi >> Yes, if the maintainer thinks its ready. > > The maintainer says that the library itself is definitely ready in its > current state. :-) This user definitely agrees! > The only problem is that it depends on cpphs, which is not in the > platform. I personally think that cpphs warrants inclusion in the > platform too, but it comes with that same LGPL (+linking exception) > "burden" that is already discussed (haskell-src-exts itself uses BSD). > > I see two possibilities: > > * Allow LGPL (+linking exception) in the HP and include cpphs. Then > haskell-src-exts can replace haskell-src immediately. I support this. > * I remove the rather small functionality in haskell-src-exts that > depends on cpphs (namely deliterating literate source files). Then a > slightly stumped haskell-src-exts could replace haskell-src after I > release such a stumped version. I seriously dislike the idea that packages must have useful features removed to end in the HP. I also want you to include CPP support in HSE, and removing cpphs makes this more unlikely. Thanks Neil From marlowsd at gmail.com Fri Aug 7 04:44:21 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Fri Aug 7 04:25:09 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <81C1DA4E-93F7-4361-B550-5BC16823D260@cs.york.ac.uk> References: <4A781B01.8080409@gmail.com> <4A78308B.2060601@o-donoghue.com> <5D533A85-8FD4-4391-B4AA-D5B38EF453FA@cs.york.ac.uk> <20090804213907.GN22887@whirlpool.galois.com> <4A78AFDB.8030806@therning.org> <20090804220523.GA23958@whirlpool.galois.com> <4A79E7AE.2020004@complete.org> <4A7AB05F.7010809@gmail.com> <4A7AD31B.9000703@gmail.com> <81C1DA4E-93F7-4361-B550-5BC16823D260@cs.york.ac.uk> Message-ID: <4A7BE965.1020504@gmail.com> On 06/08/2009 15:18, Malcolm Wallace wrote: >> No, not at all - the licensee of the library is the person statically >> linking the library into their executable and distributing it. The >> license specifically governs their obligations. > > Oh, my mistake, yes you are right. However I believe the whole para is > just a clarification of 6(a). > >> Suppose I want to distribute an executable statically linked with >> HaXml, what do I have to do to comply with clause 6? > > My reading of the phrase "any data and utility programs needed for > reproducing the executable from it" is that you may not attempt to get > round the requirements of the license by providing the proprietary parts > of your software in a format that requires the use of special tools > (like a linker) that are not otherwise freely available. This might also > be intended to exclude techniques like encryption, unless you supply the > key etc. > > So, in the specific case of HaXml, there is no additional burden on you. Ok. Might it be clearer if the LGPL exception stated that this paragraph is also excluded, along with 6(a)-6(e)? Cheers, Simon From johan.tibell at gmail.com Fri Aug 7 04:49:56 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Fri Aug 7 04:30:58 2009 Subject: Containers in the Haskell Platform In-Reply-To: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> References: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> Message-ID: <90889fe70908070149ye769a8bk6fe680cf9fdd7a1e@mail.gmail.com> Hi Alex, On Thu, Aug 6, 2009 at 9:15 AM, Alexander Dunlap wrote: > The way I see it, we have a few options. We could introduce some type > classes to cover the various container operations and ask all Platform > packages to only use operations from these classes, where applicable. > (The ListLike library would probably be a good place to start.) The > issue here is efficiency, massive class dictionaries, and type system > issues (we need MPTCs or TFs; also, since not all sequence types are > polymorphic, we probably need to have separate map and rigidMap > functions like in ListLike, and even that doesn't work for types like > uvector that can take some, but not all, concrete element types). I don't like the *Like naming much. I think that the type class deserves the shorter name and concrete instantiations should use more specialized names. Sequence wouldn't be a too bad name for data types that support indexing. Stream would be a good name for types that don't. There's an issue with regards to monads though. What type classes do we use for a Stream backed by e.g. an I/O resource? Example using a concrete element type: class ByteStream s where uncons :: s -> (Word8, s) null :: s -> Bool class ByteStreamM s where uncons :: Monad m => s -> m (Word8, s) null :: Monad m => s -> m Bool Do we need two type classes for each "concept" (e.g. stream, sequence)? > We could also implement a standard module-naming scheme: for every > package that operates on strings, for instance, we could require the > list-based code to go in Foo.Bar.String and the Text code to go in > Foo.Bar.Text. Whenever we "blessed" a new string datatype (presumably > not very often), all packages would have to add a new module. The > issue is code duplication. I would like to propose this as a general recommendation for module naming as there's already plenty of modules that do it this way. It also seems to be the most sensible place to put the data type as it groups related modules in the module hierarchy and thus in the documentation. I don't know about requiring all packages to add a new module every time there's a new type though. > > We could also implement a package naming scheme: have foo-text and > foo-string for the two different datatypes. This seems like a good scheme to me. It's the same - scheme that's proposed for modules above. > > What does everyone think about the need for standardization here and > how we ought to do it? I'm sorry that, having less experience with > Haskell than many, I can't give as much in the way of concrete > proposals, but I think it's important that we hash something out here > to get more organization in our Platform. I definitely think the container problem needs to be tackled. However, I don't know what the solution should look like. Cheers, Johan From marlowsd at gmail.com Fri Aug 7 04:58:36 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Fri Aug 7 04:39:19 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249559955.20327.9375.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> Message-ID: <4A7BECBC.6060103@gmail.com> On 06/08/2009 12:59, Duncan Coutts wrote: > On Tue, 2009-08-04 at 12:02 +0100, Simon Marlow wrote: >> On 04/08/2009 00:59, Ian Lynagh wrote: >>> On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart wrote: >>>> >>>> How would you identify the top, say, 5 libs to add? >>> >>> I would not look for libs to add. I would wait for people to come and >>> tell me that they think that particular libs are worthy of addition, and >>> then decide whether or not I agree. >> >> Ok, to kick things off then, I propose the following: >> >> Add >> >> * binary >> * getopt >> * gtk2hs > > Now that's just crazy-talk! :-) > > What/where is getopt? It's not on hackage. Elsewhere we've raised our > concerns about binary. gtk2hs is of course not cabalised. Hmm, I assumed getopt was on Hackage. Where is it then? Aargh! It's still in base :) We temporarily moved it out a while ago, and then moved it back in. Forget I mentioned it. What is stopping gtk2hs being cabalised at this stage? Is it just work, or does it need extensions to Cabal? >> Also >> >> * keep an eye on text. We certainly want it, but it's >> a young package and there's no text I/O yet. > > I'd say go for it. If the current API is good then that's enough. It's > not clear that there needs to be separate I/O modules for it. I might > suggest hiding all the fusion modules for starter though. My concern is API consistency. The way to get Text from a Handle is very roundabout: you need to read as a bytestring and then convert to Text using an encoding (and if you want the locale encoding you need text-icu I presume). Whereas the way to get a String from a Handle in the locale encoding is just hGetContents. We need to make the API more streamlined here. I haven't put a lot of thought into it, admittedly, but the first step would be to think about how to unify the codec interface, so that both packages can use the same codecs. Perhaps this isn't a showstopper. >> * decide which regex package(s) we want > > I'd like input from the regex maintainer here. In particular which > backend do we want in the platform and can we please avoid having more > than one (if we can't choose how do we expect users to choose). > >> * remove html? (we have xhtml) > > On the other hand xhtml seems to be going out of fashion. Right, but I think xhtml has had a lot more attention over the years. Perhaps it should have the option to produce HTML - after all if you drop the XML header you're nearly there. Haddock has its own local copy of html. I wonder why that is... >> * replace haskell-src with haskell-src-exts > > Yes, if the maintainer thinks its ready. > >> * remove packedstring > > Yes! And editline. Absolutely. Cheers, Simon From malcolm.wallace at cs.york.ac.uk Fri Aug 7 06:31:22 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Fri Aug 7 06:12:01 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> Message-ID: <4A312FF5-CA69-4A2B-B877-167BE4F53BE7@cs.york.ac.uk> > * I remove the rather small functionality in haskell-src-exts that > depends on cpphs (namely deliterating literate source files). If your only use of cpphs is to deliterate source files, then I recommend that you simply incorporate that functionality directly into haskell-src-exts. Although cpphs's module Language.Preprocessor.Unlit is under the LGPL, it is based on code that was published in the Haskell 1.2 Report, Appendix C, which I assume is freely copyable. Regards, Malcolm From niklas.broberg at gmail.com Fri Aug 7 06:38:33 2009 From: niklas.broberg at gmail.com (Niklas Broberg) Date: Fri Aug 7 06:19:14 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A312FF5-CA69-4A2B-B877-167BE4F53BE7@cs.york.ac.uk> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <4A312FF5-CA69-4A2B-B877-167BE4F53BE7@cs.york.ac.uk> Message-ID: >> * I remove the rather small functionality in haskell-src-exts that >> depends on cpphs (namely deliterating literate source files). > > If your only use of cpphs is to deliterate source files, then I recommend > that you simply incorporate that functionality directly into > haskell-src-exts. ?Although cpphs's module Language.Preprocessor.Unlit is > under the LGPL, it is based on code that was published in the Haskell 1.2 > Report, Appendix C, which I assume is freely copyable. That's of course an option. Though as Neil pointed out, my preference would really be to *increase* my dependency on cpphs, by introducing support for CPP in source files. So if we can actually get cpphs in the HP, that would be the best solution by far. :-) Cheers, /Niklas From johan.tibell at gmail.com Fri Aug 7 07:20:53 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Fri Aug 7 07:01:52 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <20090806064852.GA30949@whirlpool.galois.com> References: <20090804225541.GF23958@whirlpool.galois.com> <57526e770908051212n72e28426u8f33e1b2c5cde164@mail.gmail.com> <20090805203443.GF28722@whirlpool.galois.com> <57526e770908051340m3b8eb605uaeedf3c803ca524@mail.gmail.com> <20090805204707.GG28722@whirlpool.galois.com> <90889fe70908052342r687bft953071453cd7bc2c@mail.gmail.com> <20090806064852.GA30949@whirlpool.galois.com> Message-ID: <90889fe70908070420l6e998231gffb6a87517816e13@mail.gmail.com> On Thu, Aug 6, 2009 at 8:48 AM, Don Stewart wrote: > johan.tibell: >> > I'm wary of breaking the 70 packages that use Data.Binary for this, >> > rather, add this as a list of API changes for the next major release. >> >> Would this really break that many libraries? Are there many libraries >> that catch the exception that can be raised by error? >> > > Ah! Good question -- maybe we can find out... by building Hackage! While I agree with the general sentiment I don't think that would work in this case as the thrown exception isn't part of the type. -- Johan From johan.tibell at gmail.com Fri Aug 7 07:25:26 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Fri Aug 7 07:06:26 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F892@ELON17P32001A.csfb.cs-group.com> References: <20090804225541.GF23958@whirlpool.galois.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F892@ELON17P32001A.csfb.cs-group.com> Message-ID: <90889fe70908070425i5a58951byeeb2ed15fb46f764@mail.gmail.com> On Thu, Aug 6, 2009 at 11:22 AM, Sittampalam, Ganesh wrote: > Don Stewart wrote: >> Here's a ticket for Simon Marlow's proposal: >> >> ? ? http://trac.haskell.org/haskell-platform/ticket/86 >> >> Let's discuss, then have the steering committee recommend yay/nay. > > In http://www.haskell.org/pipermail/libraries/2009-July/012139.html, > Duncan said: > >> It is not the intention that the committee members get much more say > in >> policy decisions than other active contributors to discussion on the >> libraries mailing list. We hope therefore that the membership of the >> committee does not need to be terribly formal. The only difficulty > comes >> when consensus cannot be reached on an issue where making some > decision >> is widely agreed to be better than making no decision. There is no >> specified protocol for this situation at this stage. > > So I'm a bit confused about what the purpose of the steering committee's > recommendation would be. Would it amount to a decision about inclusion, > or an expression of their opinion in the hope of guiding the community > towards a consensus, or something else? The first choice contradicts > what Duncan said was the role of the committee, while the second doesn't > seem any more worthwhile than individual members expressing their own > opinions separately. In my opinion the committee should just provide the framework in which the discussion takes place (i.e. a procedure for adding packages). It should also make sure that decisions get made and documented so they can be referred to in the future in case the same issues are raised again. I'm working on such a procedure and will proposed it to the community soon. Cheers, Johan From Christian.Maeder at dfki.de Fri Aug 7 07:49:44 2009 From: Christian.Maeder at dfki.de (Christian Maeder) Date: Fri Aug 7 07:30:25 2009 Subject: Adding binary to the Haskell Platform In-Reply-To: References: <20090804225541.GF23958@whirlpool.galois.com> <20090805143800.GA27517@whirlpool.galois.com> Message-ID: <4A7C14D8.606@dfki.de> Krasimir Angelov wrote: > I know that I can use compression to reduce the size of the output but > this will make the deserialization only slower, not faster. Couldn't the binary package try to create a ByteString with maximal sharing directly? I.e. every ByteString gets an index, which is marked and used, whenever the index is shorter (as ByteString) than the ByteString it stands for. (A similar approach is used for shared ATerms, ie. see http://www.haskell.org/pipermail/glasgow-haskell-users/2005-December/009485.html) Wouldn't that be faster than the separate compression and decompression phases and faster than reading and writing such large files? Cheers Christian From ganesh.sittampalam at credit-suisse.com Fri Aug 7 08:19:22 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Fri Aug 7 08:00:47 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: References: <20090803234432.GK17991@whirlpool.galois.com><20090803235901.GA2552@matrix.chaos.earth.li><4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost><4A312FF5-CA69-4A2B-B877-167BE4F53BE7@cs.york.ac.uk> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F8A8@ELON17P32001A.csfb.cs-group.com> Niklas Broberg wrote: >>> * I remove the rather small functionality in haskell-src-exts that >>> depends on cpphs (namely deliterating literate source files). >> >> If your only use of cpphs is to deliterate source files, then I >> recommend that you simply incorporate that functionality directly >> into haskell-src-exts. ?Although cpphs's module >> Language.Preprocessor.Unlit is under the LGPL, it is based on code >> that was published in the Haskell 1.2 Report, Appendix C, which I >> assume is freely copyable. > > That's of course an option. Though as Neil pointed out, my preference > would really be to *increase* my dependency on cpphs, by introducing > support for CPP in source files. So if we can actually get cpphs in > the HP, that would be the best solution by far. :-) But the CPP that GHC supports is different (albeit less good) than cpphs. So if you want to support the extension as it is currently defined/used, you should just shell out to cpp like GHC does. Cheers, Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From malcolm.wallace at cs.york.ac.uk Fri Aug 7 10:16:01 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Fri Aug 7 09:56:43 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F8A8@ELON17P32001A.csfb.cs-group.com> References: <20090803234432.GK17991@whirlpool.galois.com><20090803235901.GA2552@matrix.chaos.earth.li><4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost><4A312FF5-CA69-4A2B-B877-167BE4F53BE7@cs.york.ac.uk> <16442B752A06A74AB4D9F9A5FF076E4B03B9F8A8@ELON17P32001A.csfb.cs-group.com> Message-ID: > But the CPP that GHC supports is different (albeit less good) than > cpphs. So if you want to support the extension as it is currently > defined/used, you should just shell out to cpp like GHC does. Shelling out to CPP is not much help if the executable that uses haskell-src-exts is running on a machine that does not have a C pre- processor installed (or if it is not configured to be found easily). Regards, Malcolm From Axel.Simon at ens.fr Fri Aug 7 10:20:51 2009 From: Axel.Simon at ens.fr (Axel Simon) Date: Fri Aug 7 10:01:48 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7BECBC.6060103@gmail.com> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <4A7BECBC.6060103@gmail.com> Message-ID: <4678F3B8-7DF9-48FA-B3D5-355FAFEAECF7@di.ens.fr> On Aug 7, 2009, at 10:58, Simon Marlow wrote: > On 06/08/2009 12:59, Duncan Coutts wrote: >> On Tue, 2009-08-04 at 12:02 +0100, Simon Marlow wrote: >>> On 04/08/2009 00:59, Ian Lynagh wrote: >>>> On Mon, Aug 03, 2009 at 04:44:32PM -0700, Donald Bruce Stewart >>>> wrote: >>>>> >>>>> How would you identify the top, say, 5 libs to add? >>>> >>>> I would not look for libs to add. I would wait for people to >>>> come and >>>> tell me that they think that particular libs are worthy of >>>> addition, and >>>> then decide whether or not I agree. >>> >>> Ok, to kick things off then, I propose the following: >>> >>> Add >>> >>> * binary >>> * getopt >>> * gtk2hs >> >> Now that's just crazy-talk! :-) >> >> What/where is getopt? It's not on hackage. Elsewhere we've raised our >> concerns about binary. gtk2hs is of course not cabalised. > [..] > What is stopping gtk2hs being cabalised at this stage? Is it just > work, or does it need extensions to Cabal? I'm no Cabal expert, so Duncan might know more. What Gtk2Hs needs is the ability to depend on executables (tools like a modified c2hs) that are built by other Cabal packages. Furthermore, we need to generate .hs files using these tools. I don't know how difficult it is to use Cabal to generate the dependencies and invoke the right tools. For instance, a file like .chs.pp is translated to .chs using CPP or hscpp, then to .hs and .chi using our own c2hs and then it is compiled using ghc. Finally, it seems that we need file-specific options in order to compile certain files. I think Cabal has no mechanism for that. I'm sure it's all doable so it probably boils down to a lack of time :-) Cheers, Axel. From bos at serpentine.com Fri Aug 7 12:36:46 2009 From: bos at serpentine.com (Bryan O'Sullivan) Date: Fri Aug 7 12:17:26 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249559955.20327.9375.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> Message-ID: On Thu, Aug 6, 2009 at 4:59 AM, Duncan Coutts wrote: > > * keep an eye on text. We certainly want it, but it's > > a young package and there's no text I/O yet. > > I'd say go for it. If the current API is good then that's enough. It's > not clear that there needs to be separate I/O modules for it. I might > suggest hiding all the fusion modules for starter though. > It's really not ready for prime time yet. I'm making changes to the API at the moment, but they're not complete, and I want to see what the new localised I/O support in 6.12 looks like to figure out whether I can hook into that in an agreeable fashion to get localised I/O without jumping through too many hoops. I'm even contemplating trimming the naming from Data.Text to Text, as part of my little war on meaningless module prefixes :-) So please, don't go for it just yet. Pitch in with patches instead! Regards, Bryan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090807/dd707b4e/attachment.html From ekmett at gmail.com Fri Aug 7 22:52:49 2009 From: ekmett at gmail.com (Edward Kmett) Date: Fri Aug 7 22:33:27 2009 Subject: Containers in the Haskell Platform In-Reply-To: <90889fe70908070149ye769a8bk6fe680cf9fdd7a1e@mail.gmail.com> References: <57526e770908060015y23c5571ci1db0410b00c9e860@mail.gmail.com> <90889fe70908070149ye769a8bk6fe680cf9fdd7a1e@mail.gmail.com> Message-ID: <7fb8f82f0908071952u70b9f04bs43ef207d2d3443ba@mail.gmail.com> Note the Steam classes you mention already have an analogue hiding in parsec 3 -- minus the null check that is. Since, there, unconsing returns m (Maybe (c,s)). rather than m (c,s), permitting the check and uncons to be performed in the same operation, but denying access to them separately. -Edward Kmett On Fri, Aug 7, 2009 at 4:49 AM, Johan Tibell wrote: > Hi Alex, > > On Thu, Aug 6, 2009 at 9:15 AM, Alexander > Dunlap wrote: > > The way I see it, we have a few options. We could introduce some type > > classes to cover the various container operations and ask all Platform > > packages to only use operations from these classes, where applicable. > > (The ListLike library would probably be a good place to start.) The > > issue here is efficiency, massive class dictionaries, and type system > > issues (we need MPTCs or TFs; also, since not all sequence types are > > polymorphic, we probably need to have separate map and rigidMap > > functions like in ListLike, and even that doesn't work for types like > > uvector that can take some, but not all, concrete element types). > > I don't like the *Like naming much. I think that the type class > deserves the shorter name and concrete instantiations should use more > specialized names. Sequence wouldn't be a too bad name for data types > that support indexing. Stream would be a good name for types that > don't. There's an issue with regards to monads though. What type > classes do we use for a Stream backed by e.g. an I/O resource? Example > using a concrete element type: > > class ByteStream s where > uncons :: s -> (Word8, s) > null :: s -> Bool > > class ByteStreamM s where > uncons :: Monad m => s -> m (Word8, s) > null :: Monad m => s -> m Bool > > Do we need two type classes for each "concept" (e.g. stream, sequence)? > > > We could also implement a standard module-naming scheme: for every > > package that operates on strings, for instance, we could require the > > list-based code to go in Foo.Bar.String and the Text code to go in > > Foo.Bar.Text. Whenever we "blessed" a new string datatype (presumably > > not very often), all packages would have to add a new module. The > > issue is code duplication. > > I would like to propose this as a general recommendation for module > naming as there's already plenty of modules that do it this way. It > also seems to be the most sensible place to put the data type as it > groups related modules in the module hierarchy and thus in the > documentation. I don't know about requiring all packages to add a new > module every time there's a new type though. > > > > > We could also implement a package naming scheme: have foo-text and > > foo-string for the two different datatypes. > > This seems like a good scheme to me. It's the same > - scheme that's proposed for modules above. > > > > > What does everyone think about the need for standardization here and > > how we ought to do it? I'm sorry that, having less experience with > > Haskell than many, I can't give as much in the way of concrete > > proposals, but I think it's important that we hash something out here > > to get more organization in our Platform. > > I definitely think the container problem needs to be tackled. However, > I don't know what the solution should look like. > > Cheers, > > Johan > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090807/72a3fb9f/attachment.html From dons at galois.com Sat Aug 8 03:53:04 2009 From: dons at galois.com (Don Stewart) Date: Sat Aug 8 03:35:45 2009 Subject: Platform et al Message-ID: <20090808075304.GA8976@whirlpool.galois.com> http://joelrayking.wordpress.com/2009/08/08/starting-out-with-haskell/ "never thought I?d actually get this excited about learning a programming language again" Yay. This work pays off. -- Don From igloo at earth.li Sat Aug 8 13:46:30 2009 From: igloo at earth.li (Ian Lynagh) Date: Sat Aug 8 13:27:05 2009 Subject: Haskell Platform: Changelogs and "available since" Message-ID: <20090808174629.GA1522@matrix.chaos.earth.li> Hi all, Reading a thread in -cafe made me think of another couple of requirements to consider for platform packages: * A changelog for the package is maintained * In the documentation for each function etc, the first version to support the feature is given. Thanks Ian From johan.tibell at gmail.com Sat Aug 8 14:37:44 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Sat Aug 8 14:18:39 2009 Subject: Haskell Platform: Changelogs and "available since" In-Reply-To: <20090808174629.GA1522@matrix.chaos.earth.li> References: <20090808174629.GA1522@matrix.chaos.earth.li> Message-ID: <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> On Sat, Aug 8, 2009 at 7:46 PM, Ian Lynagh wrote: > * In the documentation for each function etc, the first version to > ?support the feature is given. I have often missed this when trying to pick a lower bound for my package dependencies. Creating a tool that computes this automatically would help the developers do this easily. -- Johan From marlowsd at gmail.com Sat Aug 8 16:06:22 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Sat Aug 8 15:47:01 2009 Subject: Haskell Platform: Changelogs and "available since" In-Reply-To: <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> References: <20090808174629.GA1522@matrix.chaos.earth.li> <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> Message-ID: <4A7DDABE.1080200@gmail.com> Johan Tibell wrote: > On Sat, Aug 8, 2009 at 7:46 PM, Ian Lynagh wrote: >> * In the documentation for each function etc, the first version to >> support the feature is given. > > I have often missed this when trying to pick a lower bound for my > package dependencies. Creating a tool that computes this automatically > would help the developers do this easily. It was proposed as a SoC project for this summer, but didn't make the cut: http://hackage.haskell.org/trac/summer-of-code/ticket/1565 I'd be more than happy to provide pointers/guidance if anyone wants to tackle this. Cheers, Simon From duncan.coutts at worc.ox.ac.uk Mon Aug 10 08:45:22 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Aug 10 08:51:22 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> Message-ID: <1249908322.20327.9489.camel@localhost> On Fri, 2009-08-07 at 09:18 +0100, Neil Mitchell wrote: > > I see two possibilities: > > > > * Allow LGPL (+linking exception) in the HP and include cpphs. Then > > haskell-src-exts can replace haskell-src immediately. > > I support this. > > > * I remove the rather small functionality in haskell-src-exts that > > depends on cpphs (namely deliterating literate source files). Then a > > slightly stumped haskell-src-exts could replace haskell-src after I > > release such a stumped version. > > I seriously dislike the idea that packages must have useful features > removed to end in the HP. I also want you to include CPP support in > HSE, and removing cpphs makes this more unlikely. It's clear that the licensing issue is going to be controversial and will take some time. I'm not sure it is sensible to try and work it out before the next major release, given that the higher priority has to be agreeing the procedure for adding packages. If we do not get around to agreeing the licensing issue then the default position has to be no new licenses 'til we do work it out properly. As a personal opinion, I'd certainly like to see cpphs in the platform and to have Cabal use it in preference to gcc -E / cpp. Duncan From igloo at earth.li Mon Aug 10 10:56:46 2009 From: igloo at earth.li (Ian Lynagh) Date: Mon Aug 10 10:37:18 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249908322.20327.9489.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> <1249908322.20327.9489.camel@localhost> Message-ID: <20090810145646.GA11531@matrix.chaos.earth.li> On Mon, Aug 10, 2009 at 01:45:22PM +0100, Duncan Coutts wrote: > > It's clear that the licensing issue is going to be controversial and > will take some time. I'm not sure it is sensible to try and work it out > before the next major release, given that the higher priority has to be > agreeing the procedure for adding packages. If we do not get around to > agreeing the licensing issue then the default position has to be no new > licenses 'til we do work it out properly. Or to put it another way, decide that "Licence is BSD or MIT" (or whatever the list really is) is a requirement for the upcoming major release. I'd agree with that. Generally, I think that conservatism is the best answer for the platform. It will be a lot worse to put something in and then decide to take it out again, than to decide to leave it out for now and then put it in a few months later. Thanks Ian From magnus at therning.org Mon Aug 10 11:00:24 2009 From: magnus at therning.org (Magnus Therning) Date: Mon Aug 10 10:40:55 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <20090810145646.GA11531@matrix.chaos.earth.li> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> <1249908322.20327.9489.camel@localhost> <20090810145646.GA11531@matrix.chaos.earth.li> Message-ID: On Mon, Aug 10, 2009 at 3:56 PM, Ian Lynagh wrote: > On Mon, Aug 10, 2009 at 01:45:22PM +0100, Duncan Coutts wrote: >> >> It's clear that the licensing issue is going to be controversial and >> will take some time. I'm not sure it is sensible to try and work it out >> before the next major release, given that the higher priority has to be >> agreeing the procedure for adding packages. If we do not get around to >> agreeing the licensing issue then the default position has to be no new >> licenses 'til we do work it out properly. > > Or to put it another way, decide that "Licence is BSD or MIT" (or > whatever the list really is) is a requirement for the upcoming major > release. > > I'd agree with that. Generally, I think that conservatism is the best > answer for the platform. It will be a lot worse to put something in and > then decide to take it out again, than to decide to leave it out for now > and then put it in a few months later. Would it make sense to decide already now that when GHC has enough support for dynamic libs to make it easy to comply with LGPL (without linkage exceptions), then the set of "HP approved licenses" will be extended to include LGPL? Would the up-and-comming Hackell compilers (who might not have dyn lib support) be happy with that? /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus?therning?org Jabber: magnus?therning?org http://therning.org/magnus identi.ca|twitter: magthe From awick at galois.com Mon Aug 10 11:05:56 2009 From: awick at galois.com (Adam Wick) Date: Mon Aug 10 10:49:10 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249908322.20327.9489.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> <1249908322.20327.9489.camel@localhost> Message-ID: <1249916756.7566.3.camel@ned.galois.com> On Mon, 2009-08-10 at 13:45 +0100, Duncan Coutts wrote: > It's clear that the licensing issue is going to be controversial and > will take some time. I'm not sure it is sensible to try and work it out > before the next major release, If you're not going to work it out now, that means that any new packages should assume the strictest possible restriction (BSD or equivalent only), yes? It's a lot easier to add packages later than remove them. - Adam From duncan.coutts at worc.ox.ac.uk Mon Aug 10 12:22:48 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Aug 10 13:01:38 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249916756.7566.3.camel@ned.galois.com> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> <1249908322.20327.9489.camel@localhost> <1249916756.7566.3.camel@ned.galois.com> Message-ID: <1249921368.20327.9565.camel@localhost> On Mon, 2009-08-10 at 08:05 -0700, Adam Wick wrote: > On Mon, 2009-08-10 at 13:45 +0100, Duncan Coutts wrote: > > It's clear that the licensing issue is going to be controversial and > > will take some time. I'm not sure it is sensible to try and work it out > > before the next major release, > If you're not going to work it out now, that means that any new packages > should assume the strictest possible restriction (BSD or equivalent > only), yes? It's a lot easier to add packages later than remove them. Yes, I noted: If we do not get around to agreeing the licensing issue then the default position has to be no new licenses 'til we do work it out properly. Since the current licenses in platform packages are only BSD then the default position until we agree a license policy would have to be BSD-only. Duncan From duncan.coutts at worc.ox.ac.uk Mon Aug 10 18:35:51 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Aug 10 20:30:39 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <4A7BECBC.6060103@gmail.com> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <4A7BECBC.6060103@gmail.com> Message-ID: <1249943751.20327.10163.camel@localhost> On Fri, 2009-08-07 at 09:58 +0100, Simon Marlow wrote: > >> * remove html? (we have xhtml) > > > > On the other hand xhtml seems to be going out of fashion. > > Right, but I think xhtml has had a lot more attention over the years. > Perhaps it should have the option to produce HTML - after all if you > drop the XML header you're nearly there. That sounds like a good idea. It would also make it a lot easier for programs to switch between formats. > Haddock has its own local copy of html. I wonder why that is... Purge it and find out :-) Duncan From awick at galois.com Tue Aug 11 06:21:29 2009 From: awick at galois.com (Adam Wick) Date: Tue Aug 11 06:04:42 2009 Subject: Thinking about what's missing in our library coverage In-Reply-To: <1249921368.20327.9565.camel@localhost> References: <20090803234432.GK17991@whirlpool.galois.com> <20090803235901.GA2552@matrix.chaos.earth.li> <4A78154E.2010606@gmail.com> <1249559955.20327.9375.camel@localhost> <404396ef0908070118v7ac83d5foc880ab2febdc4ee4@mail.gmail.com> <1249908322.20327.9489.camel@localhost> <1249916756.7566.3.camel@ned.galois.com> <1249921368.20327.9565.camel@localhost> Message-ID: <1249986089.3204.0.camel@ned.galois.com> On Mon, 2009-08-10 at 17:22 +0100, Duncan Coutts wrote: > On Mon, 2009-08-10 at 08:05 -0700, Adam Wick wrote: > > On Mon, 2009-08-10 at 13:45 +0100, Duncan Coutts wrote: > > > It's clear that the licensing issue is going to be controversial and > > > will take some time. I'm not sure it is sensible to try and work it out > > > before the next major release, > > If you're not going to work it out now, that means that any new packages > > should assume the strictest possible restriction (BSD or equivalent > > only), yes? It's a lot easier to add packages later than remove them. > Yes, I noted: Great! -Adam From roconnor at theorem.ca Tue Aug 11 20:33:24 2009 From: roconnor at theorem.ca (roconnor@theorem.ca) Date: Tue Aug 11 20:13:51 2009 Subject: (no subject) Message-ID: According to I'm supposed to ask here for bits of heirarchy. I have no idea if this is still the procedure. As far as I know nobody follows this procedure. But I could just be ignorant. :) I'd like Data.Colour for my colour package: I should point out that also used Data.Colour, so there is a conflict to be resolved here. -- Russell O'Connor ``All talk about `theft,''' the general counsel of the American Graphophone Company wrote, ``is the merest claptrap, for there exists no property in ideas musical, literary or artistic, except as defined by statute.'' From roconnor at theorem.ca Tue Aug 11 20:50:50 2009 From: roconnor at theorem.ca (roconnor@theorem.ca) Date: Tue Aug 11 20:31:16 2009 Subject: Data.Colour In-Reply-To: References: Message-ID: Sorry about the lack of subject. Here is a subject now. On Tue, 11 Aug 2009, roconnor@theorem.ca wrote: > According to > > I'm supposed to ask here for bits of heirarchy. I have no idea if this is > still the procedure. As far as I know nobody follows this procedure. But I > could just be ignorant. :) > > I'd like Data.Colour for my colour package: > > > I should point out that also > used Data.Colour, so there is a conflict to be resolved here. > > -- Russell O'Connor ``All talk about `theft,''' the general counsel of the American Graphophone Company wrote, ``is the merest claptrap, for there exists no property in ideas musical, literary or artistic, except as defined by statute.'' From duncan.coutts at worc.ox.ac.uk Wed Aug 12 17:59:28 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Wed Aug 12 17:50:40 2009 Subject: proposal for updates in next major HP release Message-ID: <1250114368.22128.227.camel@localhost> All, The previous decision[1] of the libraries list is that major HP releases are to be on a 4 month cycle for at least the first 12 months. [1]: http://trac.haskell.org/haskell-platform/wiki/Policy That means we're aiming for the next major release being in early September, about a month away from now. This email is to propose "obvious" updates. We hope these are not controversial and that we can quickly get an OK. We suggest updating each package to its latest stable release, except where noted below. In most cases this means relatively insignificant API changes. The cases of particular interest are: regex-* packages updated from 0.7x versions to 0.9x versions QuickCheck update from version 1.2.x to version 2.x We are not proposing to switch to parsec-3.x at this time. This proposal does not cover updates to the OpenGL package(s). Remove editline package Remove packedstring package The rationale for removing editline is that it is no longer used by ghc. It was only included in the platform in the first place because early releases of ghc-6.10.x shipped it as a core library. The rationale for removing packedstring is that is has been deprecated for some years now. It was only included in the platform because the template-haskell package still depended on it. The latest version of template-haskell no longer requires it. Duncan (on behalf of the HP release team) From igloo at earth.li Wed Aug 12 18:54:30 2009 From: igloo at earth.li (Ian Lynagh) Date: Wed Aug 12 18:34:54 2009 Subject: proposal for updates in next major HP release In-Reply-To: <1250114368.22128.227.camel@localhost> References: <1250114368.22128.227.camel@localhost> Message-ID: <20090812225430.GA4582@matrix.chaos.earth.li> On Wed, Aug 12, 2009 at 10:59:28PM +0100, Duncan Coutts wrote: > > The rationale for removing packedstring is that is has been deprecated > for some years now. It was only included in the platform because the > template-haskell package still depended on it. The latest version of > template-haskell no longer requires it. But unless I'm confused, this HP release will use GHC 6.10.4, in which template-haskell does depend on packedstring. Thanks Ian From dave at zednenem.com Thu Aug 13 15:51:22 2009 From: dave at zednenem.com (David Menendez) Date: Thu Aug 13 15:31:43 2009 Subject: proposal for updates in next major HP release In-Reply-To: <1250114368.22128.227.camel@localhost> References: <1250114368.22128.227.camel@localhost> Message-ID: <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> On Wed, Aug 12, 2009 at 5:59 PM, Duncan Coutts wrote: > We suggest updating each package to its latest stable release, except > where noted below. In most cases this means relatively insignificant API > changes. The cases of particular interest are: > ? ? ? ?QuickCheck update from version 1.2.x to version 2.x There is a lot of material on the web explaining the design of QuickCheck 1, how to use it, how to get started, and so forth. I haven't seen any documentation of QC 2 beyond the Haddock pages, which don't seem very helpful for new users. For example, for shrink it says: "Produces a (possibly) empty list of all the possible immediate shrinks of the given value." So far as I can tell, the concept of a "shrink" is not actually explained anywhere, leaving the user to look at the existing instances and guess what's appropriate. (Also, if I may nit-pick, either "empty" should be in the parentheses with "possibly", or there shouldn't be parentheses.) It would be nice if the packages in the Haskell Platform met some minimum standards for documentation. -- Dave Menendez From dons at galois.com Thu Aug 13 16:35:45 2009 From: dons at galois.com (Don Stewart) Date: Thu Aug 13 16:18:17 2009 Subject: proposal for updates in next major HP release In-Reply-To: <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> Message-ID: <20090813203545.GN2827@whirlpool.galois.com> dave: > On Wed, Aug 12, 2009 at 5:59 PM, Duncan > Coutts wrote: > > > We suggest updating each package to its latest stable release, except > > where noted below. In most cases this means relatively insignificant API > > changes. The cases of particular interest are: > > > ? ? ? ?QuickCheck update from version 1.2.x to version 2.x > > There is a lot of material on the web explaining the design of > QuickCheck 1, how to use it, how to get started, and so forth. I > haven't seen any documentation of QC 2 beyond the Haddock pages, which > don't seem very helpful for new users. > > For example, for shrink it says: "Produces a (possibly) empty list of > all the possible immediate shrinks of the given value." So far as I > can tell, the concept of a "shrink" is not actually explained > anywhere, leaving the user to look at the existing instances and guess > what's appropriate. (Also, if I may nit-pick, either "empty" should be > in the parentheses with "possibly", or there shouldn't be > parentheses.) > > It would be nice if the packages in the Haskell Platform met some > minimum standards for documentation. Shrinking is explained in the original QC papers -- it is a killer feature that we somehow lived without. That said, we need an analysis of the missing pieces for our existing libraries to meet standard. From dave at zednenem.com Thu Aug 13 18:37:01 2009 From: dave at zednenem.com (David Menendez) Date: Thu Aug 13 18:17:21 2009 Subject: proposal for updates in next major HP release In-Reply-To: <20090813203545.GN2827@whirlpool.galois.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <20090813203545.GN2827@whirlpool.galois.com> Message-ID: <49a77b7a0908131537x78d662e5sd8787ba41f3c5ee6@mail.gmail.com> On Thu, Aug 13, 2009 at 4:35 PM, Don Stewart wrote: > dave: >> >> It would be nice if the packages in the Haskell Platform met some >> minimum standards for documentation. > > Shrinking is explained in the original QC papers -- it is a killer > feature that we somehow lived without. Are you referring to Section 5.4 of "QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs" by Claessen and Hughes? That says that Andy Gill else implemented shrink (under a different name) in his version of QuickCheck and says that it returns "a list of smaller, but similar values to its argument -- for example, direct subtrees." Even if a Haskell Platform user was able to track down the original paper (published in 2000 and nowhere referenced in the QuickCheck documentation), and connect the "shrink" function to Gill's "smaller" function (text search won't help), all they would learn is that shrink returns values which are "smaller" and "similar" to the argument. I think we can do better than that. Here's another question, what's the motivation for (><) in QuickCheck 2? If you look at the instance for (,) in QuickCheck 1.2, you see coarbitrary (a, b) = coarbitrary a . coarbitrary b but in QuickCheck 2.1 it's coarbitrary (x,y) = coarbitrary x >< coarbitrary y I'm sure there's a good reason for that, but as far as I can tell it isn't stated anywhere. > That said, we need an analysis of the missing pieces for our existing > libraries to meet standard. Agreed. -- Dave Menendez From igloo at earth.li Thu Aug 13 19:39:01 2009 From: igloo at earth.li (Ian Lynagh) Date: Thu Aug 13 19:19:22 2009 Subject: (no subject) In-Reply-To: References: Message-ID: <20090813233901.GA30524@matrix.chaos.earth.li> On Tue, Aug 11, 2009 at 08:33:24PM -0400, roconnor@theorem.ca wrote: > According to > > I'm supposed to ask here for bits of heirarchy. I have no idea if this > is still the procedure. As far as I know nobody follows this procedure. > But I could just be ignorant. :) > > I'd like Data.Colour for my colour package: > > > I should point out that > also used Data.Colour, so there is a conflict to be resolved here. I think that if there is a package called "colour", then it should be the one to provide Data.Colour. I don't have any opinion on whether that should be your package, AC's package, or neither (as I haven't really looked at either of them). Maybe this should be decided by whatever process we end up with for Haskell Platform inclusion? Thanks Ian From iavor.diatchki at gmail.com Fri Aug 14 08:28:54 2009 From: iavor.diatchki at gmail.com (Iavor Diatchki) Date: Fri Aug 14 08:09:13 2009 Subject: (no subject) In-Reply-To: <20090813233901.GA30524@matrix.chaos.earth.li> References: <20090813233901.GA30524@matrix.chaos.earth.li> Message-ID: <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> Hello, I know the the Haskell Platform is in the spot-light at the moment but please do not mix it in into name conventions as well! Different packages can provide modules with the same name, so it seems perfectly reasonable to let programmers choose the names for their modules. -Iavor PS: Having lived in the US for a while I would prefer Color to Colour but obviously that is not particularly important. On Thu, Aug 13, 2009 at 7:39 PM, Ian Lynagh wrote: > On Tue, Aug 11, 2009 at 08:33:24PM -0400, roconnor@theorem.ca wrote: >> According to >> >> I'm supposed to ask here for bits of heirarchy. ?I have no idea if this >> is still the procedure. ?As far as I know nobody follows this procedure. >> But I could just be ignorant. :) >> >> I'd like Data.Colour for my colour package: >> >> >> I should point out that >> also used Data.Colour, so there is a conflict to be resolved here. > > I think that if there is a package called "colour", then it should be > the one to provide Data.Colour. > > I don't have any opinion on whether that should be your package, AC's > package, or neither (as I haven't really looked at either of them). > Maybe this should be decided by whatever process we end up with for > Haskell Platform inclusion? > > > Thanks > Ian > > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > From igloo at earth.li Fri Aug 14 08:38:25 2009 From: igloo at earth.li (Ian Lynagh) Date: Fri Aug 14 08:18:45 2009 Subject: (no subject) In-Reply-To: <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> References: <20090813233901.GA30524@matrix.chaos.earth.li> <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> Message-ID: <20090814123825.GA6619@matrix.chaos.earth.li> On Fri, Aug 14, 2009 at 08:28:54AM -0400, Iavor Diatchki wrote: > Hello, > I know the the Haskell Platform is in the spot-light at the moment but > please do not mix it in into name conventions as well! Different > packages can provide modules with the same name, so it seems perfectly > reasonable to let programmers choose the names for their modules. But it is then impossible (apart from with a non-portable GHC extension) to use both packages in one program/library. It will also cause confusion. Thanks Ian From magnus at therning.org Fri Aug 14 08:57:11 2009 From: magnus at therning.org (Magnus Therning) Date: Fri Aug 14 08:37:30 2009 Subject: (no subject) In-Reply-To: <20090814123825.GA6619@matrix.chaos.earth.li> References: <20090813233901.GA30524@matrix.chaos.earth.li> <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> <20090814123825.GA6619@matrix.chaos.earth.li> Message-ID: On Fri, Aug 14, 2009 at 1:38 PM, Ian Lynagh wrote: > On Fri, Aug 14, 2009 at 08:28:54AM -0400, Iavor Diatchki wrote: >> Hello, >> I know the the Haskell Platform is in the spot-light at the moment but >> please do not mix it in into name conventions as well! ?Different >> packages can provide modules with the same name, so it seems perfectly >> reasonable to let programmers choose the names for their modules. > > But it is then impossible (apart from with a non-portable GHC extension) > to use both packages in one program/library. > > It will also cause confusion. Naming and avoiding name collisions are important. However I think this is best dealt with on a case-by-case basis. I'm happy to have a complete wild-west mentality to naming outside HP... of course the astute Haskell developer would quickly realise that it's fraught with problems to create name collisions with packages already in HP and it would preclude inclusion in HP as well. /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus?therning?org Jabber: magnus?therning?org http://therning.org/magnus identi.ca|twitter: magthe From rizsotto at gmail.com Fri Aug 14 09:34:46 2009 From: rizsotto at gmail.com (Laszlo Nagy) Date: Fri Aug 14 09:15:05 2009 Subject: Haskell Platform: Changelogs and "available since" In-Reply-To: <4A7DDABE.1080200@gmail.com> References: <20090808174629.GA1522@matrix.chaos.earth.li> <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> <4A7DDABE.1080200@gmail.com> Message-ID: <431fb4c10908140634x65bf176fncefcea5f90d70c43@mail.gmail.com> On Sat, Aug 8, 2009 at 10:06 PM, Simon Marlow wrote: > Johan Tibell wrote: >> >> On Sat, Aug 8, 2009 at 7:46 PM, Ian Lynagh wrote: >>> >>> * In the documentation for each function etc, the first version to >>> ?support the feature is given. >> >> I have often missed this when trying to pick a lower bound for my >> package dependencies. Creating a tool that computes this automatically >> would help the developers do this easily. > > It was proposed as a SoC project for this summer, but didn't make the cut: > > http://hackage.haskell.org/trac/summer-of-code/ticket/1565 > > I'd be more than happy to provide pointers/guidance if anyone wants to > tackle this. > > Cheers, > ? ? ? ?Simon I do volunteer for this. How can I get guidance? Regards, Laszlo From ndmitchell at gmail.com Fri Aug 14 11:46:48 2009 From: ndmitchell at gmail.com (Neil Mitchell) Date: Fri Aug 14 11:27:05 2009 Subject: proposal for updates in next major HP release In-Reply-To: <49a77b7a0908131537x78d662e5sd8787ba41f3c5ee6@mail.gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <20090813203545.GN2827@whirlpool.galois.com> <49a77b7a0908131537x78d662e5sd8787ba41f3c5ee6@mail.gmail.com> Message-ID: <404396ef0908140846r1decf612x2e4729268a8dc48d@mail.gmail.com> Hi >>> It would be nice if the packages in the Haskell Platform met some >>> minimum standards for documentation. >> >> Shrinking is explained in the original QC papers -- it is a killer >> feature that we somehow lived without. QuickCheck 2 is an amazing piece of work. My experience is that it finds bugs in properties that QuickCheck could never find counter-examples for. Shrinking is also fantastic. However, the change from QuickCheck 1.x to QuickCheck 2.x is a big one - it's probably of the same magnitude as haskell-src to haskell-src-exts, just with a coincidence of using the same package name. At the same time I couldn't figure out how to QuickCheck IO properties so just used unsafePerformIO - I think the pieces are there but the documentation isn't. I'm not sure if QuickCheck 2 is right to go in the HP at this particular moment - but I know I'm never using QuickCheck 1 again. Thanks, Neil From roconnor at theorem.ca Fri Aug 14 12:07:56 2009 From: roconnor at theorem.ca (roconnor@theorem.ca) Date: Fri Aug 14 11:48:15 2009 Subject: (no subject) In-Reply-To: <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> References: <20090813233901.GA30524@matrix.chaos.earth.li> <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> Message-ID: On Fri, 14 Aug 2009, Iavor Diatchki wrote: > Different packages can provide modules with the same name, so it seems > perfectly reasonable to let programmers choose the names for their > modules. What you say here seems to be somewhat at odds with what is written in the below documentation: Specifically where it says: ``If, however, you are providing a library for any other part of the hierarchy [not in User or Org], then the module names in the should be allocated.'' Your disagreement might be okay. This documentation is quite old, and might be out of date. However, if this is the case then might I suggest it either be removed from haskell.org or somehow marked as being obsolete. -- Russell O'Connor ``All talk about `theft,''' the general counsel of the American Graphophone Company wrote, ``is the merest claptrap, for there exists no property in ideas musical, literary or artistic, except as defined by statute.'' From lemming at henning-thielemann.de Fri Aug 14 12:55:58 2009 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Fri Aug 14 12:36:18 2009 Subject: Adding an ignore function to Control.Monad In-Reply-To: <20090611214354.GD18000@sliver.repetae.net> References: <9979e72e0906110628o41b02873r535ce14ba332b08@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F691@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111048m54a9e041mbb6dee6d446818e3@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F698@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111146v117caf2cr5514650b650add9a@mail.gmail.com> <20090611214354.GD18000@sliver.repetae.net> Message-ID: On Thu, 11 Jun 2009, John Meacham wrote: > Also, treating '()' specially as 'don't care' as opposed to any other > unit type doesn't sit right with me either. Sure it is an obvious choice > for a don't care value, but the fact you can use others is nicely > consistent with the idea that other than having some syntatic sugar, > built in types are not special in haskell. Agreed, it would be certainly better to have a special type like: data Ignore = Ignore From lemming at henning-thielemann.de Fri Aug 14 13:20:42 2009 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Fri Aug 14 13:01:46 2009 Subject: Adding an ignore function to Control.Monad In-Reply-To: <49a77b7a0906111345t1cbd165cx724a1f4a65400d3d@mail.gmail.com> References: <9979e72e0906110628o41b02873r535ce14ba332b08@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F691@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111048m54a9e041mbb6dee6d446818e3@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F698@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111146v117caf2cr5514650b650add9a@mail.gmail.com> <49a77b7a0906111314k22ec56cek9a9bf0a5aabe7c91@mail.gmail.com> <49a77b7a0906111345t1cbd165cx724a1f4a65400d3d@mail.gmail.com> Message-ID: On Thu, 11 Jun 2009, David Menendez wrote: > I'd ask why system is indicating failure through an exit code instead > of throwing an exception. Yes, throwing exception would also be nice. However current Haskell IO system hides the possibility of exceptions and does not tell, which exceptions may occur. To this end, various monads were developed (transformers,mtl:Control.Monad.Error, explicit-exception:Control.Monad.Exception.Synchronous, control-monad-exception). Of course, under the hood they use return values. >> It's quite easy to add 'ignore' where necessary and it safes us from >> ignoring a result by accident. Avoiding errors is the goal of using >> types, isn't it? > > If that's all there is to it, then why aren't you using Agda or Coq? I assumed, Haskell shall also be developed towards dependent types, which are intended for more safety, aren't they? Nonetheless, I find Agda an interesting project. From bulat.ziganshin at gmail.com Fri Aug 14 13:25:35 2009 From: bulat.ziganshin at gmail.com (Bulat Ziganshin) Date: Fri Aug 14 13:06:04 2009 Subject: Adding an ignore function to Control.Monad In-Reply-To: References: <9979e72e0906110628o41b02873r535ce14ba332b08@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F691@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111048m54a9e041mbb6dee6d446818e3@mail.gmail.com> <16442B752A06A74AB4D9F9A5FF076E4B03B9F698@ELON17P32001A.csfb.cs-group.com> <49a77b7a0906111146v117caf2cr5514650b650add9a@mail.gmail.com> <20090611214354.GD18000@sliver.repetae.net> Message-ID: <533544443.20090814212535@gmail.com> Hello Henning, Friday, August 14, 2009, 8:55:58 PM, you wrote: >> Also, treating '()' specially as 'don't care' as opposed to any other >> unit type doesn't sit right with me either. Sure it is an obvious choice >> for a don't care value, but the fact you can use others is nicely >> consistent with the idea that other than having some syntatic sugar, >> built in types are not special in haskell. > Agreed, it would be certainly better to have a special type like: > data Ignore = Ignore and what () means, as you think? -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com From lemming at henning-thielemann.de Fri Aug 14 13:29:52 2009 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Fri Aug 14 13:10:10 2009 Subject: Adding an ignore function to Control.Monad In-Reply-To: <20090611224417.GE18000@sliver.repetae.net> References: <20090611214354.GD18000@sliver.repetae.net> <16442B752A06A74AB4D9F9A5FF076E4B03B9F69B@ELON17P32001A.csfb.cs-group.com> <20090611224417.GE18000@sliver.repetae.net> Message-ID: On Thu, 11 Jun 2009, John Meacham wrote: > On Thu, Jun 11, 2009 at 11:21:54PM +0100, Sittampalam, Ganesh wrote: >> >> On the flip side, could you give some examples where ignoring return >> values is useful and natural? > > certainly, the very useful parser example mentioned. > > (char 'x' >> char 'y') - we want to parse x, then y Are you really interested to only ignore the result of the first 'char' parser and maintain the result of the second one? > (char 'x' `mplus` char 'y') - we want to parse one of x or y, returning > which one we got so we can change behavior based on it. Indeed using the same 'char' parser in both cases is convenient. It seems that parser monads are different from IO monad, in that it is more common to ignore results. (Or parser monads are developed with the simplicity of ignoring results in mind.) With char :: Char -> Parser Char ignore :: Functor m => m a -> m Ignore (or m ()) ignore = fmap (const Ignore) we would have to write ignore (char 'x') >> char 'y' char 'x' `mplus` char 'y' With char :: Char -> Parser Ignore (or Parser ()) echo :: Functor m => (a -> m Ignore) -> (a -> m a) echo f x = fmap (const x) $ f x we would write char 'x' >> char 'y' echo char 'x' `mplus` echo char 'y' If the parser library anticipates frequent use of ignore it could define a type class. class IgnorableChar a where wrapChar :: Char -> a class IgnorableChar Ignore where wrapChar _ = Ignore class IgnorableChar Char where wrapChar = id char :: IgnorableChar char => Char -> Parser char Cumbersome to define, but as simple to use as today's 'char' with today's '>>'. Just as a compromise for parser libraries. Ok, all these suggestions make things more complicated. Convenience vs. safety - It's certainly a matter of taste which one is more important for you. > When using the 'Writer' monad or similar it comes up a whole lot. > sometimes you are interested in the result, sometimes the written value, > sometimes both. I think one should use Monoid class directly whereever possible. I have seen frequently that people use Writer monad instead of Monoid only because of the do notation. From duncan.coutts at worc.ox.ac.uk Fri Aug 14 15:27:21 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 14 15:07:07 2009 Subject: proposal for updates in next major HP release In-Reply-To: <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> Message-ID: <1250278041.5255.15.camel@localhost> On Thu, 2009-08-13 at 15:51 -0400, David Menendez wrote: > It would be nice if the packages in the Haskell Platform met some > minimum standards for documentation. I think that's one of the many things we'll want to talk about after we agree the basic procedures for adding new packages. Currently the steering committee is comping up with a suggestion (or possibly two alternative suggestions) for the basic procedure. The committee will put the suggestion(s) to the libs list. Once that's discussed and agreed then we'll move on to the specifics of what quality standards we want to ask for. As for updating existing packages to meet whatever we are asking for new packages, yes that's something we need to address. Not all of the core packages have obvious maintainers. Duncan From duncan.coutts at worc.ox.ac.uk Fri Aug 14 15:28:03 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 14 15:07:48 2009 Subject: proposal for updates in next major HP release In-Reply-To: <404396ef0908140846r1decf612x2e4729268a8dc48d@mail.gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <20090813203545.GN2827@whirlpool.galois.com> <49a77b7a0908131537x78d662e5sd8787ba41f3c5ee6@mail.gmail.com> <404396ef0908140846r1decf612x2e4729268a8dc48d@mail.gmail.com> Message-ID: <1250278083.5255.17.camel@localhost> On Fri, 2009-08-14 at 16:46 +0100, Neil Mitchell wrote: > Hi > > >>> It would be nice if the packages in the Haskell Platform met some > >>> minimum standards for documentation. > >> > >> Shrinking is explained in the original QC papers -- it is a killer > >> feature that we somehow lived without. > > QuickCheck 2 is an amazing piece of work. My experience is that it > finds bugs in properties that QuickCheck could never find > counter-examples for. Shrinking is also fantastic. However, the change > from QuickCheck 1.x to QuickCheck 2.x is a big one - it's probably of > the same magnitude as haskell-src to haskell-src-exts, just with a > coincidence of using the same package name. At the same time I > couldn't figure out how to QuickCheck IO properties so just used > unsafePerformIO - I think the pieces are there but the documentation > isn't. > > I'm not sure if QuickCheck 2 is right to go in the HP at this > particular moment - but I know I'm never using QuickCheck 1 again. Yes, it is a significant update. That's why we're asking people to think about it and decide if they want it updated now or later. My personal opinion is that QC2 is great and we should update to it now. Duncan From duncan.coutts at worc.ox.ac.uk Fri Aug 14 15:54:05 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 14 15:37:50 2009 Subject: proposal for updates in next major HP release In-Reply-To: <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> Message-ID: <1250279645.5255.42.camel@localhost> On Thu, 2009-08-13 at 15:51 -0400, David Menendez wrote: > It would be nice if the packages in the Haskell Platform met some > minimum standards for documentation. Further on this topic... I think it's a problem with many of our libs that while we have good enough reference documentation, there is often very little in the form of introductory documentation. As you pointed out in the QC example, each function can be accurately documented without it giving any idea what some of the concepts mean or how to use the library. I think partly this is that haddock is good at and encourages reference docs but not any other kind. It is possible to put introductory sections at the top of a module, they have to go in the export list. Without wanting to blow my own trumpet too much, with the docs for the 'tar' package I tried to give more introductory explanation and examples of how to use the api: http://hackage.haskell.org/packages/archive/tar/0.3.1.0/doc/html/Codec-Archive-Tar.html Also, I think the synopsis section becomes fairly useless for APIs that are bigger than about 10 items because it doesn't give any structure or indicate what is important and what is not. For larger multi-module packages there's also no obvious place to put introductory text. The user does not necessarily know which is the "main" module to look for it in. Duncan From duncan.coutts at worc.ox.ac.uk Fri Aug 14 16:00:50 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 14 15:40:40 2009 Subject: Haskell Platform: Changelogs and "available since" In-Reply-To: <431fb4c10908140634x65bf176fncefcea5f90d70c43@mail.gmail.com> References: <20090808174629.GA1522@matrix.chaos.earth.li> <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> <4A7DDABE.1080200@gmail.com> <431fb4c10908140634x65bf176fncefcea5f90d70c43@mail.gmail.com> Message-ID: <1250280050.5255.45.camel@localhost> On Fri, 2009-08-14 at 15:34 +0200, Laszlo Nagy wrote: > >> I have often missed this when trying to pick a lower bound for my > >> package dependencies. Creating a tool that computes this automatically > >> would help the developers do this easily. > > > > It was proposed as a SoC project for this summer, but didn't make the cut: > > > > http://hackage.haskell.org/trac/summer-of-code/ticket/1565 > > > > I'd be more than happy to provide pointers/guidance if anyone wants to > > tackle this. > > > > Cheers, > > Simon > > I do volunteer for this. How can I get guidance? Great! Do you use IRC? The #ghc IRC channel, the ghc users mailing list and cabal-devel mailing list would be great places to get guidance and feedback. Duncan From simon at joyful.com Fri Aug 14 16:12:47 2009 From: simon at joyful.com (Simon Michael) Date: Fri Aug 14 15:53:17 2009 Subject: proposal for updates in next major HP release In-Reply-To: <1250279645.5255.42.camel@localhost> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <1250279645.5255.42.camel@localhost> Message-ID: Duncan Coutts wrote: > package I tried to give more introductory explanation and examples of > how to use the api: > http://hackage.haskell.org/packages/archive/tar/0.3.1.0/doc/html/Codec-Archive-Tar.html Indeed, that's very nice, and I agree with the rest of what you said. Perhaps related, I am often torn between, and duplicate info across, the various places to put documentation: - the cabal file, appears on the hackage page - module or function haddock, appears in the haddock api docs - the project's web page - the README file - non-haddock code comments, visible only to source code readers - the more elaborate user manual Currently I prioritise them approximately in the order above, using more of them as the project grows. From qdunkan at gmail.com Fri Aug 14 17:17:15 2009 From: qdunkan at gmail.com (Evan Laforge) Date: Fri Aug 14 16:57:33 2009 Subject: proposal for updates in next major HP release In-Reply-To: References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <1250279645.5255.42.camel@localhost> Message-ID: <2518b95d0908141417q37205982qcc147a2d32788491@mail.gmail.com> I like the format where haddock has reference, and then the main module's synopsis has links to external documentation like papers and tutorials. A warning if the paper's terminology differs from the implementation would be nice! As for what the "main module" is, I would put everything in the package within a single root module and then put the links to learning material in the root module's synopsis, even if the root doesn't doesn't contain anything else, or is just re-exports. On the subject of quickcheck 2, the main page (which is http://www.cs.chalmers.se/~rjmh/QuickCheck/ according to hackage) doesn't give any indication that there are two versions, doesn't say to which version the docs apply to, doesn't have any release information, "current developments" doesn't look very current, and doesn't have any dates anyway. So I had no idea there were two versions, or that the newer version was better. Yeah, I'm basically echoing David, but my experience was the same. From marlowsd at gmail.com Mon Aug 17 04:53:55 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 17 04:34:10 2009 Subject: Haskell Platform: Changelogs and "available since" In-Reply-To: <1250280050.5255.45.camel@localhost> References: <20090808174629.GA1522@matrix.chaos.earth.li> <90889fe70908081137j351887c4yd67365153d42ea5c@mail.gmail.com> <4A7DDABE.1080200@gmail.com> <431fb4c10908140634x65bf176fncefcea5f90d70c43@mail.gmail.com> <1250280050.5255.45.camel@localhost> Message-ID: <4A891AA3.50300@gmail.com> On 14/08/2009 21:00, Duncan Coutts wrote: > On Fri, 2009-08-14 at 15:34 +0200, Laszlo Nagy wrote: > >>>> I have often missed this when trying to pick a lower bound for my >>>> package dependencies. Creating a tool that computes this automatically >>>> would help the developers do this easily. >>> >>> It was proposed as a SoC project for this summer, but didn't make the cut: >>> >>> http://hackage.haskell.org/trac/summer-of-code/ticket/1565 >>> >>> I'd be more than happy to provide pointers/guidance if anyone wants to >>> tackle this. >>> >>> Cheers, >>> Simon >> >> I do volunteer for this. How can I get guidance? > > Great! > > Do you use IRC? The #ghc IRC channel, the ghc users mailing list and > cabal-devel mailing list would be great places to get guidance and > feedback. Yes - and I think it would be a good idea to throw some ideas around first, and draft a design on the wiki before starting to hack something up. Cheers, Simon From duncan.coutts at worc.ox.ac.uk Mon Aug 17 09:10:30 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Mon Aug 17 08:50:10 2009 Subject: Question of HP updates that depend on new packages Message-ID: <1250514630.5255.197.camel@localhost> All, A requirement of packages in the platform is that all dependencies of packages also be in the platform (excluding C libs). When an updated version of an existing platform package includes dependencies on packages that are currently outside the platform we must decide if those packages are to be added to the platform (or if we must stick with the older version of the package). Sometimes this will be trivial, such as if a package splits in two but keeps essentially the same API and module namespace. Sometimes it would means pulling in new packages. Normally new packages should go through the standard process for adding packages [Note: currently, the HP steering committee is nearly ready to present a draft of the procedure for adding new packages.] The case of the OpenGL 2.2.x -> 2.3 update covers both cases above. That is it includes straightforward package splits and also new packages. The last HP release included OpenGL-2.2.1.1. The latest version on hackage is OpenGL-2.3.0.0. OpenGL-2.2.x depends on the base package. OpenGL-2.3.0.0 depends on these extra packages: OpenGLRaw GLURaw ObjectName StateVar Tensor The first two are relatively straightforward. The OpenGL package has modules in the namespace Graphics.Rendering.OpenGL. The OpenGLRaw and GLURaw add modules in the namespaces Graphics.Rendering.OpenGL.Raw and Graphics.Rendering.GLU.Raw. One can easily class these new packages as merely split from the OpenGL package and this does not need much review. The other three packages are: package: ObjectName modules: Data.ObjectName package: StateVar modules: Data.StateVar package: Tensor modules: Data.Tensor These add new modules in the common Data.* namespace, indicating that they are intended for re-use more widely by other packages. The release team should not make a decision on these three packages on its own. We wish to refer these packages to the libraries list for a decision. In particular the libraries list should decide if these new packages should be rubber stamped because they are new dependencies of a package already in the platform, or if the new packages should go through the standard review process for adding packages. Duncan, on behalf of the HP release team From ganesh.sittampalam at credit-suisse.com Mon Aug 17 09:18:03 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Mon Aug 17 08:58:19 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <1250514630.5255.197.camel@localhost> References: <1250514630.5255.197.camel@localhost> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com> Duncan Coutts wrote: > When an updated version of an existing platform package includes > dependencies on packages that are currently outside the platform we > must decide if those packages are to be added to the platform (or if > we must stick with the older version of the package). The other specific considerations in this mail aside, sticking with older versions of a package sounds like a recipe for disaster. Isn't there an implicit or explicit requirement on maintainers of packages in the platform that they will only make changes that are acceptable for the platform? I realise that for the current set that have sort of ended up in the platform by default there has been no process of those maintainers accepting these constraints, but nonetheless there needs to be some understanding in place for the future. > In particular the libraries list should decide if these new packages > should be rubber stamped because they are new dependencies of a > package already in the platform, or if the new packages should go > through the standard review process for adding packages. I don't think new dependencies of this nature, where the functionality exposed is not logically part of the original package, should ever be rubber stamped. Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From marlowsd at gmail.com Mon Aug 17 10:29:14 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 17 10:09:26 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <1250514630.5255.197.camel@localhost> References: <1250514630.5255.197.camel@localhost> Message-ID: <4A89693A.8060800@gmail.com> On 17/08/2009 14:10, Duncan Coutts wrote: > All, > > A requirement of packages in the platform is that all dependencies of > packages also be in the platform (excluding C libs). > > When an updated version of an existing platform package includes > dependencies on packages that are currently outside the platform we must > decide if those packages are to be added to the platform (or if we must > stick with the older version of the package). > > Sometimes this will be trivial, such as if a package splits in two but > keeps essentially the same API and module namespace. Sometimes it would > means pulling in new packages. Normally new packages should go through > the standard process for adding packages > > [Note: currently, the HP steering committee is nearly ready to present a > draft of the procedure for adding new packages.] > > The case of the OpenGL 2.2.x -> 2.3 update covers both cases above. That > is it includes straightforward package splits and also new packages. > > The last HP release included OpenGL-2.2.1.1. The latest version on > hackage is OpenGL-2.3.0.0. > > OpenGL-2.2.x depends on the base package. > > OpenGL-2.3.0.0 depends on these extra packages: > OpenGLRaw > GLURaw > ObjectName > StateVar > Tensor > > The first two are relatively straightforward. The OpenGL package has > modules in the namespace Graphics.Rendering.OpenGL. The OpenGLRaw and > GLURaw add modules in the namespaces Graphics.Rendering.OpenGL.Raw and > Graphics.Rendering.GLU.Raw. One can easily class these new packages as > merely split from the OpenGL package and this does not need much review. > > The other three packages are: > > package: ObjectName > modules: Data.ObjectName > > package: StateVar > modules: Data.StateVar > > package: Tensor > modules: Data.Tensor > > These add new modules in the common Data.* namespace, indicating that > they are intended for re-use more widely by other packages. > > The release team should not make a decision on these three packages on > its own. We wish to refer these packages to the libraries list for a > decision. > > In particular the libraries list should decide if these new packages > should be rubber stamped because they are new dependencies of a package > already in the platform, or if the new packages should go through the > standard review process for adding packages. We should consider these 3 packages as candidates for inclusion in the platform, and deal with them as we would other candidates; i.e. subject them to the proposal/review process (which I know is in the process of being specified). In this case, we are in the slightly unfortunate position that if the packages are not accepted for inclusion, and even in the absence of any decision at all, the platform will be unable to ship the latest version of OpenGL. In the event that the new packages were rejected, then I imagine OpenGL would either have to re-absorb those modules (as hidden modules), or the platform would have to drop OpenGL since the old version would be unmaintained. I'm fairly confident it wouldn't come to that, though. Personally, I doubt the new packages would be acceptable, at least in their current form. e.g. ObjectName is a very general term for a library that seems to have quite specific functionality. Cheers, Simon From marlowsd at gmail.com Mon Aug 17 10:38:57 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 17 10:19:12 2009 Subject: (no subject) In-Reply-To: References: <20090813233901.GA30524@matrix.chaos.earth.li> <5ab17e790908140528m6d05963byd58246375b401b0c@mail.gmail.com> Message-ID: <4A896B81.1090902@gmail.com> On 14/08/2009 17:07, roconnor@theorem.ca wrote: > On Fri, 14 Aug 2009, Iavor Diatchki wrote: > >> Different packages can provide modules with the same name, so it seems >> perfectly reasonable to let programmers choose the names for their >> modules. > > What you say here seems to be somewhat at odds with what is written in > the below documentation: > > > > > Specifically where it says: > > ``If, however, you are providing a library for any other part of the > hierarchy [not in User or Org], then the module names in the should be > allocated.'' > > Your disagreement might be okay. This documentation is quite old, and > might be out of date. However, if this is the case then might I suggest > it either be removed from haskell.org or somehow marked as being obsolete. That document is quite out of date, I've now removed it to avoid confusion. There are no official policies regarding module namespace now. You can upload a package to Hackage containing whatever module names you like. However, if you choose modules that overlap with another package, then users will have some difficulty in using both packages together (not necessarily a showstopper). If you choose silly module names, that's a a library design issue in the same way that choosing silly function names would be. We really ought to have some good coding style guidelines that include how to choose module names. Cheers, Simon From marlowsd at gmail.com Mon Aug 17 10:45:51 2009 From: marlowsd at gmail.com (Simon Marlow) Date: Mon Aug 17 10:26:04 2009 Subject: proposal for updates in next major HP release In-Reply-To: <1250279645.5255.42.camel@localhost> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <1250279645.5255.42.camel@localhost> Message-ID: <4A896D1F.7090206@gmail.com> On 14/08/2009 20:54, Duncan Coutts wrote: > On Thu, 2009-08-13 at 15:51 -0400, David Menendez wrote: > >> It would be nice if the packages in the Haskell Platform met some >> minimum standards for documentation. > > Further on this topic... > > I think it's a problem with many of our libs that while we have good > enough reference documentation, there is often very little in the form > of introductory documentation. > > As you pointed out in the QC example, each function can be accurately > documented without it giving any idea what some of the concepts mean or > how to use the library. > > I think partly this is that haddock is good at and encourages reference > docs but not any other kind. It is possible to put introductory sections > at the top of a module, they have to go in the export list. Without > wanting to blow my own trumpet too much, with the docs for the 'tar' > package I tried to give more introductory explanation and examples of > how to use the api: > http://hackage.haskell.org/packages/archive/tar/0.3.1.0/doc/html/Codec-Archive-Tar.html > > Also, I think the synopsis section becomes fairly useless for APIs that > are bigger than about 10 items because it doesn't give any structure or > indicate what is important and what is not. For larger multi-module > packages there's also no obvious place to put introductory text. The > user does not necessarily know which is the "main" module to look for it > in. I've wanted to get rid of the synopsis, or make it optional, or make it click-to-expand, or something else, for a long time. But I never managed to get around to it - perhaps a Hackathon project for someone? Cheers, Simon From johan.tibell at gmail.com Mon Aug 17 11:18:48 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Mon Aug 17 10:59:17 2009 Subject: proposal for updates in next major HP release In-Reply-To: <4A896D1F.7090206@gmail.com> References: <1250114368.22128.227.camel@localhost> <49a77b7a0908131251g2801be1av57a9767fded8d0b7@mail.gmail.com> <1250279645.5255.42.camel@localhost> <4A896D1F.7090206@gmail.com> Message-ID: <90889fe70908170818j6038d4cn5c72fb101bab0f3d@mail.gmail.com> On Mon, Aug 17, 2009 at 4:45 PM, Simon Marlow wrote: > > I've wanted to get rid of the synopsis, or make it optional, or make it > click-to-expand, or something else, for a long time. ?But I never managed to > get around to it - perhaps a Hackathon project for someone? Me too. I've also been thinking about having Haddock output more semantically correct HTML to cut down on the document size and make it easier to change the CSS styling. There's an open bug about the issue in the Haddock trac: http://trac.haskell.org/haddock/ticket/108 Perhaps the whole HTML output module could be overhauled to make it easier to experiment with. Do we have a good HTML combinator library that we could use to generate the output? Cheers, Johan From jwlato at gmail.com Mon Aug 17 11:50:44 2009 From: jwlato at gmail.com (John Lato) Date: Mon Aug 17 11:30:53 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <4A89693A.8060800@gmail.com> References: <1250514630.5255.197.camel@localhost> <4A89693A.8060800@gmail.com> Message-ID: <9979e72e0908170850q6980f74amcfd3f75111a95b9d@mail.gmail.com> On Mon, Aug 17, 2009 at 3:29 PM, Simon Marlow wrote: > > We should consider these 3 packages as candidates for inclusion in the > platform, and deal with them as we would other candidates; i.e. subject > them to the proposal/review process (which I know is in the process of > being specified). > > In this case, we are in the slightly unfortunate position that if the > packages are not accepted for inclusion, and even in the absence of any > decision at all, the platform will be unable to ship the latest version > of OpenGL. I would expect this to become a common occurrence as the platform grows, unless a suitable policy is devised. I think Ganesh's suggestion of making this an explicit requirement of maintainers is a very good idea, and codifying exactly what changes are acceptable for the platform may also be helpful. Cheers, John Lato From duncan.coutts at worc.ox.ac.uk Tue Aug 18 07:10:08 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Tue Aug 18 06:49:41 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com> References: <1250514630.5255.197.camel@localhost> <16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com> Message-ID: <1250593808.5255.1665.camel@localhost> On Mon, 2009-08-17 at 14:18 +0100, Sittampalam, Ganesh wrote: > Duncan Coutts wrote: > > > When an updated version of an existing platform package includes > > dependencies on packages that are currently outside the platform we > > must decide if those packages are to be added to the platform (or if > > we must stick with the older version of the package). > > The other specific considerations in this mail aside, sticking with > older versions of a package sounds like a recipe for disaster. It's certainly not ideal. It'd conflict with the HP goal of trying to synchronise the versions of packages people are targeting and using. > Isn't there an implicit or explicit requirement on maintainers of > packages in the platform that they will only make changes that are > acceptable for the platform? It's not explicitly codified, particularly how maintainers are expected to judge what is acceptable. In the case at hand the maintainer did ask for feedback on the libraries list about the new dependencies, though I'm not sure that it was clear to people that the implication was that these new packages would be intended become part of the platform. > I realise that for the current set that have sort of ended up in the > platform by default there has been no process of those maintainers > accepting these constraints, but nonetheless there needs to be some > understanding in place for the future. I agree. As an example, in the GNOME process they have a page listing the responsibilities of maintainers. There's an implication when packages are proposed for inclusion that they have maintainers who are happy to follow the guidelines. The steering committee will be putting a recommendation on the package addition process to this list very shortly. When that gets discussed here we might also like to talk about drawing up a short set of guidelines / principles for maintainers. So yes, the general point is if we all communicate properly and understand each others expectations then we should never end up in the situation where we're forced to decide between accepting under-reviewed dependencies or sticking with an old version of a package. It's completely avoidable. > > In particular the libraries list should decide if these new packages > > should be rubber stamped because they are new dependencies of a > > package already in the platform, or if the new packages should go > > through the standard review process for adding packages. > > I don't think new dependencies of this nature, where the functionality > exposed is not logically part of the original package, should ever be > rubber stamped. That is also my opinion. Duncan From malcolm.wallace at cs.york.ac.uk Tue Aug 18 07:17:14 2009 From: malcolm.wallace at cs.york.ac.uk (Malcolm Wallace) Date: Tue Aug 18 06:57:21 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <1250593808.5255.1665.camel@localhost> References: <1250514630.5255.197.camel@localhost> <16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com> <1250593808.5255.1665.camel@localhost> Message-ID: <62C53100-A2B9-49ED-96C7-8FB807C902A5@cs.york.ac.uk> >> The other specific considerations in this mail aside, sticking with >> older versions of a package sounds like a recipe for disaster. > > It's certainly not ideal. It'd conflict with the HP goal of trying to > synchronise the versions of packages people are targeting and using. I don't see what is so awful about it. Libraries often have a "stable" version and a "development" version. Sticking with the stable version is exactly the right thing for the Platform to do, until the maintainer of the library declares that the development version is sufficiently robust to become the new stable branch. And it is not a disaster if the Platform packagers take a different view on the relative stability of different versions. Regards, Malcolm From marco-oweber at gmx.de Tue Aug 18 07:40:09 2009 From: marco-oweber at gmx.de (Marc Weber) Date: Tue Aug 18 07:20:18 2009 Subject: hasktags - If nobody minds I'll maintain it.. In-Reply-To: <1248464435-sup-1772@nixos> References: <1248464435-sup-1772@nixos> Message-ID: <1250595339-sup-2286@nixos> Igloo suggested taking over maintainership. The new darcs home of hasktags is on the community server: http://code.haskell.org/hasktags/ Feedback is welcome. What should I do with the version number? It was Version: 0.67 so I just bumped it to 0.68 although I added some more features (see previous mail). The version is based on an older version of hasktags, not the most recent one. However I verified that it finds all tags the current hackage version finds. I'll upload this version to hackage if you don't find any problems with this version. Marc Weber From ganesh.sittampalam at credit-suisse.com Tue Aug 18 08:24:56 2009 From: ganesh.sittampalam at credit-suisse.com (Sittampalam, Ganesh) Date: Tue Aug 18 08:05:23 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <62C53100-A2B9-49ED-96C7-8FB807C902A5@cs.york.ac.uk> References: <1250514630.5255.197.camel@localhost><16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com><1250593808.5255.1665.camel@localhost> <62C53100-A2B9-49ED-96C7-8FB807C902A5@cs.york.ac.uk> Message-ID: <16442B752A06A74AB4D9F9A5FF076E4B03B9F912@ELON17P32001A.csfb.cs-group.com> Malcolm Wallace wrote: >>> The other specific considerations in this mail aside, sticking with >>> older versions of a package sounds like a recipe for disaster. >> >> It's certainly not ideal. It'd conflict with the HP goal of trying to >> synchronise the versions of packages people are targeting and using. > > I don't see what is so awful about it. Libraries often have a > "stable" version and a "development" version. Sticking with the > stable version is exactly the right thing for the Platform to do, > until the maintainer of the library declares that the development > version is sufficiently robust to become the new stable branch. And > it is not a disaster if the Platform packagers take a different view > on the relative stability of different versions. Sorry for being unclear - I meant being indefinitely stuck on an old version, rather than temporarily while a new stable version was under development. Cheers, Ganesh =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html =============================================================================== From duncan.coutts at worc.ox.ac.uk Tue Aug 18 08:56:15 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Tue Aug 18 09:57:52 2009 Subject: Question of HP updates that depend on new packages In-Reply-To: <62C53100-A2B9-49ED-96C7-8FB807C902A5@cs.york.ac.uk> References: <1250514630.5255.197.camel@localhost> <16442B752A06A74AB4D9F9A5FF076E4B03B9F900@ELON17P32001A.csfb.cs-group.com> <1250593808.5255.1665.camel@localhost> <62C53100-A2B9-49ED-96C7-8FB807C902A5@cs.york.ac.uk> Message-ID: <1250600175.5255.1790.camel@localhost> On Tue, 2009-08-18 at 12:17 +0100, Malcolm Wallace wrote: > >> The other specific considerations in this mail aside, sticking with > >> older versions of a package sounds like a recipe for disaster. > > > > It's certainly not ideal. It'd conflict with the HP goal of trying to > > synchronise the versions of packages people are targeting and using. > > I don't see what is so awful about it. Libraries often have a > "stable" version and a "development" version. Sticking with the > stable version is exactly the right thing for the Platform to do, > until the maintainer of the library declares that the development > version is sufficiently robust to become the new stable branch. Sure, in general there can be stable versions and development / preview versions. In the current case, the maintainer recommends the latest version as the stable one and the one to use. There would be some unfortunate fragmentation in this case if no solution is found. > And it is not a disaster if the Platform packagers take a different view > on the relative stability of different versions. That's true, and there are cases like that, but if lots of people agree with the package maintainer (or depend on the latest features) then in practise we do get fragmentation. Duncan From ross at soi.city.ac.uk Wed Aug 19 11:20:23 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Wed Aug 19 11:00:28 2009 Subject: Proposal #3271: new methods for Data.Sequence In-Reply-To: References: Message-ID: <20090819152023.GA7760@soi.city.ac.uk> Two more things: I think that tails and inits may be more complex than necessary. When I see partial pattern matches like in zipDigit/zipNode, I worry that the hidden invariants are both extra complexity for a human reader and a missed opportunity for the compiler. I think you're doing it because you want the two traversals to go in opposite directions, so that you don't need to start with the total size of the prefix or suffix, but it turns out that you're computing these anyway. So you might as well combine the pairs of traversals in each case. The names of the sequential search functions: takeWhile :: (a -> Bool) -> Seq a -> Seq a takeWhileEnd :: (a -> Bool) -> Seq a -> Seq a dropWhile :: (a -> Bool) -> Seq a -> Seq a dropWhileEnd :: (a -> Bool) -> Seq a -> Seq a span :: (a -> Bool) -> Seq a -> (Seq a, Seq a) spanEnd :: (a -> Bool) -> Seq a -> (Seq a, Seq a) break :: (a -> Bool) -> Seq a -> (Seq a, Seq a) breakEnd :: (a -> Bool) -> Seq a -> (Seq a, Seq a) elemIndex :: Eq a => a -> Seq a -> Maybe Int elemIndices :: Eq a => a -> Seq a -> [Int] elemLastIndex :: Eq a => a -> Seq a -> Maybe Ind elemIndicesDesc :: Eq a => a -> Seq a -> [Int] findIndex :: (a -> Bool) -> Seq a -> Maybe Int findIndices :: (a -> Bool) -> Seq a -> [Int] findLastIndex :: (a -> Bool) -> Seq a -> Maybe Int findIndicesDesc :: (a -> Bool) -> Seq a -> [Int] conflict with the symmetrical view of sequences elsewhere in the interface. Sequences are viewed as having left and right ends, not front and back or first and last. (Of course the existing functions take and drop have similar issues, but at least they didn't involve new names.) So I'd prefer L and R variants of each (perhaps with spanl/spanr and breakl/breakr for consistency with other similar names). From ross at soi.city.ac.uk Wed Aug 19 11:40:20 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Wed Aug 19 11:20:26 2009 Subject: proposal #3335: make some Applicative functions into methods, and split off Data.Functor In-Reply-To: <20090629124545.GA6378@soi.city.ac.uk> References: <20090629124545.GA6378@soi.city.ac.uk> Message-ID: <20090819154020.GB7760@soi.city.ac.uk> On Mon, Jun 29, 2009 at 01:45:45PM +0100, Ross Paterson wrote: > The proposal is that the following functions > > (<$) :: Functor f => a -> f b -> f a > (*>) :: Applicative f => f a -> f b -> f b > (<*) :: Applicative f => f a -> f b -> f a > some :: Alternative f => f a -> f [a] > many :: Alternative f => f a -> f [a] > > are moved into the corresponding classes, with the existing implementations > as default definitions. Henning asked for concrete examples (i.e. code) where this would give a substantial performance win while still defining the same function, including termination properties. (But it seems reasonable not to rely on RULES to improve the asymptotic complexity.) I gave an example for (<$). Does anyone have concrete examples for the others? From ekmett at gmail.com Wed Aug 19 13:04:13 2009 From: ekmett at gmail.com (Edward Kmett) Date: Wed Aug 19 12:44:16 2009 Subject: proposal #3335: make some Applicative functions into methods, and split off Data.Functor In-Reply-To: <20090819154020.GB7760@soi.city.ac.uk> References: <20090629124545.GA6378@soi.city.ac.uk> <20090819154020.GB7760@soi.city.ac.uk> Message-ID: <7fb8f82f0908191004k7e330991xda5bbecea6f95116@mail.gmail.com> (*>) and (<*) could be used to apply recognizing parsers for the discarded half. This makes a huge gain for uu-parsinglib. uu-parsinglib's P_m monad could be extended just as it has done with P_f and P_h to also wrap its existing R monad, which would let it apply the parser as a recognizer efficiently. And for parsimony it allows me to treat that side of the alternative grammar as a right seminearring ignoring the argument, this increases sharing opportunities for my grammar fragments, because pure nodes in recognizers can be treated as epsilons in the grammar and safely elided. -Edward Kmett On Wed, Aug 19, 2009 at 11:40 AM, Ross Paterson wrote: > On Mon, Jun 29, 2009 at 01:45:45PM +0100, Ross Paterson wrote: > > The proposal is that the following functions > > > > (<$) :: Functor f => a -> f b -> f a > > (*>) :: Applicative f => f a -> f b -> f b > > (<*) :: Applicative f => f a -> f b -> f a > > some :: Alternative f => f a -> f [a] > > many :: Alternative f => f a -> f [a] > > > > are moved into the corresponding classes, with the existing > implementations > > as default definitions. > > Henning asked for concrete examples (i.e. code) where this would give > a substantial performance win while still defining the same function, > including termination properties. (But it seems reasonable not to rely > on RULES to improve the asymptotic complexity.) > > I gave an example for (<$). Does anyone have concrete examples for > the others? > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090819/ce52243b/attachment.html From ross at soi.city.ac.uk Wed Aug 19 16:21:25 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Wed Aug 19 16:01:31 2009 Subject: proposal #3335: make some Applicative functions into methods, and split off Data.Functor In-Reply-To: <7fb8f82f0908191004k7e330991xda5bbecea6f95116@mail.gmail.com> References: <20090629124545.GA6378@soi.city.ac.uk> <20090819154020.GB7760@soi.city.ac.uk> <7fb8f82f0908191004k7e330991xda5bbecea6f95116@mail.gmail.com> Message-ID: <20090819202125.GA9785@soi.city.ac.uk> On Wed, Aug 19, 2009 at 01:04:13PM -0400, Edward Kmett wrote: > (*>) and (<*) could be used to apply recognizing parsers for the > discarded half. This makes a huge gain for uu-parsinglib. > uu-parsinglib's P_m monad could be extended just as it has done with > P_f and P_h to also wrap its existing R monad, which would let it > apply the parser as a recognizer efficiently. > > And for parsimony it allows me to treat that side of the alternative > grammar as a right seminearring ignoring the argument, this increases > sharing opportunities for my grammar fragments, because pure nodes in > recognizers can be treated as epsilons in the grammar and safely elided. code? From ekmett at gmail.com Wed Aug 19 17:00:22 2009 From: ekmett at gmail.com (Edward Kmett) Date: Wed Aug 19 16:40:25 2009 Subject: proposal #3335: make some Applicative functions into methods, and split off Data.Functor In-Reply-To: <20090819202125.GA9785@soi.city.ac.uk> References: <20090629124545.GA6378@soi.city.ac.uk> <20090819154020.GB7760@soi.city.ac.uk> <7fb8f82f0908191004k7e330991xda5bbecea6f95116@mail.gmail.com> <20090819202125.GA9785@soi.city.ac.uk> Message-ID: <7fb8f82f0908191400l6d4c8fbcnfd04de439a82da61@mail.gmail.com> On Wed, Aug 19, 2009 at 4:21 PM, Ross Paterson wrote: > On Wed, Aug 19, 2009 at 01:04:13PM -0400, Edward Kmett wrote: > > (*>) and (<*) could be used to apply recognizing parsers for the > > discarded half. This makes a huge gain for uu-parsinglib. > > uu-parsinglib's P_m monad could be extended just as it has done with > > P_f and P_h to also wrap its existing R monad, which would let it > > apply the parser as a recognizer efficiently. > > > > And for parsimony it allows me to treat that side of the alternative > > grammar as a right seminearring ignoring the argument, this increases > > sharing opportunities for my grammar fragments, because pure nodes in > > recognizers can be treated as epsilons in the grammar and safely elided. > > code? The parsimony case is a bit drastic to post here, because I'd have to basically post the whole library and I haven't released it yet, or rewritten it to accomodate these extra Applicative actions. However, I can try to do justice to how uu-parsinglib could use the new members. It currently has several parsers, which have types i'll abridge here. newtype P_h st a = P_h (forall r . (a -> st -> Steps r) -> st -> Steps r) newtype P_f st a = P_f (forall r . (st -> Steps r) -> st -> Steps (a, r)) newtype R st a = R (forall r . (st -> Steps r) -> st -> Steps r) newtype P_m state a = P_m (P_h state a, P_f state a) It uses an 'ExtApplicative' class to let it mix recognizers (R's) with other parsers when you will just be discarding the recognized branch of the result. Note P_f and R are both Applicative, not Monadic. I'll just handle (<*) to avoid clutter below. class Applicative p => ExtApplicative p where (<<*) :: p a -> R (State p) b -> p a instance ExtApplicative (P_h st) where P_h p <<* R r = P_h ( p. (r.)) instance ExtApplicative (P_f st) where P_f p <<* R r = P_f (\ k st -> p (r k) st) R just discards its phantom type argument. So it is trivially a Functor. instance Functor (R st) where fmap _ (R r) = R r Also note that the ExtApplicative case above could not be defined with P_f rather than R. P_f has to deal with its argument, and isn't able to when you would try to apply it like R above. When used applicatively however... instance Functor (P_f st) where fmap f (P_f p) = P_f (\k inp -> Apply (\(a,r) -> (f a, r)) (p k inp)) This could be made into a more palatable functor by Yoneda encoding some of the Step GADT constructors, to carry around any mappings, but that is irrelevant to this exposition. The P_m monad uses a mechanism for binding history parsers to future parsers, which basically lets the context-free future be glued onto a context-sensitive history. instance Applicative (P_m st) => Monad (P_m st) where P_m (P_h p, _) >>= a2q = P_m ( P_h (\k -> p (\ a -> unP_m_h (a2q a) k)) , P_f (\k -> p (\ a -> unP_m_f (a2q a) k)) ) But the same thing can be done with some modifications to P_m to add a possible recognizer (R) as an end-state. These represent a monadic computation with the final batch of applicative or right seminearring operations that end it separated out. newtype P_m' state a = P_m (P_h state a, P_f state a, R state a) instance Applicative (P_m st) => Monad (P_m st) where P_m' (P_h p, _) >>= a2q = P_m' ( P_h (\k -> p (\ a -> unP_m'_h (a2q a) k)) , P_f (\k -> p (\ a -> unP_m'_f (a2q a) k)) , P_r (\k -> p (\ a -> unP_m'_r (a2q a) k)) ) And then you can drop in special cases for (*>) and (<*) which mirror the existing code for the ExtApplicative operators of the same name in uu-parsinglib. instance Applicative (P_m st) where P_m (hp, fp,rp) <* P_m (_,_,r) = P_m (hp <<* r, fp <<* r, rp <* r) Now, the a parser written with a substantially unmodified uu-parsinglib can efficiently evaluate the side of the computation that is being ignored because any epsilon productions in that side come for free, so all the fiddly little fmapping that goes on in the Applicative is ignored. Doaitse could probably do this better justice than I, as I only have a passing familiarity with the internals of uu-parsinglib. parsimony can derive a similar benefit by accumulating a right seminnearring parser as a grammar-algebra off of the base functor for my grammars and applying that grammar when possible for <*'d fragments in a similar fashion, but as it only deals with context-free attribute grammars, it has a simpler job to do. -Edward Kmett > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090819/1eb5f011/attachment-0001.html From ekmett at gmail.com Wed Aug 19 17:03:39 2009 From: ekmett at gmail.com (Edward Kmett) Date: Wed Aug 19 16:43:42 2009 Subject: proposal #3335: make some Applicative functions into methods, and split off Data.Functor In-Reply-To: <7fb8f82f0908191400l6d4c8fbcnfd04de439a82da61@mail.gmail.com> References: <20090629124545.GA6378@soi.city.ac.uk> <20090819154020.GB7760@soi.city.ac.uk> <7fb8f82f0908191004k7e330991xda5bbecea6f95116@mail.gmail.com> <20090819202125.GA9785@soi.city.ac.uk> <7fb8f82f0908191400l6d4c8fbcnfd04de439a82da61@mail.gmail.com> Message-ID: <7fb8f82f0908191403j1dd9bbfbm11a8cf900f7c2860@mail.gmail.com> Please pretend I sprinkled primes liberally through the last two code fragments. ;) On Wed, Aug 19, 2009 at 5:00 PM, Edward Kmett wrote: > On Wed, Aug 19, 2009 at 4:21 PM, Ross Paterson wrote: > >> On Wed, Aug 19, 2009 at 01:04:13PM -0400, Edward Kmett wrote: >> > (*>) and (<*) could be used to apply recognizing parsers for the >> > discarded half. This makes a huge gain for uu-parsinglib. >> > uu-parsinglib's P_m monad could be extended just as it has done with >> > P_f and P_h to also wrap its existing R monad, which would let it >> > apply the parser as a recognizer efficiently. >> > >> > And for parsimony it allows me to treat that side of the alternative >> > grammar as a right seminearring ignoring the argument, this increases >> > sharing opportunities for my grammar fragments, because pure nodes in >> > recognizers can be treated as epsilons in the grammar and safely elided. >> >> code? > > > The parsimony case is a bit drastic to post here, because I'd have to > basically post the whole library and I haven't released it yet, or rewritten > it to accomodate these extra Applicative actions. > > However, I can try to do justice to how uu-parsinglib could use the new > members. It currently has several parsers, which have types i'll abridge > here. > > newtype P_h st a = P_h (forall r . (a -> st -> Steps r) -> st -> > Steps r) > newtype P_f st a = P_f (forall r . (st -> Steps r) -> st -> Steps (a > , r)) > newtype R st a = R (forall r . (st -> Steps r) -> st -> Steps r) > newtype P_m state a = P_m (P_h state a, P_f state a) > > It uses an 'ExtApplicative' class to let it mix recognizers (R's) with > other parsers when you will just be discarding the recognized branch of the > result. Note P_f and R are both Applicative, not Monadic. > > I'll just handle (<*) to avoid clutter below. > > class Applicative p => ExtApplicative p where > (<<*) :: p a -> R (State p) b -> p a > > instance ExtApplicative (P_h st) where > P_h p <<* R r = P_h ( p. (r.)) > instance ExtApplicative (P_f st) where > P_f p <<* R r = P_f (\ k st -> p (r k) st) > > R just discards its phantom type argument. So it is trivially a Functor. > > instance Functor (R st) where > fmap _ (R r) = R r > > Also note that the ExtApplicative case above could not be defined with P_f > rather than R. P_f has to deal with its argument, and isn't able to when > you would try to apply it like R above. When used applicatively however... > > instance Functor (P_f st) where > fmap f (P_f p) = P_f (\k inp -> Apply (\(a,r) -> (f a, r)) (p k > inp)) > > This could be made into a more palatable functor by Yoneda encoding some of > the Step GADT constructors, to carry around any mappings, but that is > irrelevant to this exposition. > > The P_m monad uses a mechanism for binding history parsers to future > parsers, which basically lets the context-free future be glued onto a > context-sensitive history. > > instance Applicative (P_m st) => Monad (P_m st) where > P_m (P_h p, _) >>= a2q = > P_m ( P_h (\k -> p (\ a -> unP_m_h (a2q a) k)) > , P_f (\k -> p (\ a -> unP_m_f (a2q a) k)) > ) > But the same thing can be done with some modifications to P_m to add a > possible recognizer (R) as an end-state. These represent a monadic > computation with the final batch of applicative or right seminearring > operations that end it separated out. > > newtype P_m' state a = P_m (P_h state a, P_f state a, R state a) > instance Applicative (P_m st) => Monad (P_m st) where > P_m' (P_h p, _) >>= a2q = > P_m' ( P_h (\k -> p (\ a -> unP_m'_h (a2q a) k)) > , P_f (\k -> p (\ a -> unP_m'_f (a2q a) k)) > , P_r (\k -> p (\ a -> unP_m'_r (a2q a) k)) > ) > And then you can drop in special cases for (*>) and (<*) which > mirror the existing code for the ExtApplicative operators of the same name > in uu-parsinglib. > > instance Applicative (P_m st) where > P_m (hp, fp,rp) <* P_m (_,_,r) = P_m (hp <<* r, fp <<* r, rp <* r) > > > Now, the a parser written with a substantially unmodified uu-parsinglib can > efficiently evaluate the side of the computation that is being ignored > because any epsilon productions in that side come for free, so all the > fiddly little fmapping that goes on in the Applicative is ignored. > > Doaitse could probably do this better justice than I, as I only have a > passing familiarity with the internals of uu-parsinglib. > > parsimony can derive a similar benefit by accumulating a right > seminnearring parser as a grammar-algebra off of the base functor for my > grammars and applying that grammar when possible for <*'d fragments in a > similar fashion, but as it only deals with context-free attribute grammars, > it has a simpler job to do. > > -Edward Kmett > > >> _______________________________________________ >> Libraries mailing list >> Libraries@haskell.org >> http://www.haskell.org/mailman/listinfo/libraries >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090819/f02cb40b/attachment.html From wasserman.louis at gmail.com Wed Aug 19 18:22:40 2009 From: wasserman.louis at gmail.com (Louis Wasserman) Date: Wed Aug 19 18:03:01 2009 Subject: Ticket #3271: New methods for Data.Sequence Message-ID: Ross, This new versionshould fix your concerns. Is there anything else? Louis Wasserman wasserman.louis@gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090819/29ba08f7/attachment.html From duncan.coutts at worc.ox.ac.uk Wed Aug 19 22:04:20 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Wed Aug 19 21:43:47 2009 Subject: Recommendation for the procedure to add platform packages Message-ID: <1250733860.9379.2142.camel@localhost> All, The Haskell Platform steering committee have been very busy over the past two weeks drafting, discussing and redrafting. The result is our recommendation to the libraries list for a procedure for adding new packages to the Haskell Platform. Before you all go off and read it, we'd like to say what feedback we are looking for and what the next steps will be. The document we've come up with describes the procedure but only some very basic quality requirements for packages. Our aim is to have the libraries list come to an agreement on the procedure first and then we can discuss codifying a more comprehensive set of package requirements and guidelines. The main document contains: * the procedure itself, which is relatively short * a rationale, cross-linked to the procedure * a procedure to help us make decisions http://trac.haskell.org/haskell-platform/wiki/AddingPackages There is an accompanying "how to" guide to help the people who will actually be making proposals: http://trac.haskell.org/haskell-platform/wiki/AddingPackages/HowTo There is also an example proposal: http://trac.haskell.org/haskell-platform/wiki/Proposals/example So please send in your comments and of course feel free to ask questions and seek clarifications. Our hope is that we'll be able to agree this procedure stuff relatively quickly and then move onto package quality issues and indeed to actually start proposing packages. Let me also say that I've been very pleased with how the steering committee has worked. People have put in a lot of effort. It took 2 weeks, nearly 60 emails and over 100 edits to the draft. We didn't always agree but we talked things through thoroughly and turned our early draft into something much better that we're all satisfied with. Duncan, on behalf of the rest of the steering committee: Iavor Diatchki, Isaac Dupree, Thomas Schilling, Johan Tibell and Adam Wick. From leather at cs.uu.nl Thu Aug 20 02:51:04 2009 From: leather at cs.uu.nl (Sean Leather) Date: Thu Aug 20 02:31:25 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <1250733860.9379.2142.camel@localhost> References: <1250733860.9379.2142.camel@localhost> Message-ID: <3c6288ab0908192351t7d41a848o8b7642dc9c33bf10@mail.gmail.com> > The Haskell Platform steering committee have been very busy over the > past two weeks drafting, discussing and redrafting. > Looks nice! > Our aim is to have the libraries list come to > an agreement on the procedure first and then we can discuss codifying a > more comprehensive set of package requirements and guidelines. Sorry to diverge from this, but I just wanted to note that something to the effect of "following the license policy" should be mentioned in the package requirements. Yes, the "interim license policy" is not far down, but adding this would make it clearer, I think. Regards, Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090820/8631b15f/attachment-0001.html From magnus at therning.org Thu Aug 20 03:56:11 2009 From: magnus at therning.org (Magnus Therning) Date: Thu Aug 20 03:36:12 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <1250733860.9379.2142.camel@localhost> References: <1250733860.9379.2142.camel@localhost> Message-ID: I found this "Compile on all operating systems and compilers that the platform targets." to be a little confusing. HP 2009.2.0.2 for Windows does include Win32 (providing System.Win32). Of course there's not much point in including Win32 in HP for a Unix platform. I assume the same is true for unix (providing System.Posix), but in the opposite direction, so to speak. It would be a shame to not include these two _platform-specific_ packages in HP. OTOH, maybe I'm not using the word "compile" in the correct sense... Would a platform-specific package qualify if building it on _other_ platforms resulted in a successful build, but with empty output? /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus?therning?org Jabber: magnus?therning?org http://therning.org/magnus identi.ca|twitter: magthe From ross at soi.city.ac.uk Thu Aug 20 06:31:54 2009 From: ross at soi.city.ac.uk (Ross Paterson) Date: Thu Aug 20 06:11:56 2009 Subject: Ticket #3271: New methods for Data.Sequence In-Reply-To: References: Message-ID: <20090820103154.GA5561@soi.city.ac.uk> Fine by me. I'll check over the haddock docs and apply the patch. From seanmcl at gmail.com Thu Aug 20 09:12:02 2009 From: seanmcl at gmail.com (Sean McLaughlin) Date: Thu Aug 20 08:52:01 2009 Subject: Cabal: literate hscolour Message-ID: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> Hello, I can't find a way to pass the -lit flag to hscoulour using neither my cabal configuration file nor a command line option to runhaskell Setup.hs haddock. I'd really like to not have my literate comments be colored according to Haskell lexical conventions. Is there a way to do this? Thanks! Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090820/d73d0560/attachment.html From duncan.coutts at worc.ox.ac.uk Thu Aug 20 09:12:40 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 20 08:52:05 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <3c6288ab0908192351t7d41a848o8b7642dc9c33bf10@mail.gmail.com> References: <1250733860.9379.2142.camel@localhost> <3c6288ab0908192351t7d41a848o8b7642dc9c33bf10@mail.gmail.com> Message-ID: <1250773960.9379.2831.camel@localhost> On Thu, 2009-08-20 at 08:51 +0200, Sean Leather wrote: > > The Haskell Platform steering committee have been very busy > over the > past two weeks drafting, discussing and redrafting. > > Looks nice! Thanks. > Our aim is to have the libraries list come to > an agreement on the procedure first and then we can discuss > codifying a > more comprehensive set of package requirements and guidelines. > > Sorry to diverge from this, but I just wanted to note that something > to the effect of "following the license policy" should be mentioned in > the package requirements. Yes, the "interim license policy" is not far > down, but adding this would make it clearer, I think. Technically the "interim license policy" is a sub-section of the "package requirements" section, not that this is terribly clear from the size of fonts used. So point taken. Lets clarify this when we get a proper license policy, or if we need to expand and reorganise the package requirements section in general. Duncan From duncan.coutts at worc.ox.ac.uk Thu Aug 20 09:12:44 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 20 08:52:09 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: References: <1250733860.9379.2142.camel@localhost> Message-ID: <1250773964.9379.2832.camel@localhost> On Thu, 2009-08-20 at 08:56 +0100, Magnus Therning wrote: > I found this > > "Compile on all operating systems and compilers that the platform targets." > > to be a little confusing. HP 2009.2.0.2 for Windows does include > Win32 (providing System.Win32). Of course there's not much point in > including Win32 in HP for a Unix platform. I assume the same is true > for unix (providing System.Posix), but in the opposite direction, so > to speak. It would be a shame to not include these two > _platform-specific_ packages in HP. You're right that we somehow need to allow these exceptions but without encouraging non-portable packages in general. > OTOH, maybe I'm not using the word "compile" in the correct sense... > Would a platform-specific package qualify if building it on _other_ > platforms resulted in a successful build, but with empty output? I think "build and work" would be a better expression of the intention. Duncan From duncan.coutts at worc.ox.ac.uk Thu Aug 20 12:49:41 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Thu Aug 20 13:25:07 2009 Subject: Cabal: literate hscolour In-Reply-To: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> Message-ID: <1250786981.9379.3191.camel@localhost> On Thu, 2009-08-20 at 09:12 -0400, Sean McLaughlin wrote: > Hello, > > > I can't find a way to pass the -lit flag to hscoulour using neither > my cabal configuration file nor a command > line option to runhaskell Setup.hs haddock. I'd really like to not > have my literate comments be colored according > to Haskell lexical conventions. Is there a way to do this? So it's fine for us to call hscolour differently for .lhs vs .hs files. The annoying thing here is that to "do the right thing" Cabal would need to distinguish bird-track style literate files (hscolour -lit) from tex style (hscolour -lit-tex). I don't think this is reasonable. The hscolour program is the one doing the lexing so it is in a position to work out which literate style it is using, or indeed if it's using a mixture. So I suggest you file a ticket against hscolour asking for -lit and -lit-tex to be merged. It might be worth asking if hscolour should by default just do the right thing, ie use -lit on .lhs input files. If not then file a separate ticket for Cabal to use -lit on .lhs files. In particular I'm not suggesting any configuration knob in Cabal for this. We should just do the right thing. Seem like a reasonable plan? Duncan From igloo at earth.li Thu Aug 20 21:23:55 2009 From: igloo at earth.li (Ian Lynagh) Date: Thu Aug 20 21:03:55 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <1250733860.9379.2142.camel@localhost> References: <1250733860.9379.2142.camel@localhost> Message-ID: <20090821012355.GA24225@matrix.chaos.earth.li> On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote: > > There is also an example proposal: > > http://trac.haskell.org/haskell-platform/wiki/Proposals/example > > So please send in your comments and of course feel free to ask questions I read the example first, so I'll give my comments on it first: My first impression is that there is an awful lot of text, and that it's mostly in paragraph form (i.e. I need to actually read it, rather than the key points being highlighted). I think this will reduce the level of review that proposals get. This is a proposal for the 'tar' package to be included in the next major release of the Haskell platform. Should specify version number. Review comments should be sent to the libraries mailing list by ${deadline - 4 weeks} This sounds like it is the wrong way round to me. The deadline for review comments should be "${now + 2 weeks}" (for some value of 2, and larger values for complicated/large proposals). If you make the proposal early, then it can be agreed earlier. If you make it too late for the discussion and consensus to happen by the deadline then it can wait until the next release. (that's pretty much how library submission proposals work). Abstract I think this should be replaced with a link to the hackage page, which shows the abstract from the Cabal file. You might argue that having all the info in one place is convenient for reviewers, but * I am unconvinced that it is much easier * Packages that are proposed are probably widely used anyway, so this is just noise to a significant proportion of people * by only linking to it you remove the temptation to alter the text in the proposal but not apply the improvments to the Cabal file. Rationale I don't think the actual rationale part of this section is useful. It mostly says "The foo library is useful because some people want to foo". If during the discussion we find that the rationale is unclear then a rationale could be added to the proposal, but mostly I expect it will be self-evident (and if not then the abstract in the Cabal file probably needs some work). The list of users is useful (less convinced by the list of packages you think /ought/ to use it), but a bullet-pointed list would be much easier to read. Introduction to the API What I said about the abstract applies here, only moreso. If an intro to the API is useful, then one should already exist. People should not be given the opportunity to /write/ an introduction to the API here; it should already be in the haddock docs on hackage, or on the project page on community, or in some similar location. Therefore this too should just be a link. Again, this will remove the temptation to update the proposal but not the real docs following the discussion. It will also mean people don't waste time reformatting docs in wiki markup. Design decisions Ditto. Open issues This is the interesting bit of the proposal, IMO. What I want to see for a proposal is just something like: Package: mylib 1.2 Is useful? [green] [link to below] Yes[/link] [/green] Has intro to API? [green] [link to hackage]Yes[/link] [/green] Has design decision doc? [yellow] No [/yellow] Builds on all platforms? [green] Yes [/green] Builds with all impls? [yellow] No [/yellow] Good API? [green] No objections[/green] Is useful ("Is useful?" above links to here) --------- Users include: * darcs * tar Has design decision ------------------- The package is too simple for such a doc to be useful. [...] Builds with all impls --------------------- hugs is missing the WibbleTheThingy extension, so cannot build the package. But H2010 will include WibbleTheThingy, so hugs is expected to implement it soon. and the goal of the discussion would then be to get a consensus on whether the yellow sections are severe enough to not accept the package (and also to give people a chance to disagree that some of the green entries really are green, e.g. "Is useful?" is subjective). I've mentioned this link on IRC before, but for those who haven't seen it, this is partly inspired by: http://release.debian.org/etch/arch_qualify.html (which shows at a glance which arches are acceptable to be in a Debian release, and where the issues and potential issues are. The link in the first row gives more detail on some of the points). > > http://trac.haskell.org/haskell-platform/wiki/AddingPackages > I've only skimmed this, due to the length, but my impression is that it is too heavy-weight. e.g. http://www.haskell.org/haskellwiki/Library_submissions has this to say about consensus: If someone has done all this, they are entitled to expect feedback; silence during the discussion period can be interpreted as consent. If consensus was achieved, [...] and doesn't seem to have had problems with getting consensus, or agreeing whether we have got it. The "Achieving consensus" section of AddingPackages is 947 words according to wc (that's about double the length of the entire Library_submissions page). Also, the fact a credits section was deemed necessary suggests to me that too much work is involved. Thanks Ian From Malcolm.Wallace at cs.york.ac.uk Fri Aug 21 06:24:49 2009 From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace) Date: Fri Aug 21 06:04:48 2009 Subject: Cabal: literate hscolour In-Reply-To: <1250786981.9379.3191.camel@localhost> References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> <1250786981.9379.3191.camel@localhost> Message-ID: <4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk> > The annoying thing here is that to "do the right thing" Cabal would > need > to distinguish bird-track style literate files (hscolour -lit) from > tex > style (hscolour -lit-tex). I don't think this is reasonable. I've fixed hscolour - from the newly release version 1.14 there is now no difference between the -lit and -lit-tex options. Both will deal with either style of literate input. > It might be worth asking if hscolour should by > default just do the right thing, ie use -lit on .lhs input files. Hscolour-1.14 also makes this fix. If your source file name ends in .lhs or .ly or .lx, it will be automatically processed with -lit. You can override the guess, by using an explicit -nolit (or -lit) to force the particular behaviour you want. Another new change, is that you can now supply multiple filenames as input, and their colourised contents will be concatenated to the output. Regards, Malcolm From seanmcl at gmail.com Fri Aug 21 09:12:16 2009 From: seanmcl at gmail.com (Sean McLaughlin) Date: Fri Aug 21 08:52:14 2009 Subject: Cabal: literate hscolour In-Reply-To: <4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk> References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> <1250786981.9379.3191.camel@localhost> <4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk> Message-ID: <6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com> Hi Malcolm, Thanks for the quick fix! I installed the newest version. There are a couple of problems for me. The first is that there is no color. My old files processed with hscolour 1.13 had headers like src/Imogen/Infon/Fol/Subst.lhs The files now don't have this header, and they are not showing up colored. Second, I'm probably being a pain in the ass, but I'd also like to keep my literate comments preformatted. Right now they are mashed all together and are unreadable. Would it be possible to add an option for preformatting, in case you prefer the default to be unformatted? Thanks! Sean On Fri, Aug 21, 2009 at 6:24 AM, Malcolm Wallace < Malcolm.Wallace@cs.york.ac.uk> wrote: > The annoying thing here is that to "do the right thing" Cabal would need >> to distinguish bird-track style literate files (hscolour -lit) from tex >> style (hscolour -lit-tex). I don't think this is reasonable. >> > > I've fixed hscolour - from the newly release version 1.14 there is now no > difference between the -lit and -lit-tex options. Both will deal with > either style of literate input. > > It might be worth asking if hscolour should by >> default just do the right thing, ie use -lit on .lhs input files. >> > > Hscolour-1.14 also makes this fix. If your source file name ends in .lhs > or .ly or .lx, it will be automatically processed with -lit. You can > override the guess, by using an explicit -nolit (or -lit) to force the > particular behaviour you want. > > Another new change, is that you can now supply multiple filenames as input, > and their colourised contents will be concatenated to the output. > > Regards, > Malcolm > > _______________________________________________ > Libraries mailing list > Libraries@haskell.org > http://www.haskell.org/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.haskell.org/pipermail/libraries/attachments/20090821/b7394bca/attachment-0001.html From duncan.coutts at worc.ox.ac.uk Fri Aug 21 14:56:01 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 21 14:35:26 2009 Subject: Call for consensus: Question of HP updates that depend on new packages In-Reply-To: <1250514630.5255.197.camel@localhost> References: <1250514630.5255.197.camel@localhost> Message-ID: <1250880961.30910.1293.camel@localhost> On Mon, 2009-08-17 at 14:10 +0100, Duncan Coutts wrote: > In particular the libraries list should decide if these new packages > should be rubber stamped because they are new dependencies of a package > already in the platform, or if the new packages should go through the > standard review process for adding packages. Given the responses it seems that we think that new packages should go through the standard review process for adding packages. In particular "where the functionality exposed is not logically part of the original package" since there is an exception for straightforward package splits. I suggest the dividing line between package splits and genuinely new dependencies be left to the judgement of the release team. So maintainers who would like advice can talk to the release team. Are there any unresolved concerns? If not then we'll go with this as the policy. Duncan From duncan.coutts at worc.ox.ac.uk Fri Aug 21 15:58:56 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Fri Aug 21 15:38:17 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <20090821012355.GA24225@matrix.chaos.earth.li> References: <1250733860.9379.2142.camel@localhost> <20090821012355.GA24225@matrix.chaos.earth.li> Message-ID: <1250884736.30910.1484.camel@localhost> On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote: > On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote: > > > > There is also an example proposal: > > > > http://trac.haskell.org/haskell-platform/wiki/Proposals/example > > > > So please send in your comments and of course feel free to ask questions > > I read the example first, so I'll give my comments on it first: The policy is significantly less prescriptive than the example might lead you to believe. The essential requirement is given as: The proposal forms the nub of the argument for why the package should be included in the Haskell Platform. It should contain or link to enough information so that reviewers can make an informed decision on whether the package should be accepted. With further specific details given in the "Proposal content" section: http://trac.haskell.org/haskell-platform/wiki/AddingPackages#Proposalcontent I should also note that I'm happy to improve my tar proposal to pick up some of the points you make. In addition to being an example proposal I intend to submit it as a real proposal, so improvements benefit both the real proposal and the example. > My first impression is that there is an awful lot of text, and that it's > mostly in paragraph form (i.e. I need to actually read it, rather than > the key points being highlighted). I think this will reduce the level of > review that proposals get. Below you suggest that instead the proposal should link to existing documentation (and that such documentation be required to exist). I presume the documentation linked to would have the same amount of text. Assuming that's what you mean, I'm not sure what you're comparing to that has less text that we might expect would give us an increased level of review. > This is a proposal for the 'tar' package to be included in the next > major release of the Haskell platform. > > Should specify version number. I expect that usually the exact number is not especially relevant. The exact released version now is not necessarily the same as the final one that gets included in the case that reviewers make suggestions. If there's more than one active branch then yes it's relevant so reviewers know which one to look at. Otherwise reviewers can reasonably assume the version on the hackage page. The guidance notes also mention that if reviewers are expected to have to look at a development version, that the repo should be given in the proposal. > Review comments should be sent to the libraries mailing list by > ${deadline - 4 weeks} > > This sounds like it is the wrong way round to me. The deadline for > review comments should be "${now + 2 weeks}" (for some value of 2, and > larger values for complicated/large proposals). If you make the proposal > early, then it can be agreed earlier. If you make it too late for the > discussion and consensus to happen by the deadline then it can wait > until the next release. Suggesting that comments be made by a time relative to the proposal time rather than the final deadline would also be consistent with the policy. The constraints are that people be given a reasonable amount of time to review things and that discussions and issues are resolved before the final deadline. I'd be happy to change the guidance to suggest now + time rather than final deadline - time. > (that's pretty much how library submission proposals work). Sure. Though I think we need to clarify how much time is reasonable for review rather than letting the proposers pick whatever they like. > Abstract > > I think this should be replaced with a link to the hackage page, which > shows the abstract from the Cabal file. You might argue that having all > the info in one place is convenient for reviewers, but > * I am unconvinced that it is much easier > * Packages that are proposed are probably widely used anyway, so this is > just noise to a significant proportion of people > * by only linking to it you remove the temptation to alter the text in > the proposal but not apply the improvments to the Cabal file. The policy allows the info to be inline in the proposal or linked to. It's up to the proposal author: Where appropriate, existing sources of information about the package may be copied or linked to. > Rationale > > I don't think the actual rationale part of this section is useful. It > mostly says "The foo library is useful because some people want to foo". > If during the discussion we find that the rationale is unclear then a > rationale could be added to the proposal, but mostly I expect it will be > self-evident (and if not then the abstract in the Cabal file probably > needs some work). It's intended to cover these proposal requirements: The proposal forms the nub of the argument for why the package should be included in the Haskell Platform. It should explain the gap in the existing set of platform packages that the new package addresses. Alternatively if the package would duplicate or replace any existing platform packages then the reasons why the new package is better should be clearly explained. In this case the proposal should be tied to another proposal to deprecate the existing package(s) (or parts thereof) and thought should be given to how the transition should be managed. [note-7.2] The guidance notes say: The rationale is the core of your argument as to why this package should be added to the platform. Your task here is to convince people. What makes the package useful? What are the use cases? Being able to point to existing uses is always good. Similarly, existing comments or reviews from users can be helpful. Old release announcements can be a good source of inspiration here. If your package is a replacement for an existing package (or part of one) then you'll want to convince people why the new one is an improvement. I agree that what I've written for the rationale for the tar proposal is of somewhat dubious utility. In fact personally I don't like the term "rationale" for the section. > The list of users is useful (less convinced by the list of packages you > think /ought/ to use it), but a bullet-pointed list would be much easier > to read. Well I'm convinced darcs ought to use it ;-). I'm citing it there as an example of an alternative to using the tar package, to use an external tar program (and the various disadvantages that has). In general I don't think it's an unreasonable argument to point out cases where existing packages could be improved by making use of the new package rather than using some existing home-grown solution. Of course this is all a judgement for the proposal author for what they think makes a convincing argument (and for the reviewers to decide if they're convinced). I'll keep it in mind that you don't find it convincing :-). > Introduction to the API > > What I said about the abstract applies here, only moreso. If an intro to > the API is useful, then one should already exist. It's explicitly ok to link to one where one already exists. The policy and guidance notes say as much. > People should not be given the opportunity to /write/ an introduction > to the API here; it should already be in the haddock docs on hackage, > or on the project page on community, or in some similar location. If we add that as a package requirement then it would be reasonable to change the advice to say simply to link to it. Currently the only doc requirement we've put in is: Library packages should have Haddock API documentation for all exported functions and types. I expect once we've agreed the procedure that we'll get onto more discussing more detailed requirements on things like documentation. In the tar example I had already made an introductory section to the API in the haddock docs. However on trying to present it again in the proposal (by doing a fair bit of copy and pasting) I think I made some improvements to how and the order in which concepts are presented. Writing with a particular audience in mind can help a lot. > Therefore this too should just be a link. Again, this will remove the > temptation to update the proposal but not the real docs following the > discussion. It will also mean people don't waste time reformatting > docs in wiki markup. The guidance notes say: Of course, if you end up making something new for the proposal then it'd be great to keep it somewhere so that new users can benefit from it too. The obvious place is in the Haddock docs, or a project website. > Design decisions > > Ditto. The same points apply. If we want to require that packages must have this kind of documentation then it ought to be spelled out explicitly as a package requirement. Currently it's just part of the proposal because it's a less onerous requirement. > Open issues > > This is the interesting bit of the proposal, IMO. What I want to see for > a proposal is just something like: That's a quite different kind of thing from what the open issues section is intended for. The policy says: Open issues and objections raised by reviewers should be tracked on the proposal wiki under a separate section. [note-3.2] and An explicit checklist of the package requirements below is not required. The proposal should state however that all the requirements are met, or for any requirements that are not met, a reason why they are not met. [note-7.6] What you're asking for is a checklist. That idea was particularly controversial within the steering committee and it was not adopted. Personally I was weakly in favour if a checklist but the concerns were that it restricted the arguments and explanations that the proposal author could make. The consensus was for a much more free-form proposal rather than forcing things into a checklist. > and the goal of the discussion would then be to get a consensus on > whether the yellow sections are severe enough to not accept the package > (and also to give people a chance to disagree that some of the green > entries really are green, e.g. "Is useful?" is subjective). I think that would come under the complaint that that is too narrow for the argument for why the package should be included. Perhaps other members of the steering committee would like to comment on this point. On the other hand if it's not part of the proposal then I think you'd get a better reception. I know that the release team will want to use its own status board for the aspects of packages that particularly affect the release team (like building on various arches). > I've mentioned this link on IRC before, but for those who haven't seen > it, this is partly inspired by: > http://release.debian.org/etch/arch_qualify.html > (which shows at a glance which arches are acceptable to be in a Debian > release, and where the issues and potential issues are. The link in the > first row gives more detail on some of the points). My concern if this is the main focus of discussions is that it covers the "does it work" aspect reasonably well, it doesn't cover the "is it any good? how could it be better?" aspects. > > > > http://trac.haskell.org/haskell-platform/wiki/AddingPackages > > > > I've only skimmed this, due to the length, but my impression is that it > is too heavy-weight. e.g. > http://www.haskell.org/haskellwiki/Library_submissions > has this to say about consensus: > > If someone has done all this, they are entitled to expect feedback; > silence during the discussion period can be interpreted as consent. > > If consensus was achieved, [...] > > and doesn't seem to have had problems with getting consensus, or > agreeing whether we have got it. That's the main reason we agreed to go with a consensus method. The policy states: The standard of consensus required is the same as that defined for the existing library submissions process. [note-5.4] Our early draft simply said that and nothing else. Several members of the steering committee and others who read drafts were unsure or unconvinced that consensus would be a usable method. We agreed to try it and see if it works. In addition we added the option to use this consensus protocol. > The "Achieving consensus" section of > AddingPackages is 947 words according to wc (that's about double the > length of the entire Library_submissions page). You will notice that in the simple case there is essentially no formality at all: The first stage is presenting the proposal followed by open group discussion. This stage lasts long enough to give everyone the opportunity to send in their review comments. There follows a call for consensus. This is essentially the same as the libraries process (including the bit about expecting feedback and silence is consent). We only need to go further when there are remaining disagreements. So I do not agree that it is too heavy-weight. It starts out very light and gets more structured as the severity of the disagreement increases and as the final deadline approaches. > Also, the fact a credits section was deemed necessary suggests to me > that too much work is involved. It was agreed because it's a cheap and easy way to promote the work of review which is often undervalued. We also think that serious review is a great way to improve package quality and entry to the platform is an appropriate moment to do it. There is clearly a trade off between: * work for package proposers / maintainers * work for reviewers * quality of package review and quality of decisions My impression so far is that opinions are reasonably evenly split on all three. To me, that indicates we're roughly in the right place. What are you suggesting specifically that would reduce the amount of work for reviewers? Would those things increase work for the proposal authors / package maintainers? Would those things increase or decrease the level of depth/quality of review and decisions? Of course if there are any suggestions for straightforward wins in this trade-off then we should consider them seriously. Duncan From ml at isaac.cedarswampstudios.org Fri Aug 21 16:12:52 2009 From: ml at isaac.cedarswampstudios.org (Isaac Dupree) Date: Fri Aug 21 15:52:56 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <1250884736.30910.1484.camel@localhost> References: <1250733860.9379.2142.camel@localhost> <20090821012355.GA24225@matrix.chaos.earth.li> <1250884736.30910.1484.camel@localhost> Message-ID: <4A8EFFC4.2030002@isaac.cedarswampstudios.org> >> Also, the fact a credits section was deemed necessary suggests to me >> that too much work is involved. > > It was agreed because it's a cheap and easy way to promote the work of > review which is often undervalued. Exactly! Probably the main motivation for package authors is not that they get credited... but, nevertheless, they get credited. It's helpful (even, generally, as a matter of programmer/open-source culture) to try and make equally important work be equally glamorous. -Isaac From igloo at earth.li Fri Aug 21 20:17:35 2009 From: igloo at earth.li (Ian Lynagh) Date: Fri Aug 21 19:57:33 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <1250884736.30910.1484.camel@localhost> References: <1250733860.9379.2142.camel@localhost> <20090821012355.GA24225@matrix.chaos.earth.li> <1250884736.30910.1484.camel@localhost> Message-ID: <20090822001735.GA17487@matrix.chaos.earth.li> On Fri, Aug 21, 2009 at 08:58:56PM +0100, Duncan Coutts wrote: > On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote: > > On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote: > > > > > > There is also an example proposal: > > > > > > http://trac.haskell.org/haskell-platform/wiki/Proposals/example > > > > > > So please send in your comments and of course feel free to ask questions > > > > I read the example first, so I'll give my comments on it first: > > The policy is significantly less prescriptive than the example might > lead you to believe. Looking again at AddingPackages, I think you are right. That said, if I were making a proposal, I would probably base it on an example (or a template, which is essentially the same thing). The AddingPackages page confuses me. If I need to read all the notes, then the page seems very hard to read, and if I don't then could the rationale be put on a separate page? Also, the [note-1.3]s could all be [rationale] or [rationale-1.3], right? That way I wouldn't think that I have to read them unless I am actually interested in the rationale. Currently the page looks daunting to me. > I should also note that I'm happy to improve my tar proposal to pick up > some of the points you make. In addition to being an example proposal I > intend to submit it as a real proposal, so improvements benefit both the > real proposal and the example. I've made an alternative proposal for the tar package here: http://trac.haskell.org/haskell-platform/wiki/Proposals/alternate_example I've made up some of the answers, and also just taken most of the suggested requirements to be the package requirement list. (That the trac colour scheme means that some of the hyperlinks are hard to identify, but we can sort that out somehow). I would be interested to hear people's opinions on how they compare. > > This is a proposal for the 'tar' package to be included in the next > > major release of the Haskell platform. > > > > Should specify version number. > > I expect that usually the exact number is not especially relevant. The > exact released version now is not necessarily the same as the final one > that gets included in the case that reviewers make suggestions. > > If there's more than one active branch then yes it's relevant so > reviewers know which one to look at. Otherwise reviewers can reasonably > assume the version on the hackage page. I don't see the harm in giving the intended version number in the proposal. It may be that 1.18 is originall proposed and 1.20 ends up being what goes into the next release, if the changes are either following the decisions made during the discussion, or small enough that the release team would happily automatically upgrade 1.18 to 1.20 between two major releases. > The policy allows the info to be inline in the proposal or linked to. Right, I saw that, but I think it should be /required/ to be a link. > > Introduction to the API > > > > People should not be given the opportunity to /write/ an introduction > > to the API here; it should already be in the haddock docs on hackage, > > or on the project page on community, or in some similar location. > > If we add that as a package requirement then it would be reasonable to > change the advice to say simply to link to it. Currently the only doc > requirement we've put in is: > > Library packages should have Haddock API documentation for all > exported functions and types. I didn't mean to give an opinion on whether or not an intro to the API should be required or not, but: * if it is required then I think it should be linked to * if it isn't then I certainly don't think the proposal should have to include one (because then there would be no reason not to require that it exists). Based on our earlier IRC discussion: So is the API intro in the haddock docs? yes OK, ta that's where I copied most of the stuff in the proposal from but re-edited, hopefully to be better I also left out detail to make it a shorter and easier intro it sounds to me like this is an example of exactly what I wanted to avoid. You've spent some time making a nice friendly introduction to the tar API, but in 6 months time when someone comes to use the tar package they won't find this documentation, as it only exists in a "package addition" proposal buried on the haskell-platform wiki. If you're going to go to all the effort of making an API intro, you ought to put it on the package's homepage. > I expect once we've agreed the procedure that we'll get onto more > discussing more detailed requirements on things like documentation. Drifting off-topic, but I think that the two discussions are orthogonal (i.e. we could have them in either order), but the "detailed requirements" discussion is the more important one. People can make proposals without a process being nailed down (and that may even help us work out what the process should be), but how can we decide if a package should be included if we don't know what the acceptance criteria are? So I think the other discussion is much closer on the critical path. > > Therefore this too should just be a link. Again, this will remove the > > temptation to update the proposal but not the real docs following the > > discussion. It will also mean people don't waste time reformatting > > docs in wiki markup. > > The guidance notes say: > > Of course, if you end up making something new for the proposal > then it'd be great to keep it somewhere so that new users can > benefit from it too. The obvious place is in the Haddock docs, > or a project website. Yes, and people will honestly intend to merge the changes and new docs back while they are writing them. But 90% of the time they won't. > > Open issues > > > > This is the interesting bit of the proposal, IMO. What I want to see for > > a proposal is just something like: > > That's a quite different kind of thing from what the open issues section > is intended for. The policy says: > > Open issues and objections raised by reviewers should be tracked > on the proposal wiki under a separate section. [note-3.2] > > and > > An explicit checklist of the package requirements below is not > required. > > What you're asking for is a checklist. Yes indeed. As in my example: http://trac.haskell.org/haskell-platform/wiki/Proposals/alternate_example > That idea was particularly > controversial within the steering committee and it was not adopted. > Personally I was weakly in favour if a checklist but the concerns were > that it restricted the arguments and explanations that the proposal > author could make. The consensus was for a much more free-form proposal > rather than forcing things into a checklist. For items that benefit from free-form text, you can link to a section beyond the checklist, as in my example. Where there would no benefit, a simple "Yes" in the table suffices, and doesn't waste any reviewer time. > > and the goal of the discussion would then be to get a consensus on > > whether the yellow sections are severe enough to not accept the package > > (and also to give people a chance to disagree that some of the green > > entries really are green, e.g. "Is useful?" is subjective). > > I think that would come under the complaint that that is too narrow for > the argument for why the package should be included. I don't understand what you mean by that. > > I've mentioned this link on IRC before, but for those who haven't seen > > it, this is partly inspired by: > > http://release.debian.org/etch/arch_qualify.html > > (which shows at a glance which arches are acceptable to be in a Debian > > release, and where the issues and potential issues are. The link in the > > first row gives more detail on some of the points). > > My concern if this is the main focus of discussions is that it covers > the "does it work" aspect reasonably well, it doesn't cover the "is it > any good? how could it be better?" aspects. You can have an "Any API issues?" question, which if the answer is not "No" will link to one or more free-form text sections. But if the answer is "No" then it's clear at a glance that it is no. Much like the "Other issues" item in my example. > The standard of consensus required is the same as that defined > for the existing library submissions process. [note-5.4] > > Our early draft simply said that and nothing else. Several members of > the steering committee and others who read drafts were unsure or > unconvinced that consensus would be a usable method. We agreed to try it > and see if it works. In addition we added the option to use this > consensus protocol. > > You will notice that in the simple case there is essentially no > formality at all: > > The first stage is presenting the proposal followed by open > group discussion. This stage lasts long enough to give everyone > the opportunity to send in their review comments. There follows > a call for consensus. In fact, I think that's the main problem I have with both the process document and the example proposal: They try to anticipate all questions that might be asked, and answer them. But the common case is that the question wouldn't be asked, so this is just more stuff for someone to write, and and more stuff for many people to read. That's why here: > I agree that what I've written for the rationale for the tar proposal is > of somewhat dubious utility. In fact personally I don't like the term you've had to write stuff "of somewhat dubious utility": because you're answering a boilerplate question that didn't need asking in this case. And in the case of consensus, I don't think we need to lay down rules now for what happens if the current system doesn't work. For one thing, it hasn't been an issue so far. For another, when we have the first case of it not working, it might be clear that the solution would be to do something else anyway. > What are you suggesting specifically that would reduce the amount of > work for reviewers? I am suggesting that there be less text written. > Would those things increase work for the proposal > authors / package maintainers? It should reduce work for the authors / package maintainers. > Would those things increase or decrease > the level of depth/quality of review and decisions? I believe it would increase the number of reviewers, giving better decisions. Thanks Ian From thomas.dubuisson at gmail.com Sat Aug 22 00:49:58 2009 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Sat Aug 22 00:29:53 2009 Subject: Potential Network SIG Message-ID: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> Hello All, If you are CCed it's because you are listed as a maintainer of a network-* package that I consider related to the Haskell network library. I'm hoping to roll much of the functionality of network-{bytestring, multicast, fancy} etc into a single package that the community will agree on (namely, "network"). Johan suggested starting a SIG to hammer out a design for a new Network API seeing as the current API, a straight-forward Berkeley binding, doesn't seem to please anyone in a Haskell context. If you want to partake then this e-mail if your heads up. If there is some formal method of setting up a Haskell SIG then please let me know. My thoughts on some important parts are below - I'm sure not everyone will agree as these thoughts directly contradict some designs found in current libraries. 1) Separate low level functions / bindings from high level / productive code by placing each in different modules. The low level bindings should remain available for those cases we fail to have the needed functionality in our high level packages. That said, I'm hoping to cover more than the 80% of users with any new design. 2) Maintain type safety by using type classes for most things. Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as constructors of the same data type), I think we should allow for the possibility that some users of the library will be limited to just one IP version without resorting to partial functions. I suggest type classes to cover this aspect (class Address, class Port, etc). 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency. As in network-bytestring, any new API should be performance concious enough to avoid String. 4) Support more features Features such as Multicast, Header inclusion (IP_HDRINCL), address binding, etc. IOW, most the IP_ and SO_ options of socket (7) and ip (7) man pages. It would be rather nice if we were able to expose these in a friendly way - but with our cross platform concerns that might not be a good idea (e.g. I'm not familiar with windows). 5) Separate IO-less data declarations from IO and any platform dependent code in different packages. I've already got some work in this area via network-data. Not claiming its current design will stand the test of time, its just an example of keeping the data structures separate from any IO operations that will need platform specific work. 6) Integrate with the rest of hackage. This means instance of PrettyClass, Parsec, and Binary. I noticed a number of parsing utilities for IP addresses - lots of duplicated effort here. I'm currently looking at how the network library is being used - particularly when Network.Socket is invoked. So I guess I'll go sort through some of the source code for scurry, happstack, and adhoc-network. Thomas From mail at joachim-breitner.de Sat Aug 22 04:18:35 2009 From: mail at joachim-breitner.de (Joachim Breitner) Date: Sat Aug 22 03:58:35 2009 Subject: darcs patch: Add justIf to Data.Maybe Message-ID: Hi, I propose the attached change, with a discussion timeframe until end of September. The corresponding ticket is http://hackage.haskell.org/trac/ghc/ticket/3446 Thanks, Joachim Fri Aug 21 19:17:12 CEST 2009 Joachim Breitner * Add justIf to Data.Maybe -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/x-darcs-patch Size: 9698 bytes Desc: A darcs patch for your repository! Url : http://www.haskell.org/pipermail/libraries/attachments/20090822/8f7c74a1/attachment.bin From jon.fairbairn at cl.cam.ac.uk Sat Aug 22 05:13:12 2009 From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn) Date: Sat Aug 22 04:53:27 2009 Subject: darcs patch: Add justIf to Data.Maybe References: Message-ID: Joachim Breitner writes: > Hi, > > I propose the attached change, with a discussion timeframe until end of > September. The corresponding ticket is > http://hackage.haskell.org/trac/ghc/ticket/3446 > > Thanks, > Joachim I have been meaning to get round to proposing check for Data.Monad: > check :: (MonadPlus m) => (a -> Bool) -> a -> m a > check p a > | p a = return a > | otherwise = mzero Now justIf = flip (check . const) So I suggest that it would be better to add check first(, and then possibly switch the order of arguments to justIf). -- J?n Fairbairn Jon.Fairbairn@cl.cam.ac.uk From mail at joachim-breitner.de Sat Aug 22 06:14:38 2009 From: mail at joachim-breitner.de (Joachim Breitner) Date: Sat Aug 22 05:54:38 2009 Subject: darcs patch: Add justIf to Data.Maybe In-Reply-To: References: Message-ID: <1250936078.4339.3.camel@localhost> Hi, Am Samstag, den 22.08.2009, 10:13 +0100 schrieb Jon Fairbairn: > Joachim Breitner writes: > > I propose the attached change, with a discussion timeframe until end of > > September. The corresponding ticket is > > http://hackage.haskell.org/trac/ghc/ticket/3446 > > > > Thanks, > > Joachim > > I have been meaning to get round to proposing check for Data.Monad: > > > check :: (MonadPlus m) => (a -> Bool) -> a -> m a > > check p a > > | p a = return a > > | otherwise = mzero > > Now justIf = flip (check . const) > > So I suggest that it would be better to add check first(, and then > possibly switch the order of arguments to justIf). are you proposing that, if check is available, no justIf is needed in the libraries? Or are you just proposing a different implementation? Greetings, Joachim -- Joachim Breitner e-Mail: mail@joachim-breitner.de Homepage: http://www.joachim-breitner.de ICQ#: 74513189 Jabber-ID: nomeata@joachim-breitner.de -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Dies ist ein digital signierter Nachrichtenteil Url : http://www.haskell.org/pipermail/libraries/attachments/20090822/3f5d2ad5/attachment.bin From jon.fairbairn at cl.cam.ac.uk Sat Aug 22 10:15:39 2009 From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn) Date: Sat Aug 22 09:55:58 2009 Subject: darcs patch: Add justIf to Data.Maybe References: <1250936078.4339.3.camel@localhost> Message-ID: Joachim Breitner writes: > Hi, > > Am Samstag, den 22.08.2009, 10:13 +0100 schrieb Jon Fairbairn: >> Joachim Breitner writes: >> > I propose the attached change, with a discussion timeframe until end of >> > September. The corresponding ticket is >> > http://hackage.haskell.org/trac/ghc/ticket/3446 >> > >> > Thanks, >> > Joachim >> >> I have been meaning to get round to proposing check for Data.Monad: >> >> > check :: (MonadPlus m) => (a -> Bool) -> a -> m a >> > check p a >> > | p a = return a >> > | otherwise = mzero >> >> Now justIf = flip (check . const) >> >> So I suggest that it would be better to add check first(, and then >> possibly switch the order of arguments to justIf). > > are you proposing that, if check is available, no justIf is needed in > the libraries? I wouldn't be proposing that, having promised not to post on the topic of naming short functions for ten years. > Or are you just proposing a different implementation? I /am/ proposing that. I suppose I have to work out how to make a proper proposal for the addition of check? -- J?n Fairbairn Jon.Fairbairn@cl.cam.ac.uk From duncan.coutts at worc.ox.ac.uk Sat Aug 22 11:10:37 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sat Aug 22 10:50:06 2009 Subject: darcs patch: Add justIf to Data.Maybe In-Reply-To: References: Message-ID: <1250953837.30910.2729.camel@localhost> On Sat, 2009-08-22 at 10:18 +0200, Joachim Breitner wrote: > Hi, > > I propose the attached change > Fri Aug 21 19:17:12 CEST 2009 Joachim Breitner > * Add justIf to Data.Maybe justIf :: a -> Bool -> Maybe a justIf x True = Just x justIf x False = Nothing Which apparently is intended to be used like: v `justIf` a Another instance of this idiom, this one from Cabal, is check :: Bool -> PackageCheck -> Maybe PackageCheck check False _ = Nothing check True pc = Just pc which generalises to check :: Bool -> a -> Maybe a check False _ = Nothing check True x = Just x which is used without `back ticks` like: check (not (null deprecatedExtensions)) $ PackageDistSuspicious $ "Deprecated extensions: " ++ {- some fairly big expression -} In this use case the result is typically a lot bigger than the condition, so the condition goes first rather than last. As J?n was suggesting we might want to generalise to MonadPlus rather than putting it in Data.Maybe, it becomes rather similar to the existing function 'guard': guard :: MonadPlus m => Bool -> m () guard True = return () guard False = mzero check :: MonadPlus m => Bool -> a -> m a check True x = return x check False _ = mzero Using a Bool for the condition is a tad more general than a predicate like: check :: MonadPlus m => (a -> Bool) -> a -> m a In particular, the use case in Cabal does not fit the predicate pattern because the condition is not a condition on the value being returned but on something else in the environment. BTW, I also don't want to take too strong a position on the name ;-) Duncan From duncan.coutts at worc.ox.ac.uk Sat Aug 22 10:43:54 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sat Aug 22 10:50:09 2009 Subject: Potential Network SIG In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> Message-ID: <1250952234.30910.2678.camel@localhost> On Fri, 2009-08-21 at 21:49 -0700, Thomas DuBuisson wrote: > Hello All, > > If you are CCed it's because you are listed as a maintainer of a > network-* package that I consider related to the Haskell network > library. I'm hoping to roll much of the functionality of > network-{bytestring, multicast, fancy} etc into a single package that > the community will agree on (namely, "network"). That sounds like an excellent idea. Go for it! > Johan suggested starting a SIG to hammer out a design for a new > Network API seeing as the current API, a straight-forward Berkeley > binding, doesn't seem to please anyone in a Haskell context. If you > want to partake then this e-mail if your heads up. If there is some > formal method of setting up a Haskell SIG then please let me know. Your email suggesting it is the "formal" method. :-) So yes, get the network people together, talk things over for as long as you need and post your conclusions to this list so we know what is going on and get a chance to comment. If you end up needing some kind of transition or compatibility plan (e.g. if you end up changing APIs etc) then you'll want to talk to the Haskell Platform release team. The aim would be that if we need some transition then we plan it deliberately and get all platform packages to switch in one major release cycle. Duncan From jon.fairbairn at cl.cam.ac.uk Sat Aug 22 11:22:40 2009 From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn) Date: Sat Aug 22 11:02:55 2009 Subject: darcs patch: Add justIf to Data.Maybe References: <1250953837.30910.2729.camel@localhost> Message-ID: Duncan Coutts writes: > On Sat, 2009-08-22 at 10:18 +0200, Joachim Breitner wrote: >> Hi, >> >> I propose the attached change > >> Fri Aug 21 19:17:12 CEST 2009 Joachim Breitner >> * Add justIf to Data.Maybe > As J?n was suggesting we might want to generalise to MonadPlus rather > than putting it in Data.Maybe, it becomes rather similar to the existing > function 'guard': [...] some alpha conversion > > checkA :: MonadPlus m => Bool -> a -> m a > checkA True x = return x > checkA False _ = mzero > > Using a Bool for the condition is a tad more general than a predicate > like: > > checkB :: MonadPlus m => (a -> Bool) -> a -> m a I'd say the generality comparison goes the other way: > In particular, the use case in Cabal does not fit the predicate pattern > because the condition is not a condition on the value being returned but > on something else in the environment. But you can write any checkA e as checkB $ const e whereas a function like all_spaces = checkB (all . isSpace) is hard to write using checkA (suppose you are doing something like ?map (checkB predicate)?) since checkA doesn't look at the second argument. > BTW, I also don't want to take too strong a position on the name ;-) me neither, but check (all . isOK) reads nicely ;-) -- J?n Fairbairn Jon.Fairbairn@cl.cam.ac.uk http://www.chaos.org.uk/~jf/Stuff-I-dont-want.html (updated 2009-01-31) From mail at joachim-breitner.de Sat Aug 22 12:10:23 2009 From: mail at joachim-breitner.de (Joachim Breitner) Date: Sat Aug 22 11:50:19 2009 Subject: darcs patch: Add justIf to Data.Maybe In-Reply-To: <1250953837.30910.2729.camel@localhost> References: <1250953837.30910.2729.camel@localhost> Message-ID: <1250957423.4339.22.camel@localhost> Hi, Am Samstag, den 22.08.2009, 16:10 +0100 schrieb Duncan Coutts: > In this use case the result is typically a lot bigger than the > condition, so the condition goes first rather than last. At first I wrote justIf this way (justIf cond $ value), and then noticed the how nice "value `justIf` cond" sounds when read out aloud. I?m fine with either order, and your point about argument size is a valid one. Greetings, Joachim -- Joachim Breitner e-Mail: mail@joachim-breitner.de Homepage: http://www.joachim-breitner.de ICQ#: 74513189 Jabber-ID: nomeata@joachim-breitner.de -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Dies ist ein digital signierter Nachrichtenteil Url : http://www.haskell.org/pipermail/libraries/attachments/20090822/b56b3c14/attachment.bin From taruti at taruti.net Sat Aug 22 14:26:02 2009 From: taruti at taruti.net (taruti) Date: Sat Aug 22 14:05:57 2009 Subject: Potential Network SIG In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> Message-ID: <1250964740-sup-8537@oz.taruti.net> Excerpts from Thomas DuBuisson's message of Sat Aug 22 07:49:58 +0300 2009: > Johan suggested starting a SIG to hammer out a design for a new > Network API seeing as the current API, a straight-forward Berkeley > binding, doesn't seem to please anyone in a Haskell context. If you > want to partake then this e-mail if your heads up. If there is some > formal method of setting up a Haskell SIG then please let me know. Sounds good, some notes below. > 1) Separate low level functions / bindings from high level / > productive code by placing each in different modules. Good. > 2) Maintain type safety by using type classes for most things. > > Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as > constructors of the same data type), I think we should allow for the > possibility that some users of the library will be limited to just one > IP version without resorting to partial functions. I suggest type > classes to cover this aspect (class Address, class Port, etc). The limitation to one IP-version is a runtime issue not a compile time one. All the platforms may support either IPv4 or IPv6, both or none depending on the machine the binary is run on. I think it should be possible to say "use v4", "use v6" and "use something" on all target platforms. A typeclass might work fine if designed correctly. Also a typeclass makes extensions easier if designed carefully. > 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency. > As in network-bytestring, any new API should be performance concious > enough to avoid String. Ambivalent here. Does it make more sense to have a send :: StringLike s => ... or sendS :: String -> ... sendBS :: ByteString -> ... sendLBS :: L.ByteString -> ... Also if we have separate functions we need yet another set of functions when Text is ready. > 4) Support more features > Features such as Multicast, Header inclusion (IP_HDRINCL), address > binding, etc. IOW, most the IP_ and SO_ options of socket (7) and ip > (7) man pages. It would be rather nice if we were able to expose > these in a friendly way - but with our cross platform concerns that > might not be a good idea (e.g. I'm not familiar with windows). Most of these seem to be for the low level binding? > 5) Separate IO-less data declarations from IO and any platform > dependent code in different packages. Would create lots of nearly identical implementation packages. Most of the implementation differences across platforms are quite small. > 6) Integrate with the rest of hackage. > This means instance of PrettyClass, Parsec, and Binary. I noticed a > number of parsing utilities for IP addresses - lots of duplicated > effort here. Sounds good. - Taru Karttunen From jon.fairbairn at cl.cam.ac.uk Sun Aug 23 05:10:35 2009 From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn) Date: Sun Aug 23 04:50:49 2009 Subject: Proposal: add new function "check" to Control.Monad Message-ID: Trac ticket #3453. Two week time frame. Add check function to Control.Monad check :: (MonadPlus m) => (a -> Bool) -> a -> m a check p a | p a = return a | otherwise = mzero Rationale: The example that suggested the function to me is this: readMaybe :: Read a => String -> Maybe a readMaybe = join . fmap no_trailing_garbage . listToMaybe . reads where no_trailing_garbage = fmap fst . check (all isSpace . snd) but check is clearly more generally useful than guard. (guard = flip (check . const) ()) Discussion: I also note in the comments to check that we can define List.filter like this filter = (concat .) . map . check Now, concat is just join specialised to lists, and map [is fmap, but...] is liftM, so we would expect a function mfilter = (join .) . liftM . check to be useful. Should that also be added? -- J?n Fairbairn Jon.Fairbairn@cl.cam.ac.uk From duncan.coutts at worc.ox.ac.uk Sat Aug 22 18:43:20 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Aug 23 08:22:27 2009 Subject: Potential Network SIG In-Reply-To: <1250964740-sup-8537@oz.taruti.net> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> <1250964740-sup-8537@oz.taruti.net> Message-ID: <1250981000.30910.3194.camel@localhost> On Sat, 2009-08-22 at 21:26 +0300, taruti wrote: > > 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency. > > As in network-bytestring, any new API should be performance concious > > enough to avoid String. > > Ambivalent here. Does it make more sense to have a > send :: StringLike s => ... > or > sendS :: String -> ... > sendBS :: ByteString -> ... > sendLBS :: L.ByteString -> ... I've never understood why people want these type classes. Just pick the right one. Each of those types has functions for converting to each other. Let the caller do the conversion if any needs doing, it's just a function call. > Also if we have separate functions we need yet another set > of functions when Text is ready. Which also comes with functions for converting to ByteString or String. Duncan From judah.jacobson at gmail.com Sun Aug 23 12:22:31 2009 From: judah.jacobson at gmail.com (Judah Jacobson) Date: Sun Aug 23 12:02:41 2009 Subject: Proposal #3455: Add a setting to change how Unicode encoding errors are handled Message-ID: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com> I proposal that we augment ghc-6.12.1's support for Unicode Handles by adding the following functions to System.IO: hSetOnEncodingError :: Handle -> OnEncodingError -> IO () hGetOnEncodingError :: Handle -> IO OnEncodingError as well as the enumeration `OnEncodingError` with three constructors: - `ThrowEncodingError`: Throw an exception at the first encoding or decoding error. - `SkipEncodingError`: Skip all invalid bytes or characters. - `TranslitEncodingError`: Replace undecodable bytes with u+FFFD, and unencodable characters with '?'. I have implemented this functionality in a patch attached to the ticket. Haddock docs are here: http://code.haskell.org/~judah/new-io-docs/System-IO.html#23 The choice of error handler is orthogonal to the choice of encoder. Additionally, the same setting is used for both read and write modes. For portability, the handlers are written in pure Haskell rather than using GNU iconv's //TRANSLIT feature. Note that the text package, for example, provides more sophisticated error-handling options. However, I think the above choices are useful enough without making the API too complicated. Discussion deadline: September 9 Ticket: http://hackage.haskell.org/trac/ghc/ticket/3455 Best, -Judah From judah.jacobson at gmail.com Sun Aug 23 12:27:15 2009 From: judah.jacobson at gmail.com (Judah Jacobson) Date: Sun Aug 23 12:07:24 2009 Subject: Proposal #3456: Add FilePath -> String decoder Message-ID: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com> Currently, FilePaths on POSIX systems are represented as raw bytes in a String. When this last came up on the mailing list, the general consensus was to make FilePath an abstract datatype representing paths as Strings on Windows and raw bytes on POSIX systems. However, such a change is sure to break some existing code. As a small step towards that goal, I propose adding the following two functions to the System.IO module: filePathToString :: FilePath -> IO String getFilePathToStringFunc :: IO (FilePath -> String) I've implemented those functions in a patch attached to the trac ticket. Haddock docs are here: http://code.haskell.org/~judah/new-io-docs/System-IO.html#v%3AfilePathToString On POSIX those functions decode according to the current locale, so they ought to be in the IO monad. I think that the latter function will be easier to integrate into existing pure code. On Windows, those functions are just `return` and `return id`, respectively, since GHC already treats FilePaths as Strings on that platform. Discussion deadline: September 9 Ticket: http://hackage.haskell.org/trac/ghc/ticket/3456 Best, -Judah From mauricio.antunes at gmail.com Sun Aug 23 14:09:18 2009 From: mauricio.antunes at gmail.com (=?ISO-8859-1?Q?Maur=ED=ADcio_CA?=) Date: Sun Aug 23 13:55:00 2009 Subject: Binding to inline functions Message-ID: I understand we can't use 'foreign import ccall' to wrap inline C functions. Do you think it could be possible to have an option in cabal to generate such functions in an object file when #included in a C file, in a compiler independent, portable way? Thanks, Maur?cio From duncan.coutts at worc.ox.ac.uk Sun Aug 23 14:49:23 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Aug 23 14:29:08 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <20090822001735.GA17487@matrix.chaos.earth.li> References: <1250733860.9379.2142.camel@localhost> <20090821012355.GA24225@matrix.chaos.earth.li> <1250884736.30910.1484.camel@localhost> <20090822001735.GA17487@matrix.chaos.earth.li> Message-ID: <1251053363.30910.4813.camel@localhost> Ian, You raise a number of points. I'll address the easy ones here and summarise the others in a separate email. What I've left out is: * the issue about whether things should be in the proposal or should be required to exist as separate documents that are linked from the proposal. * the issue of a checklist / status table in the proposal Summary of the bits where I think we already agree: * use [rationale-1.3] links rather than [note-1.3] for better clarity * suggest the proposal mention the package's version number * the 'tar' example could be improved and shortened On Sat, 2009-08-22 at 01:17 +0100, Ian Lynagh wrote: > On Fri, Aug 21, 2009 at 08:58:56PM +0100, Duncan Coutts wrote: > > On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote: > > > I read the example first, so I'll give my comments on it first: > > > > The policy is significantly less prescriptive than the example might > > lead you to believe. > > Looking again at AddingPackages, I think you are right. That said, if I > were making a proposal, I would probably base it on an example (or a > template, which is essentially the same thing). > > The AddingPackages page confuses me. If I need to read all the notes, It is not essential to read them to be able to understand the policy. They are there to answer the question "why was this requirement included?" and possibly also to clarify the meaning by stating the intentions. > then the page seems very hard to read, and if I don't then could the > rationale be put on a separate page? A separate page would make it much slower to jump back and forth between the rationale and the policy. The intro says: * The rationale section gives an explanation and justification for the policy. Is there a wording that would make it clearer that you do not have to read it unless you want an explanation for a point in the policy? > Also, the [note-1.3]s could all be [rationale] or [rationale-1.3], > right? That way I wouldn't think that I have to read them unless I am > actually interested in the rationale. That's a good idea, we should do that. > Currently the page looks daunting to me. We already split off the "how to" guide into a separate page. We could possibly do the same with the consensus protocol. I think the rationale should stay with the policy however. > > > Should specify version number. > > > > I expect that usually the exact number is not especially relevant. The > > exact released version now is not necessarily the same as the final one > > that gets included in the case that reviewers make suggestions. > > > > If there's more than one active branch then yes it's relevant so > > reviewers know which one to look at. Otherwise reviewers can reasonably > > assume the version on the hackage page. > > I don't see the harm in giving the intended version number in the > proposal. Right, there's no harm and it's fairly easy for the author to add that info. I'm happy to suggest it in the "how to" guide. I don't think it's such crucial info that it needs to be explicitly required by the policy. > Drifting off-topic, but I think that the two discussions are orthogonal > (i.e. we could have them in either order), but the "detailed > requirements" discussion is the more important one. People can make > proposals without a process being nailed down (and that may even help us > work out what the process should be), but how can we decide if a package > should be included if we don't know what the acceptance criteria are? The acceptance criteria is clearly specified. It is not that a set of minimal requirements be met. It is that the reviewers, using their judgement come to a consensus to accept the package. > So I think the other discussion is much closer on the critical path. I don't think it is on the critical path. I think it is quite possible to start reviewing packages and try to codify what we agree additional criteria should be. If there are significant disagreements between reviewers while reviewing packages then these would indicate areas where it may be helpful to codify a requirement or guideline. There is certainly an advantage to having a more comprehensive set of requirements and guidelines because it'll help people proposing packages to know what to aim for and should help reviewers to agree with each other about a package. > In fact, I think that's the main problem I have with both the process > document and the example proposal: They try to anticipate all questions > that might be asked, and answer them. But the common case is that the > question wouldn't be asked, so this is just more stuff for someone to > write, and and more stuff for many people to read. This is true, but is the balance of benefit such that we should leave various questions open in favour of making it slightly quicker to read the policy? I note that the policy is only 1500 words. We deliberately tried to keep it short by separating it from the rationale and from the practical advice. Is there anything specific in the policy you would remove that would make the thing appreciably shorter without making it too ambiguous? My guess is that you'll say the consensus protocol. > That's why here: > > > I agree that what I've written for the rationale for the tar proposal is > > of somewhat dubious utility. In fact personally I don't like the term > > you've had to write stuff "of somewhat dubious utility": because you're > answering a boilerplate question that didn't need asking in this case. I think it's partly that I originally wrote the example for a draft of the policy that was significantly more prescriptive about the structure of the proposal and I did not update it significantly when I adapted it for the final draft. > And in the case of consensus, I don't think we need to lay down rules > now for what happens if the current system doesn't work. For one thing, > it hasn't been an issue so far. That is what we had in an intermediate draft. We added the facility for more formality in contentious cases because several members of the steering committee and other people who read the draft expressed the concern that despite consensus working ok for the libraries process, that it might not work so well for deciding on package additions. Is there any disadvantage to having this option available? Do you think it is a bad option or merely unnecessary? Is it just the fact that it makes the wiki page longer? > For another, when we have the first case of it not working, it might > be clear that the solution would be to do something else anyway. That option is always open anyway. The rationale says: [note-5.4] The exact definition of what constitutes a consensus is possibly a contentious point. The obvious choice is to do what we already do with the existing library submissions process. It uses a relatively vague definition of consensus and yet has worked reasonably well in practice. If it does not work out in practice for the package addition process then the libraries list may want to revisit this point. > > What are you suggesting specifically that would reduce the amount of > > work for reviewers? > > I am suggesting that there be less text written. Less text in the proposal or less text in total? You've suggested that the proposal link to several external documents. > > Would those things increase work for the proposal > > authors / package maintainers? > > It should reduce work for the authors / package maintainers. Even taking into account that you would expect them to write these external documents? Where is the reduction coming from? > > Would those things increase or decrease > > the level of depth/quality of review and decisions? > > I believe it would increase the number of reviewers, giving better > decisions. I'm afraid I'm not clear about where the reduction is coming from in what authors / package maintainers have to write and reviewers are expected to read. Is it just the "rationale waffle" stuff from the 'tar' example that you mean? Duncan From duncan.coutts at worc.ox.ac.uk Sun Aug 23 15:31:17 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Aug 23 15:10:35 2009 Subject: Recommendation for the procedure to add platform packages In-Reply-To: <20090822001735.GA17487@matrix.chaos.earth.li> References: <1250733860.9379.2142.camel@localhost> <20090821012355.GA24225@matrix.chaos.earth.li> <1250884736.30910.1484.camel@localhost> <20090822001735.GA17487@matrix.chaos.earth.li> Message-ID: <1251055877.30910.4928.camel@localhost> All, I'll try and summarise the concerns that Ian has raised: * That documentation written for the proposal will get lost. The "how to" says that if people write a new API intro, then it should be integrated into the haddock docs or saved on a project home page. Ian thinks this suggestion/advice is not sufficient to ensure that the docs do not get lost and thinks that some procedural device is necessary. Ian thinks that the best mechanism is to require the API intro be a separate doc that is linked from the proposal. Thus the proposal authors would not have the opportunity to "improve" the docs for the proposal without those improvements also being saved as part of the package. Ian accepts that there may be other mechanisms to make sure docs written for the proposal do not get lost but would have greatest confidence in the method he suggested. * That the package proposals would be too long and too wordy. There are a few aspects to this: 1. That text included inline in the proposal is less readable than the same amount of text in separate documents linked from the proposal (like API intro); 2. That the "rationale" suggested in the "how to" and example is mostly redundant/unnecessary; 3. That things described in words could be summarised by a "yes" in a checklist. Specifically that a checklist augmented with text sections is a shorter and better proposal format than a free-form proposal. Related to this is a belief that the criteria for inclusion can and should be reduced to evaluating a list of requirements. Some of those requirements would be objective, some subjective, but all relatively specific. Passing all requirements would be the criteria for inclusion. * That the policy document itself is too long and too detailed. That it anticipates eventualities that may or may not arise in practise. That the overall length is a problem because it is off-putting, with the danger that people simply will not read it. Ian, is this a fair summary? Is there anything I've missed that I didn't address specifically in the previous email? Duncan From duncan.coutts at worc.ox.ac.uk Sun Aug 23 15:34:18 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Aug 23 15:13:30 2009 Subject: Binding to inline functions In-Reply-To: References: Message-ID: <1251056058.30910.4937.camel@localhost> On Sun, 2009-08-23 at 15:09 -0300, Maur??cio CA wrote: > I understand we can't use 'foreign import ccall' > to wrap inline C functions. Do you think it could > be possible to have an option in cabal to generate > such functions in an object file when #included in > a C file, in a compiler independent, portable way? It might be better to include this feature into hsc2hs and/or c2hs (which may in turn require some help from Cabal). Duncan From duncan.coutts at worc.ox.ac.uk Sun Aug 23 20:53:29 2009 From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts) Date: Sun Aug 23 20:32:45 2009 Subject: Proposal #3455: Add a setting to change how Unicode encoding errors are handled In-Reply-To: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com> References: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com> Message-ID: <1251075209.30910.5603.camel@localhost> On Sun, 2009-08-23 at 09:22 -0700, Judah Jacobson wrote: > I proposal that we augment ghc-6.12.1's support for Unicode Handles > by adding the following functions to System.IO: > > hSetOnEncodingError :: Handle -> OnEncodingError -> IO () > hGetOnEncodingError :: Handle -> IO OnEncodingError I agree that it is important. > Note that the text package, for example, provides more sophisticated > error-handling options. However, I think the above choices are useful > enough without making the API too complicated. Personally I would prefer we postpone this decision with the aim that we unify the Text encoding / decoding between the IO system and text package. Specifically I suggest we put this decision off until after ICFP where we hope that the authors of ghc's new text IO system and the authors of the text package can get together with other interested individuals to discuss some more unified system. It should be possible to make an encoder abstraction that can be used purely in the text package for conversion between ByteString <-> Text, and also used in the IO system for text mode Handles. How to handle encoding and translation errors would have to be part of the design of that encoder abstraction. My impression is that an ST version of the current text encoder abstraction in the GHC text IO system would also be usable for pure conversions in the text package. Duncan From johan.tibell at gmail.com Mon Aug 24 03:53:14 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Mon Aug 24 03:33:24 2009 Subject: Potential Network SIG In-Reply-To: <1250981000.30910.3194.camel@localhost> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> <1250964740-sup-8537@oz.taruti.net> <1250981000.30910.3194.camel@localhost> Message-ID: <90889fe70908240053r779400c9race5e3fa0ba94fca@mail.gmail.com> On Sun, Aug 23, 2009 at 12:43 AM, Duncan Coutts wrote: > On Sat, 2009-08-22 at 21:26 +0300, taruti wrote: > >> > 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency. >> > As in network-bytestring, any new API should be performance concious >> > enough to avoid String. >> >> Ambivalent here. Does it make more sense to have a >> send :: StringLike s => ... >> or >> sendS :: String -> ... >> sendBS :: ByteString -> ... >> sendLBS :: L.ByteString -> ... > > I've never understood why people want these type classes. Just pick the > right one. Each of those types has functions for converting to each > other. Let the caller do the conversion if any needs doing, it's just a > function call. Arguably, ByteString is the right one, at least for the lower level binding. ByteString map well to the types used by the underlying system calls and it is semantically the right type. I can see a reason to provide both the strict and the lazy version as going from lazy to strict might be inefficient as it involves copying. This is why network-bytestring supports both. >> Also if we have separate functions we need yet another set >> of functions when Text is ready. > > Which also comes with functions for converting to ByteString or String. I'm not even sure what sending 'String's or 'Text's would mean without specifying an encoding. We shouldn't be afraid to use composition to prevent a combinatorial explosion i.e. don't have a sendAsUtf8, sendAsUtf16 but instead do send $ Text.encode Utf8 "myString" Cheers, Johan From sebf at informatik.uni-kiel.de Mon Aug 24 03:59:24 2009 From: sebf at informatik.uni-kiel.de (Sebastian Fischer) Date: Mon Aug 24 03:39:46 2009 Subject: Proposal: add new function "check" to Control.Monad In-Reply-To: References: Message-ID: <2E8CC191-9C5F-4C22-857B-0C8F432C0DA5@informatik.uni-kiel.de> > Add check function to Control.Monad > > check :: (MonadPlus m) => (a -> Bool) -> a -> m a > check p a > | p a = return a > | otherwise = mzero I agree that such a function is occasionally useful. But I doubt your rationale: > Rationale: [...] > > but check is clearly more generally useful than guard. (guard = flip > (check . const) ()) This is a bit like saying that concatMap is more generally useful than map and concat because: map f = concatMap ((:[]).f) concat = concatMap id Although this is correct, map and concat are smaller pieces that can be easily combined to concatMap: concatMap f = concat . map f So the question is whether check is useful enough to be included as a shortcut for a combination of simpler primitives (as was decided for concatMap). If 'check' is added then I would prefer this definition: check f x = guard (f x) >> return x It emphasises how 'check' is combined from simpler parts. > so we would expect a function > > mfilter = (join .) . liftM . check > > to be useful. Should that also be added? I would not object and prefer this definition: mfilter f m = m >>= check f It seems simpler. Cheers, Sebastian -- Underestimating the novelty of the future is a time-honored tradition. (D.G.) From johan.tibell at gmail.com Mon Aug 24 03:59:57 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Mon Aug 24 03:40:06 2009 Subject: Potential Network SIG In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> Message-ID: <90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com> Hi, Thanks for getting this started Thomas. I've added some of my thought inline: On Sat, Aug 22, 2009 at 6:49 AM, Thomas DuBuisson wrote: > 1) Separate low level functions / bindings from high level / > productive code by placing each in different modules. I think this is crucial to avoid having a percentage of the users have to write their own bindings because we simplified away their use case. In order for the library to be extensible it should also allow access to the underlying file descriptor. That way someone could e.g. write a library for the 'select' system call and have it work with sockets. > 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency. > As in network-bytestring, any new API should be performance concious > enough to avoid String. Agreed. String is also semantically the wrong types. The OS socket API doesn't understand Unicode. > 6) Integrate with the rest of hackage. > This means instance of PrettyClass, Parsec, and Binary. ?I noticed a > number of parsing utilities for IP addresses - lots of duplicated > effort here. I must disagree here. We don't want hard coupling to lots of other libraries. i.e. we don't need convenience functions like: get :: Socket -> Binary a These ends are better reach by the client writing the glue code. It's easy to get a combinatorial explosion in the number of functions if we try to provide an integration point for each library out there. > I'm currently looking at how the network library is being used - > particularly when Network.Socket is invoked. ?So I guess I'll ?go sort > through some of the source code for scurry, happstack, and > adhoc-network. I think the best way to proceed is to discuss specific. Lets dig up some examples of things that are suboptimal in the current design and write down some alternative designs on a wiki page. Cheers, Johan From gale at sefer.org Mon Aug 24 04:03:09 2009 From: gale at sefer.org (Yitzchak Gale) Date: Mon Aug 24 03:43:18 2009 Subject: Proposal: add new function "check" to Control.Monad In-Reply-To: References: Message-ID: <2608b8a80908240103v35a2d36andeee9cf20fb3f5e0@mail.gmail.com> J?n Fairbairn wrote: > Trac ticket #3453. Two week time frame. > Add check function to Control.Monad > check :: (MonadPlus m) => (a -> Bool) -> a -> m a I remember needing this on a number of occasions. > ? mfilter = (join .) . liftM . check I might write that as: mFilter = (=<<) . check That is an intriguing dual to filterM. Regards, Yitz From gale at sefer.org Mon Aug 24 04:43:04 2009 From: gale at sefer.org (Yitzchak Gale) Date: Mon Aug 24 04:23:13 2009 Subject: Potential Network SIG In-Reply-To: <90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> <90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com> Message-ID: <2608b8a80908240143v1f20677hf48cff54aabaf74b@mail.gmail.com> Johan Tibell wrote: > We shouldn't be afraid to use composition to > prevent a combinatorial explosion i.e. don't have a sendAsUtf8, > sendAsUtf16 but instead do > send $ Text.encode Utf8 "myString" OK, makes sense. But then, that should appear very explicitly in the documentation. It will be the most common usage, and people using it will not be focused on Network, Bytestring, and Text, but on other things. They should not be forced to thrash around and do a research project just to do something simple. Thomas DuBuisson wrote: >> 6) Integrate with the rest of hackage. >> This means instance of PrettyClass, Parsec, and Binary. > I must disagree here. We don't want hard coupling to lots of other > libraries. It won't be a problem if the convenience glue is in separate packages that pull in all of the dependencies. But then, these other packages need to be mentioned in the documentation for the main package. Regards, Yitz From johan.tibell at gmail.com Mon Aug 24 05:23:45 2009 From: johan.tibell at gmail.com (Johan Tibell) Date: Mon Aug 24 05:03:53 2009 Subject: Potential Network SIG In-Reply-To: <2608b8a80908240143v1f20677hf48cff54aabaf74b@mail.gmail.com> References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> <90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com> <2608b8a80908240143v1f20677hf48cff54aabaf74b@mail.gmail.com> Message-ID: <90889fe70908240223x38fe14a8vbd969d434cded935@mail.gmail.com> On Mon, Aug 24, 2009 at 10:43 AM, Yitzchak Gale wrote: > Johan Tibell wrote: >> We shouldn't be afraid to use composition to >> prevent a combinatorial explosion i.e. don't have a sendAsUtf8, >> sendAsUtf16 but instead do >> send $ Text.encode Utf8 "myString" > > OK, makes sense. But then, that should appear very > explicitly in the documentation. It will be the most common > usage, and people using it will not be focused on Network, > Bytestring, and Text, but on other things. They should not > be forced to thrash around and do a research project just > to do something simple. I agree. The network package could use a lot more documentation in general. For example, I put a echo client/server pair example in my network-bytestring package as the network library lacked even something that basic. > Thomas DuBuisson wrote: >>> 6) Integrate with the rest of hackage. >>> This means instance of PrettyClass, Parsec, and Binary. > >> I must disagree here. We don't want hard coupling to lots of other >> libraries. > > It won't be a problem if the convenience glue is in separate > packages that pull in all of the dependencies. But then, these > other packages need to be mentioned in the documentation > for the main package. That is an option. Perhaps something we could look into once we dealt with the basic design of the network library itself? -- Johan From Malcolm.Wallace at cs.york.ac.uk Mon Aug 24 08:10:35 2009 From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace) Date: Mon Aug 24 07:50:29 2009 Subject: Cabal: literate hscolour In-Reply-To: <6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com> References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> <1250786981.9379.3191.camel@localhost> <4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk> <6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com> Message-ID: > My old files processed with hscolour 1.13 had headers like > > "> > > > > src/Imogen/Infon/Fol/Subst.lhs > > > > > The files now don't have this header, and they are not showing up > colored. Ah, yes, that was one of the consequences of using -lit: it has always implied -partial. However, it is clearly a bug that there was no way to get an HTML / CSS / LaTeX prologue and epilogue in conjunction with the -lit option. Now fixed. The behaviour of -nopartial is now the default for all inputs, even literate files. People who want the old behaviour must now ask for it explicitly, using -lit -partial. > Second, I'm probably being a pain in the ass, but I'd also like to > keep my literate comments preformatted. Right now they are mashed > all together I'm not sure what you mean. Could you send an example? (It is possible that it is a DOS/Windows line-ending bug.) I will make a new release 1.15 of HsColour once I have a fix for this second bug. Regards, Malcolm From mauricio.antunes at gmail.com Mon Aug 24 10:28:35 2009 From: mauricio.antunes at gmail.com (=?ISO-8859-1?Q?Maur=ED=ADcio_CA?=) Date: Mon Aug 24 10:08:49 2009 Subject: Binding to inline functions In-Reply-To: <1251056058.30910.4937.camel@localhost> References: <1251056058.30910.4937.camel@localhost> Message-ID: >> I understand we can't use 'foreign import ccall' >> to wrap inline C functions. Do you think it could >> be possible to have an option in cabal to generate >> such functions in an object file when #included in >> a C file, in a compiler independent, portable way? > It might be better to include this feature into hsc2hs and/or c2hs > (which may in turn require some help from Cabal). Do you imagine a nice way to include that in hsc2hs or c2hs? (Also: inline functions are exactly like normal functions, except at compilation time. For those who just want to wrap a few functions, isn't it too much to require them to learn a tool just to use them?) Thanks, Maur?cio From seanmcl at gmail.com Mon Aug 24 11:55:20 2009 From: seanmcl at gmail.com (Sean McLaughlin) Date: Mon Aug 24 11:35:07 2009 Subject: Cabal: literate hscolour In-Reply-To: References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com> <1250786981.9379.3191.camel@localhost> <4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk> <6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com> Message-ID: <6579f8680908240855s4df26049gcaed5823749a9a5@mail.gmail.com> Hi Malcolm, > > > Second, I'm probably being a pain in the ass, but I'd also like to keep my >> literate comments preformatted. Right now they are mashed all together >> > > I'm not sure what you mean. Could you send an example? (It is possible > that it is a DOS/Windows line-ending bug.) > When you put the literate comments directly in html, for instance -- ---------------------------------------------- -- Sequents -- ---------------------------------------------- Here are a lot of comments about the sequent calculus. > data Seq = ... This is rendered as -- -------------------------------- -- Sequents -- --------------------------------------- Here are a lot ... I'd like the format of the comments to stay the same as in the file. The easiest way I suppose is to wrap 
 tags around them.  That's all I meant.

Thanks for your help,

Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/libraries/attachments/20090824/aff9e0eb/attachment.html
From nonowarn at gmail.com  Mon Aug 24 12:46:46 2009
From: nonowarn at gmail.com (Yusaku Hashimoto)
Date: Mon Aug 24 12:26:33 2009
Subject: Potential Network SIG
In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
Message-ID: 

Hello,

On Sat, Aug 22, 2009 at 1:49 PM, Thomas
DuBuisson wrote:
> 2) Maintain type safety by using type classes for most things.
>
> Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as
> constructors of the same data type), I think we should allow for the
> possibility that some users of the library will be limited to just one
> IP version without resorting to partial functions. ?I suggest type
> classes to cover this aspect (class Address, class Port, etc).

I was trying to add another type safety for socket operations, and you
can get it by ` darcs get
http://patch-tag.com/r/phantom-socket/pullrepo phantom-socket`.

My approach is having state of socket in its type, and having states
of socket before/after actions in its types. This forces programmers
not to write wrong operations like listening on socket before binding.

E.g., Function for creating socket which is connected to somewhere has
the type `HostName -> PortNumber -> IO (Socket Connected)`. Note type
Socket has parameter in its type of returning value, this represents
the state of connected socket.

And a function for listening with specifying backlog's size has the
type `BackLog -> SockAct Bound Listening ()`. SockAct is an indexed
monad[1] has socket state before/after its action in its *type*. Here,
SockAct has three parameters. The first is the before-state of the
action, the second is the after-state, and the last is the type of
value wrapped in monad.

Working example is at here[2]. If you swapped some lines in
ixdo-block, you would see compile error, not run-time error. (ixdo is
do-notation in indexed monad, it is provided by ixdopp, preprocessor
for de-sugaring ixdo-block before actual compiling)

But it ensures only compile-time's safety, can't ensure run-time's
safety (consider if client closes socket while sending). Currently, if
run-time and compile-time's states become not same, program just
crushes.

So I think my implementation can't be used in real world, but I hope
this would give you some ideas for type-safe network libraries.

[1]: http://hackage.haskell.org/packages/archive/category-extras/0.44.4/doc/html/Control-Monad-Indexed.html
[2]: http://patch-tag.com/r/phantom-socket/snapshot/current/content/pretty/test/Test.hs

Cheers,
-nwn
From Malcolm.Wallace at cs.york.ac.uk  Mon Aug 24 13:30:28 2009
From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace)
Date: Mon Aug 24 13:10:20 2009
Subject: Cabal: literate hscolour
In-Reply-To: <6579f8680908240855s4df26049gcaed5823749a9a5@mail.gmail.com>
References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com>
	<1250786981.9379.3191.camel@localhost>
	<4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk>
	<6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com>
	
	<6579f8680908240855s4df26049gcaed5823749a9a5@mail.gmail.com>
Message-ID: <31699B12-0590-4C00-B102-9EC7B5063483@cs.york.ac.uk>

> -- ----------------------------------------------
> -- Sequents
> -- ----------------------------------------------
>
> This is rendered as
>
> -- -------------------------------- -- Sequents --  
> --------------------------------------- 
>
> I'd like the format of the comments to stay the same as in the  
> file.  The easiest way I suppose is to wrap 
 tags around them.   
> That's all I meant.

Now I understand.  The literate file format is designed for you to use  
your own markup for the commentary.  In particular, many people use  
real HTML tags (or LaTeX commands etc) for the comments, and I have no  
wish to impede them.  One obvious solution, that would inconvenience  
no-one else, would be to add the 
 tags yourself.

If you wish your comments to be essentially a plain-text format, but  
still to render nicely in HTML, then I suggest you look into markdown http://daringfireball.net/projects/markdown/ 
  and in particular its implementation in Haskell called pandoc http://johnmacfarlane.net/pandoc/

Pandoc does syntax highlighting for Haskell (and other languages),  
using the highlighting-kate package, rather than HsColour.

Regards,
     Malcolm

From duncan.coutts at worc.ox.ac.uk  Mon Aug 24 13:42:07 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Mon Aug 24 13:37:01 2009
Subject: Cabal: literate hscolour
In-Reply-To: <31699B12-0590-4C00-B102-9EC7B5063483@cs.york.ac.uk>
References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com>
	<1250786981.9379.3191.camel@localhost>
	<4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk>
	<6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com>
	
	<6579f8680908240855s4df26049gcaed5823749a9a5@mail.gmail.com>
	<31699B12-0590-4C00-B102-9EC7B5063483@cs.york.ac.uk>
Message-ID: <1251135727.30910.6814.camel@localhost>

On Mon, 2009-08-24 at 18:30 +0100, Malcolm Wallace wrote:

> > I'd like the format of the comments to stay the same as in the  
> > file.  The easiest way I suppose is to wrap 
 tags around them.   
> > That's all I meant.
> 
> Now I understand.  The literate file format is designed for you to use  
> your own markup for the commentary.  In particular, many people use  
> real HTML tags (or LaTeX commands etc) for the comments, and I have no  
> wish to impede them.  One obvious solution, that would inconvenience  
> no-one else, would be to add the 
 tags yourself.

Malcolm, this is all true but we need some sensible default case, eg for
all the source links in the hackage docs in the case where people are
not using any specific markup. If there were a "pre" mode then I think
this would be the default that Cabal should use (since it currently does
not know any more detail about what form the .lhs files are).

Duncan

From bos at serpentine.com  Mon Aug 24 14:00:31 2009
From: bos at serpentine.com (Bryan O'Sullivan)
Date: Mon Aug 24 13:40:32 2009
Subject: Potential Network SIG
In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
Message-ID: 

On Fri, Aug 21, 2009 at 9:49 PM, Thomas DuBuisson <
thomas.dubuisson@gmail.com> wrote:

>
> Johan suggested starting a SIG to hammer out a design for a new
> Network API seeing as the current API, a straight-forward Berkeley
> binding, doesn't seem to please anyone in a Haskell context.


Thanks for getting the conversation started!

Your agenda is very ambitious, which worries me a little. As a "how to get
things moving" tip, I'd very strongly suggest trying to make progress on the
lowest level bindings first, as they will be the most concrete, and the
least likely to provoke prolonged discussion and disagreement.


> 2) Maintain type safety by using type classes for most things.
>
> Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as
> constructors of the same data type), I think we should allow for the
> possibility that some users of the library will be limited to just one
> IP version without resorting to partial functions.  I suggest type
> classes to cover this aspect (class Address, class Port, etc).
>

I don't understand what this might mean. Code examples or type signatures
are going to be much easier to follow than English descriptions.


> 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency.
>

That's already a step up from the lowest-level bindings, which should be
using Ptr a.


> 4) Support more features
> Features such as Multicast, Header inclusion (IP_HDRINCL), address
> binding, etc.  IOW, most the IP_ and SO_ options of socket (7) and ip
> (7) man pages.  It would be rather nice if we were able to expose
> these in a friendly way - but with our cross platform concerns that
> might not be a good idea (e.g. I'm not familiar with windows).
>

Providing Network.Windows and Network.Linux and Network.BSD etc modules
would work fine for non-portable platform-specific features (of which there
are many).

As for providing instances for the likes of Binary, there are good reasons
not to do that in the core network package, because it will drag in
dependencies on quite a few third-party packages that are either not in core
(binary) or have questions over their futures (Parsec).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/libraries/attachments/20090824/3e3bf4b4/attachment.html
From Malcolm.Wallace at cs.york.ac.uk  Mon Aug 24 14:23:14 2009
From: Malcolm.Wallace at cs.york.ac.uk (Malcolm Wallace)
Date: Mon Aug 24 14:03:01 2009
Subject: Cabal: literate hscolour
In-Reply-To: <1251135727.30910.6814.camel@localhost>
References: <6579f8680908200612i375222a1x73c84760d4b9536e@mail.gmail.com>
	<1250786981.9379.3191.camel@localhost>
	<4012E42D-7D69-4405-8301-D69DCF805B60@cs.york.ac.uk>
	<6579f8680908210612o5cfcefc9t15027ab9694e06d@mail.gmail.com>
	
	<6579f8680908240855s4df26049gcaed5823749a9a5@mail.gmail.com>
	<31699B12-0590-4C00-B102-9EC7B5063483@cs.york.ac.uk>
	<1251135727.30910.6814.camel@localhost>
Message-ID: <4CDD6C2A-807F-4D7B-A8D7-8B527B2F707E@cs.york.ac.uk>

>>> I'd like the format of the comments to stay the same as in the
>>> file.  The easiest way I suppose is to wrap 
 tags around them.
>
> we need some sensible default case, eg for all the source links in  
> the hackage docs in the case where people are not using any specific  
> markup. If there were a "pre" mode then I think this would be the  
> default that Cabal should use (since it currently does not know any  
> more detail about what form the .lhs files are).

OK, if there is demand of more than one, then I can add a new  
option.  :-)

Regards,
     Malcolm

From thomas.dubuisson at gmail.com  Mon Aug 24 14:34:01 2009
From: thomas.dubuisson at gmail.com (Thomas DuBuisson)
Date: Mon Aug 24 14:13:48 2009
Subject: Potential Network SIG
In-Reply-To: 
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
Message-ID: <4c44d90b0908241134l70879fdcmaddb09638feccafb@mail.gmail.com>

Bryan O'Sullivan wrote:
> Your agenda is very ambitious, which worries me a little. As a "how to get
> things moving" tip, I'd very strongly suggest trying to make progress on the
> lowest level bindings first, as they will be the most concrete, and the
> least likely to provoke prolonged discussion and disagreement.

Thanks for the tip - I'll probably do that.  It's my intention to
reuse much of the current Network low level bindings.  It still
doesn't have all the constants we want... and it has some extra fluff
such as inet_ntoa, but its a great place to start.

>> 2) Maintain type safety by using type classes for most things.
>>
>> Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as
>> constructors of the same data type), I think we should allow for the
>> possibility that some users of the library will be limited to just one
>> IP version without resorting to partial functions. ?I suggest type
>> classes to cover this aspect (class Address, class Port, etc).
>
> I don't understand what this might mean. Code examples or type signatures
> are going to be much easier to follow than English descriptions.

Network.Fancy (and Network.Socket) defines a data structure morally
equivalent to:

    data Address = IPv4 Word32 | IPv6 Word128 | ...

I Object!  Code that uses addresses _might_ be very specific to IPv4
or IPv6 - such code should be able to restrict itself to the proper
category of addresses, headers etc without resorting to partial
functions.  I propose a solution akin to:

    class Address a where ...
    instance Address IPv4
    instance Address IPv6

This could allow connect functions, such as those in Network.Fancy, to
look like the below signature.

     connectDgram :: Address a => a -> IO Socket

While still giving type safety to those of us who do low level munging
with protocol headers.  I'm still not clear on how the routines would
be built - this idea was born from a desire for type safety when
building headers.

>> 4) Support more features
>> Features such as Multicast, Header inclusion (IP_HDRINCL), address
>> binding, etc. ?IOW, most the IP_ and SO_ options of socket (7) and ip
>> (7) man pages. ?It would be rather nice if we were able to expose
>> these in a friendly way - but with our cross platform concerns that
>> might not be a good idea (e.g. I'm not familiar with windows).
>
> Providing Network.Windows and Network.Linux and Network.BSD etc modules
> would work fine for non-portable platform-specific features (of which there
> are many).

I was actually thinking of even rarer platforms that might not even
have a concept of Sockets (halvm anyone? House, hOp?) - keeping
important data declarations separate from IO allows such platforms to
provide their own IO but not have to rebuild everything in a manner
incompatible with the rest of hackage.

> As for providing instances for the likes of Binary, there are good reasons
> not to do that in the core network package, because it will drag in
> dependencies on quite a few third-party packages that are either not in core
> (binary) or have questions over their futures (Parsec).

I understand the issue, but it would be a shame if a new network
package failed to have functional Address parsing - its just such a
basic thing.  I suppose this gives even more weight to a dual package
solution.

Thomas
From johan.tibell at gmail.com  Mon Aug 24 15:48:59 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Mon Aug 24 15:29:06 2009
Subject: Potential Network SIG
In-Reply-To: 
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> 
	
Message-ID: <90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>

On Mon, Aug 24, 2009 at 8:00 PM, Bryan O'Sullivan wrote:
> On Fri, Aug 21, 2009 at 9:49 PM, Thomas DuBuisson
>  wrote:
>> 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency.
>
> That's already a step up from the lowest-level bindings, which should be
> using Ptr a.

My plan for network-bytestring has always been to offer a
Network.Socket.Buffer with function such as 'recvInto' that work on
buffers. The higher level bytestring interface can then be implemented
in terms of this interface.

I've been struggling a bit with how to expose less frequently used
functionality in the BSD socket API such as the 'flags' parameter to
'send'. It feels a bit unfortunate to have to pass a "no flags" value
in the most frequent case. Optional keyword parameters would be useful
here.

>> 4) Support more features
>> Features such as Multicast, Header inclusion (IP_HDRINCL), address
>> binding, etc. ?IOW, most the IP_ and SO_ options of socket (7) and ip
>> (7) man pages. ?It would be rather nice if we were able to expose
>> these in a friendly way - but with our cross platform concerns that
>> might not be a good idea (e.g. I'm not familiar with windows).
>
> Providing Network.Windows and Network.Linux and Network.BSD etc modules
> would work fine for non-portable platform-specific features (of which there
> are many).

I think this makes sense. I read some slides from a presentation on
the next Java file API. The realization the Java community seemed to
have come to is that you need to offer APIs with both a cross platform
part and a platform specific part, preferably in such a way that both
can coexist somewhat peacefully (i.e. not two completely separate
class hierarchies).

Cheers,

Johan
From byorgey at seas.upenn.edu  Mon Aug 24 16:21:19 2009
From: byorgey at seas.upenn.edu (Brent Yorgey)
Date: Mon Aug 24 16:01:06 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: 
References: 
Message-ID: <20090824202119.GA28087@seas.upenn.edu>

> +-- | The 'justIf' function is a shortcut for the idiom
> +-- @if a then Just v else v@, which can be written as @v `justIf` a@.

Just wanted to point out that there's a typo in the comment: surely
you mean @if a then Just v else Nothing@.

-Brent
From bos at serpentine.com  Mon Aug 24 17:22:12 2009
From: bos at serpentine.com (Bryan O'Sullivan)
Date: Mon Aug 24 17:02:01 2009
Subject: Potential Network SIG
In-Reply-To: <90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
	<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
Message-ID: 

On Mon, Aug 24, 2009 at 12:48 PM, Johan Tibell wrote:

>
> My plan for network-bytestring has always been to offer a
> Network.Socket.Buffer with function such as 'recvInto' that work on
> buffers. The higher level bytestring interface can then be implemented
> in terms of this interface.
>

How would a Buffer differ from a combination of Ptr a and CSize? (My implied
argument is that it shouldn't.)

I've been struggling a bit with how to expose less frequently used
> functionality in the BSD socket API such as the 'flags' parameter to
> 'send'. It feels a bit unfortunate to have to pass a "no flags" value
> in the most frequent case. Optional keyword parameters would be useful
> here.
>

send vs sendWithFlags would be workable, since we're not going to get
keyword or optional arguments in a happy-making form in the language any
time soon. Of course, just making people use defaultFlags (as getAddrInfo
does) isn't too awful, either.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/libraries/attachments/20090824/803029fe/attachment.html
From duncan.coutts at worc.ox.ac.uk  Mon Aug 24 17:59:06 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Mon Aug 24 17:38:16 2009
Subject: Potential Network SIG
In-Reply-To: <90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	<90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com>
Message-ID: <1251151146.30910.7124.camel@localhost>

On Mon, 2009-08-24 at 09:59 +0200, Johan Tibell wrote:

> > 6) Integrate with the rest of hackage.
> > This means instance of PrettyClass, Parsec, and Binary.  I noticed a
> > number of parsing utilities for IP addresses - lots of duplicated
> > effort here.
> 
> I must disagree here. We don't want hard coupling to lots of other
> libraries. i.e. we don't need convenience functions like:
> 
> get :: Socket -> Binary a
> 
> These ends are better reach by the client writing the glue code. It's
> easy to get a combinatorial explosion in the number of functions if we
> try to provide an integration point for each library out there.


Yes, I think this is a trap that we often fall into, that we think we
make libraries better by providing exactly the one function that people
need, in some kind of "high level" "all in one" operation.

As an example, I'm going to pick on Don because I know he can take
it ;-).

The download package provides:

openURI :: String -> IO (Either String ByteString)

which is great, but also these:

openAsTags :: String -> IO (Either String [Tag])
openAsXML  :: String -> IO (Either String [Content])
openAsFeed :: String -> IO (Either String Feed)

which are all instances of (roughly) this pattern:

openAsFoo = Foo.parseFoo `fmap` openURI

The point is:


  *** don't fear the composition!! ***


Let the user / caller do the composition. The API is simpler and easier
to understand if the user does the composition rather than providing
every use case. Also the package dependencies are much simpler.

Sure, point out in the docs that the user could use it that way with
other libs but don't provide every composition in the name of
convenience.

A Microsoft blogger has a nice short (and rather sarky) article on this
topic:

"Programming means that sometimes you have to snap two blocks together"
http://blogs.msdn.com/oldnewthing/archive/2009/08/04/9856634.aspx


Duncan

From igloo at earth.li  Mon Aug 24 19:00:08 2009
From: igloo at earth.li (Ian Lynagh)
Date: Mon Aug 24 18:39:56 2009
Subject: Recommendation for the procedure to add platform packages
In-Reply-To: <1251053363.30910.4813.camel@localhost>
References: <1250733860.9379.2142.camel@localhost>
	<20090821012355.GA24225@matrix.chaos.earth.li>
	<1250884736.30910.1484.camel@localhost>
	<20090822001735.GA17487@matrix.chaos.earth.li>
	<1251053363.30910.4813.camel@localhost>
Message-ID: <20090824230008.GA3992@matrix.chaos.earth.li>

On Sun, Aug 23, 2009 at 07:49:23PM +0100, Duncan Coutts wrote:
> 
> The intro says:
> 
>       * The rationale section gives an explanation and justification for
>         the policy.
> 
> Is there a wording that would make it clearer that you do not have to
> read it unless you want an explanation for a point in the policy?

I didn't read that line. I'll expand on this (and the other questions
about process description length) in a reply to your other e-mail.

> > Drifting off-topic, but I think that the two discussions are orthogonal
> > (i.e. we could have them in either order), but the "detailed
> > requirements" discussion is the more important one. People can make
> > proposals without a process being nailed down (and that may even help us
> > work out what the process should be), but how can we decide if a package
> > should be included if we don't know what the acceptance criteria are?
> 
> The acceptance criteria is clearly specified.

Well, it says (somewhat tautologically):

    A package is considered as accepted if, by the discussion deadline,
    the libraries mailing list reach consensus to accept it.

but I don't see how we can reach consensus about a package that (for
example) is not -Wall clean if we haven't decided whether or not all
packages in the platform must be -Wall clean.

> It is not that a set of
> minimal requirements be met. It is that the reviewers, using their
> judgement come to a consensus to accept the package.

I agree that some of the requirements will be fuzzy (such as "good
API"), and we may even overlook black-and-white requirements in
particular cases if there is a good reason.

> > > Would those things increase work for the proposal
> > > authors / package maintainers?
> > 
> > It should reduce work for the authors / package maintainers.
> 
> Even taking into account that you would expect them to write these
> external documents? Where is the reduction coming from?

Like you said at the start of your mail, "the 'tar' example could be
improved and shortened", so to some extent at least we are in agreement,
and the issue was just that the example was written for an earlier, more
rigid process.

I suspect that I would say that it could be shortened even more than you
are thinking, with more of your paragraphs being condensed into a line
in the checklist (in the common case where the package obviously
satisfies the criteria).

> Less text in the proposal or less text in total?

A mixture of both.


Thanks
Ian

From igloo at earth.li  Mon Aug 24 19:18:26 2009
From: igloo at earth.li (Ian Lynagh)
Date: Mon Aug 24 18:58:12 2009
Subject: Recommendation for the procedure to add platform packages
In-Reply-To: <1251055877.30910.4928.camel@localhost>
References: <1250733860.9379.2142.camel@localhost>
	<20090821012355.GA24225@matrix.chaos.earth.li>
	<1250884736.30910.1484.camel@localhost>
	<20090822001735.GA17487@matrix.chaos.earth.li>
	<1251055877.30910.4928.camel@localhost>
Message-ID: <20090824231826.GB3992@matrix.chaos.earth.li>

On Sun, Aug 23, 2009 at 08:31:17PM +0100, Duncan Coutts wrote:
> 
>         Passing all requirements would be the
>         criteria for inclusion.

I would say "Passing all requirements would be sufficient for
inclusion", but we may decide to include packages that don't meet all
the requirements.

>       * That the policy document itself is too long and too detailed.
>         
>         That it anticipates eventualities that may or may not arise in
>         practise.
>         
>         That the overall length is a problem because it is off-putting,
>         with the danger that people simply will not read it.

Right. I was admittedly reading it at ~2am, but IIRC the way I read it
was:

* Hmm, the widget on the scrollbar is very small. Reading this is going
  to take a while.
* Drag widget down a bit. See "[note-6.3]". Hmm, jumping up and down
  between the text and all the notes is really going to break the flow
  if I try and read it properly.
* Drag widget down more. See sections like "Acceptance", "Proposal
  content" and "Package requirements", not realising I was looking at
  rationale.
* Drag widget down more, and see the "Achieving consensus" major
  section. Boggle at the size as compared to the library submissions
  proposal.
* Read a little text, probably from the early "Procedure" section.

By contrast, I would probably have read this:
http://trac.haskell.org/haskell-platform/wiki/AddingPackagesCore


If you do want to keep the rationale in the same document then making it
expandable inline with JavaScript may be better. (It would still work in
non-JS browsers, but be very verbose in them).

> Ian, is this a fair summary? Is there anything I've missed that I didn't
> address specifically in the previous email?

Yes, I think it's a fair summary and covers everything.


Thanks
Ian

From naesten at gmail.com  Mon Aug 24 20:36:31 2009
From: naesten at gmail.com (Samuel Bronson)
Date: Mon Aug 24 20:16:16 2009
Subject: ioToST
Message-ID: 

Dear libraries@...,

Shouldn't Control.Monad.ST export ioToST :: IO a -> ST RealWorld a, as
defined in GHC.IO?

Hugs for some reason does not have it, but it would seem trivial to
write, given their stToIO:

stToIO              :: ST RealWorld a -> IO a
stToIO (ST f)        = IO f

It doesn't look like anyone else even *has* ST...
From bulat.ziganshin at gmail.com  Mon Aug 24 23:36:18 2009
From: bulat.ziganshin at gmail.com (Bulat Ziganshin)
Date: Mon Aug 24 23:16:09 2009
Subject: ioToST
In-Reply-To: 
References: 
Message-ID: <9410602916.20090825073618@gmail.com>

Hello Samuel,

Tuesday, August 25, 2009, 4:36:31 AM, you wrote:

> Shouldn't Control.Monad.ST export ioToST :: IO a -> ST RealWorld a, as
> defined in GHC.IO?

no. ST is a subset of IO monad with a limited set of operators
guaranteeing referential transparence, as a result runST is a pure
operation. if you provide ability to run in ST monad arbitrary
operations, this guarantee will break. there are pother way to do the
same - use IO operations and unsafePerformIO


-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin@gmail.com

From s.clover at gmail.com  Mon Aug 24 23:49:00 2009
From: s.clover at gmail.com (Sterling Clover)
Date: Mon Aug 24 23:28:35 2009
Subject: Potential Network SIG
In-Reply-To: <90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
	<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
Message-ID: <771B85EE-5D41-4C56-8275-6D62F67932EE@gmail.com>

The hardest part of network operations, I've found, is handling the  
myriad of exceptional conditions safely, in such a way that there's  
control over the number of sockets used and the timeouts in  
connections (each step potentially needs its own timeout, in fact,  
and one has to be careful with the interaction of asynchronous  
exceptions here, or move to a different sort of model) while also  
providing strong guarantees that resources won't leak. I.e. depending  
on the stage of the connection process and type of exception,  
different resources need to be freed.

network-fancy seems to be on the right track here, but I think  
withSocket should absolutely take an optional timeout parameter (not  
for the whole action, just for the connection). I'm also not too sure  
that it won't leak as currently written -- i.e. if there's an error  
in the connection loop itself, is the socket freed?

Of course something depending on the socket could still be returned  
out of withSocket even after its underlying socket was closed. On the  
other hand, if users really wanted guarantees, I think the signature  
of withSocket is expressive enough that iteratees could be used  
within the continuation it was passed. Lightweight monadic regions  
would also be possible as an alternate API on top of this, as a  
separate lib.

The server functions for network-fancy also seem pretty good.  
Although it would be nice to have an MVar or other mechanism to shut  
down the servers cleanly, rather than relying on asynchronous  
exceptions, which might cause odd interactions. In particular,  
listening on sockets can likely block exceptions, which means that if  
you want to be able to shut down a server, you need to throw it an  
exception, and then ping it to force the listen call to return.  
Either we need some nonblocking way to listen, or that functionality  
needs to be wrapped up in some neat abstraction.

Speaking of which, are we sure about what calls should be unsafe and  
safe? The small price of marking things unsafe can lead to big  
problems with blocking thread context switches, and thus grinding the  
rest of an app to a halt. A key criteria for the network library, in  
my opinion, should be that no network call on a given socket and  
thread blocks any other work of the program, including another  
network call on another socket and thread.

Additionally, while sockets are too nitty-gritty for a high-level  
interface (although a socket with phantom types might not be),  
Handles feel to me to be too tricky to reliably map onto, and I'd  
much prefer something in between. Perhaps  an explicit pair of a lazy  
bytestring and a function of type (ByteString -> IO ()) (both, of  
course, properly buffered.)?

Of course, this doesn't map onto some of the multicast and datagram  
type functionality, which would need other abstractions.

Cheers,
S.

On Aug 24, 2009, at 3:48 PM, Johan Tibell wrote:

> On Mon, Aug 24, 2009 at 8:00 PM, Bryan  
> O'Sullivan wrote:
>> On Fri, Aug 21, 2009 at 9:49 PM, Thomas DuBuisson
>>  wrote:
>>> 3) Use Bytestrings (and have corrosponding .Lazy modules) for  
>>> efficiency.
>>
>> That's already a step up from the lowest-level bindings, which  
>> should be
>> using Ptr a.
>
> My plan for network-bytestring has always been to offer a
> Network.Socket.Buffer with function such as 'recvInto' that work on
> buffers. The higher level bytestring interface can then be implemented
> in terms of this interface.
>
> I've been struggling a bit with how to expose less frequently used
> functionality in the BSD socket API such as the 'flags' parameter to
> 'send'. It feels a bit unfortunate to have to pass a "no flags" value
> in the most frequent case. Optional keyword parameters would be useful
> here.
>
>>> 4) Support more features
>>> Features such as Multicast, Header inclusion (IP_HDRINCL), address
>>> binding, etc.  IOW, most the IP_ and SO_ options of socket (7)  
>>> and ip
>>> (7) man pages.  It would be rather nice if we were able to expose
>>> these in a friendly way - but with our cross platform concerns that
>>> might not be a good idea (e.g. I'm not familiar with windows).
>>
>> Providing Network.Windows and Network.Linux and Network.BSD etc  
>> modules
>> would work fine for non-portable platform-specific features (of  
>> which there
>> are many).
>
> I think this makes sense. I read some slides from a presentation on
> the next Java file API. The realization the Java community seemed to
> have come to is that you need to offer APIs with both a cross platform
> part and a platform specific part, preferably in such a way that both
> can coexist somewhat peacefully (i.e. not two completely separate
> class hierarchies).
>
> Cheers,
>
> Johan
> _______________________________________________
> Libraries mailing list
> Libraries@haskell.org
> http://www.haskell.org/mailman/listinfo/libraries

From dan.doel at gmail.com  Mon Aug 24 23:59:18 2009
From: dan.doel at gmail.com (Dan Doel)
Date: Mon Aug 24 23:39:09 2009
Subject: ioToST
In-Reply-To: <9410602916.20090825073618@gmail.com>
References: 
	<9410602916.20090825073618@gmail.com>
Message-ID: <200908242359.18474.dan.doel@gmail.com>

On Monday 24 August 2009 11:36:18 pm Bulat Ziganshin wrote:
> Hello Samuel,
>
> Tuesday, August 25, 2009, 4:36:31 AM, you wrote:
> > Shouldn't Control.Monad.ST export ioToST :: IO a -> ST RealWorld a, as
> > defined in GHC.IO?
>
> no. ST is a subset of IO monad with a limited set of operators
> guaranteeing referential transparence, as a result runST is a pure
> operation. if you provide ability to run in ST monad arbitrary
> operations, this guarantee will break. there are pother way to do the
> same - use IO operations and unsafePerformIO

The proposed ioToST yields a value of type 'ST RealWorld a', which is not 
usable with runST, as it is not sufficiently polymorphic, and thus cannot be 
used to write unsafePerformIO.

The only way you can eventually do anything with a value of the above type 
(short of unsafeCoerce and such) is to use stToIO to put it back into IO, and 
run it from main.

-- Dan
From taruti at taruti.net  Tue Aug 25 03:39:03 2009
From: taruti at taruti.net (taruti)
Date: Tue Aug 25 03:18:50 2009
Subject: Potential Network SIG
In-Reply-To: <4c44d90b0908241134l70879fdcmaddb09638feccafb@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
	<4c44d90b0908241134l70879fdcmaddb09638feccafb@mail.gmail.com>
Message-ID: <1251184096-sup-274@oz.taruti.net>

Excerpts from Thomas DuBuisson's message of Mon Aug 24 21:34:01 +0300 2009:
> I Object!  Code that uses addresses _might_ be very specific to IPv4
> or IPv6 - such code should be able to restrict itself to the proper
> category of addresses, headers etc without resorting to partial
> functions.  I propose a solution akin to:
> 
>     class Address a where ...
>     instance Address IPv4
>     instance Address IPv6

Also we could have

data ConnectOptions t where ...
instance Address t => Address (ConnectOptions t) where ...

> I understand the issue, but it would be a shame if a new network
> package failed to have functional Address parsing - its just such a
> basic thing.  I suppose this gives even more weight to a dual package
> solution.

IP-address parsing does not need Parsec. It is really quite simple.

Which type system extensions can we use? e.g. GADTs are attractive
but not supported by Hugs...

Here is a little bit of formulation for a possible high-level API.

Connections 

> type HostName = String
> type Port     = Word16

> data IPv4 = IPv4 HostName Port
> data IPv6 = IPv6 HostName Port
> data IP   = IP HostName Port
> data Unix = Unix FilePath
> data AddressOptions t where ...

> data Stream
> data Packet

> class Address t where
>   connectStream :: t -> IO (Socket Stream)
>   connectPacket :: t -> IO (Socket Packet)

> instance Connect IPv4 where ...
> instance Connect IPv6 where ...
> instance Connect IP   where ...
> instance Connect (AddressOptions t) where ...
> instance Connect Unix where ...
>
> withStream :: Address a => a -> (Socket Stream -> IO a) -> IO a
> withPacket :: Address a => a -> (Socket Packet -> IO a) -> IO a

> socketToHandle :: Socket Stream -> IO Handle

Sending and receiving

> -- | Send a bytestring
> send :: Socket t -> L.ByteString -> IO ()
> -- | Receive a bytestring with a maximum length
> recv :: Socket t -> Int -> IO L.ByteString
> -- | Receive from
> --   FIXME SomeAddress here is a kludge, what would be better?
> recvFrom :: Socket t -> Int -> IO (L.ByteString, SomeAddress)
> close :: Socket t -> IO ()
> sendTo :: Address a => Socket Packet -> a -> L.ByteString -> IO ()

Server API

> data ServerOptions t where ...
> -- FIXME
> server :: ServerOptions t -> (Socket t -> SomeAddress -> IO ()) -> IO Foobar

Miscellaneous highlevel api

> sleepForever :: IO ()
> getCurrentHost :: IO HostName
> nameToNumeric :: Address a => a -> IO a
> numericToName :: Address a => a -> IO a

- Taru Karttunen
From taruti at taruti.net  Tue Aug 25 03:54:39 2009
From: taruti at taruti.net (taruti)
Date: Tue Aug 25 03:34:24 2009
Subject: Potential Network SIG
In-Reply-To: 
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
Message-ID: <1251186675-sup-2085@oz.taruti.net>

Excerpts from Yusaku Hashimoto's message of Mon Aug 24 19:46:46 +0300 2009:
> I was trying to add another type safety for socket operations, and you
> can get it by ` darcs get
> http://patch-tag.com/r/phantom-socket/pullrepo phantom-socket`.
> 

I think we should not have these operations.
A) it makes it hard to pass sockets between threads
B) the state distinction is useful only for servers and servers could 
   be handled much better with a record and hiding the operations 
   from users.

- Taru Karttunen
From johan.tibell at gmail.com  Tue Aug 25 04:10:41 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Tue Aug 25 03:50:46 2009
Subject: Potential Network SIG
In-Reply-To: 
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> 
	 
	<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com> 
	
Message-ID: <90889fe70908250110q14b39da7w624da736fe20cd0f@mail.gmail.com>

On Mon, Aug 24, 2009 at 11:22 PM, Bryan O'Sullivan wrote:
> On Mon, Aug 24, 2009 at 12:48 PM, Johan Tibell 
> wrote:
>>
>> My plan for network-bytestring has always been to offer a
>> Network.Socket.Buffer with function such as 'recvInto' that work on
>> buffers. The higher level bytestring interface can then be implemented
>> in terms of this interface.
>
> How would a Buffer differ from a combination of Ptr a and CSize? (My implied
> argument is that it shouldn't.)

It wouldn't. A raw Ptr a and a size (but possibly not CSize?) was what
I had in mind.

recvInto :: Socket -> Ptr a -> CSize -> IO ()

(Or perhaps just call it 'recv' since it would live in a separate module.)

>> I've been struggling a bit with how to expose less frequently used
>> functionality in the BSD socket API such as the 'flags' parameter to
>> 'send'. It feels a bit unfortunate to have to pass a "no flags" value
>> in the most frequent case. Optional keyword parameters would be useful
>> here.
>
> send vs sendWithFlags would be workable, since we're not going to get
> keyword or optional arguments in a happy-making form in the language any
> time soon. Of course, just making people use defaultFlags (as getAddrInfo
> does) isn't too awful, either.

sendWithFlags might be the best solution given what we have to work
with. It would give us twice as many functions in the
send/recv/sendTo/recvFrom part of the API which is unfortunate. If
there were more than one optional parameter I would, as you mentioned,
go with a record type for the options as it's more extensible (i.e.
doesn't break user code as long as the user only uses the accessors)
and doesn't increase the number of functions.

-- Johan
From johan.tibell at gmail.com  Tue Aug 25 04:49:43 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Tue Aug 25 04:29:48 2009
Subject: Potential Network SIG
In-Reply-To: <1251151146.30910.7124.camel@localhost>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> 
	<90889fe70908240059s45f153e4m7f07a47c371fd733@mail.gmail.com> 
	<1251151146.30910.7124.camel@localhost>
Message-ID: <90889fe70908250149m5baa2dabn80604b3424e7a061@mail.gmail.com>

On Mon, Aug 24, 2009 at 11:59 PM, Duncan
Coutts wrote:
> On Mon, 2009-08-24 at 09:59 +0200, Johan Tibell wrote:
> Yes, I think this is a trap that we often fall into, that we think we
> make libraries better by providing exactly the one function that people
> need, in some kind of "high level" "all in one" operation.
>
> As an example, I'm going to pick on Don because I know he can take
> it ;-).

I think I mentioned this to Don months ago when I decided to against
using the library due to all the dependencies. :)

> Let the user / caller do the composition. The API is simpler and easier
> to understand if the user does the composition rather than providing
> every use case. Also the package dependencies are much simpler.
>
> Sure, point out in the docs that the user could use it that way with
> other libs but don't provide every composition in the name of
> convenience.
>
> A Microsoft blogger has a nice short (and rather sarky) article on this
> topic:
>
> "Programming means that sometimes you have to snap two blocks together"
> http://blogs.msdn.com/oldnewthing/archive/2009/08/04/9856634.aspx

Another example: At work we have a File class that has a whopping 20
methods for opening a file. Most methods are combinations of different
ways to do error handling, locking, setting attributes and options.
The whole class has more than 100 methods!

I've seen arguments on the form

    Lots of people write programs that contains the function foo = bar
. baz. Therefor we should put foo in library X.

on the libraries list lately. I think this argument is bogus most of
the time. If you want 'foo', stick it at the top of your module. New
functions should add functionality that is difficult to express using
the current set of functions in the library.

-- Johan
From johan.tibell at gmail.com  Tue Aug 25 05:01:28 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Tue Aug 25 04:41:35 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: <20090824202119.GA28087@seas.upenn.edu>
References: 
	<20090824202119.GA28087@seas.upenn.edu>
Message-ID: <90889fe70908250201t3b68e4feu388448c66e39d48@mail.gmail.com>

Hi,

Inspired by a discussion we've been having around the network package
and its API I would like to ask: Are all these small little functions
really worth it?

They're trivially defined using the current API and they make the
current API bigger and hence more complicated. Is there a way to
figure out which are the most general functions expose them and let
developers use composition to define the rest?

Just a thought,

Johan
From gale at sefer.org  Tue Aug 25 07:11:38 2009
From: gale at sefer.org (Yitzchak Gale)
Date: Tue Aug 25 06:51:43 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: <90889fe70908250201t3b68e4feu388448c66e39d48@mail.gmail.com>
References: 
	<20090824202119.GA28087@seas.upenn.edu> 
	<90889fe70908250201t3b68e4feu388448c66e39d48@mail.gmail.com>
Message-ID: <2608b8a80908250411v225c697dk9d88bb38aff963c9@mail.gmail.com>

Johan Tibell wrote:
> Are all these small little functions
> really worth it?
> They're trivially defined using the current API...
> let developers use composition to define the rest?

I completely agree with you. But in this case, even
though these functions really are trivial, they come
up *all* the time, and there is just something awkward
about them.

We certainly won't add more than one of the two:

justIf x b = guard b >> return x
check p x = guard (p x) >> return x

justIf = flip $ check . const
check = ap justIf

Perhaps the following is a bit of an explanation
of why they feel awkward:

 @pl \ p x -> guard (p x) >> return x
 (`ap` return) . (((>>) . guard) .)

 @pl \ p x -> if p x then return x else mzero
 flip flip mzero . (`ap` return) . (if' .)

Please note that if we choose to add justIf, it should
be spelled with an uppercase 'I' to be consistent with
Haskell conventions.

Thanks,
Yitz
From marlowsd at gmail.com  Tue Aug 25 07:32:00 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Tue Aug 25 07:11:51 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: 
References: 	<1250953837.30910.2729.camel@localhost>
	
Message-ID: <4A93CBB0.9030706@gmail.com>

On 22/08/2009 16:22, Jon Fairbairn wrote:
> Duncan Coutts  writes:
>
>> On Sat, 2009-08-22 at 10:18 +0200, Joachim Breitner wrote:
>>> Hi,
>>>
>>> I propose the attached change
>>
>>> Fri Aug 21 19:17:12 CEST 2009  Joachim Breitner
>>>    * Add justIf to Data.Maybe
>
>> As J?n was suggesting we might want to generalise to MonadPlus rather
>> than putting it in Data.Maybe, it becomes rather similar to the existing
>> function 'guard':
>
> [...] some alpha conversion
>>
>> checkA :: MonadPlus m =>  Bool ->  a ->  m a
>> checkA True  x  = return x
>> checkA False _  = mzero
>>
>> Using a Bool for the condition is a tad more general than a predicate
>> like:
>>
>> checkB :: MonadPlus m =>  (a ->  Bool) ->  a ->  m a
>
> I'd say the generality comparison goes the other way:
>
>> In particular, the use case in Cabal does not fit the predicate pattern
>> because the condition is not a condition on the value being returned but
>> on something else in the environment.
>
> But you can write any
>
>     checkA e
> as
>     checkB $ const e

Right, but  checkB p x = checkA (p x) x.

Also, checkA p = (guard p >>) . return

> whereas a function like
>
>     all_spaces = checkB (all . isSpace)

I think that for consistency with if/then/else and guard we should use 
checkA.  It's not that hard to write

      all_spaces xs = checkA (all (isSpace xs)) xs

if that's really what you want.

I wouldn't call checkB more "general"; to me it's just a minor 
convenience issue, which is outweighed by consistency and simplicity.

> is hard to write using checkA (suppose you are doing something like ?map
> (checkB predicate)?) since checkA doesn't look at the second argument.

the "applying check to a list" case is not hard either:

   zipWithM checkA (map p xs) xs

again, if that's what you want to do.

Cheers,
	Simon
From marlowsd at gmail.com  Tue Aug 25 08:03:21 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Tue Aug 25 07:43:13 2009
Subject: Potential Network SIG
In-Reply-To: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
Message-ID: <4A93D309.3030308@gmail.com>

On 22/08/2009 05:49, Thomas DuBuisson wrote:
> Hello All,
>
> If you are CCed it's because you are listed as a maintainer of a
> network-* package that I consider related to the Haskell network
> library.  I'm hoping to roll much of the functionality of
> network-{bytestring, multicast, fancy} etc into a single package that
> the community will agree on (namely, "network").
>
> Johan suggested starting a SIG to hammer out a design for a new
> Network API seeing as the current API, a straight-forward Berkeley
> binding, doesn't seem to please anyone in a Haskell context.  If you
> want to partake then this e-mail if your heads up.  If there is some
> formal method of setting up a Haskell SIG then please let me know.
>
>
> My thoughts on some important parts are below - I'm sure not everyone
> will agree as these thoughts directly contradict some designs found in
> current libraries.
>
> 1) Separate low level functions / bindings from high level /
> productive code by placing each in different modules.
>
> The low level bindings should remain available for those cases we fail
> to have the needed functionality in our high level packages.  That
> said, I'm hoping to cover more than the 80% of users with any new
> design.
>
> 2) Maintain type safety by using type classes for most things.
>
> Unlike Network.Fancy and Network.Socket (which have IPv4 and IPv6 as
> constructors of the same data type), I think we should allow for the
> possibility that some users of the library will be limited to just one
> IP version without resorting to partial functions.  I suggest type
> classes to cover this aspect (class Address, class Port, etc).
>
> 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency.
> As in network-bytestring, any new API should be performance concious
> enough to avoid String.

Idealogically speaking, this is not a choice you should make in the 
network library.  The network library should deal with setting up 
sockets, and delegate the actual I/O to the I/O library.

Right now, that means making Handles from Sockets (which is something 
the current network library provides).  And then you use the bytestring 
library to write bytestrings to the Handle.  In the future we'll have a 
way to write text to a Handle too.

Now, I wouldn't be surprised if this doesn't cover all the use cases. 
Maybe people want to use the low-level send/recv.  But I expect that for 
most applications, going via Handle will be the right thing, and we 
should look at how to accommodate the other use cases.

Cheers,
	Simon
From marlowsd at gmail.com  Tue Aug 25 08:07:27 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Tue Aug 25 07:47:18 2009
Subject: Potential Network SIG
In-Reply-To: <771B85EE-5D41-4C56-8275-6D62F67932EE@gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>		<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
	<771B85EE-5D41-4C56-8275-6D62F67932EE@gmail.com>
Message-ID: <4A93D3FF.4010002@gmail.com>

On 25/08/2009 04:49, Sterling Clover wrote:

> Additionally, while sockets are too nitty-gritty for a high-level
> interface (although a socket with phantom types might not be), Handles
> feel to me to be too tricky to reliably map onto, and I'd much prefer
> something in between. Perhaps an explicit pair of a lazy bytestring and
> a function of type (ByteString -> IO ()) (both, of course, properly
> buffered.)?

Could you elaborate a bit more on "Handles feel to me to be too tricky 
to reliably map onto"?  I'd like to understand what problems there are, 
and see if there's some way to address them.

Cheers,
	Simon
From marlowsd at gmail.com  Tue Aug 25 08:10:09 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Tue Aug 25 07:50:00 2009
Subject: Proposal #3455: Add a setting to change how Unicode encoding
 errors are handled
In-Reply-To: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com>
References: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com>
Message-ID: <4A93D4A1.60602@gmail.com>

On 23/08/2009 17:22, Judah Jacobson wrote:
> I proposal that we augment ghc-6.12.1's support for Unicode Handles
> by adding the following functions to System.IO:
>
> hSetOnEncodingError :: Handle ->  OnEncodingError ->  IO ()
> hGetOnEncodingError :: Handle ->  IO OnEncodingError
>
> as well as the enumeration `OnEncodingError` with three constructors:
>
>   - `ThrowEncodingError`: Throw an exception at the first encoding or
>   decoding
>     error.
>   - `SkipEncodingError`: Skip all invalid bytes or characters.
>   - `TranslitEncodingError`: Replace undecodable bytes with u+FFFD, and
>   unencodable characters with '?'.
>
> I have implemented this functionality in a patch attached to the
> ticket.  Haddock docs
> are here:
> http://code.haskell.org/~judah/new-io-docs/System-IO.html#23
>
>
> The choice of error handler is orthogonal to the choice of encoder.
> Additionally, the same setting is used for both read and write modes.  For
> portability, the handlers are written in pure Haskell rather than using
> GNU iconv's //TRANSLIT feature.
>
> Note that the text package, for example, provides more sophisticated
> error-handling options.  However, I think the above choices are useful
> enough without making the API too complicated.

I replied on the ticket, reproduced here for readers of libraries@:

It looks like the main question here is whether the IOError should be 
returned explicitly (as in your patch), or whether we should just catch 
the exception. All things being equal, catching the exception would be 
simpler, as it wouldn't require any changes in the codecs. Is there a 
reason why you didn't do it that way? Perhaps because you want to be 
sure that the exception is really an encoding error, and not some other 
kind of exception? If that's the case, then we should introduce a new 
exception for encoding errors (that's probably a good idea anyway).

Cheers,
	Simon
From marlowsd at gmail.com  Tue Aug 25 08:18:15 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Tue Aug 25 07:58:06 2009
Subject: Binding to inline functions
In-Reply-To: <1251056058.30910.4937.camel@localhost>
References: 
	<1251056058.30910.4937.camel@localhost>
Message-ID: <4A93D687.4050001@gmail.com>

On 23/08/2009 20:34, Duncan Coutts wrote:
> On Sun, 2009-08-23 at 15:09 -0300, Maur??cio CA wrote:
>> I understand we can't use 'foreign import ccall'
>> to wrap inline C functions. Do you think it could
>> be possible to have an option in cabal to generate
>> such functions in an object file when #included in
>> a C file, in a compiler independent, portable way?
>
> It might be better to include this feature into hsc2hs and/or c2hs
> (which may in turn require some help from Cabal).

I think it would be easy to do in GHC.  We already have the machinery to 
generate the _stub.c files and compile them.

The main question is what the syntax should look like.  I was toying with

foreign import capi "foo" foo :: ...

where "capi" means the C API as opposed to the ABI, which is what ccall 
means.  This expresses the meaning without saying anything about the 
implementation - implementations that compile via C might not need a 
separate wrapper, so something like 'foreign import ccall "cwrapper 
foo"' would be wrong.

I wonder whether not being able to specify the actual calling convention 
will be a problem, however.  We are relying on the calling convention of 
foo being implicit.  Are there cases where this might cause a problem?

Cheers,
	Simon
From lemming at henning-thielemann.de  Tue Aug 25 08:30:35 2009
From: lemming at henning-thielemann.de (Henning Thielemann)
Date: Tue Aug 25 08:10:52 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: 
References: 
Message-ID: 


On Sat, 22 Aug 2009, Joachim Breitner wrote:

> Hi,
>
> I propose the attached change, with a discussion timeframe until end of
> September. The corresponding ticket is
> http://hackage.haskell.org/trac/ghc/ticket/3446

I proposed the same with reversed parameter order and the name 'toMaybe' 
years ago and it was rejected.

http://www.haskell.org/pipermail/libraries/2004-July/002381.html

However, I have added it to my own utility library.

http://code.haskell.org/~thielema/utility/src/Data/Maybe/HT.hs
From mauricio.antunes at gmail.com  Tue Aug 25 08:45:28 2009
From: mauricio.antunes at gmail.com (=?ISO-8859-1?Q?Maur=ED=ADcio_CA?=)
Date: Tue Aug 25 08:25:36 2009
Subject: Binding to inline functions
In-Reply-To: <4A93D687.4050001@gmail.com>
References: 	<1251056058.30910.4937.camel@localhost>
	<4A93D687.4050001@gmail.com>
Message-ID: 

>>> I understand we can't use 'foreign import ccall'
>>> to wrap inline C functions. Do you think it could
>>> be possible to have an option in cabal to generate
>>> such functions in an object file when #included in
>>> a C file, in a compiler independent, portable way?

> I think it would be easy to do in GHC.  We already have the machinery to 
> generate the _stub.c files and compile them.
> 
> The main question is what the syntax should look like.  I was toying with
> 
> foreign import capi "foo" foo :: ...

Under that syntax, how would GHC know where to find
declaration/definition for a function? Since it's not,
by hypothesis, on an object or library file, a C file
will have to be compiled to get both the declaration
and the definition.

Thanks,
Maur?cio

From bos at serpentine.com  Tue Aug 25 12:18:59 2009
From: bos at serpentine.com (Bryan O'Sullivan)
Date: Tue Aug 25 11:58:44 2009
Subject: Potential Network SIG
In-Reply-To: <4A93D309.3030308@gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	<4A93D309.3030308@gmail.com>
Message-ID: 

On Tue, Aug 25, 2009 at 5:03 AM, Simon Marlow  wrote:

> Right now, that means making Handles from Sockets (which is something the
> current network library provides).  And then you use the bytestring library
> to write bytestrings to the Handle.  In the future we'll have a way to write
> text to a Handle too.
>

I remember seeing notes about the new I/O code being about 20-25% slower
than the old, due to support for character set transcoding. If my
recollection is correct, would that hold true for writing ByteStrings, too?
(Yes, I'm somewhat performance obsessed.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/libraries/attachments/20090825/e46b42f3/attachment.html
From taruti at taruti.net  Tue Aug 25 13:05:28 2009
From: taruti at taruti.net (taruti)
Date: Tue Aug 25 12:45:13 2009
Subject: Potential Network SIG
In-Reply-To: <4A93D309.3030308@gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	<4A93D309.3030308@gmail.com>
Message-ID: <1251219678-sup-7700@oz.taruti.net>

Excerpts from Simon Marlow's message of Tue Aug 25 15:03:21 +0300 2009:
> Now, I wouldn't be surprised if this doesn't cover all the use cases. 
> Maybe people want to use the low-level send/recv.  But I expect that for 
> most applications, going via Handle will be the right thing, and we 
> should look at how to accommodate the other use cases.

For example packet (datagram) sockets cannot use Handles at all since
they need have explicit boundaries between packets. Thus stream
oriented handles don't fit them.

Another problematic case is setting/getting socket options - it is not
possible to tell in the type system whether a particular Handle
represents a socket or not.

Handles will be ok for high-level TCP applications if the performance
is good enough.

- Taru Karttunen
From johan.tibell at gmail.com  Tue Aug 25 16:23:19 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Tue Aug 25 16:03:23 2009
Subject: Potential Network SIG
In-Reply-To: <4A93D309.3030308@gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com> 
	<4A93D309.3030308@gmail.com>
Message-ID: <90889fe70908251323t3caaba87wa9760ff82d176aa6@mail.gmail.com>

On Tue, Aug 25, 2009 at 2:03 PM, Simon Marlow wrote:
> On 22/08/2009 05:49, Thomas DuBuisson wrote:
>> 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency.
>> As in network-bytestring, any new API should be performance concious
>> enough to avoid String.
>
> Idealogically speaking, this is not a choice you should make in the network
> library. ?The network library should deal with setting up sockets, and
> delegate the actual I/O to the I/O library.
>
> Right now, that means making Handles from Sockets (which is something the
> current network library provides). ?And then you use the bytestring library
> to write bytestrings to the Handle. ?In the future we'll have a way to write
> text to a Handle too.
>
> Now, I wouldn't be surprised if this doesn't cover all the use cases. Maybe
> people want to use the low-level send/recv. ?But I expect that for most
> applications, going via Handle will be the right thing, and we should look
> at how to accommodate the other use cases.

In my mind an improved I/O library would look something like this:

> -- At the very bottom is a type class 'RawIO' which represents a
> -- variety of stream-like types.
> class RawIO a where
>     readInto :: Ptr Word8 -> Int -> IO ()
>     write :: ByteString -> IO ()
>
>     read :: Int -> IO ByteString
>     read n = ByteString.createAndTrim n (\p -> readInto p n)

This definition is very minimal and most likely need to be expanded
with operations such as 'close' and perhaps also 'seek'. The methods
would map to the system calls for e.g. files and sockets. A particular
instance could would exceptions for unsupported methods (e.g. a file
opened as read-only would throw exceptions if 'write' is called).

> -- A simple wrapper for file descriptors.
> data File = File CInt
>
> instance RawIO File where
>     readInto = cRead
>     write = cWrite
>
> -- This is only for stream like sockets. Datagram sockets still need to use the lower level API.
> instance RawIO Socket where
>     readInto = cRecv
>     write = send
>
> -- Assuming 'Buffer' is a mutable byte buffer type.
> -- This is useful for e.g. testing. ByteString could also be
> -- made an instance that throws exceptions for unimplemented
> -- methods (e.g. write in this case).
> instance RawIO Buffer where
>    readInto = readFromBufferInto
>    write = writeToBuffer

We can now layer buffering on top.

> -- Buffers for reading and writing are kept in a data type 'BufferedIO'.
> -- This data type need not be exposed.
> data BufferedIO = forall a. RawIO a => BufferedIO Buffer Buffer a
>
> instance RawIO BufferedIO where
>     readInto = readFromBufferInto  -- Calls RawIO.readInto if needed
>     write = writeToBuffer  -- Calls RawIO.write if needed
>
> -- Allocates buffers and returns a BufferedIO
> buffered :: RawIO a => a -> a
> buffered = ...

We might opt for a type class for buffered I/O in case we want to
expose any methods in addition to those exported by RawIO.

We can now layer text I/O on top of buffered I/O:

> class TextIO a where
>     read :: Int -> IO Text
>     write :: Text -> IO ()
>     readLine :: IO Text
>
> -- To do this efficiently BufferedIO might need to expose its buffer.
> -- Alternatively TextIO can be layered directly on top of RawIO and
> -- manage its own buffers.
> text :: (BufferedIO a, TextIO b) => a -> b
> text = ...

A few people have express interest in discussing this at ICFP. Perhaps
we could draft a proposal and put it on a wiki page for others to
review.

-- Johan
From duncan.coutts at worc.ox.ac.uk  Tue Aug 25 17:28:38 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Tue Aug 25 17:07:44 2009
Subject: Potential Network SIG
In-Reply-To: 
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	<4A93D309.3030308@gmail.com>
	
Message-ID: <1251235718.30910.8559.camel@localhost>

On Tue, 2009-08-25 at 09:18 -0700, Bryan O'Sullivan wrote:
> On Tue, Aug 25, 2009 at 5:03 AM, Simon Marlow 
> wrote:
>         
>         Right now, that means making Handles from Sockets (which is
>         something the current network library provides).  And then you
>         use the bytestring library to write bytestrings to the
>         Handle.  In the future we'll have a way to write text to a
>         Handle too.
>         
> 
> I remember seeing notes about the new I/O code being about 20-25%
> slower than the old, due to support for character set transcoding. If
> my recollection is correct, would that hold true for writing
> ByteStrings, too? (Yes, I'm somewhat performance obsessed.)

As far as I know the only slow down is when there is actual text
decoding going on, ie not for ByteString I/O.

Simon has promised to explain all in his upcoming talk at the Haskell
Implementers' Workshop. :-)

That will also be the obvious time to discuss what a new public I/O
interface might look like. There's lots of fun new machinery in GHC's
I/O system but we need to talk about public APIs.

Duncan

From marlowsd at gmail.com  Wed Aug 26 03:47:01 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Wed Aug 26 03:26:53 2009
Subject: Potential Network SIG
In-Reply-To: <1251235718.30910.8559.camel@localhost>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>	
	<4A93D309.3030308@gmail.com>	
	
	<1251235718.30910.8559.camel@localhost>
Message-ID: <4A94E875.1080905@gmail.com>

On 25/08/2009 22:28, Duncan Coutts wrote:
> On Tue, 2009-08-25 at 09:18 -0700, Bryan O'Sullivan wrote:
>> On Tue, Aug 25, 2009 at 5:03 AM, Simon Marlow
>> wrote:
>>
>>          Right now, that means making Handles from Sockets (which is
>>          something the current network library provides).  And then you
>>          use the bytestring library to write bytestrings to the
>>          Handle.  In the future we'll have a way to write text to a
>>          Handle too.
>>
>>
>> I remember seeing notes about the new I/O code being about 20-25%
>> slower than the old, due to support for character set transcoding. If
>> my recollection is correct, would that hold true for writing
>> ByteStrings, too? (Yes, I'm somewhat performance obsessed.)
>
> As far as I know the only slow down is when there is actual text
> decoding going on, ie not for ByteString I/O.

Right - and when I measured the final version, it was actually faster 
than the old version, but only on x86-64 (for reasons unknown).  This 
was measuring hPutStr and hGetContents though, which if you are 
performance-conscious you wouldn't be using anyway.

Bytestring I/O to Handles bypasses the encoding/decoding layer, and if 
the I/O is big enough it also bypasses the intermediate buffer and 
writes the bytes directly from the Bytestring.  All this is unchanged 
relative to the old I/O library in GHC 6.10.

Still, I would expect Handles to have a performance penalty for doing 
lots of small writes, due to the overhead of taking the lock, various 
exception handlers and checking whether the Char buffer needs to be 
flushed.  If you need to do lots of small writes then you'd need to go 
to a lower layer or add another layer of buffering (as GHC does - see 
compiler/utils/BufWrite.hs in the GHC sources).

> Simon has promised to explain all in his upcoming talk at the Haskell
> Implementers' Workshop. :-)
>
> That will also be the obvious time to discuss what a new public I/O
> interface might look like. There's lots of fun new machinery in GHC's
> I/O system but we need to talk about public APIs.

Definitely.

Cheers,
	Simon
From marlowsd at gmail.com  Wed Aug 26 04:13:30 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Wed Aug 26 03:53:22 2009
Subject: Potential Network SIG
In-Reply-To: <90889fe70908251323t3caaba87wa9760ff82d176aa6@mail.gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	<4A93D309.3030308@gmail.com>
	<90889fe70908251323t3caaba87wa9760ff82d176aa6@mail.gmail.com>
Message-ID: <4A94EEAA.9080807@gmail.com>

On 25/08/2009 21:23, Johan Tibell wrote:
> On Tue, Aug 25, 2009 at 2:03 PM, Simon Marlow  wrote:
>> On 22/08/2009 05:49, Thomas DuBuisson wrote:
>>> 3) Use Bytestrings (and have corrosponding .Lazy modules) for efficiency.
>>> As in network-bytestring, any new API should be performance concious
>>> enough to avoid String.
>>
>> Idealogically speaking, this is not a choice you should make in the network
>> library.  The network library should deal with setting up sockets, and
>> delegate the actual I/O to the I/O library.
>>
>> Right now, that means making Handles from Sockets (which is something the
>> current network library provides).  And then you use the bytestring library
>> to write bytestrings to the Handle.  In the future we'll have a way to write
>> text to a Handle too.
>>
>> Now, I wouldn't be surprised if this doesn't cover all the use cases. Maybe
>> people want to use the low-level send/recv.  But I expect that for most
>> applications, going via Handle will be the right thing, and we should look
>> at how to accommodate the other use cases.
>
> In my mind an improved I/O library would look something like this:
>
>> -- At the very bottom is a type class 'RawIO' which represents a
>> -- variety of stream-like types.
>> class RawIO a where
>>      readInto :: Ptr Word8 ->  Int ->  IO ()
>>      write :: ByteString ->  IO ()
>>
>>      read :: Int ->  IO ByteString
>>      read n = ByteString.createAndTrim n (\p ->  readInto p n)

This is quite similar to the class of the same name in the GHC I/O library:

-- | A low-level I/O provider where the data is bytes in memory.
class RawIO a where
   read                :: a -> Ptr Word8 -> Int -> IO Int
   readNonBlocking     :: a -> Ptr Word8 -> Int -> IO (Maybe Int)
   write               :: a -> Ptr Word8 -> Int -> IO ()
   writeNonBlocking    :: a -> Ptr Word8 -> Int -> IO Int

I think the Bytestring API should be a layer on top of this.

> This definition is very minimal and most likely need to be expanded
> with operations such as 'close' and perhaps also 'seek'.

close/seek etc. are methods of the IODevice class in GHC's IO library.

See http://darcs.haskell.org/packages/base/GHC/IO/Device.hs

We have implementations of these classes for file descriptors, and it is 
my intention to have other implementations too: memory-mapped files, 
Windows HANDLEs, Bytestring (for testing), and Chan Word8 (for testing 
again: you write to the Handle, and the decoded bytes come out of the Chan).

These APIs aren't currently "public", in the sense that they are 
exported by modules in the GHC.* hierarchy.  I hope they'll help as a 
concrete start to the discussion of where the I/O library should be 
going, though.

> We can now layer buffering on top.
>
>> -- Buffers for reading and writing are kept in a data type 'BufferedIO'.
>> -- This data type need not be exposed.
>> data BufferedIO = forall a. RawIO a =>  BufferedIO Buffer Buffer a
>>
>> instance RawIO BufferedIO where
>>      readInto = readFromBufferInto  -- Calls RawIO.readInto if needed
>>      write = writeToBuffer  -- Calls RawIO.write if needed
>>
>> -- Allocates buffers and returns a BufferedIO
>> buffered :: RawIO a =>  a ->  a
>> buffered = ...

This is where things get a bit hairy.  The upper layers often want to 
know about the buffer, for instance when it needs to be flushed, or for 
performance reasons - e.g. encoding/decoding needs to have direct access 
to both buffers.  So in GHC's I/O library buffering is a new class 
BufferedIO, consumed by the higher layer, and you can make a BufferedIO 
instance trivially given a RawIO instance.  Not everything has a RawIO 
instance though: memory-mapped files just appear as buffers.

Incedentally, there have been various designs around this theme in the 
past, e.g. http://www.haskell.org/haskellwiki/Library/Streams (with 
various problems IMO, but there are some good ideas there).

Cheers,
	Simon
From marlowsd at gmail.com  Wed Aug 26 04:18:55 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Wed Aug 26 03:58:43 2009
Subject: Binding to inline functions
In-Reply-To: 
References: 	<1251056058.30910.4937.camel@localhost>	<4A93D687.4050001@gmail.com>
	
Message-ID: <4A94EFEF.8000801@gmail.com>

On 25/08/2009 13:45, Maur??cio CA wrote:
>>>> I understand we can't use 'foreign import ccall'
>>>> to wrap inline C functions. Do you think it could
>>>> be possible to have an option in cabal to generate
>>>> such functions in an object file when #included in
>>>> a C file, in a compiler independent, portable way?
>
>> I think it would be easy to do in GHC. We already have the machinery
>> to generate the _stub.c files and compile them.
>>
>> The main question is what the syntax should look like. I was toying with
>>
>> foreign import capi "foo" foo :: ...
>
> Under that syntax, how would GHC know where to find
> declaration/definition for a function? Since it's not,
> by hypothesis, on an object or library file, a C file
> will have to be compiled to get both the declaration
> and the definition.

Yes, absolutely.  GHC will inject a C wrapper function into the _stub.c 
file, compile it, and then call the wrapper from Haskell.

Cheers,
	Simon
From johan.tibell at gmail.com  Wed Aug 26 04:38:30 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Wed Aug 26 04:18:32 2009
Subject: Proposal #3456: Add FilePath -> String decoder
In-Reply-To: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com>
References: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com>
Message-ID: <90889fe70908260138ic8ebd0ap2627e9b5d30e6ccb@mail.gmail.com>

Hi Judah,

A few comments:

- I would spell 'filePathToString' as just 'toString' and use the
module system to provide a namespace (i.e. stick it in
System.FilePath).

- A function that takes the encoding as a parameter instead of
fetching it from the current locale seems more useful. However,
creating such a function is a bit problematic since passing an
explicit encoding doesn't really make sense on Windows where a path is
already represented as Unicode. Perhaps the only solution is to have
System.FilePath.Posix.toString and System.FilePath.Windows.toString
with different type signatures.

- As long as FilePath is just a type synonym the function is unsafe as
it's equivalent to decodeWithCurrentLocale :: String -> IO String
which fails if the argument string is actually used to store Unicode
data (rather than bytes stored as code points < 256). This is probably
intentional but it might be worth mentioning it in the Haddock
documentation.

Cheers,

Johan
From gale at sefer.org  Wed Aug 26 07:55:39 2009
From: gale at sefer.org (Yitzchak Gale)
Date: Wed Aug 26 07:35:42 2009
Subject: Proposal #3456: Add FilePath -> String decoder
In-Reply-To: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com>
References: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com>
Message-ID: <2608b8a80908260455x3aacebe0q7ff89a511b1da7b@mail.gmail.com>

> Ticket: http://hackage.haskell.org/trac/ghc/ticket/3456

That ticket is a duplicate of #3307, please reference your
patch there.

There has already been some discussion of this proposal
in that bug, and in the referenced thread on Haskell Cafe.

-Yitz
From gale at sefer.org  Wed Aug 26 09:14:54 2009
From: gale at sefer.org (Yitzchak Gale)
Date: Wed Aug 26 08:54:57 2009
Subject: Proposal #3456: Add FilePath -> String decoder
In-Reply-To: <90889fe70908260138ic8ebd0ap2627e9b5d30e6ccb@mail.gmail.com>
References: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com> 
	<90889fe70908260138ic8ebd0ap2627e9b5d30e6ccb@mail.gmail.com>
Message-ID: <2608b8a80908260614m493128c9vef79c0b8a15187d3@mail.gmail.com>

Johan Tibell wrote:
> Perhaps the only solution is to have
> System.FilePath.Posix.toString and System.FilePath.Windows.toString
> with different type signatures.

I'm not sure there's any point. As Duncan pointed out,
we are not just talking about the file system, we are
talking about interaction between the file system and
a user interface - how file paths should appear to
users. So it also depends on what UI you are using.

For example, GTK2 on Unix always uses UTF-8
to display file paths no matter what the current locale -
unless you've set a certain environment variable.

Most X terminals display file paths using the current
locale.

I'm not sure what the current situation is in Qt.

On Mac OS X, HFS+ stores file names as UTF-16, and
file paths in POSIX calls are interpreted as UTF-8. But
canonical Unicode is used, so the actual file path might
not be the same as what you provided if it includes
combining characters.

I think that Windows also converts the file path
to (some kind of) canonical Unicode in the presence of
combining characters.

So we should probably add stringToFilePath as well -
encode on vanilla POSIX, canoncialize and
encode on Mac OS X, canonicalize on Windows.
We need to research exactly which canonical form
is used on each platform. Unfortunately, that may
depend upon the file system. Also, based on past
experience, I fear that on Windows "canonical"
may mean something different than anything
published.

I am now beginning to lean towards Ketil's suggestion
that on POSIX platforms we should always use
UTF-8. We then need a prominent warning in the
documentation that if you need something else,
like the current locale, decode it yourself.

Note that it is becoming increasingly rare for people
to use non-UTF-8 locales anywhere in the world,
and even then it's likely ignored by many UIs.
So I'm inclined against cluttering the API with
convenience functions for other encodings, as Johan
is suggesting.

As a way forward - I propose:

1. Accept Judah's patch, modified always to use UTF-8.

2. Add strident warnings in the documentation that:

   o If you need a different encoding on POSIX, do it
     yourself.

   o If FilePath does not come from the file
     system, it may not match the actual file path used
     in the file system due to Unicode canonicalization.

3. Open a feature request for stringToFilePath as
   described above.

Regards,
Yitz
From s.clover at gmail.com  Wed Aug 26 11:43:13 2009
From: s.clover at gmail.com (Sterling Clover)
Date: Wed Aug 26 11:22:55 2009
Subject: Potential Network SIG
In-Reply-To: <4A93D3FF.4010002@gmail.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>
	
	<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>
	<771B85EE-5D41-4C56-8275-6D62F67932EE@gmail.com>
	<4A93D3FF.4010002@gmail.com>
Message-ID: <25151428.post@talk.nabble.com>



Simon Marlow-7 wrote:
> 
> Could you elaborate a bit more on "Handles feel to me to be too tricky 
> to reliably map onto"?  I'd like to understand what problems there are, 
> and see if there's some way to address them.
> 

I'm not a network/sockets expert by any means, but I've had to deal with
doing some fiddly stuff in Haskell, and ended up just not trusting the
Handle model, even though I know there's been some improvement and quite a
few bugs have been fixed, etc. Additionally, these comments are based on
what exists now in GHC 6.10.

That said, my first gripe is that lots of operations on Handles are really
only sane for actual files. Maybe these could be abstracted out, by
parameterizing Handles over a phantom type, or by typeclassing? The file
size operations are obviously meaningless, as are the get/set posn
functions, and seeks, but hIsEOF and hLookAhead also seem problematic, not
least because they force a buffer for even ostensibly unbuffered handles.
Additionally, at least on windows, sometimes there is an IOError
OtherException with error text "failed (No Error)" when an EOF should be
provided. [That's really more a behavioural quirk/bug than a failing of
Handles as such, I suppose]

As far as buffering goes, Handles currently couple buffering modes, which is
potentially frustrating if one wants, e.g., no buffering on recv, but block
buffering on send. 

Additionally, for many, though not all applications, it makes sense to
separate the ability to close a handle from the ability to read or write to
it, particularly for high-level, e.g., server APIs.

On the other hand, I'm less concerned with the ability to set options, since
it seems to be that socket options can almost invariably be set as part of
the initial connection/setup phase, and once you have a Handle or whatever,
then the ability to change them isn't really necessary.

Cheers,
Sterl.
-- 
View this message in context: http://www.nabble.com/Potential-Network-SIG-tp25090622p25151428.html
Sent from the Haskell - Libraries mailing list archive at Nabble.com.

From kili at outback.escape.de  Wed Aug 26 12:19:10 2009
From: kili at outback.escape.de (Matthias Kilian)
Date: Wed Aug 26 11:59:46 2009
Subject: darcs patch: Allow for configurable iconv include and library
	locat...
Message-ID: <200908261619.n7QGJAfQ006158@nutty.outback.escape.de>

Wed Aug 26 17:44:06 CEST 2009  Matthias Kilian 
  * Allow for configurable iconv include and library locations.
  This should help to fix the build on OpenBSD.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-darcs-patch
Size: 9990 bytes
Desc: A darcs patch for your repository!
Url : http://www.haskell.org/pipermail/libraries/attachments/20090826/79df9fc0/attachment.bin
From kili at outback.escape.de  Wed Aug 26 12:25:20 2009
From: kili at outback.escape.de (Matthias Kilian)
Date: Wed Aug 26 12:05:45 2009
Subject: darcs patch: Allow for configurable iconv include and library
	locat...
Message-ID: <200908261625.n7QGPK3k001466@nutty.outback.escape.de>

Wed Aug 26 17:44:06 CEST 2009  Matthias Kilian 
  * Allow for configurable iconv include and library locations.
  This should help to fix the build on OpenBSD.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-darcs-patch
Size: 9990 bytes
Desc: A darcs patch for your repository!
Url : http://www.haskell.org/pipermail/libraries/attachments/20090826/79ba3592/attachment-0001.bin
From marlowsd at gmail.com  Wed Aug 26 12:42:19 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Wed Aug 26 12:22:06 2009
Subject: Potential Network SIG
In-Reply-To: <25151428.post@talk.nabble.com>
References: <4c44d90b0908212149y650a3861j93352a24bdce12f1@mail.gmail.com>		<90889fe70908241248g201596a8p5509020aa88bcba1@mail.gmail.com>	<771B85EE-5D41-4C56-8275-6D62F67932EE@gmail.com>	<4A93D3FF.4010002@gmail.com>
	<25151428.post@talk.nabble.com>
Message-ID: <4A9565EB.6020402@gmail.com>

On 26/08/2009 16:43, Sterling Clover wrote:
>
>
> Simon Marlow-7 wrote:
>>
>> Could you elaborate a bit more on "Handles feel to me to be too tricky
>> to reliably map onto"?  I'd like to understand what problems there are,
>> and see if there's some way to address them.
>>
>
> I'm not a network/sockets expert by any means, but I've had to deal with
> doing some fiddly stuff in Haskell, and ended up just not trusting the
> Handle model, even though I know there's been some improvement and quite a
> few bugs have been fixed, etc. Additionally, these comments are based on
> what exists now in GHC 6.10.
>
> That said, my first gripe is that lots of operations on Handles are really
> only sane for actual files. Maybe these could be abstracted out, by
> parameterizing Handles over a phantom type, or by typeclassing? The file
> size operations are obviously meaningless, as are the get/set posn
> functions, and seeks, but hIsEOF and hLookAhead also seem problematic, not
> least because they force a buffer for even ostensibly unbuffered handles.

Ok, so is it fair to summarise this as saying you'd prefer there to be 
more static type safety to these operations, rather than the current 
dynamic type checking?  I couldn't agree more.

> Additionally, at least on windows, sometimes there is an IOError
> OtherException with error text "failed (No Error)" when an EOF should be
> provided. [That's really more a behavioural quirk/bug than a failing of
> Handles as such, I suppose]

That sounds like a bug, if you can reproduce it then please report it.

> As far as buffering goes, Handles currently couple buffering modes, which is
> potentially frustrating if one wants, e.g., no buffering on recv, but block
> buffering on send.

Buffering is always invisible on input - if there is any input 
available, you'll see it immediately.  It has performance implications 
only - but I can't imagine you'd want to deliberately reduce performance 
by turning off buffering (in fact, I think the new I/O library doesn't 
even honour NoBuffering on input Handles).

> Additionally, for many, though not all applications, it makes sense to
> separate the ability to close a handle from the ability to read or write to
> it, particularly for high-level, e.g., server APIs.

Yes - more detailed types would be nice.

> On the other hand, I'm less concerned with the ability to set options, since
> it seems to be that socket options can almost invariably be set as part of
> the initial connection/setup phase, and once you have a Handle or whatever,
> then the ability to change them isn't really necessary.

Still, we could provide a way to recover the Socket from a Handle if 
necessary (it would fail on a Handle that wasn't a Socket, of course).

The other question to consider is I/O multiplexing 
(select()/epoll()/etc.).  Right now if you use Handles with file 
descriptors you get this for free (not epoll() yet, but hopefully in the 
future).  If you do your own I/O, then you have to implement this too. 
Ideally the multiplexing of I/O should be available as a separate 
library that you can hook into from your own I/O code, and it would be 
used by Handles as well as from other libraries that need low-level I/O.

One other reason to want Handles: if you want to do something that 
involves changing encodings on the fly (e.g. reading an HTTP response) 
then Handles with hSetEncoding do the job nicely, the buffering and 
re-encoding is all handled behind the scenes for you.  This sort of 
thing would be tricky with bytestring and text.

Cheers,
	Simon
From iavor.diatchki at gmail.com  Wed Aug 26 12:49:33 2009
From: iavor.diatchki at gmail.com (Iavor Diatchki)
Date: Wed Aug 26 12:29:14 2009
Subject: Proposal: add new function "check" to Control.Monad
In-Reply-To: <2608b8a80908240103v35a2d36andeee9cf20fb3f5e0@mail.gmail.com>
References: 
	<2608b8a80908240103v35a2d36andeee9cf20fb3f5e0@mail.gmail.com>
Message-ID: <5ab17e790908260949w30bad06ej9d5f3e8150cb43ac@mail.gmail.com>

Hello,
I don't really think that we need this function, it is plenty easy to
just write with "guard".  Furthermore, if you choose to add it, then
please do not use "check" for the name.  This is way too generic a
name which I often use for local functions that do some validation.
-Iavor

On Mon, Aug 24, 2009 at 1:03 AM, Yitzchak Gale wrote:
> J?n Fairbairn wrote:
>> Trac ticket #3453. Two week time frame.
>> Add check function to Control.Monad
>> check :: (MonadPlus m) => (a -> Bool) -> a -> m a
>
> I remember needing this on a number of occasions.
>
>> ? mfilter = (join .) . liftM . check
>
> I might write that as:
>
> mFilter = (=<<) . check
>
> That is an intriguing dual to filterM.
>
> Regards,
> Yitz
> _______________________________________________
> Libraries mailing list
> Libraries@haskell.org
> http://www.haskell.org/mailman/listinfo/libraries
>
From jon.fairbairn at cl.cam.ac.uk  Wed Aug 26 13:52:59 2009
From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn)
Date: Wed Aug 26 13:32:41 2009
Subject: Proposal: add new function "check" to Control.Monad
In-Reply-To: Your message of "Mon\, 24 Aug 2009 09\:59\:24 +0200."
	<2E8CC191-9C5F-4C22-857B-0C8F432C0DA5@informatik.uni-kiel.de>
Message-ID: <17202.1251309179@calligramme.charmers>

I've been having network problems, and in addition, I usually
use gmane to access this list, but Sebastian and Yitzchak's
messages never turned up there.

On 2009-08-24 at 09:59+0200 Sebastian Fischer wrote:
> > Add check function to Control.Monad
> >
> >    check :: (MonadPlus m) => (a -> Bool) -> a -> m a
> >    check p a
> >        | p a = return a
> >        | otherwise = mzero
> 
> I agree that such a function is occasionally useful. But I
> doubt your rationale:

> > Rationale: [...]
> >
> > but check is clearly more generally useful than guard. (guard = flip
> > (check . const) ())
> 
> This is a bit like saying that concatMap is more generally
> useful than map and concat because:

I think I wasn't clear. When I said "clearly" I thought that the
general usefulness was clear; the remark about guard was an
"and" rather than a "because". What makes it clear to me is the
types: check is more polymorphic in the sense that it has a type
variable where guard has ().

>     map f = concatMap ((:[]).f)
>     concat = concatMap id
> 
> Although this is correct, map and concat are smaller pieces
> that can be easily combined to concatMap:

>     concatMap f = concat . map f
> 
> So the question is whether check is useful enough to be
> included as a shortcut for a combination of simpler
> primitives (as was decided for  concatMap).

Well, while I agree with the general principle, (and have argued
for it strongly in the past), I don't agree with the way you are
using "simple" here. When designing a library, one should choose
the primitives so that future definitions are simple, both in
the complexity of the terms needed to define them and in the
thought needed to define them, rather than the simplicity of
definition of the primitives themselves. The reason that I
suggested check is that it seems to me that it gives both of
those things in a way that guard doesn't (and people may have to
work a bit when weighing this, owing to the long-standing
familiarity of guard). The reason I didn't suggest removal of
guard was that long-standing familiarity.

To my mind, MonadPlus m => m () is a very specific and rather
peculiar type (consider [()], for example) and I think guard
suggests a rather imperative outlook because of this. Getting
from m () to Maybe Char strikes me as a longer thought process
than getting from m a to Maybe Char.

> > so we would expect a function
> >
> >   mfilter = (join .) . liftM . check
> >
> > to be useful.  Should that also be added?
> 
> I would not object and prefer this definition:
> 
>     mfilter f m = m >>= check f
> 
> It seems simpler.

I agree.  The equation above was just the derivation.

 J?n

-- 
J?n Fairbairn                              Jon.Fairbairn at cl.cam.ac.uk
From jon.fairbairn at cl.cam.ac.uk  Wed Aug 26 13:58:23 2009
From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn)
Date: Wed Aug 26 13:38:05 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: Your message of "Tue\, 25 Aug 2009 12\:32\:00 BST."
	<4A93CBB0.9030706@gmail.com>
Message-ID: <17512.1251309503@calligramme.charmers>

On 2009-08-25 at 12:32BST Simon Marlow wrote:
> I think that for consistency with if/then/else and guard we
> should use checkA.

See my response to Sebastian.

> It's not that hard to write
> 
>      all_spaces xs = checkA (all (isSpace xs)) xs
> 
> if that's really what you want.

Not /hard/, but it involves using xs twice, which should tell us
something.

> I wouldn't call checkB more "general"; to me it's just a
> minor convenience issue, which is outweighed by consistency
> and simplicity.

See response to Sebastian.

> > is hard to write using checkA (suppose you are doing something like ?map
> > (checkB predicate)?) since checkA doesn't look at the second argument.
> 
> the "applying check to a list" case is not hard either:
> 
>   zipWithM checkA (map p xs) xs

which involves duplication again, and on top of that zipWithM,
which while not /hard/ is hardly conceptually simple.

 J?n

-- 
J?n Fairbairn                              Jon.Fairbairn at cl.cam.ac.uk
From johan.tibell at gmail.com  Wed Aug 26 14:03:48 2009
From: johan.tibell at gmail.com (Johan Tibell)
Date: Wed Aug 26 13:43:49 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: <2608b8a80908250411v225c697dk9d88bb38aff963c9@mail.gmail.com>
References: 
	<20090824202119.GA28087@seas.upenn.edu> 
	<90889fe70908250201t3b68e4feu388448c66e39d48@mail.gmail.com> 
	<2608b8a80908250411v225c697dk9d88bb38aff963c9@mail.gmail.com>
Message-ID: <90889fe70908261103p3e058629p3089a4bc0365856e@mail.gmail.com>

On Tue, Aug 25, 2009 at 1:11 PM, Yitzchak Gale wrote:
> Johan Tibell wrote:
>> Are all these small little functions
>> really worth it?
>> They're trivially defined using the current API...
>> let developers use composition to define the rest?
>
> I completely agree with you. But in this case, even
> though these functions really are trivial, they come
> up *all* the time, and there is just something awkward
> about them.
>
> We certainly won't add more than one of the two:
>
> justIf x b = guard b >> return x
> check p x = guard (p x) >> return x

As you showed they are awkward to express in point-free style but as
you demonstrated above they're trivially expressed using the monadic
operators and guard. That several people want several different
variation also suggests that the proposed version perhaps isn't useful
enough to warrant inclusion in the library.

Cheers,

Johan
From jon.fairbairn at cl.cam.ac.uk  Wed Aug 26 14:07:43 2009
From: jon.fairbairn at cl.cam.ac.uk (Jon Fairbairn)
Date: Wed Aug 26 13:47:24 2009
Subject: Proposal: add new function "check" to Control.Monad
In-Reply-To: Your message of "Wed\, 26 Aug 2009 09\:49\:33 PDT."
	<5ab17e790908260949w30bad06ej9d5f3e8150cb43ac@mail.gmail.com>
Message-ID: <18058.1251310063@calligramme.charmers>

On 2009-08-26 at 09:49PDT Iavor Diatchki wrote:
> Hello,
> I don't really think that we need this function, it is plenty easy to
> just write with "guard". 

See my response to Sebastian, but also, none of the definitions
of check in terms of guard meet my criteria for being simple
enough, namely no lambdas and only a small amount of plumbing
(ie don't get round the "no lambdas" part by compiling it to S
and K ;-)

> Furthermore, if you choose to add it, then please do not use
> "check" for the name. This is way too generic a name which I
> often use for local functions that do some validation.

I was unsure of the name, for that very reason, so am open to
suggestions.

 J?n

-- 
J?n Fairbairn                              Jon.Fairbairn at cl.cam.ac.uk
From dave at zednenem.com  Wed Aug 26 15:03:44 2009
From: dave at zednenem.com (David Menendez)
Date: Wed Aug 26 14:43:28 2009
Subject: Proposal: add new function "check" to Control.Monad
In-Reply-To: <18058.1251310063@calligramme.charmers>
References: <5ab17e790908260949w30bad06ej9d5f3e8150cb43ac@mail.gmail.com>
	<18058.1251310063@calligramme.charmers>
Message-ID: <49a77b7a0908261203y57698896m1ceeac9abc60625a@mail.gmail.com>

On Wed, Aug 26, 2009 at 2:07 PM, Jon
Fairbairn wrote:
> On 2009-08-26 at 09:49PDT Iavor Diatchki wrote:
>> Hello,
>> I don't really think that we need this function, it is plenty easy to
>> just write with "guard".
>
> See my response to Sebastian, but also, none of the definitions
> of check in terms of guard meet my criteria for being simple
> enough, namely no lambdas and only a small amount of plumbing
> (ie don't get round the "no lambdas" part by compiling it to S
> and K ;-)

I'm usually opposed to adding small functions to the standard library,
but check is something I've defined for myself dozens of times.
(Although I usually call it "require".)

In my experience, check is very natural if you're writing code using
>>= (and especially =<<), but guard is possibly more natural with
do-notation. Similarly, guard scales better if you have multiple
conditions. e.g.,

    foo >>= require even >>= bar

is more convenient than

    foo >>= \x -> guard (even x) >> bar x

but

    foo >>= require (\x -> even x && x < y) >>= bar

is no better than

    foo >>= \x -> guard (even x && x < y) >> bar x


On the other hand, the more applicative style,

    bar =<< require even =<< foo

doesn't easily translate into using guard.


Finally, it's worth considering adding filter (or "mfilter", "sieve",
whatever). It's trivial to define check/require in terms of filter,
and the resulting code looks even cleaner.

    filter even foo >>= bar
    filter (\x -> even x && x < y) foo >>= bar
    bar =<< filter even foo

-- 
Dave Menendez 

From sebf at informatik.uni-kiel.de  Wed Aug 26 15:44:54 2009
From: sebf at informatik.uni-kiel.de (Sebastian Fischer)
Date: Wed Aug 26 15:24:44 2009
Subject: Proposal: add new function "check" to Control.Monad
In-Reply-To: <17202.1251309179@calligramme.charmers>
References: <17202.1251309179@calligramme.charmers>
Message-ID: <351FDBA1-8512-457D-BA61-FC67422EAB66@informatik.uni-kiel.de>


On Aug 26, 2009, at 7:52 PM, Jon Fairbairn wrote:

> When designing a library, one should choose
> the primitives so that future definitions are simple, both in
> the complexity of the terms needed to define them and in the
> thought needed to define them, rather than the simplicity of
> definition of the primitives themselves. The reason that I
> suggested check is that it seems to me that it gives both of
> those things in a way that guard doesn't

I see. Also the type of guard is indeed unusual. Historically, guard  
seems inspired by conditions in list comprehensions but I would  
appreciate a sensible refactoring.

When designing MonadPlus combinators from scratch I would probably  
implement

   filter :: MonadPlus m => (a -> Bool) -> m a -> m a

as a generalisation of the list function with the same name. After  
Davids remarks I'm not sure whether I would need

   check f = filter f . return

I agree to you reasoning to prefer check over guard but go one step  
further and prefer filter over check.

I would vote for adding a generalized filter to Control.Monad. Using  
the name of the Prelude function on lists is justified because it is a  
generalisation just like many functions in Data.Foldable which also  
reuse Prelude names.

Cheers,
Sebastian

-- 
Underestimating the novelty of the future is a time-honored tradition.
(D.G.)



From duncan.coutts at worc.ox.ac.uk  Wed Aug 26 17:16:24 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Wed Aug 26 17:12:42 2009
Subject: darcs patch: Add justIf to Data.Maybe
In-Reply-To: <17512.1251309503@calligramme.charmers>
References: <17512.1251309503@calligramme.charmers>
Message-ID: <1251321384.30910.10061.camel@localhost>

On Wed, 2009-08-26 at 18:58 +0100, Jon Fairbairn wrote:

> > It's not that hard to write
> > 
> >      all_spaces xs = checkA (all (isSpace xs)) xs
> > 
> > if that's really what you want.
> 
> Not /hard/, but it involves using xs twice, which should tell us
> something.

I'm not sure it does tell us much, given that there are also cases where
we need to use it only once, and would be forced to use \_ -> to ignore
it in the predicate.

The two forms are clearly inter-convertible and one form is more
convenient in some situations while the other is more convenient in
others.

Duncan

From marlowsd at gmail.com  Thu Aug 27 06:35:56 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Thu Aug 27 06:15:40 2009
Subject: darcs patch: Allow for configurable iconv include and library
 locat...
In-Reply-To: <200908261625.n7QGPK3k001466@nutty.outback.escape.de>
References: <200908261625.n7QGPK3k001466@nutty.outback.escape.de>
Message-ID: <4A96618C.5080001@gmail.com>

On 26/08/2009 17:25, Matthias Kilian wrote:
> Wed Aug 26 17:44:06 CEST 2009  Matthias Kilian
>    * Allow for configurable iconv include and library locations.
>    This should help to fix the build on OpenBSD.

Thanks, I'll validate and apply.

Simon
From duncan.coutts at worc.ox.ac.uk  Fri Aug 28 18:21:27 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Fri Aug 28 21:22:34 2009
Subject: Binding to inline functions
In-Reply-To: <4A93D687.4050001@gmail.com>
References: 
	<1251056058.30910.4937.camel@localhost>  <4A93D687.4050001@gmail.com>
Message-ID: <1251498087.30910.15118.camel@localhost>

On Tue, 2009-08-25 at 13:18 +0100, Simon Marlow wrote:
> On 23/08/2009 20:34, Duncan Coutts wrote:
> > On Sun, 2009-08-23 at 15:09 -0300, Maur??cio CA wrote:
> >> I understand we can't use 'foreign import ccall'
> >> to wrap inline C functions. Do you think it could
> >> be possible to have an option in cabal to generate
> >> such functions in an object file when #included in
> >> a C file, in a compiler independent, portable way?
> >
> > It might be better to include this feature into hsc2hs and/or c2hs
> > (which may in turn require some help from Cabal).
> 
> I think it would be easy to do in GHC.  We already have the machinery to 
> generate the _stub.c files and compile them.
> 
> The main question is what the syntax should look like.  I was toying with
> 
> foreign import capi "foo" foo :: ...

While maybe it can be done in ghc, is that the best place to put it? Do
we do so without extending the FFI spec?

The advantage of using something like hsc2hs is that we can still
compile the same code in hugs, nhc, jhc and uhc.

> I wonder whether not being able to specify the actual calling convention 
> will be a problem, however.  We are relying on the calling convention of 
> foo being implicit.  Are there cases where this might cause a problem?

If we do it via hsc2hs or c2hs then there's no need to specify any
calling convention. We just get the one declared in the header files.
This means we can handle stdcall, ccall and even crazy platform-specific
variations like __attribute__((regparam, 3)).

A generalisation of this is needed for calling "functions" defined by
macros, and C varargs functions. A similar case, that is not fully
automatic, is cases like binding functions that pass C structures by
value. The "C structs by value" problem is already on the IHG wishlist.

Both hsc2hs and c2hs support a way of outputting extra C snippets and
binding to them, though there are some limitations. Currently Cabal does
not support this feature in hsc2hs and the limitation in c2hs is that
one can only add extra C snippets to the header, not to something that
goes into a .c file to be compiled.

So, I think we should consider making these kinds of use cases easy via
c2hs and hsc2hs (and Cabal).

Duncan

From duncan.coutts at worc.ox.ac.uk  Fri Aug 28 18:50:27 2009
From: duncan.coutts at worc.ox.ac.uk (Duncan Coutts)
Date: Fri Aug 28 21:22:45 2009
Subject: Proposal #3456: Add FilePath -> String decoder
In-Reply-To: <2608b8a80908260614m493128c9vef79c0b8a15187d3@mail.gmail.com>
References: <6d74b0d20908230927w996fcb3m4cd0737f0a4eb9c2@mail.gmail.com>
	<90889fe70908260138ic8ebd0ap2627e9b5d30e6ccb@mail.gmail.com>
	<2608b8a80908260614m493128c9vef79c0b8a15187d3@mail.gmail.com>
Message-ID: <1251499827.30910.15177.camel@localhost>

On Wed, 2009-08-26 at 16:14 +0300, Yitzchak Gale wrote:
> Johan Tibell wrote:
> > Perhaps the only solution is to have
> > System.FilePath.Posix.toString and System.FilePath.Windows.toString
> > with different type signatures.
> 
> I'm not sure there's any point. As Duncan pointed out,
> we are not just talking about the file system, we are
> talking about interaction between the file system and
> a user interface - how file paths should appear to
> users. So it also depends on what UI you are using.


Mmm, this stuff is complex :-(

In general I like the idea of the proposal that we have functions for
converting between String and FilePath. As it says in the proposal, it
gets us closer to being able to treat FilePath as abstract.

Of course the devil is in the detail. Getting it right, and making it
portable and usable is hard.

> I am now beginning to lean towards Ketil's suggestion
> that on POSIX platforms we should always use
> UTF-8. We then need a prominent warning in the
> documentation that if you need something else,
> like the current locale, decode it yourself.

That's nice in that it makes the function pure, or equivalently so that
it does not need a locale parameter.

> Note that it is becoming increasingly rare for people
> to use non-UTF-8 locales anywhere in the world,
> and even then it's likely ignored by many UIs.
> So I'm inclined against cluttering the API with
> convenience functions for other encodings, as Johan
> is suggesting.
> 
> As a way forward - I propose:
> 
> 1. Accept Judah's patch, modified always to use UTF-8.

If we don't have the locale stuff then doesn't the API become a lot
simpler?

Instead of:

filePathToString :: FilePath -> IO String
getFilePathToStringFunc :: IO (FilePath -> String)

We'd have:

filePathToString :: FilePath -> String

Presumably on POSIX we will follow the glib approach of using '?'
replacement chars, since the conversion to string is aimed at human
consumption. Doing this makes the function total but lossy.

And I didn't notice anything in the proposal about the other direction,
converting String to FilePath. Surely we need both.

stringToFilePath :: String -> FilePath

A nice thing about using UTF8 on POSIX is we know this function cannot
fail, unlike conversions into a locale encoding. Presumably on POSIX
this does not do any kind of Unicode canonicalisation, while on OSX and
Windows it would do the appropriate kind.


At this point I expect Johan to jump up and down and say these should
be:

import qualified System.FilePath as FilePath

FilePath.toString   :: FilePath -> String
FilePath.fromString :: String -> FilePath


In principle I guess it'd be ok to add versions in the
System.FilePath.Posix module that take an extra encoding parameter, but
it can't be the portable version since the encoding is fixed for OSX and
Windows. It's also jolly inconvenient, and as you've pointed out, of
diminishing importance.

> 2. Add strident warnings in the documentation that:
> 
>    o If you need a different encoding on POSIX, do it
>      yourself.
>
>    o If FilePath does not come from the file
>      system, it may not match the actual file path used
>      in the file system due to Unicode canonicalization.

Similar points apply to trying to round-trip via
toString . fromString :: String -> String
fromString . toString :: FilePath -> FilePath

The String -> String transform would do some Unicode canonicalisation on
Windows and OSX.

The FilePath -> FilePath would be identity on Windows and OSX for
strings coming from the file system. On POSIX however we can get utf8
decoding errors which will give us replacement chars.

So the advice in this section of the documentation should probably be
similar to the glib docs, where it says that you should keep both forms
in some circumstances. You can present the file name to the user though
a graphical or command line ui, but also so you can still access the
same file later (eg to save it). Especially in document-oriented GUI
apps, it's very annoying if you open, edit and save, but saving either
fails because it cannot re-encode, or ends up writing a different file
(different in Unicode canonicalisation or having replacement chars).


Duncan

From kili at outback.escape.de  Sat Aug 29 05:41:40 2009
From: kili at outback.escape.de (Matthias Kilian)
Date: Sat Aug 29 05:21:35 2009
Subject: darcs patch: Fix unportable use of sleep(1)
Message-ID: <200908290941.n7T9fer1025948@nutty.outback.escape.de>

Sat Aug 29 11:37:20 CEST 2009  Matthias Kilian 
  * Fix unportable use of sleep(1)
  `sleep 1s' is probably a GNUism; `sleep 1' should do the same in a
  portable way.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-darcs-patch
Size: 1054 bytes
Desc: A darcs patch for your repository!
Url : http://www.haskell.org/pipermail/libraries/attachments/20090829/79b8d815/attachment.bin
From marlowsd at gmail.com  Sat Aug 29 08:05:48 2009
From: marlowsd at gmail.com (Simon Marlow)
Date: Sat Aug 29 07:45:44 2009
Subject: Binding to inline functions
In-Reply-To: <1251498087.30910.15118.camel@localhost>
References: 	
	<1251056058.30910.4937.camel@localhost>
	<4A93D687.4050001@gmail.com>
	<1251498087.30910.15118.camel@localhost>
Message-ID: <4A99199C.40406@gmail.com>

Duncan Coutts wrote:
> On Tue, 2009-08-25 at 13:18 +0100, Simon Marlow wrote:
>> On 23/08/2009 20:34, Duncan Coutts wrote:
>>> On Sun, 2009-08-23 at 15:09 -0300, Maur??cio CA wrote:
>>>> I understand we can't use 'foreign import ccall'
>>>> to wrap inline C functions. Do you think it could
>>>> be possible to have an option in cabal to generate
>>>> such functions in an object file when #included in
>>>> a C file, in a compiler independent, portable way?
>>> It might be better to include this feature into hsc2hs and/or c2hs
>>> (which may in turn require some help from Cabal).
>> I think it would be easy to do in GHC.  We already have the machinery to 
>> generate the _stub.c files and compile them.
>>
>> The main question is what the syntax should look like.  I was toying with
>>
>> foreign import capi "foo" foo :: ...
> 
> While maybe it can be done in ghc, is that the best place to put it? Do
> we do so without extending the FFI spec?
> 
> The advantage of using something like hsc2hs is that we can still
> compile the same code in hugs, nhc, jhc and uhc.

Yes, it could be done in hsc2hs.  I don't think the GHC build system or 
Cabal currently handles the _hsc.c stub files that hsc2hs produces (at 
least, if it does, I can't see where).  That can be fixed, of course.

So the reason I thought of adding it to GHC is because in a sense it's 
just another calling convention for the FFI to support.  If it was done 
in hsc2hs then the syntax becomes heavier - you need both a foreign 
import and an hsc2hs directive of some kind, compared to just a foreign 
import.  In c2hs the syntax could be lighter, but c2hs isn't widely 
available.

>> I wonder whether not being able to specify the actual calling convention 
>> will be a problem, however.  We are relying on the calling convention of 
>> foo being implicit.  Are there cases where this might cause a problem?
> 
> If we do it via hsc2hs or c2hs then there's no need to specify any
> calling convention. We just get the one declared in the header files.
> This means we can handle stdcall, ccall and even crazy platform-specific
> variations like __attribute__((regparam, 3)).

And the same would be true if it was done in GHC.  The point I failed to 
  make very well was that you don't get to declare the calling 
convention, so you rely on the header file supplying it.  Of course, if 
you do this in hsc2hs you could pick some syntax to declare the calling 
convention.  This is only a minor point.

> A generalisation of this is needed for calling "functions" defined by
> macros, and C varargs functions. A similar case, that is not fully
> automatic, is cases like binding functions that pass C structures by
> value. The "C structs by value" problem is already on the IHG wishlist.

I'm thinking of this as all part of the same thing: a "function" that 
can be "called" from C code by writing f(arg1, arg2, ...).  My 'foreign 
import capi' would let you call any of these, as would a hsc2hs extension.

Ok, I don't really mind where it's implemented, especially if someone 
else does the work :-)

For doing it in GHC:
   - lighter syntax
   - no extra build system or Cabal support necessary

For doing it in hsc2hs:
   - multiple compilers get to take advantage

Cheers,
	Simon

From dons at galois.com  Sat Aug 29 18:30:55 2009
From: dons at galois.com (Don Stewart)
Date: Sat Aug 29 18:12:42 2009
Subject: Hackage rankings : August 2009
Message-ID: <20090829223055.GB21689@whirlpool.galois.com>

Monthly statistics on the most popular Haskell applications and
libraries on Hackage. August 2009 edition now up:

    http://donsbot.wordpress.com/2009/08/29/haskell-popularity-rankings-september-2009/

-- Don
From vandijk.roel at gmail.com  Mon Aug 31 15:01:52 2009
From: vandijk.roel at gmail.com (Roel van Dijk)
Date: Mon Aug 31 14:41:18 2009
Subject: Fwd: How to change -optc-O2 when compiling C files
In-Reply-To: 
References: 
Message-ID: 

I sent this message to cabal-devel. After reading the cabal website
more carefully I noticed I should sent questions here. Apologies for
the noise in cabal-devel.


Hello,

I have cabal file which specifies some c-sources to compile together
with the usual Haskell stuff. I noticed that cabal automatically
passes -optc-O2 to GHC when compiling C files. If I specify some
additional cc-options in my cabal file they just get prepended to the
list of options passed to ghc. How do I get rid of the -optc-O2?

With my current cc-options "-D_OPENMP -O3 -funroll-loops" GHC is
invoked like this during a "cabal build":

/usr/bin/ghc -Ilevmar-2.4 -package base-3.0.3.1 -optc-D_OPENMP
-optc-O3 -optc-funroll-loops -optc-O2 -odir dist/build -c
levmar-2.4/lm.c

I don't know which -optc-O? will be picked by the C compiler. Normally
-O2 is ok, but in this case I really want -O3.

Thanks in advance,
Roel van Dijk
From judah.jacobson at gmail.com  Mon Aug 31 19:29:51 2009
From: judah.jacobson at gmail.com (Judah Jacobson)
Date: Mon Aug 31 19:09:36 2009
Subject: Proposal #3455: Add a setting to change how Unicode encoding 
	errors are handled
In-Reply-To: <4A93D4A1.60602@gmail.com>
References: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com> 
	<4A93D4A1.60602@gmail.com>
Message-ID: <6d74b0d20908311629s5bdeb8f8y682d4d8ea541344c@mail.gmail.com>

On Tue, Aug 25, 2009 at 5:10 AM, Simon Marlow wrote:
> On 23/08/2009 17:22, Judah Jacobson wrote:
>>
>> I proposal that we augment ghc-6.12.1's support for Unicode Handles
>> by adding the following functions to System.IO:
>>
>> hSetOnEncodingError :: Handle -> ?OnEncodingError -> ?IO ()
>> hGetOnEncodingError :: Handle -> ?IO OnEncodingError
>>
>> as well as the enumeration `OnEncodingError` with three constructors:
>>
>> ?- `ThrowEncodingError`: Throw an exception at the first encoding or
>> ?decoding
>> ? ?error.
>> ?- `SkipEncodingError`: Skip all invalid bytes or characters.
>> ?- `TranslitEncodingError`: Replace undecodable bytes with u+FFFD, and
>> ?unencodable characters with '?'.
>>
>> I have implemented this functionality in a patch attached to the
>> ticket. ?Haddock docs
>> are here:
>> http://code.haskell.org/~judah/new-io-docs/System-IO.html#23
>>
>>
>> The choice of error handler is orthogonal to the choice of encoder.
>> Additionally, the same setting is used for both read and write modes. ?For
>> portability, the handlers are written in pure Haskell rather than using
>> GNU iconv's //TRANSLIT feature.
>>
>> Note that the text package, for example, provides more sophisticated
>> error-handling options. ?However, I think the above choices are useful
>> enough without making the API too complicated.
>
> I replied on the ticket, reproduced here for readers of libraries@:
>
> It looks like the main question here is whether the IOError should be
> returned explicitly (as in your patch), or whether we should just catch the
> exception. All things being equal, catching the exception would be simpler,
> as it wouldn't require any changes in the codecs. Is there a reason why you
> didn't do it that way? Perhaps because you want to be sure that the
> exception is really an encoding error, and not some other kind of exception?
> If that's the case, then we should introduce a new exception for encoding
> errors (that's probably a good idea anyway).

I agree that we should create a new exception type.  Given the errors
currently thrown by the library, I assume that it doesn't need to be
anything more than a newtype wrapping a String message.

If the text package and ghc's IO library are merged into a new system,
then it would probably be better to explicitly return the error --
that way we can have pure ByteString <-> Text conversion functions.
But for the current state of the library (where the encoding type is
only exposed under GHC.* and makes few stability promises) it probably
doesn't make a big difference.

-Judah
From ekmett at gmail.com  Mon Aug 31 19:59:32 2009
From: ekmett at gmail.com (Edward Kmett)
Date: Mon Aug 31 19:38:58 2009
Subject: Proposal #3455: Add a setting to change how Unicode encoding 
	errors are handled
In-Reply-To: <6d74b0d20908311629s5bdeb8f8y682d4d8ea541344c@mail.gmail.com>
References: <6d74b0d20908230922u7677afe9iade0480ed4b035f8@mail.gmail.com>
	<4A93D4A1.60602@gmail.com>
	<6d74b0d20908311629s5bdeb8f8y682d4d8ea541344c@mail.gmail.com>
Message-ID: <7fb8f82f0908311659h6addbcefw8bd752f70a7ef8fc@mail.gmail.com>

On Mon, Aug 31, 2009 at 7:29 PM, Judah Jacobson wrote:

> On Tue, Aug 25, 2009 at 5:10 AM, Simon Marlow wrote:
> > On 23/08/2009 17:22, Judah Jacobson wrote:
> >> I proposal that we augment ghc-6.12.1's support for Unicode Handles
> >> by adding the following functions to System.IO:
> >>
> >> hSetOnEncodingError :: Handle ->  OnEncodingError ->  IO ()
> >> hGetOnEncodingError :: Handle ->  IO OnEncodingError
> >>
> >> as well as the enumeration `OnEncodingError` with three constructors:
> >>
> >>  - `ThrowEncodingError`: Throw an exception at the first encoding or
> >>  decoding
> >>    error.
> >>  - `SkipEncodingError`: Skip all invalid bytes or characters.
> >>  - `TranslitEncodingError`: Replace undecodable bytes with u+FFFD, and
> >>  unencodable characters with '?'.
>

As a brief, possibly irrelevant aside:

There is one other option for how to handle Unicode en/decoding errors that
I've used and seen used.
It is the basis of Markus Kuhn's "UTF-8B" encoding whereby parse errors are
read as 0xdc00 + the raw byte, which when you go to emit them, you can emit
them directly into the stream as raw bytes. This permits a perfect round
trip from UTF-8 to String to UTF-8, regardless of encoding errors. The
codepoints from 0xdc80-0xdcff don't conflict with UTF-16, because they are
in the unmapped d800-dfff range and in ISO 10646-1 section R.4 it notes that
the mapping of those code positions in UTF8 are undefined, so an
implementation is free to do with them as it pleases. The main good thing
that comes with this representation is that no information is discarded. It
doesn't hurt that this also sidesteps the other uses of the d800-dfff range
like the illegal Oracle-style "CESU-8" encoding of surrogate pairs, etc.

http://mail.nl.linux.org/linux-utf8/2000-07/msg00040.html
http://bsittler.livejournal.com/10381.html

-Edward Kmett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/libraries/attachments/20090831/6b0303b6/attachment.html