# Typeclassopedia

(Difference between revisions)
 Revision as of 12:39, 26 September 2012 (edit) (→Intuition)← Previous diff Revision as of 21:18, 5 October 2012 (edit) (undo) (→Semigroup: add some content to semigroup section)Next diff → Line 758: Line 758: For example, the natural numbers under addition form a semigroup: the sum of any two natural numbers is a natural number, and $(a+b)+c = a+(b+c)\$ for any natural numbers $a\$, $b\$, and $c\,\$. The integers under multiplication also form a semigroup, as do the integers (or rationals, or reals) under $\max\$ or $\min\$, Boolean values under conjunction and disjunction, lists under concatenation, functions from a set to itself under composition ... Semigroups show up all over the place, once you know to look for them. For example, the natural numbers under addition form a semigroup: the sum of any two natural numbers is a natural number, and $(a+b)+c = a+(b+c)\$ for any natural numbers $a\$, $b\$, and $c\,\$. The integers under multiplication also form a semigroup, as do the integers (or rationals, or reals) under $\max\$ or $\min\$, Boolean values under conjunction and disjunction, lists under concatenation, functions from a set to itself under composition ... Semigroups show up all over the place, once you know to look for them. - Under construction, more coming soon... + ==Definition== + + Semigroups are not (yet?) defined in the base package, but the {{HackagePackage|id=semigroups}} package provides a standard definition. + + The definition of the Semigroup type class ([http://hackage.haskell.org/packages/archive/semigroups/latest/doc/html/Data-Semigroup.html haddock]) is as follows: + + + class Semigroup a where + (<>) :: a -> a -> a + + sconcat :: NonEmpty a -> a + sconcat = sconcat (a :| as) = go a as where + go b (c:cs) = b <> go c cs + go b [] = b + + times1p :: Whole n => n -> a -> a + times1p = ... + + + The really important method is (<>), representing the associative binary operation. The other two methods have default implementations in terms of (<>), and are included in the type class in case some instances can give more efficient implementations than the default. sconcat reduces a nonempty list using (<>); times1p n is equivalent to (but more efficient than) sconcat . replicate n. See the [http://hackage.haskell.org/packages/archive/semigroups/latest/doc/html/Data-Semigroup.html haddock documentation] for more information on sconcat and times1p. + + ==Laws== + + The only law is that (<>) must be associative: + + + (x <> y) <> z = x <> (y <> z) + + + More coming soon... =Monoid= =Monoid=

## Revision as of 21:18, 5 October 2012

By Brent Yorgey, byorgey@cis.upenn.edu

Originally published 12 March 2009 in issue 13 of the Monad.Reader. Ported to the Haskell wiki in November 2011 by Geheimdienst.

This is now the official version of the Typeclassopedia and supersedes the version published in the Monad.Reader. Please help update and extend it by editing it yourself or by leaving comments, suggestions, and questions on the talk page.

# 1 Abstract

The standard Haskell libraries feature a number of type classes with algebraic or category-theoretic underpinnings. Becoming a fluent Haskell hacker requires intimate familiarity with them all, yet acquiring this familiarity often involves combing through a mountain of tutorials, blog posts, mailing list archives, and IRC logs.

The goal of this document is to serve as a starting point for the student of Haskell wishing to gain a firm grasp of its standard type classes. The essentials of each type class are introduced, with examples, commentary, and extensive references for further reading.

# 2 Introduction

Have you ever had any of the following thoughts?

• What the heck is a monoid, and how is it different from a monad?
• I finally figured out how to use Parsec with do-notation, and someone told me I should use something called Applicative instead. Um, what?
• Someone in the #haskell IRC channel used (***), and when I asked lambdabot to tell me its type, it printed out scary gobbledygook that didn’t even fit on one line! Then someone used fmap fmap fmap and my brain exploded.
• When I asked how to do something I thought was really complicated, people started typing things like zip.ap fmap.(id &&& wtf) and the scary thing is that they worked! Anyway, I think those people must actually be robots because there’s no way anyone could come up with that in two seconds off the top of their head.

If you have, look no further! You, too, can write and understand concise, elegant, idiomatic Haskell code with the best of them.

There are two keys to an expert Haskell hacker’s wisdom:

1. Understand the types.
2. Gain a deep intuition for each type class and its relationship to other type classes, backed up by familiarity with many examples.

It’s impossible to overstate the importance of the first; the patient student of type signatures will uncover many profound secrets. Conversely, anyone ignorant of the types in their code is doomed to eternal uncertainty. “Hmm, it doesn’t compile ... maybe I’ll stick in an fmap here ... nope, let’s see ... maybe I need another (.) somewhere? ... um ...”

The second key—gaining deep intuition, backed by examples—is also important, but much more difficult to attain. A primary goal of this document is to set you on the road to gaining such intuition. However—

This document can only be a starting point, since good intuition comes from hard work, not from learning the right metaphor. Anyone who reads and understands all of it will still have an arduous journey ahead—but sometimes a good starting point makes a big difference.

It should be noted that this is not a Haskell tutorial; it is assumed that the reader is already familiar with the basics of Haskell, including the standard Prelude, the type system, data types, and type classes.

The type classes we will be discussing and their interrelationships:

Semigroup can be found in the semigroups package, Apply in the semigroupoids package, and Comonad in the comonad package.

• Solid arrows point from the general to the specific; that is, if there is an arrow from Foo to Bar it means that every Bar is (or should be, or can be made into) a Foo.
• Dotted arrows indicate some other sort of relationship.
• Monad and ArrowApply are equivalent.
• Semigroup, Apply and Comonad are greyed out since they are not actually (yet?) in the standard Haskell libraries .

One more note before we begin. The original spelling of “type class” is with two words, as evidenced by, for example, the Haskell 98 Revised Report, early papers on type classes like Type classes in Haskell and Type classes: exploring the design space, and Hudak et al.’s history of Haskell. However, as often happens with two-word phrases that see a lot of use, it has started to show up as one word (“typeclass”) or, rarely, hyphenated (“type-class”). When wearing my prescriptivist hat, I prefer “type class”, but realize (after changing into my descriptivist hat) that there's probably not much I can do about it.

We now begin with the simplest type class of all: Functor.

# 3 Functor

The Functor class (haddock) is the most basic and ubiquitous type class in the Haskell libraries. A simple intuition is that a Functor represents a “container” of some sort, along with the ability to apply a function uniformly to every element in the container. For example, a list is a container of elements, and we can apply a function to every element of a list, using map. As another example, a binary tree is also a container of elements, and it’s not hard to come up with a way to recursively apply a function to every element in a tree.

Another intuition is that a Functor represents some sort of “computational context”. This intuition is generally more useful, but is more difficult to explain, precisely because it is so general. Some examples later should help to clarify the Functor-as-context point of view.

In the end, however, a Functor is simply what it is defined to be; doubtless there are many examples of Functor instances that don’t exactly fit either of the above intuitions. The wise student will focus their attention on definitions and examples, without leaning too heavily on any particular metaphor. Intuition will come, in time, on its own.

## 3.1 Definition

Here is the type class declaration for Functor:

class Functor f where
fmap :: (a -> b) -> f a -> f b

Functor is exported by the Prelude, so no special imports are needed to use it.

First, the f a and f b in the type signature for fmap tell us that f isn’t just a type; it is a type constructor which takes another type as a parameter. (A more precise way to say this is that the kind of f must be * -> *.) For example, Maybe is such a type constructor: Maybe is not a type in and of itself, but requires another type as a parameter, like Maybe Integer. So it would not make sense to say instance Functor Integer, but it could make sense to say instance Functor Maybe.

Now look at the type of fmap: it takes any function from a to b, and a value of type f a, and outputs a value of type f b. From the container point of view, the intention is that fmap applies a function to each element of a container, without altering the structure of the container. From the context point of view, the intention is that fmap applies a function to a value without altering its context. Let’s look at a few specific examples.

## 3.2 Instances

∗ Recall that [] has two meanings in Haskell: it can either stand for the empty list, or, as here, it can represent the list type constructor (pronounced “list-of”). In other words, the type [a] (list-of-a) can also be written [] a.

∗ You might ask why we need a separate map function. Why not just do away with the current list-only map function, and rename fmap to map instead? Well, that’s a good question. The usual argument is that someone just learning Haskell, when using map incorrectly, would much rather see an error about lists than about Functors.

As noted before, the list constructor [] is a functor ; we can use the standard list function map to apply a function to each element of a list . The Maybe type constructor is also a functor, representing a container which might hold a single element. The function fmap g has no effect on Nothing (there are no elements to which g can be applied), and simply applies g to the single element inside a Just. Alternatively, under the context interpretation, the list functor represents a context of nondeterministic choice; that is, a list can be thought of as representing a single value which is nondeterministically chosen from among several possibilities (the elements of the list). Likewise, the Maybe functor represents a context with possible failure. These instances are:

instance Functor [] where
fmap _ []     = []
fmap g (x:xs) = g x : fmap g xs
-- or we could just say fmap = map

instance Functor Maybe where
fmap _ Nothing  = Nothing
fmap g (Just a) = Just (g a)

As an aside, in idiomatic Haskell code you will often see the letter f used to stand for both an arbitrary Functor and an arbitrary function. In this document, f represents only Functors, and g or h always represent functions, but you should be aware of the potential confusion. In practice, what f stands for should always be clear from the context, by noting whether it is part of a type or part of the code.

There are other Functor instances in the standard libraries; below are a few. Note that some of these instances are not exported by the Prelude; to access them, you can import Control.Monad.Instances.

• Either e is an instance of Functor; Either e a represents a container which can contain either a value of type a, or a value of type e (often representing some sort of error condition). It is similar to Maybe in that it represents possible failure, but it can carry some extra information about the failure as well.
• ((,) e) represents a container which holds an “annotation” of type e along with the actual value it holds. It might be clearer to write it as (e,), by analogy with an operator section like (1+), but that syntax is not allowed in types (although it is allowed in expressions with the TupleSections extension enabled). However, you can certainly think of it as (e,).
• ((->) e) (which can be thought of as (e ->); see above), the type of functions which take a value of type e as a parameter, is a Functor. As a container, (e -> a) represents a (possibly infinite) set of values of a, indexed by values of e. Alternatively, and more usefully, ((->) e) can be thought of as a context in which a value of type e is available to be consulted in a read-only fashion. This is also why ((->) e) is sometimes referred to as the reader monad; more on this later.
• IO is a Functor; a value of type IO a represents a computation producing a value of type a which may have I/O effects. If m computes the value x while producing some I/O effects, then fmap g m will compute the value g x while producing the same I/O effects.
• Many standard types from the containers library (such as Tree, Map, and Sequence) are instances of Functor. A notable exception is Set, which cannot be made a Functor in Haskell (although it is certainly a mathematical functor) since it requires an Ord constraint on its elements; fmap must be applicable to any types a and b. However, Set (and other similarly restricted data types) can be made an instance of a suitable generalization of Functor, either by making a and b arguments to the Functor type class themselves, or by adding an associated constraint.
Exercises
1. Implement Functor instances for Either e and ((->) e).
2. Implement Functor instances for ((,) e) and for Pair, defined as
data Pair a = Pair a a

Explain their similarities and differences.

3. Implement a Functor instance for the type ITree, defined as
data ITree a = Leaf (Int -> a)
| Node [ITree a]
4. Give an example of a type of kind * -> * which cannot be made an instance of Functor (without using undefined).
5. Is this statement true or false?
The composition of two Functors is also a Functor.
If false, give a counterexample; if true, prove it by exhibiting some appropriate Haskell code.

## 3.3 Laws

As far as the Haskell language itself is concerned, the only requirement to be a Functor is an implementation of fmap with the proper type. Any sensible Functor instance, however, will also satisfy the functor laws, which are part of the definition of a mathematical functor. There are two:

fmap id = id
fmap (g . h) = (fmap g) . (fmap h)

∗ Technically, these laws make f and fmap together an endofunctor on Hask, the category of Haskell types (ignoring , which is a party pooper). See Wikibook: Category theory.

Together, these laws ensure that fmap g does not change the structure of a container, only the elements. Equivalently, and more simply, they ensure that fmap g changes a value without altering its context .

The first law says that mapping the identity function over every item in a container has no effect. The second says that mapping a composition of two functions over every item in a container is the same as first mapping one function, and then mapping the other.

As an example, the following code is a “valid” instance of Functor (it typechecks), but it violates the functor laws. Do you see why?

-- Evil Functor instance
instance Functor [] where
fmap _ [] = []
fmap g (x:xs) = g x : g x : fmap g xs

Any Haskeller worth their salt would reject this code as a gruesome abomination.

Unlike some other type classes we will encounter, a given type has at most one valid instance of Functor. This can be proven via the free theorem for the type of fmap. In fact, GHC can automatically derive Functor instances for many data types.

A similar argument also shows that any Functor instance satisfying the first law (fmap id = id) will automatically satisfy the second law as well. Practically, this means that only the first law needs to be checked (usually by a very straightforward induction) to ensure that a Functor instance is valid.

Exercises
1. Although it is not possible for a Functor instance to satisfy the first Functor law but not the second, the reverse is possible. Give an example of a (bogus) Functor instance which satisfies the second law but not the first.
2. Which laws are violated by the evil Functor instance for list shown above: both laws, or the first law alone? Give specific counterexamples.

## 3.4 Intuition

There are two fundamental ways to think about fmap. The first has already been mentioned: it takes two parameters, a function and a container, and applies the function “inside” the container, producing a new container. Alternately, we can think of fmap as applying a function to a value in a context (without altering the context).

Just like all other Haskell functions of “more than one parameter”, however, fmap is actually curried: it does not really take two parameters, but takes a single parameter and returns a function. For emphasis, we can write fmap’s type with extra parentheses: fmap :: (a -> b) -> (f a -> f b). Written in this form, it is apparent that fmap transforms a “normal” function (g :: a -> b) into one which operates over containers/contexts (fmap g :: f a -> f b). This transformation is often referred to as a lift; fmap “lifts” a function from the “normal world” into the “f world”.

A good starting point for reading about the category theory behind the concept of a functor is the excellent Haskell wikibook page on category theory.

# 4 Applicative

A somewhat newer addition to the pantheon of standard Haskell type classes, applicative functors represent an abstraction lying in between Functor and Monad in expressivity, first described by McBride and Paterson. The title of their classic paper, Applicative Programming with Effects, gives a hint at the intended intuition behind the Applicative type class. It encapsulates certain sorts of “effectful” computations in a functionally pure way, and encourages an “applicative” programming style. Exactly what these things mean will be seen later.

## 4.1 Definition

Recall that Functor allows us to lift a “normal” function to a function on computational contexts. But fmap doesn’t allow us to apply a function which is itself in a context to a value in a context. Applicative gives us just such a tool, (<*>). It also provides a method, pure, for embedding values in a default, “effect free” context. Here is the type class declaration for Applicative, as defined in Control.Applicative:

class Functor f => Applicative f where
pure  :: a -> f a
(<*>) :: f (a -> b) -> f a -> f b

Note that every Applicative must also be a Functor. In fact, as we will see, fmap can be implemented using the Applicative methods, so every Applicative is a functor whether we like it or not; the Functor constraint forces us to be honest.

∗ Recall that ($) is just function application: f$ x = f x.

g <$> x = pure g <*> x. ## 4.3 Instances Most of the standard types which are instances of Functor are also instances of Applicative. Maybe can easily be made an instance of Applicative; writing such an instance is left as an exercise for the reader. The list type constructor [] can actually be made an instance of Applicative in two ways; essentially, it comes down to whether we want to think of lists as ordered collections of elements, or as contexts representing multiple results of a nondeterministic computation (see Wadler’s How to replace failure by a list of successes). Let’s first consider the collection point of view. Since there can only be one instance of a given type class for any particular type, one or both of the list instances of Applicative need to be defined for a newtype wrapper; as it happens, the nondeterministic computation instance is the default, and the collection instance is defined in terms of a newtype called ZipList. This instance is: newtype ZipList a = ZipList { getZipList :: [a] } instance Applicative ZipList where pure = undefined -- exercise (ZipList gs) <*> (ZipList xs) = ZipList (zipWith ($) gs xs)

To apply a list of functions to a list of inputs with (<*>), we just match up the functions and inputs elementwise, and produce a list of the resulting outputs. In other words, we “zip” the lists together with function application, ($); hence the name ZipList. The other Applicative instance for lists, based on the nondeterministic computation point of view, is: instance Applicative [] where pure x = [x] gs <*> xs = [ g x | g <- gs, x <- xs ] Instead of applying functions to inputs pairwise, we apply each function to all the inputs in turn, and collect all the results in a list. Now we can write nondeterministic computations in a natural style. To add the numbers 3 and 4 deterministically, we can of course write (+) 3 4. But suppose instead of 3 we have a nondeterministic computation that might result in 2, 3, or 4; then we can write pure (+) <*> [2,3,4] <*> pure 4 or, more idiomatically, (+) <$> [2,3,4] <*> pure 4.

There are several other Applicative instances as well:

• IO is an instance of Applicative, and behaves exactly as you would think: to execute m1 <*> m2, first m1 is executed, resulting in a function f, then m2 is executed, resulting in a value x, and finally the value f x is returned as the result of executing m1 <*> m2.
• ((,) a) is an Applicative, as long as a is an instance of Monoid (section Monoid). The a values are accumulated in parallel with the computation.
• The Applicative module defines the Const type constructor; a value of type Const a b simply contains an a. This is an instance of Applicative for any Monoid a; this instance becomes especially useful in conjunction with things like Foldable (section Foldable).
• The WrappedMonad and WrappedArrow newtypes make any instances of Monad (section Monad) or Arrow (section Arrow) respectively into instances of Applicative; as we will see when we study those type classes, both are strictly more expressive than Applicative, in the sense that the Applicative methods can be implemented in terms of their methods.
Exercises
1. Implement an instance of Applicative for Maybe.
2. Determine the correct definition of pure for the ZipList instance of Applicative—there is only one implementation that satisfies the law relating pure and (<*>).

## 4.4 Intuition

McBride and Paterson’s paper introduces the notation $[[g \; x_1 \; x_2 \; \cdots \; x_n]]\$ to denote function application in a computational context. If each $x_i\$ has type $f \; t_i\$ for some applicative functor $f\$, and $g\$ has type $t_1 \to t_2 \to \dots \to t_n \to t\$, then the entire expression $[[g \; x_1 \; \cdots \; x_n]]\$ has type $f \; t\$. You can think of this as applying a function to multiple “effectful” arguments. In this sense, the double bracket notation is a generalization of fmap, which allows us to apply a function to a single argument in a context.

Why do we need Applicative to implement this generalization of fmap? Suppose we use fmap to apply g to the first parameter x1. Then we get something of type f (t2 -> ... t), but now we are stuck: we can’t apply this function-in-a-context to the next argument with fmap. However, this is precisely what (<*>) allows us to do.

This suggests the proper translation of the idealized notation $[[g \; x_1 \; x_2 \; \cdots \; x_n]]\$ into Haskell, namely

g <$> x1 <*> x2 <*> ... <*> xn, recalling that Control.Applicative defines (<$>) as convenient infix shorthand for fmap. This is what is meant by an “applicative style”—effectful computations can still be described in terms of function application; the only difference is that we have to use the special operator (<*>) for application instead of simple juxtaposition.

Note that pure allows embedding “non-effectful” arguments in the middle of an idiomatic application, like

g <$> x1 <*> pure x2 <*> x3 which has type f d, given g :: a -> b -> c -> d x1 :: f a x2 :: b x3 :: f c The double brackets are commonly known as “idiom brackets”, because they allow writing “idiomatic” function application, that is, function application that looks normal but has some special, non-standard meaning (determined by the particular instance of Applicative being used). Idiom brackets are not supported by GHC, but they are supported by the Strathclyde Haskell Enhancement, a preprocessor which (among many other things) translates idiom brackets into standard uses of (<$>) and (<*>). This can result in much more readable code when making heavy use of Applicative.

15
> getProduct (mconcat . map Product $[1..5]) 120 This example code is silly, of course; we could just write sum [1..5] and product [1..5]. Nevertheless, these instances are useful in more generalized settings, as we will see in the section Foldable. • Any and All are newtype wrappers providing Monoid instances for Bool (under disjunction and conjunction, respectively). • There are three instances for Maybe: a basic instance which lifts a Monoid instance for a to an instance for Maybe a, and two newtype wrappers First and Last for which mappend selects the first (respectively last) non-Nothing item. • Endo a is a newtype wrapper for functions a -> a, which form a monoid under composition. • There are several ways to “lift” Monoid instances to instances with additional structure. We have already seen that an instance for a can be lifted to an instance for Maybe a. There are also tuple instances: if a and b are instances of Monoid, then so is (a,b), using the monoid operations for a and b in the obvious pairwise manner. Finally, if a is a Monoid, then so is the function type e -> a for any e; in particular, g mappend h is the function which applies both g and h to its argument and then combines the results using the underlying Monoid instance for a. This can be quite useful and elegant (see example). • The type Ordering = LT || EQ || GT is a Monoid, defined in such a way that mconcat (zipWith compare xs ys) computes the lexicographic ordering of xs and ys (if xs and ys have the same length). In particular, mempty = EQ, and mappend evaluates to its leftmost non-EQ argument (or EQ if both arguments are EQ). This can be used together with the function instance of Monoid to do some clever things (example). • There are also Monoid instances for several standard data structures in the containers library (haddock), including Map, Set, and Sequence. Monoid is also used to enable several other type class instances. As noted previously, we can use Monoid to make ((,) e) an instance of Applicative: instance Monoid e => Applicative ((,) e) where pure x = (mempty, x) (u, f) <*> (v, x) = (u mappend v, f x) Monoid can be similarly used to make ((,) e) an instance of Monad as well; this is known as the writer monad. As we’ve already seen, Writer and WriterT are a newtype wrapper and transformer for this monad, respectively. Monoid also plays a key role in the Foldable type class (see section Foldable). ## 9.4 Other monoidal classes: Alternative, MonadPlus, ArrowPlus The Alternative type class (haddock) is for Applicative functors which also have a monoid structure: class Applicative f => Alternative f where empty :: f a (<|>) :: f a -> f a -> f a Of course, instances of Alternative should satisfy the monoid laws. Likewise, MonadPlus (haddock) is for Monads with a monoid structure: class Monad m => MonadPlus m where mzero :: m a mplus :: m a -> m a -> m a The MonadPlus documentation states that it is intended to model monads which also support “choice and failure”; in addition to the monoid laws, instances of MonadPlus are expected to satisfy mzero >>= f = mzero v >> mzero = mzero which explains the sense in which mzero denotes failure. Since mzero should be the identity for mplus, the computation m1 mplus m2 succeeds (evaluates to something other than mzero) if either m1 or m2 does; so mplus represents choice. The guard function can also be used with instances of MonadPlus; it requires a condition to be satisfied and fails (using mzero) if it is not. A simple example of a MonadPlus instance is [], which is exactly the same as the Monoid instance for []: the empty list represents failure, and list concatenation represents choice. In general, however, a MonadPlus instance for a type need not be the same as its Monoid instance; Maybe is an example of such a type. A great introduction to the MonadPlus type class, with interesting examples of its use, is Doug Auclair’s MonadPlus: What a Super Monad! in the Monad.Reader issue 11. There used to be a type class called MonadZero containing only mzero, representing monads with failure. The do-notation requires some notion of failure to deal with failing pattern matches. Unfortunately, MonadZero was scrapped in favor of adding the fail method to the Monad class. If we are lucky, someday MonadZero will be restored, and fail will be banished to the bit bucket where it belongs (see MonadPlus reform proposal). The idea is that any do-block which uses pattern matching (and hence may fail) would require a MonadZero constraint; otherwise, only a Monad constraint would be required. Finally, ArrowZero and ArrowPlus (haddock) represent Arrows (see below) with a monoid structure: class Arrow arr => ArrowZero arr where zeroArrow :: b arr c class ArrowZero arr => ArrowPlus arr where (<+>) :: (b arr c) -> (b arr c) -> (b arr c) ## 9.5 Further reading Monoids have gotten a fair bit of attention recently, ultimately due to a blog post by Brian Hurt, in which he complained about the fact that the names of many Haskell type classes (Monoid in particular) are taken from abstract mathematics. This resulted in a long haskell-cafe thread arguing the point and discussing monoids in general. ∗ May its name live forever. However, this was quickly followed by several blog posts about Monoid . First, Dan Piponi wrote a great introductory post, Haskell Monoids and their Uses. This was quickly followed by Heinrich Apfelmus’s Monoids and Finger Trees, an accessible exposition of Hinze and Paterson’s classic paper on 2-3 finger trees, which makes very clever use of Monoid to implement an elegant and generic data structure. Dan Piponi then wrote two fascinating articles about using Monoids (and finger trees): Fast Incremental Regular Expressions and Beyond Regular Expressions In a similar vein, David Place’s article on improving Data.Map in order to compute incremental folds (see the Monad Reader issue 11) is also a good example of using Monoid to generalize a data structure. Some other interesting examples of Monoid use include building elegant list sorting combinators, collecting unstructured information, and a brilliant series of posts by Chung-Chieh Shan and Dylan Thurston using Monoids to elegantly solve a difficult combinatorial puzzle (followed by part 2, part 3, part 4). As unlikely as it sounds, monads can actually be viewed as a sort of monoid, with join playing the role of the binary operation and return the role of the identity; see Dan Piponi’s blog post. # 10 Foldable The Foldable class, defined in the Data.Foldable module (haddock), abstracts over containers which can be “folded” into a summary value. This allows such folding operations to be written in a container-agnostic way. ## 10.1 Definition The definition of the Foldable type class is: class Foldable t where fold :: Monoid m => t m -> m foldMap :: Monoid m => (a -> m) -> t a -> m foldr :: (a -> b -> b) -> b -> t a -> b foldl :: (a -> b -> a) -> a -> t b -> a foldr1 :: (a -> a -> a) -> t a -> a foldl1 :: (a -> a -> a) -> t a -> a This may look complicated, but in fact, to make a Foldable instance you only need to implement one method: your choice of foldMap or foldr. All the other methods have default implementations in terms of these, and are presumably included in the class in case more efficient implementations can be provided. ## 10.2 Instances and examples The type of foldMap should make it clear what it is supposed to do: given a way to convert the data in a container into a Monoid (a function a -> m) and a container of a’s (t a), foldMap provides a way to iterate over the entire contents of the container, converting all the a’s to m’s and combining all the m’s with mappend. The following code shows two examples: a simple implementation of foldMap for lists, and a binary tree example provided by the Foldable documentation. instance Foldable [] where foldMap g = mconcat . map g data Tree a = Empty | Leaf a | Node (Tree a) a (Tree a) instance Foldable Tree where foldMap f Empty = mempty foldMap f (Leaf x) = f x foldMap f (Node l k r) = foldMap f l ++ f k ++ foldMap f r where (++) = mappend The foldr function has a type similar to the foldr found in the Prelude, but more general, since the foldr in the Prelude works only on lists. The Foldable module also provides instances for Maybe and Array; additionally, many of the data structures found in the standard containers library (for example, Map, Set, Tree, and Sequence) provide their own Foldable instances. ## 10.3 Derived folds Given an instance of Foldable, we can write generic, container-agnostic functions such as: -- Compute the size of any container. containerSize :: Foldable f => f a -> Int containerSize = getSum . foldMap (const (Sum 1)) -- Compute a list of elements of a container satisfying a predicate. filterF :: Foldable f => (a -> Bool) -> f a -> [a] filterF p = foldMap (\a -> if p a then [a] else []) -- Get a list of all the Strings in a container which include the -- letter a. aStrings :: Foldable f => f String -> [String] aStrings = filterF (elem 'a') The Foldable module also provides a large number of predefined folds, many of which are generalized versions of Prelude functions of the same name that only work on lists: concat, concatMap, and, or, any, all, sum, product, maximum(By), minimum(By), elem, notElem, and find. The reader may enjoy coming up with elegant implementations of these functions using fold or foldMap and appropriate Monoid instances. There are also generic functions that work with Applicative or Monad instances to generate some sort of computation from each element in a container, and then perform all the side effects from those computations, discarding the results: traverse_, sequenceA_, and others. The results must be discarded because the Foldable class is too weak to specify what to do with them: we cannot, in general, make an arbitrary Applicative or Monad instance into a Monoid. If we do have an Applicative or Monad with a monoid structure—that is, an Alternative or a MonadPlus—then we can use the asum or msum functions, which can combine the results as well. Consult the Foldable documentation for more details on any of these functions. Note that the Foldable operations always forget the structure of the container being folded. If we start with a container of type t a for some Foldable t, then t will never appear in the output type of any operations defined in the Foldable module. Many times this is exactly what we want, but sometimes we would like to be able to generically traverse a container while preserving its structure—and this is exactly what the Traversable class provides, which will be discussed in the next section. ## 10.4 Further reading The Foldable class had its genesis in McBride and Paterson’s paper introducing Applicative, although it has been fleshed out quite a bit from the form in the paper. An interesting use of Foldable (as well as Traversable) can be found in Janis Voigtländer’s paper Bidirectionalization for free!. # 11 Traversable ## 11.1 Definition The Traversable type class, defined in the Data.Traversable module (haddock), is: class (Functor t, Foldable t) => Traversable t where traverse :: Applicative f => (a -> f b) -> t a -> f (t b) sequenceA :: Applicative f => t (f a) -> f (t a) mapM :: Monad m => (a -> m b) -> t a -> m (t b) sequence :: Monad m => t (m a) -> m (t a) As you can see, every Traversable is also a foldable functor. Like Foldable, there is a lot in this type class, but making instances is actually rather easy: one need only implement traverse or sequenceA; the other methods all have default implementations in terms of these functions. A good exercise is to figure out what the default implementations should be: given either traverse or sequenceA, how would you define the other three methods? (Hint for mapM: Control.Applicative exports the WrapMonad newtype, which makes any Monad into an Applicative. The sequence function can be implemented in terms of mapM.) ## 11.2 Intuition The key method of the Traversable class, and the source of its unique power, is sequenceA. Consider its type: sequenceA :: Applicative f => t (f a) -> f (t a) This answers the fundamental question: when can we commute two functors? For example, can we turn a tree of lists into a list of trees? (Answer: yes, in two ways. Figuring out what they are, and why, is left as an exercise. A much more challenging question is whether a list of trees can be turned into a tree of lists.) The ability to compose two monads depends crucially on this ability to commute functors. Intuitively, if we want to build a composed monad M a = m (n a) out of monads m and n, then to be able to implement join :: M (M a) -> M a, that is, join :: m (n (m (n a))) -> m (n a), we have to be able to commute the n past the m to get m (m (n (n a))), and then we can use the joins for m and n to produce something of type m (n a). See Mark Jones’s paper for more details. ## 11.3 Instances and examples What’s an example of a Traversable instance? The following code shows an example instance for the same Tree type used as an example in the previous Foldable section. It is instructive to compare this instance with a Functor instance for Tree, which is also shown. data Tree a = Empty | Leaf a | Node (Tree a) a (Tree a) instance Traversable Tree where traverse g Empty = pure Empty traverse g (Leaf x) = Leaf <$> g x
traverse g (Node l x r) = Node <$> traverse g l <*> g x <*> traverse g r instance Functor Tree where fmap g Empty = Empty fmap g (Leaf x) = Leaf$ g x
fmap     g (Node l x r) = Node (fmap g l)
(g x)
(fmap g r)

It should be clear that the Traversable and Functor instances for Tree are almost identical; the only difference is that the Functor instance involves normal function application, whereas the applications in the Traversable instance take place within an Applicative context, using (<$>) and (<*>). In fact, this will be true for any type. Any Traversable functor is also Foldable, and a Functor. We can see this not only from the class declaration, but by the fact that we can implement the methods of both classes given only the Traversable methods. A good exercise is to implement fmap and foldMap using only the Traversable methods; the implementations are surprisingly elegant. The Traversable module provides these implementations as fmapDefault and foldMapDefault. The standard libraries provide a number of Traversable instances, including instances for [], Maybe, Map, Tree, and Sequence. Notably, Set is not Traversable, although it is Foldable. ## 11.4 Further reading The Traversable class also had its genesis in McBride and Paterson’s Applicative paper, and is described in more detail in Gibbons and Oliveira, The Essence of the Iterator Pattern, which also contains a wealth of references to related work. # 12 Category Category is another fairly new addition to the Haskell standard libraries; you may or may not have it installed depending on the version of your base package. It generalizes the notion of function composition to general “morphisms”. The definition of the Category type class (from Control.Categoryhaddock) is shown below. For ease of reading, note that I have used an infix type constructor arr, in parallel with the infix function type constructor (->). This syntax is not part of Haskell 2010. The second definition shown is the one used in the standard libraries. For the remainder of this document, I will use the infix type constructor arr for Category as well as Arrow. class Category arr where id :: a arr a (.) :: (b arr c) -> (a arr b) -> (a arr c) -- The same thing, with a normal (prefix) type constructor class Category cat where id :: cat a a (.) :: cat b c -> cat a b -> cat a c Note that an instance of Category should be a type constructor which takes two type arguments, that is, something of kind * -> * -> *. It is instructive to imagine the type constructor variable cat replaced by the function constructor (->): indeed, in this case we recover precisely the familiar identity function id and function composition operator (.) defined in the standard Prelude. Of course, the Category module provides exactly such an instance of Category for (->). But it also provides one other instance, shown below, which should be familiar from the previous discussion of the Monad laws. Kleisli m a b, as defined in the Control.Arrow module, is just a newtype wrapper around a -> m b. newtype Kleisli m a b = Kleisli { runKleisli :: a -> m b } instance Monad m => Category (Kleisli m) where id = Kleisli return Kleisli g . Kleisli h = Kleisli (h >=> g) The only law that Category instances should satisfy is that id and (.) should form a monoid—that is, id should be the identity of (.), and (.) should be associative. Finally, the Category module exports two additional operators: (<<<), which is just a synonym for (.), and (>>>), which is (.) with its arguments reversed. (In previous versions of the libraries, these operators were defined as part of the Arrow class.) ## 12.1 Further reading The name Category is a bit misleading, since the Category class cannot represent arbitrary categories, but only categories whose objects are objects of Hask, the category of Haskell types. For a more general treatment of categories within Haskell, see the category-extras package. For more about category theory in general, see the excellent Haskell wikibook page, Steve Awodey’s new book, Benjamin Pierce’s Basic category theory for computer scientists, or Barr and Wells’s category theory lecture notes. Benjamin Russell’s blog post is another good source of motivation and category theory links. You certainly don’t need to know any category theory to be a successful and productive Haskell programmer, but it does lend itself to much deeper appreciation of Haskell’s underlying theory. # 13 Arrow The Arrow class represents another abstraction of computation, in a similar vein to Monad and Applicative. However, unlike Monad and Applicative, whose types only reflect their output, the type of an Arrow computation reflects both its input and output. Arrows generalize functions: if arr is an instance of Arrow, a value of type b arr c can be thought of as a computation which takes values of type b as input, and produces values of type c as output. In the (->) instance of Arrow this is just a pure function; in general, however, an arrow may represent some sort of “effectful” computation. ## 13.1 Definition The definition of the Arrow type class, from Control.Arrow (haddock), is: class Category arr => Arrow arr where arr :: (b -> c) -> (b arr c) first :: (b arr c) -> ((b, d) arr (c, d)) second :: (b arr c) -> ((d, b) arr (d, c)) (***) :: (b arr c) -> (b' arr c') -> ((b, b') arr (c, c')) (&&&) :: (b arr c) -> (b arr c') -> (b arr (c, c')) ∗ In versions of the base package prior to version 4, there is no Category class, and the Arrow class includes the arrow composition operator (>>>). It also includes pure as a synonym for arr, but this was removed since it conflicts with the pure from Applicative. The first thing to note is the Category class constraint, which means that we get identity arrows and arrow composition for free: given two arrows g :: b arr c and h :: c arr d, we can form their composition g >>> h :: b arr d . As should be a familiar pattern by now, the only methods which must be defined when writing a new instance of Arrow are arr and first; the other methods have default definitions in terms of these, but are included in the Arrow class so that they can be overridden with more efficient implementations if desired. ## 13.2 Intuition Let’s look at each of the arrow methods in turn. Ross Paterson’s web page on arrows has nice diagrams which can help build intuition. • The arr function takes any function b -> c and turns it into a generalized arrow b arr c. The arr method justifies the claim that arrows generalize functions, since it says that we can treat any function as an arrow. It is intended that the arrow arr g is “pure” in the sense that it only computes g and has no “effects” (whatever that might mean for any particular arrow type). • The first method turns any arrow from b to c into an arrow from (b,d) to (c,d). The idea is that first g uses g to process the first element of a tuple, and lets the second element pass through unchanged. For the function instance of Arrow, of course, first g (x,y) = (g x, y). • The second function is similar to first, but with the elements of the tuples swapped. Indeed, it can be defined in terms of first using an auxiliary function swap, defined by swap (x,y) = (y,x). • The (***) operator is “parallel composition” of arrows: it takes two arrows and makes them into one arrow on tuples, which has the behavior of the first arrow on the first element of a tuple, and the behavior of the second arrow on the second element. The mnemonic is that g *** h is the product (hence *) of g and h. For the function instance of Arrow, we define (g *** h) (x,y) = (g x, h y). The default implementation of (***) is in terms of first, second, and sequential arrow composition (>>>). The reader may also wish to think about how to implement first and second in terms of (***). • The (&&&) operator is “fanout composition” of arrows: it takes two arrows g and h and makes them into a new arrow g &&& h which supplies its input as the input to both g and h, returning their results as a tuple. The mnemonic is that g &&& h performs both g and h (hence &) on its input. For functions, we define (g &&& h) x = (g x, h x). ## 13.3 Instances The Arrow library itself only provides two Arrow instances, both of which we have already seen: (->), the normal function constructor, and Kleisli m, which makes functions of type a -> m b into Arrows for any Monad m. These instances are: instance Arrow (->) where arr g = g first g (x,y) = (g x, y) newtype Kleisli m a b = Kleisli { runKleisli :: a -> m b } instance Monad m => Arrow (Kleisli m) where arr f = Kleisli (return . f) first (Kleisli f) = Kleisli (\ ~(b,d) -> do c <- f b return (c,d) ) ## 13.4 Laws There are quite a few laws that instances of Arrow should satisfy : arr id = id arr (h . g) = arr g >>> arr h first (arr g) = arr (g *** id) first (g >>> h) = first g >>> first h first g >>> arr (id *** h) = arr (id *** h) >>> first g first g >>> arr fst = arr fst >>> g first (first g) >>> arr assoc = arr assoc >>> first g assoc ((x,y),z) = (x,(y,z)) Note that this version of the laws is slightly different than the laws given in the first two above references, since several of the laws have now been subsumed by the Category laws (in particular, the requirements that id is the identity arrow and that (>>>) is associative). The laws shown here follow those in Paterson’s Programming with Arrows, which uses the Category class. ∗ Unless category-theory-induced insomnolence is your cup of tea. The reader is advised not to lose too much sleep over the Arrow laws , since it is not essential to understand them in order to program with arrows. There are also laws that ArrowChoice, ArrowApply, and ArrowLoop instances should satisfy; the interested reader should consult Paterson: Programming with Arrows. ## 13.5 ArrowChoice Computations built using the Arrow class, like those built using the Applicative class, are rather inflexible: the structure of the computation is fixed at the outset, and there is no ability to choose between alternate execution paths based on intermediate results. The ArrowChoice class provides exactly such an ability: class Arrow arr => ArrowChoice arr where left :: (b arr c) -> (Either b d arr Either c d) right :: (b arr c) -> (Either d b arr Either d c) (+++) :: (b arr c) -> (b' arr c') -> (Either b b' arr Either c c') (|||) :: (b arr d) -> (c arr d) -> (Either b c arr d) A comparison of ArrowChoice to Arrow will reveal a striking parallel between left, right, (+++), (|||) and first, second, (***), (&&&), respectively. Indeed, they are dual: first, second, (***), and (&&&) all operate on product types (tuples), and left, right, (+++), and (|||) are the corresponding operations on sum types. In general, these operations create arrows whose inputs are tagged with Left or Right, and can choose how to act based on these tags. • If g is an arrow from b to c, then left g is an arrow from Either b d to Either c d. On inputs tagged with Left, the left g arrow has the behavior of g; on inputs tagged with Right, it behaves as the identity. • The right function, of course, is the mirror image of left. The arrow right g has the behavior of g on inputs tagged with Right. • The (+++) operator performs “multiplexing”: g +++ h behaves as g on inputs tagged with Left, and as h on inputs tagged with Right. The tags are preserved. The (+++) operator is the sum (hence +) of two arrows, just as (***) is the product. • The (|||) operator is “merge” or “fanin”: the arrow g ||| h behaves as g on inputs tagged with Left, and h on inputs tagged with Right, but the tags are discarded (hence, g and h must have the same output type). The mnemonic is that g ||| h performs either g or h on its input. The ArrowChoice class allows computations to choose among a finite number of execution paths, based on intermediate results. The possible execution paths must be known in advance, and explicitly assembled with (+++) or (|||). However, sometimes more flexibility is needed: we would like to be able to compute an arrow from intermediate results, and use this computed arrow to continue the computation. This is the power given to us by ArrowApply. ## 13.6 ArrowApply The ArrowApply type class is: class Arrow arr => ArrowApply arr where app :: (b arr c, b) arr c If we have computed an arrow as the output of some previous computation, then app allows us to apply that arrow to an input, producing its output as the output of app. As an exercise, the reader may wish to use app to implement an alternative “curried” version, app2 :: b arr ((b arr c) arr c). This notion of being able to compute a new computation may sound familiar: this is exactly what the monadic bind operator (>>=) does. It should not particularly come as a surprise that ArrowApply and Monad are exactly equivalent in expressive power. In particular, Kleisli m can be made an instance of ArrowApply, and any instance of ArrowApply can be made a Monad (via the newtype wrapper ArrowMonad). As an exercise, the reader may wish to try implementing these instances: instance Monad m => ArrowApply (Kleisli m) where app = -- exercise newtype ArrowApply a => ArrowMonad a b = ArrowMonad (a () b) instance ArrowApply a => Monad (ArrowMonad a) where return = -- exercise (ArrowMonad a) >>= k = -- exercise ## 13.7 ArrowLoop The ArrowLoop type class is: class Arrow a => ArrowLoop a where loop :: a (b, d) (c, d) -> a b c trace :: ((b,d) -> (c,d)) -> b -> c trace f b = let (c,d) = f (b,d) in c It describes arrows that can use recursion to compute results, and is used to desugar the rec construct in arrow notation (described below). Taken by itself, the type of the loop method does not seem to tell us much. Its intention, however, is a generalization of the trace function which is also shown. The d component of the first arrow’s output is fed back in as its own input. In other words, the arrow loop g is obtained by recursively “fixing” the second component of the input to g. It can be a bit difficult to grok what the trace function is doing. How can d appear on the left and right sides of the let? Well, this is Haskell’s laziness at work. There is not space here for a full explanation; the interested reader is encouraged to study the standard fix function, and to read Paterson’s arrow tutorial. ## 13.8 Arrow notation Programming directly with the arrow combinators can be painful, especially when writing complex computations which need to retain simultaneous reference to a number of intermediate results. With nothing but the arrow combinators, such intermediate results must be kept in nested tuples, and it is up to the programmer to remember which intermediate results are in which components, and to swap, reassociate, and generally mangle tuples as necessary. This problem is solved by the special arrow notation supported by GHC, similar to do notation for monads, that allows names to be assigned to intermediate results while building up arrow computations. An example arrow implemented using arrow notation, taken from Paterson, is: class ArrowLoop arr => ArrowCircuit arr where delay :: b -> (b arr b) counter :: ArrowCircuit arr => Bool arr Int counter = proc reset -> do rec output <- idA -< if reset then 0 else next next <- delay 0 -< output + 1 idA -< output This arrow is intended to represent a recursively defined counter circuit with a reset line. There is not space here for a full explanation of arrow notation; the interested reader should consult Paterson’s paper introducing the notation, or his later tutorial which presents a simplified version. ## 13.9 Further reading An excellent starting place for the student of arrows is the arrows web page, which contains an introduction and many references. Some key papers on arrows include Hughes’s original paper introducing arrows, Generalising monads to arrows, and Paterson’s paper on arrow notation. Both Hughes and Paterson later wrote accessible tutorials intended for a broader audience: Paterson: Programming with Arrows and Hughes: Programming with Arrows. Although Hughes’s goal in defining the Arrow class was to generalize Monads, and it has been said that Arrow lies “between Applicative and Monad” in power, they are not directly comparable. The precise relationship remained in some confusion until analyzed by Lindley, Wadler, and Yallop, who also invented a new calculus of arrows, based on the lambda calculus, which considerably simplifies the presentation of the arrow laws (see The arrow calculus). Some examples of Arrows include Yampa, the Haskell XML Toolkit, and the functional GUI library Grapefruit. Some extensions to arrows have been explored; for example, the BiArrows of Alimarine et al., for two-way instead of one-way computation. The Haskell wiki has links to many additional research papers relating to Arrows. # 14 Comonad The final type class we will examine is Comonad. The Comonad class is the categorical dual of Monad; that is, Comonad is like Monad but with all the function arrows flipped. It is not actually in the standard Haskell libraries, but it has seen some interesting uses recently, so we include it here for completeness. ## 14.1 Definition The Comonad type class, defined in the Control.Comonad module of the comonad library, is: class Functor f => Copointed f where extract :: f a -> a class Copointed w => Comonad w where duplicate :: w a -> w (w a) extend :: (w a -> b) -> w a -> w b As you can see, extract is the dual of return, duplicate is the dual of join, and extend is the dual of (>>=) (although its arguments are in a different order). The definition of Comonad is a bit redundant (after all, the Monad class does not need join), but this is so that a Comonad can be defined by fmap, extract, and either duplicate or extend. Each has a default implementation in terms of the other. A prototypical example of a Comonad instance is: -- Infinite lazy streams data Stream a = Cons a (Stream a) instance Functor Stream where fmap g (Cons x xs) = Cons (g x) (fmap g xs) instance Copointed Stream where extract (Cons x _) = x -- 'duplicate' is like the list function 'tails' -- 'extend' computes a new Stream from an old, where the element -- at position n is computed as a function of everything from -- position n onwards in the old Stream instance Comonad Stream where duplicate s@(Cons x xs) = Cons s (duplicate xs) extend g s@(Cons x xs) = Cons (g s) (extend g xs) -- = fmap g (duplicate s) ## 14.2 Further reading Dan Piponi explains in a blog post what cellular automata have to do with comonads. In another blog post, Conal Elliott has examined a comonadic formulation of functional reactive programming. Sterling Clover’s blog post Comonads in everyday life explains the relationship between comonads and zippers, and how comonads can be used to design a menu system for a web site. Uustalu and Vene have a number of papers exploring ideas related to comonads and functional programming: # 15 Acknowledgements A special thanks to all of those who taught me about standard Haskell type classes and helped me develop good intuition for them, particularly Jules Bean (quicksilver), Derek Elkins (ddarius), Conal Elliott (conal), Cale Gibbard (Cale), David House, Dan Piponi (sigfpe), and Kevin Reid (kpreid). I also thank the many people who provided a mountain of helpful feedback and suggestions on a first draft of the Typeclassopedia: David Amos, Kevin Ballard, Reid Barton, Doug Beardsley, Joachim Breitner, Andrew Cave, David Christiansen, Gregory Collins, Mark Jason Dominus, Conal Elliott, Yitz Gale, George Giorgidze, Steven Grady, Travis Hartwell, Steve Hicks, Philip Hölzenspies, Edward Kmett, Eric Kow, Serge Le Huitouze, Felipe Lessa, Stefan Ljungstrand, Eric Macaulay, Rob MacAulay, Simon Meier, Eric Mertens, Tim Newsham, Russell O’Connor, Conrad Parker, Walt Rorie-Baety, Colin Ross, Tom Schrijvers, Aditya Siram, C. Smith, Martijn van Steenbergen, Joe Thornber, Jared Updike, Rob Vollmert, Andrew Wagner, Louis Wasserman, and Ashley Yakeley, as well as a few only known to me by their IRC nicks: b_jonas, maltem, tehgeekmeister, and ziman. I have undoubtedly omitted a few inadvertently, which in no way diminishes my gratitude. Finally, I would like to thank Wouter Swierstra for his fantastic work editing the Monad.Reader, and my wife Joyia for her patience during the process of writing the Typeclassopedia. # 16 About the author Brent Yorgey (blog, homepage) is (as of November 2011) a fourth-year Ph.D. student in the programming languages group at the University of Pennsylvania. He enjoys teaching, creating EDSLs, playing Bach fugues, musing upon category theory, and cooking tasty lambda-treats for the denizens of #haskell. # 17 Colophon The Typeclassopedia was written by Brent Yorgey and initally published in March 2009. Painstakingly converted to wiki syntax by User:Geheimdienst in November 2011, after asking Brent’s permission. If something like this tex to wiki syntax conversion ever needs to be done again, here are some vim commands that helped: • %s/\\section{$$[^}]*$$}/=\1=/gc • %s/\\subsection{$$[^}]*$$}/==\1==/gc • %s/^ *\\item /\r* /gc • %s/---/—/gc • %s/\$$$[^]*$$\$/[itex]\1\\ <\/math>/gc Appending “\ ” forces images to be rendered. Otherwise, Mediawiki would go back and forth between one font for short [itex] tags, and another more Tex-like font for longer tags (containing more than a few characters)"" • %s/|$$[^|]*$$|/<code>\1<\/code>/gc • %s/\\dots/.../gc • %s/^\\label{.*$//gc
• %s/\\emph{$$[^}]*$$}/''\1''/gc
• %s/\\term{$$[^}]*$$}/''\1''/gc

The biggest issue was taking the academic-paper-style citations and turning them into hyperlinks with an appropriate title and an appropriate target. In most cases there was an obvious thing to do (e.g. online PDFs of the cited papers or Citeseer entries). Sometimes, however, it’s less clear and you might want to check the original Typeclassopedia PDF with the original bibliography file.

To get all the citations into the main text, I first tried processing the source with Tex or Lyx. This didn’t work due to missing unfindable packages, syntax errors, and my general ineptitude with Tex.

I then went for the next best solution, which seemed to be extracting all instances of “\cite{something}” from the source and in that order pulling the referenced entries from the .bib file. This way you can go through the source file and sorted-references file in parallel, copying over what you need, without searching back and forth in the .bib file. I used:

• egrep -o "\cite\{[^\}]*\}" ~/typeclassopedia.lhs | cut -c 6- | tr "," "\n" | tr -d "}" > /tmp/citations
• for i in $(cat /tmp/citations); do grep -A99 "$i" ~/typeclassopedia.bib|egrep -B99 '^\}\$' -m1 ; done > ~/typeclasso-refs-sorted