# Automatic Differentiation

(Difference between revisions)
 Revision as of 21:39, 12 May 2011 (edit)← Previous diff Current revision (21:40, 12 May 2011) (edit) (undo) Line 20: Line 20: * [http://hackage.haskell.org/cgi-bin/hackage-scripts/package/rad rad] (reverse mode) * [http://hackage.haskell.org/cgi-bin/hackage-scripts/package/rad rad] (reverse mode) * [[Vector-space]] (forward mode tower) * [[Vector-space]] (forward mode tower) - * http://comonad.com/haskell/monoids/dist/doc/html/monoids/Data-Ring-Module-AutomaticDifferentiation.html (forward mode) + * [http://comonad.com/haskell/monoids/dist/doc/html/monoids/Data-Ring-Module-AutomaticDifferentiation.html Data.Ring.Module.AutomaticDifferentiation](forward mode) == Power Series == == Power Series ==

## Current revision

Automatic Differentiation enables you to compute both the value of a function at a point and its derivative(s) at the same time.

When using Forward Mode this roughly means that a numerical value is equipped with its derivative with respect to one of your input, which is updated accordingly on every function application. Let the number x0 be equipped with the derivative x1: $\langle x_0,x_1 \rangle$. For example the sinus is defined as:

• $\sin\langle x_0,x_1 \rangle = \langle \sin x_0, x_1\cdot\cos x_0\rangle$

Replacing this single derivative with a lazy list of them can enable you to compute an entire derivative tower at the same time.

However, it becomes more difficult for vector functions, when computing the derivatives in reverse, when computing towers, and/or when trying to minimize the number of computations needed to compute all of the kth partial derivatives of an n-ary function.

Forward mode is suitable when you have fewer arguments than outputs, because it requires multiple applications of the function, one for each input.

Reverse mode is suitable when you have fewer results than inputs, because it requires multiple applications of the function, one for each output.

Implementations:

## 1 Power Series

If you can compute all of the derivatives of a function, you can compute Taylor series from it.