Feature overview

Primitives

Functions are built and evaluated using three primitives: Variable, Function, and Point.

Variable x = new Variable();
Variable y = new Variable();
Function f = x * y;
Point p = new Point(new VariableAssignment(x, 2.0), new VariableAssignment(y, 3.0));
double x = f.Value(p); // Value is 6

Longer function expressions can be constructed from more primitive ones using operator overloads and static methods defined by the abstract Function class. The static methods are used like those in the System.Math class, except they return another function (which derives from Function) instead of a double. To compute the exponential, we may write:

Function g = Function.Exp(f);
double x = g.Value(p); // Value is 403.43

Another operator being defined by Function is the evaluation operator |. When this binary operator is used with a Variable and a double, a VariableAssignment object is created and returned. This way we may define and use the point above simply by writing:

Point p = new Point(x | 2.0, y | 3.0);
f.Value(p);

Or even simpler, using another overload of the Value method:

f.Value(x | 2.0, y | 3.0);

All primitives in FuncLib are immutable, so they can’t be modified once created. An instance of Point can safely be passed to other methods without having to worry about if the object is modified.

Computing derivatives

New functions can also be defined by taking derivatives of existing ones. To compute the first order partial derivative of g with respect to x, we can write:

Function g1 = g.Derivative(x);

We may continue and compute the partial derivative of this new function with respect to y by writing:

Function g2 = g1.Derivative(y);

Of course we can write this in one single statement as

Function g2 = g.Derivative(x).Derivative(y);

Or, using another overload of the Derivative method, we may use the short-hand notation:

Function g2 = g.Derivative(x, y);

Similarly, we can compute second order partial derivative with respect to x twice using:

g.Derivative(x, 2);

Where’s no intrinsic limitation of the order of differentiation or the number of variables being used. The library may even be smart and determine that some of the Function objects can be reused if the same mathematical expression occurs. The canonical example of this is the exponential, since the derivative of the exponential is just the exponential itself. Hence the computation of the 10000th order derivative creates just one single object:

Function f = Function.Exp(x);
f.Derivative(x, 10000);

Partial values

The Value method requires that all variables used by the function are assigned to some value (otherwise a VariableNotAssignedException exception is thrown). It would make sense to define a new function by only assigned values to some of the variables, and let the function depend on the remaining variables. This is the purpose of the PartialValue. Unlike the Value method, which returns a double, the PartialValue returns another instance of Function.

Now f doesn't depend on x anymore, so taking the derivative wi

Compilation

Using named variables

Variable x = new Variable(“x”);
Point p = new Point(x | 0.5);

Serialization

All of the primitives in FuncLib support serialization, including instances of the optimizers. This way a problem may be transferred to another computer for computation. Compiled functions are recompiled from the generated source code upon deserialization.

Reverse-mode and forward-mode

Functions in FuncLib are represented by a recursive objective structure and derivatives are evaluated by what is known as reverse-mode automatic differentiation (unless they’re compiled to a flat structure). This may not be feasible for very complicated functions: we could reach the limit of the stack or we may run out of memory for the object representation of the derivatives. To overcome this limitation, FuncLib also provides forward-mode automatic differentiation though the classes DualNumber (representing up to second derivatives, suitable for numerical optimization) and ExtendedDualNumber (up to third derivatives).

The preferred method in FuncLib since it's easier to represent in a object-oriented structure. The FuncLib primitives Variable and Function are reverse-mode objects. It's also possible to compile this structure to very fast IL code.

To overcome some limitations with the reverse-mode implementation, support for forward-mode computations is also provided with the DualNumber, DualMatrix, and DualVector classes.

See: Performance considerations

Last edited Aug 24, 2011 at 8:04 PM by bakkedal, version 2

Comments

No comments yet.