Project Description

C# automatic differentiation and numerical optimization library, transparent use with operator overloading, unlimited order of differentiation and unlimited number of variables, very flexible support for matrix algebra, on-the-fly function compilation to IL code for very fast evaluation. Various numerical optimization methods are supported.

Samples

Hello World

A small Hello World sample.

// Variables.
Variable x = new Variable();
Variable y = new Variable();

// Define a function.
Function f = x * Function.Exp(x * y);

// Define a new function as one of the second order partial derivatives.
Function fx = f.Derivative(x);
Function fxy = fx.Derivative(y);

// Evaluate this function at (x, y) = (2, 3).
Console.WriteLine(fxy.Value(x | 2.0, y | 3.0));

// Or equivalently without operator overloading.
Console.WriteLine(fxy.Value(new Point(new VariableAssignment(x, 2.0), new VariableAssignment(y, 3.0))));

Numerical optimization

More about optimization in FuncLib here: Optimization

FuncLib provides an easy to use framework for numerical optimization. Currently, the BFGS method and the powerful Ipopt are supported; in case of Ipopt through an interface to the precompiled Ipopt binaries. Constraints are specified intuitively using C# operator overloading (BFGS supports bounds on variables only, whereas Ipopt supports general non-linear constraints). The sparsity structure of the derivatives is determined automatically or may be specified explicitly. Much time has been spend on making the underlying representation as efficient as possible while still having a user-friendly programmer interface.

// Variables.
Variable x1 = new Variable();
Variable x2 = new Variable();
Variable x3 = new Variable();
Variable x4 = new Variable();

// Objective function and non-linear constraints.
Function f = x1 * x4 * (x1 + x2 + x3) + x3;
Function g1 = x1 * x2 * x3 * x4;
Function g2 = Function.Sqr(x1) + Function.Sqr(x2) + Function.Sqr(x3) + Function.Sqr(x4);

// Prepare the optimizer with variables bounds and non-linear equality and inequality constraints.
IpoptOptimizer o = new IpoptOptimizer();
o.Variables.Add(x1, x2, x3, x4);
o.ObjectiveFunction = f;
o.Constraints.Add(g1 >= 25.0);
o.Constraints.Add(g2 == 40.0);
o.Constraints.Add(x1 => 1.0, x1 <= 5.0);
o.Constraints.Add(x2 => 1.0, x2 <= 5.0);
o.Constraints.Add(x3 => 1.0, x3 <= 5.0);
o.Constraints.Add(x4 => 1.0, x4 <= 5.0);

// Run optimization starting from (x1, x2, x3, x4) = (1, 5, 5, 1). Not required to satisfy the constraints.
IOptimizerResult or = o.Run(x1 | 1.0, x2 | 5.0, x3 | 5.0, x4 | 1.0);

Console.WriteLine("f(x1, x2, x3, x4) = " + or.OptimalValue);
Console.WriteLine("g1(x1, x2, x3, x4) = " + g1.Value(or.OptimalPoint));
Console.WriteLine("g2(x1, x2, x3, x4) = " + g2.Value(or.OptimalPoint));
Console.WriteLine("x1 = " + or.OptimalPoint[x1]);
Console.WriteLine("x2 = " + or.OptimalPoint[x2]);
Console.WriteLine("x3 = " + or.OptimalPoint[x3]);
Console.WriteLine("x4 = " + or.OptimalPoint[x4]);

Matrix algebra with functions

FuncLib provides versatile matrix algebra capabilities while still allowing unlimited automatic differentiation. This is just a small sample to get a feeling of how it works.

// A matrix with mixed functions and constants.
FunctionMatrix a = new FunctionMatrix(new Function[,] { { f, g }, { 1.0, f * g } });

// Inverse matrix.
FunctionMatrix b = FunctionMatrix.Inverse(a);

// Partial derivative with respect to x of each entry.
FunctionMatrix bx = b.Derivative(x);

// Evaluate first entry of the resulting matrix.
Function h = bx[0, 0];
Console.WriteLine(h.Value(x | 2.0, y | 3.0));

Compiled functions

Any function represented by a recursive object structure may be replaced by a flat function of highly optimized IL code. The compiled IL code is encapsulated in the same object structure for transparency.

// Compile to IL code using the variables given.
CompiledFunction h = Compiler.Compile(g, x, y);

// Evaluate again. Now the order of the variables is fixed like an ordinary C# method.
Console.WriteLine(h.Value(2.0, 3.0));

// Or evaluate as any non-compiled function.
Console.WriteLine(h.Value(x | 2.0, y | 3.0));

// Show generated code.
Console.WriteLine(h.GeneratedCode);

Source code notes

It's easy to get up running. Just download the latest version and open the solution in Visual Studio 2010. The Express edition is freely available. Older editions are not directly supported at present but it's easy to create a new solution and import all the C# files. Remember to run in Release mode if high performance is important.

Last edited Jun 22, 2012 at 10:43 AM by bakkedal, version 38