CodePlexProject Hosting for Open Source Software

(A hybrid of DualNumber combined with Function inside may also be considered.)

The DualNumber representation works by keeping track of the first and second derivatives while performing operations, starting by specifying the known properties of the variables.

// Define DualNumber as variables. We specify our knowledge about the partial derivatives. The Hessian is implicitly zero. DualNumber x = new DualNumber(2.0, new Vector(new double[] { 1.0, 0.0 })); DualNumber y = new DualNumber(3.0, new Vector(new double[] { 0.0, 1.0 })); // Computer a new value through some aritmetics. This is our function. DualNumber f = x * DualNumber.Exp(x * y); // Show one of the second order partial derivatives after the operations above. Console.WriteLine(f.Hessian[0, 1]);

Since all intermediate computations are thrown away during the process, the memory usage keeps low, assuming that garbage collection works efficiently. I've used DualNumber, DualMatrix, and DualVector extensively for very complicated problems myself, and the garbage collection part has worked just fine. No worries here.

To do numerical optimization on a DualNumber structure, DualNumberFunction can be used to perform the proper transformation.

// Traditional variables (i.e. not DualNumber). Variable x = new Variable(); Variable y = new Variable(); // Create the Function using DualNumberFunction. Function f = DualNumberFunction.Create( delegate(IDualNumberTransform transform) { // When this delegate is called a DualNumber is assigned to represent each Variable. DualNumber x0 = transform[x]; DualNumber y0 = transform[y]; // Compute the actual function using DualNumber. In this case the Rosenbrock function. return DualNumber.Sqr(1.0 - x0) + 100.0 * DualNumber.Sqr(y0 - DualNumber.Sqr(x0)); }, x, y); // Now just the usual procedure to start the optimizer. IpoptOptimizer o = new IpoptOptimizer(); o.Variables.Add(x, y); o.ObjectiveFunction = f; o.Run(x | 0.5, y | 0.3);

There is one problem associated with the large amount of short-lived objects created by DualNumber, though, and this manifests itself when doing parallel computations in multiple threads. Each object creation (and memory allocation) requires obtaining a lock internally in .NET, and this has shown to cause the CPU utilization to drop to around 50% for 12 cores; the other 50% of the time the threads are waiting for obtaining this lock. One solution to this issue is to perform the computations in separate processes with their own virtual memory space and only synchronize the final value of the function.

Last edited Aug 11, 2011 at 8:17 PM by bakkedal, version 13