variable subsitution

May 19, 2014 at 12:13 PM
Hello,

is there any support for variable substitution? Something like

var x = new Variable();
var f = 2.0 * Function.Sin(x);

var y = new Variable();
var z = new Variable();
var g = f(y) + f(z);

var result = g.Value(y | 2.0, z | 3.0);

Thanks for making your work publicly available!

Regards,

Eric.
Coordinator
May 19, 2014 at 5:32 PM
Hi Eric

I'm actually working on this same problem myself at the moment. Obviously, you would have to use the chain rule somehow, preferable performed automatically by the library. It's probably fairly simple with only one variable, but more complicated to make general for any number of variable and any order of derivatives. If doing it manually it's much easier to use the forward-mode automatic differentiation as implemented in the DualNumber class. I'll let you know if find a way to do this generically.

Morten
May 20, 2014 at 7:15 AM
Hi Morten,

Having looked at the library more closely I think what I wrote above would be hard to implement because the Function class, despite its name, really represents a mathematical expression, whereas mathematical functions are higher-order objects (composed of an n-uple of argument variables and a mathematical expression).

So I guess the distinction between function composition (for which the chain rule makes sense) and variable substitution within an expression is useful. The former is certainly very complicated, and would require some kind of concept of "variable scope" within the evaluation and compilation functions. The latter on the other hand is a sort of "re-factoring" operation and should be fairly simple to implement. I will try to do it when I have a sec.

I haven't looked at the forward-mode differentiation and DualNumber, I not sure I understand what that's for.

Eric.
May 20, 2014 at 2:39 PM
I have got the following working
var p = new Variable("p");
var x = new Variable("x");
var y = new Variable("y");
var z = new Variable("z");

var f = Function.Pow(x, p);
var g = f[x | y * z, p | 2.0] + f[x | Function.Sin(y), p | 3.0];

// the following expression evaluates to (4*3)^2 + Sin(4)^3
var result = g.Value(y | 4.0, z | 3.0);

// the following expression evaluates to true
var test = (f == f[x | x]) && (f == f[z | 3.0]);
by adding these new classes
    public interface IVariableSubstitution
    {
        Variable Variable
        {
            get;
        }

        Function Substitute
        {
            get;
        }
    }

    public class VariableSubstitution : IVariableSubstitution
    {
        public VariableSubstitution(Variable variable, Function substitute)
        {
            Variable = variable;
            Substitute = substitute;
        }

        public Variable Variable
        {
            get;
            protected set;
        }

        public Function Substitute
        {
            get;
            protected set;
        }
    }
inserting the following code into the Function class
        protected class SubstitutionContext
        {
            Dictionary<Function, Function> cache = new Dictionary<Function, Function>();

            public SubstitutionContext(IVariableSubstitution[] substitutions)
            {
                foreach(var s in substitutions)
                    cache.Add(s.Variable, s.Substitute);
            }

            public bool Contains(Function f)
            {
                return cache.ContainsKey(f);
            }

            public Function this[Function f]
            {
                get
                {
                    Function g;
                    if(cache.TryGetValue(f, out g))
                        return g;
                    else
                    {
                        cache[f] = g = f.Substitute(this);
                        return g;
                    }
                }
            }
        }

        protected virtual Function Substitute(SubstitutionContext context)
        {
            throw new NotSupportedException();
        }

        public Function this[params IVariableSubstitution[] substitutions]
        {
            get
            {
                SubstitutionContext context = new SubstitutionContext(substitutions);
                if(context.Contains(this))
                    return context[this];
                else
                    return this.Substitute(context);
            }
        }
and in each function class implementing a Substitute function, for example in AddFunction
            protected override Function Substitute(Function.SubstitutionContext context)
            {
                var cf = context[f];
                var cg = context[g];
                if(cf != f || cg != g)
                    return cf + cg;
                return this;
            }
Let me know what you think about this approach!
Coordinator
May 20, 2014 at 2:56 PM
That not a bad idea.

I think I'll generalize the PartialValue method to allow replacing variables with a Function object instead of just numbers (and rename it to Substitute, to be explicit). However, I'm not sure derivatives works correctly in your code, as I don't see the chain rule being applied anywhere, but that's probably easy to implement.
May 20, 2014 at 3:02 PM
Derivatives should work, since the function gets literally replaced with a new one with the substitutions applied (so there's no need for nor indeed any way to apply chain rule, since, like i mentioned above, there's no conceptual function composition but outright expression refactoring).

Let me know if you want my code.

By the way I also did the following in my local copy: in the Derivative method of each function class I check if the derivative of the inner functions is zero, if so intelligently return a zero function. I have found this greatly speeds up execution (even of compiled functions) in my particular use case, because it's a very high dimensional function expressed as a sum of many terms, but many of the variables only act on one term so this way you avoid evaluating all the derivatives of all the terms which would work out to zero anyway. I don't know if that's clear.

So let me know if you want me to e-mail you it or whatever.

Regards

Eric.