First of all congratulations, I like this a lot and it works really well for what I wanted, and is nice and easy to use.
I have plugged your library together with the Math.NET numerics libaries and got Levenberg Marqardt working in a test harness.
so some suggestions are
1 a string parser to allow user defined functions to be submitted and compiled
2 Make use of Math.net library functionality for matrices etc they have done a lot of work on solvers etc