Improve Constants Optimization for Symbolic Regression

[mkommend]

• Constants optimization should work without linear scaling
• Constants optimization should report the number of function and gradient evaluations

[mkommend]

r15447: Adapted constants optimization and auto diff converter to not add linear scaling terms.

[mkommend]

r15448: Added counts of function and gradient evaluations performed by constants optimization as results.

Maximum gradient evaluations = maximum iterations + 1
Maximum function evaluations = maximum iterations * 2 + 1;

[gkronber]

Reviewed r15448.

Are 'result parameters' already available in the trunk?

Reading the code it is odd to find a 'ResultParameter'. I think at least the properties should be renamed (FunctionEvaluationsResultParameter -> FunctionEvaluationsResult, FunctionEvaluationsResultParameterName -> FunctionEvaluationsResultName)

[mkommend]

Replying to gkronber:

Are 'result parameters' already available in the trunk?

Resultparameter were introduced with #2281 over a year ago!

Reading the code it is odd to find a 'ResultParameter'. I think at least the properties should be renamed (FunctionEvaluationsResultParameter -> FunctionEvaluationsResult, FunctionEvaluationsResultParameterName -> FunctionEvaluationsResultName)

I have already implemented the proposed changes, but during a final review and test I decided against committing them. The reason therefore is that an execution context must be present (and manually set line 169) to allow access to the result and the postfix parameter indicates that.

[gkronber]

r15480: moved the scaling parameters to the end of the parameter vector to be able to remove the c.Skip(2).ToArray() call and removed unnecessary .ToArray() calls

[gkronber]

Reviewed r15447. Concerning r15448 I have the feeling that the algorithm is slower because of the updates of function and gradient evaluation results. However, I have not really tested this.

Please check r15480 and release.

[mkommend]

r15480 is a good idea (too get rid of the .ToArray() calls), but is not implemented correctly.

It might be that r15448 slows down constants optimization. An option would be to make this analysis optional and disabled per default.

[gkronber]

r15481: reverted changeset r15480 partially to fix unit test

[gkronber]

After some tests I conclude that the order of parameters in lsfit() has an influence on the optimization results. Therefore, we cannot put the scaling parameters at the end of the parameter vector without breaking backwards compatibility.

Please add a parameter to make reporting of the number of gradient and function evaluations optional.

[mkommend]

r15483: Added option for reporting function and gradient evaluation in the constants optimization evaluator.

Unfortunately, you have been right and reporting the evaluations has a huge impact on the performance. Quick tests on my work station for the OSGP SymReg sample show execution times of ~ 5.1 min with and ~ 3.02 min without reporting (fixed seed of 0).

[gkronber]

Reviewed r15483. Thanks for testing this.

[mkommend]

r15515: Merged r15447, r15448, r15480, r15481, r15483 into stable.