Possibility to do multiple random restarts for non-linear regression (curve fitting) algorithm

[gkronber]

r14258: added random restarts for NonlinearRegression (curve fitting) algorithm

[mkommend]

r14316: Changed nonlinear regression to perform at least one optimization.

[mkommend]

Review comments:

• It should be configurable whether constants are reinitialized with N(0,1) or the actual value is taking as starting point for the optimization
• Remaining code looks OK.

[gkronber]

There is no straight forward way to initialize parameters.

The initial scale of the parameter supplied by the user should ideally be preserved. This can be achieved by using a random factor which is multiplied to the starting value. However, simple multiplication with a positive factor does not allow to switch signs.

Following this line of thinking, I suggest to sample

• f = Math.Exp( N (0,1))
• s = U(0,1) > 0? 1 : -1

and initialize each parameter p_0 = p_start * f * s where p_start is the used supplied starting value.

[gkronber]

r14319:

• added switch to determine if numeric parameters should be initialized randomly.
• fixed a bug in the infix parser

[gkronber]

Connected to #2677

[mkommend]

Reviewed r14258, r14316, and r14319.

[gkronber]

r14564: merged r14258, r14316, r14319 and 14347 from trunk to stable