The number of iterations for constant optimization is fixed in the simplifier view

[gkronber]

Currently, the number of steps for constants optimization in the simplifier view for symbolic regression solutions is fixed to 50. Sometimes it is necessary to click the 'Optimize'-button multiple times because constants optimization does not converge within the first 50 steps.

Wouldn't it be better to leave the number of iterations open and stop constants optimization only when the algorithm is converged? alglib already supports this.

[gkronber]

@mkommend: please comment

[mkommend]

This idea sounds reasonable. However, there are a few points that should be considered:

• Does the constants optimization always converge?
• How much longer does it take until constants optimization is stopped?
• Currently constants optimization already stops before the 50 iterations are reached, if convergence is reached.
• It also affects #1666, that shows a progress bar for constants optimization. A progress that stops at 100% is not possible anymore, because the max. number of iterations would not be known in advance.
  • What about increasing (doubling) the number of iterations?

[abeham]

According to the ​docs the alglib.lsfitresults method tells you if further improvement is possible in which case it could be run again:

    Info    -   completion code:
                    * -7    gradient verification failed.
                            See LSFitSetGradientCheck() for more information.
                    *  2    relative step is no more than EpsX.
                    *  5    MaxIts steps was taken
                    *  7    stopping conditions are too stringent,
                            further improvement is impossible

[gkronber]

I think it would be OK to just use a large number for maximum iterations as the algorithm stops if it converges before the maximum iterations are reached. Maybe we set the number to 1000?

It should not be necessary to click "Optimize" multiple times.

[pfleck]

In #2845 we also want to fix/implement a "cancelable" progress. We could use this to give the user the option to stop the constant-opt. process and use the result of the latest iteration.

[mkommend]

I've tried a 100 iterations locally for some models and this still hasn't been enough so that further improvement by additional iterations can be achieved. A severe draw back was that CoOp takes a long time for larger models / datasets and while it is running the whole solution is blocked and the progress is displayed.

The best solution would be to automatically restart CoOp if further improvements can be achieved (see abeham's response) or let it stop based on the improvements achieved or the gradient information instead of using a fixed number of iterations as stopping criterion. However, this is only suitable if CoOp becomes cancel-able / stoppable so that the GUI is not blocked.

[gkronber]

r16798: made the constant optimization process stoppable, repeat 50 optimization steps until the result is unchanged.

This is just a suggested improvement based on the comments above. Please review.

[mkommend]

Reviewed r16798.

[mkommend]

r17057: Merged r16798 into stable.
r17058: Added merge info for r16798.