Overfitting analyzer for symbolic regression

[gkronber]

Follow up to ticket #1142

[gkronber]

Copied overfitting analyzer for symbolic regression from feature exploration branch with r5192.

[gkronber]

Introduced base class for operators that evaluate symbolic regression models on a validation set with r5197.

[gkronber]

Changed FixedValidationBestScaledSymbolicRegressionSolutionAnalyzer to extend from SymbolicRegressionValidationAnalyzer with r5198.

[gkronber]

Added overfitting analyzer to default operators of symbolic regression problems to make sure it also works correctly in cross-validation with r5260.

[mkommend]

Reviewed the r5192, r5197, r5198 and r5260.

Comments:

• The FixedValidationBestScaledSymbolicRegressionSolutionAnalyzer does not check if the estimation limits parameter have a value.
• In my opinion all ValueLookupParameter and LookupParameters must have null checks before accessing their actual value.
• The following source is too complicated:

      double correlationThreshold;
        if (OverfittingParameter.ActualValue != null && OverfittingParameter.ActualValue.Value) {
          // if is already overfitting => have to reach the upper threshold to switch back to non-overfitting state
          correlationThreshold = UpperCorrelationThresholdParameter.ActualValue.Value;
        } else {
          // if currently in non-overfitting state => have to reach to lower threshold to switch to overfitting state
          correlationThreshold = LowerCorrelationThresholdParameter.ActualValue.Value;
        }
        bool overfitting = r < correlationThreshold;
        OverfittingParameter.ActualValue = new BoolValue(overfitting);
  

I think the following is more readable and does the same thing:

      //start with no overfitting
      if (OverfittingParameter.ActualValue == null) OverfittingParameter.ActualValue = new BoolValue(false);
      double lowerCorrelationThreshold = LowerCorrelationThresholdParameter.ActualValue.Value;
      double upperCorrelationThreshold = UpperCorrelationThresholdParameter.ActualValue.Value;

      if (!(lowerCorrelationThreshold < r && r < upperCorrelationThreshold)) {
        bool overfitting = r < lowerCorrelationThreshold;
        OverfittingParameter.ActualValue = new BoolValue(overfitting);
      }

[gkronber]

r5436: clarified code in overfitting analyzer.

[gkronber]

r5437: added default values for upper and lower estimation limits if they are not specified where necessary.

[gkronber]

Please review the changeset r5436:5437 and set the status of the ticket to ready-to-release if there are no more objections.

[mkommend]

Changesets r5436 and r5437 look fine.