Refactor certain parts for data-based modeling

[gkronber]

• Move translation to AutoDiff out of evaluator
• Create ISymbolicExpressionTree for linear model from vector of coefficients
• Create two-dimensional array from ProblemData

[gkronber]

r14378:

• created a folder for all classes related to transformation from and to trees
• created a transformator which takes a tree and uses AutoDiff to produce a function and gradient function for the tree.
• moved code from SymbolicRegressionConstantOptimizationEvaluator to TreeToAutoDiffTermTransformator to make AutoDiff for trees more accessible

[gkronber]

r14390:

• renaming of folder "Transformation" to "Converters" to distinguish between transformations for variables (from data preprocessing) and classes for transformation of trees.
• renamed SymbolicDataAnalysisExpressionTreeSimplifier -> TreeSimplifier
• Implemented a converter to create a linar model as a symbolic expression tree

[gkronber]

r14391:

• extended converter for linear models to support lagged variables and changed AR(k) to use this method

[gkronber]

Todo: remove class scaling (needs persistence backwards compat in GPR)

[gkronber]

r14393:

• removed AlglibUtil.cs and added an extension method .ToArray(names, rows) to IDataset instead.
• refactored transformation so that it is possible to apply an transformation without resetting the parameters
• Used transformations instead of Scaling as far as possible.
• Moved TakeEvery extension method to HL.Common

[gkronber]

r14394: renamed TreeSimplifier in unit test

[gkronber]

r14396: added methods to get training and test input matrices from ProblemData

[gkronber]

r14400: reverse merge of r14378, r14390, r14391, r14393, r14394, r14396 because the changes cannot be merged over to the #2650 branch

[gkronber]

This is needed by #745.

However, #2650 should be merged to the trunk first before we apply the changes reverted in r14400 because there are many conflicting changes.

[mkommend]

r14507: Added caching of parameters and removed resetting of training and test partition in AdjustProblemDataProperties (used for changing the problem data of a solution) in DataAnalysisProblemData.

[gkronber]

r14840: applied changes from r14378 again

[gkronber]

r14843: applied r14390, r14391, r14393, r14394, r14396 again (resolving conflicts)

[gkronber]

r14848: fixed build fail of test solution

[gkronber]

r14851: removed unnecessary variables

[gkronber]

r14854: introduced original code for scaling from AlglibUtil into GaussianProcessModel to guarantee backwards compatibility regarding unit test results

[mkommend]

#2697: Made symbol fields readonly in TreeSimplifier.

[mkommend]

Review comments

Enumerable Extensions

• Why has the ext. method TakeEvery been added, although it is never used?
  gkronber: Good question. I traced this back to r7154. Seems we never needed this. Can be removed.
  mkommend: Addressed with comment:19.

HL.Problems.DataAnalysis.Symbolic

• Delete empty and unused folder Transformation
  mkommend: Addressed with comment:20.

HeuristicLab.Problems.DataAnalysis Transformations

• Why do we need transformations?!?!
• Can't we delete all of them?
• If not, why don't we move them to the DataPreprocessing and forget about them?

gkronber: They are used in DataPreprocessing. After the changes in this ticket they are also used for scaling of input variables (instead of the Scaling class). Any symbolic regression model can have transformations. If we want to keep support for this at least the interface is necessary in the symbolic namespace.

Edit: After discussion we decided to move all transformations to the DataPreprocessing namespace and remove support for transformations from symbolic models.

Converters

• I don't really get the benefit of the Convert class, besides listing somehow related conversion operations.
  gkronber: addressed with comment:22
• AFAIK the TreeSimplifier is stateless? If that's the case why don't we provide a static method for simplification. e.g. var simplifiedTree = TreeSimplifier.Simplify(originalTree);.
  gkronber: addressed with comment:23
• TreeToAutoDiffTermConverter encapsulates the functionality nicely. This new refactoring would also enable to use exception for control flow handling in TryConvertToAutoDiff(node,term) instead of always checking the transformation result (true or false).
  gkronber: addressed with comment:24

Finished reviewing the changes in this tickets.

[mkommend]

Replying to mkommend:

Enumerable Extensions

• Why has the ext. method TakeEvery been added, although it is never used?

gkronber: Good question. I traced this back to r7154. Seems we never needed this. Can be removed.

r14944: Removed TakeEvery methods from EnumerableExtensions.cs

[mkommend]

Replying to mkommend:

HL.Problems.DataAnalysis.Symbolic

• Delete empty and unused folder Transformation

r14945: Removed empty folder HL.Problems.DataAnalysis.Symbolic\Transformation.

[gkronber]

r14948: removed static Convert class

[gkronber]

r14949: made TreeSimplifier static

[gkronber]

r14950: code improvement in TreeToAutoDiffTermConverter

[gkronber]

Since MCTS has been removed (already merged to stable #2581) there are conflicts when merging this ticket to stable.

[gkronber]

r15140: merged r14507 from trunk to stable

[gkronber]

r15141: merged r14840 from trunk to stable

[gkronber]

r15142: merged r14843 (resolving conflicts in csproj file for HL.Algorithms.DataAnalysis because MCTS has been removed)

[gkronber]

r15143: merged r14845, r14851, r14854 from trunk to stable

[gkronber]

r15144: merged r14944, r14945, r14948 from trunk to stable

[gkronber]

r14938 also belongs to this ticket

[gkronber]

r15145: merged r14938,r14949,r14950 from trunk to stable (all changesets merged)