Implement multiobjective test functions

[abeham]

There are a number of popular test functions for multiobjective optimization that would fit in the scope of Problems.TestFunctions.

These functions are, e.g. ZDT1-6 (except ZDT5 which uses binary representation), Kursawe, Golinski, Schaffer, Fonseca, DTLZ1-7, ... with some of them having constraints as well.

[mkommend]

r13394: Created new plugin for multi-objective test functions.

[bwerth]

r13421: Implemented skeleton structure and first testfunctions(Fonesca, SchafferN1 & SchafferN2)

[bwerth]

r13448 Implemented more Testfunctions; enabled setting of SolutionSize

[bwerth]

r13451 extensive Testing; fixed minor bugs

[bwerth]

r13515 minor bugfixes and added unittests

[bwerth]

r13562 added more unittests and comments

[mkommend]

r13563: Added reference to System.Drawing in HL.Problems.MultiObjectiveTestFunctions.

[bwerth]

r13620 regorganized testfunctions, added view for qualities

[bwerth]

r13622 recommited project file, renamed IMOSolution to MOSolution

[mkommend]

r13623: Added instance provider base class for MOTestFunctions.

[bwerth]

r13672 added Analyzers, reworked PFStore, added licence information, cleaned code

[bwerth]

r13673 fixed bug in DTLZ1, DTLZ3 and DTLZ7

[bwerth]

r13725 minor bugfixes, added Parameters to Analyzers, convenience Tooltips for ScatterPlot

[bwerth]

r13726 fixed Naming in FastHyperVolumeTests.cs

[bwerth]

r13729 annotated unit tests

[bwerth]

r13771 bugfix + additional relative HV calculation added

[bwerth]

r13776 added ConstrainedFunctions

[bwerth]

r13894 set pareto-Series to lower Opacity so overlapping series can be identified

[bwerth]

r13936 added more functions (IHR1-4, IHR6, CIGTAB, ELLI)

[bwerth]

r13988 reformatted project, bugfixes

[bwerth]

r13989 removed unused usings

[bwerth]

r13991 removed unused function

[mkommend]

r14018: Rewrote and adapted the multi-objective calculators.

[mkommend]

Review comments

I only had time to review and work on the calculator classes. Review of remaining classes and functionality is still pending.

Calculators

• Inconsistent usage of points and front for member names. Please check and unify.

• GenerationalDistance.cs
  • Is the comment "TODO build a kd-tree, sort the array, do something intelligent here" still valid
    • Why has not been something intelligent done?
  • How should the parameter p be set? A comment (XMLDoc) on this would be helpful
    • The method is everywhere called with p == 1.0
  • IMHO the Calculate method calculates math(1/n (\sum d[i])^{1/p})? which is not math(1/n \sum d[i]^{1/p})? as described in the XMLDoc.
    • This is only true for p == 1.0.
• HyperVolume.cs
  • Does Calculate2D work with differing maximization values (e.g., min dim0, max dim1)?
  • CalculateMD should reuse the domination functionality of NonDominatedSelect (line 97)
  • IsDominated must be removed. The method provides little to no benefits.
• Crowding.cs
  • No unit tests
  • There should be a method that calculates the crowding distance between a point and a set of points / front. This method should then be reused for calculating the crowding distance of set of points / front. The crowding distance calculation should yield the same results as the CrowdingDistanceAssignment operator of the NSGA-II!
  • Why have inifinity values to be excluded while calculating the average? Is it normal for the individual calculation to result in infinity?
  • Line 58 - 67 can be rewritten as: return pointsums.Where(d => !double.IsPositiveInfinity(d)).DefaultIfEmpty(double.PositiveInfinity).Average();

• What is the difference between HyperVolume.CalculateMD and MultiDimensionalHypervolume?
  • The code looks the same to me.

[bwerth]

r14030 several fixes according to the review comments in comment 31

[mkommend]

r14044: Refactored and improved analyzers for multi-objective test functions.

[mkommend]

Review of Analyzers

• Why is the BestKnownFrontParameter defined in the MOTFAnalyzer, although it is only used by the hyper volume analyzers?

• HypervolumeAnalyzer
  • Why is the best known hyper volume saved in a parameter of the Analyzer? Should be in the TestFunction / Problem.
  • The ReferencePoint should be taken from the TestFunction (no value parameter).
  • Reuse DoubleValues by setting their values instead of creating new ones (results). Similar to the changes in r14044.
  • Could NonDominatedSelect.selectNonDominatedVectors (line 93) return an empty enumberable? If not the check in line 95 is obsolete.

• NormalizedHypervolumeAnalyzer
  • Most of the comments for the hypervolume analyzers apply to this class as well.
  • BestKnownFront Zitzler ?!?
  • Always use FixedValueParameters for ValueParameters<T> where T in HeuristicLab.Data and T can be modified.

[mkommend]

r14065: Refactored instance providers and data descriptors for MOTF. r14066: Forgot to commit the project file.

[mkommend]

r14067: Refactored multi-objective test functions. r14068: Added checks for min and max objectives to test functions.

[mkommend]

Review of TestFunctions

• IHR
  • What is the purpose of the Method Y protected RealVector Y(RealVector x) { return x; }
  • Why is the method G defined as abstract and implemented in derived classes, but never called?
  • F2 never uses its first parameter r
  • Evaluate calls F2(r,y) where y = Y(r) and Y just returns its argument. Therefore y == r. Why does F2 need two arguments at all?
  • IHR5 ?
• ELLI
  • Line 70 x is never used.
• SchafferN2
  • Optimal ParetoFront throws NotImplementedException!
• ZDT5 ?

[mkommend]

r14069: Moved Pareto fronts to subfolders in the according test functions folder and changed PFStore.

[mkommend]

r14073: Worked on Multi-objective test function problem and adapted plugin dependencies.

[mkommend]

r14085: Further refactoring of testfunction problem and analyzers.

[bwerth]

r14090 several fixes according to the review comments 35 and 38

[bwerth]

r14092 removed NormalizedHypervolumeAnalyzer and IMOFrontModel.cs, refactored ScatterPlotAnalyzer ,fixed bug in HypervolumeAnalyzer

[bwerth]

r14093 altered Cloning in ScatterPlotContent

[bwerth]

r14097 refactored Analyzers to use ResultParamters

[mkommend]

r14108: Fixed bugs in Spacing and Hypervolume calculation, corrected plugin dependencies and minor changes to the scatter plot view.

[mkommend]

r14111: Change plugin and folder name from HeuristicLab.Problems.MultiObjectiveTestFunctions to HeuristicLab.Problems.Testfunctions.MultiObjective and adapted namespaces and projects accordingly.

Persistence break in the branch for those already using the multi-objective test functions!

[mkommend]

r14120: Integrated multi-objective test functions in the trunk.

[mkommend]

r14121: Copied unit tests from branch and adapted path of multi-objective test function project file.

[mkommend]

r14122: Renamed Testfunctions folder to TestFunctions to comply to name reference in the solution and that the compile works also with case-sensitive file systems (e.g., Linux).

[bburlacu]

r14124, r14125: Accidental commit, reverted. r14126: Remove duplicate PreBuildEvent in the project file.

[mkommend]

r14127: Readded accidentally deleted TestFunctions.

[bburlacu]

r14130: Remove HintPath for DataVisualization dll and unnecessary source control xml tags from project file.

[pfleck]

r14132 Corrected namespace of the MultiObjectiveTestFunctionParetoFrontScatterPlotView.

[mkommend]

This ticket depends on #2281 and #2594.

The following changesets must be merged to stable: 14120, 14121, 14122, 14124, 14125, 14126, 14127, 14130, 14132

[pfleck]

r14150 Changed BestKnownFront parameter to OptionalValueParameter to allow null if no best front is known (DTLZ8 and SchafferN2 and DTLZs with objectives > 2).

[mkommend]

r14164: Reverse merged changes in r14125 to hive solution and settings.

[mkommend]

r14165: Merged r14120, r14121, r14122, r14124, r14125, r14126, r14127, r14130, r14132, r14150, r14164 into stable.