Opened 14 years ago
Closed 8 years ago
#1087 closed feature request (done)
Implement multiobjective test functions
Reported by: | abeham | Owned by: | mkommend |
---|---|---|---|
Priority: | high | Milestone: | HeuristicLab 3.3.14 |
Component: | Problems.TestFunctions | Version: | 3.3.14 |
Keywords: | Cc: | b.werth@gmx.at |
Description
There are a number of popular test functions for multiobjective optimization that would fit in the scope of Problems.TestFunctions.
These functions are, e.g. ZDT1-6 (except ZDT5 which uses binary representation), Kursawe, Golinski, Schaffer, Fonseca, DTLZ1-7, ... with some of them having constraints as well.
Change History (61)
comment:1 Changed 14 years ago by swagner
- Summary changed from Implement Problems: Multiobjective Testfunctions to Implement multiobjective test functions
comment:2 Changed 14 years ago by swagner
- Owner changed from abeham to gkronber
- Status changed from new to assigned
comment:3 Changed 11 years ago by gkronber
- Owner changed from gkronber to swagner
- Priority changed from medium to high
comment:4 Changed 9 years ago by mkommend
- Owner changed from swagner to bwerth
comment:5 Changed 9 years ago by mkommend
- Version changed from 3.3 to branch
comment:6 Changed 9 years ago by mkommend
comment:7 Changed 9 years ago by bwerth
r13421: Implemented skeleton structure and first testfunctions(Fonesca, SchafferN1 & SchafferN2)
comment:8 Changed 9 years ago by bwerth
r13448 Implemented more Testfunctions; enabled setting of SolutionSize
comment:9 Changed 9 years ago by bwerth
r13451 extensive Testing; fixed minor bugs
comment:10 Changed 9 years ago by bwerth
r13515 minor bugfixes and added unittests
comment:11 Changed 9 years ago by bwerth
r13562 added more unittests and comments
comment:12 Changed 9 years ago by mkommend
r13563: Added reference to System.Drawing in HL.Problems.MultiObjectiveTestFunctions.
comment:13 Changed 9 years ago by bwerth
r13620 regorganized testfunctions, added view for qualities
comment:14 Changed 9 years ago by bwerth
r13622 recommited project file, renamed IMOSolution to MOSolution
comment:15 Changed 9 years ago by mkommend
r13623: Added instance provider base class for MOTestFunctions.
comment:16 Changed 9 years ago by bwerth
r13672 added Analyzers, reworked PFStore, added licence information, cleaned code
comment:17 Changed 9 years ago by bwerth
r13673 fixed bug in DTLZ1, DTLZ3 and DTLZ7
comment:18 Changed 9 years ago by bwerth
r13725 minor bugfixes, added Parameters to Analyzers, convenience Tooltips for ScatterPlot
comment:19 Changed 9 years ago by bwerth
- Owner changed from bwerth to mkommend
- Status changed from assigned to reviewing
comment:20 Changed 9 years ago by mkommend
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.14
comment:21 Changed 9 years ago by bwerth
r13726 fixed Naming in FastHyperVolumeTests.cs
comment:22 Changed 9 years ago by bwerth
r13729 annotated unit tests
comment:23 Changed 8 years ago by bwerth
r13771 bugfix + additional relative HV calculation added
comment:24 Changed 8 years ago by bwerth
r13776 added ConstrainedFunctions
comment:25 Changed 8 years ago by bwerth
r13894 set pareto-Series to lower Opacity so overlapping series can be identified
comment:26 Changed 8 years ago by bwerth
r13936 added more functions (IHR1-4, IHR6, CIGTAB, ELLI)
comment:27 Changed 8 years ago by bwerth
r13988 reformatted project, bugfixes
comment:28 Changed 8 years ago by bwerth
r13989 removed unused usings
comment:29 Changed 8 years ago by bwerth
r13991 removed unused function
comment:30 Changed 8 years ago by mkommend
r14018: Rewrote and adapted the multi-objective calculators.
comment:31 Changed 8 years ago by mkommend
Review comments
I only had time to review and work on the calculator classes. Review of remaining classes and functionality is still pending.
Calculators
- Inconsistent usage of points and front for member names. Please check and unify.
- GenerationalDistance.cs
- Is the comment "TODO build a kd-tree, sort the array, do something intelligent here" still valid
- Why has not been something intelligent done?
- How should the parameter p be set? A comment (XMLDoc) on this would be helpful
- The method is everywhere called with p == 1.0
- IMHO the Calculate method calculates math(1/n (\sum d[i])^{1/p})? which is not math(1/n \sum d[i]^{1/p})? as described in the XMLDoc.
- This is only true for p == 1.0.
- Is the comment "TODO build a kd-tree, sort the array, do something intelligent here" still valid
- HyperVolume.cs
- Does Calculate2D work with differing maximization values (e.g., min dim0, max dim1)?
- CalculateMD should reuse the domination functionality of NonDominatedSelect (line 97)
- IsDominated must be removed. The method provides little to no benefits.
- Crowding.cs
- No unit tests
- There should be a method that calculates the crowding distance between a point and a set of points / front. This method should then be reused for calculating the crowding distance of set of points / front. The crowding distance calculation should yield the same results as the CrowdingDistanceAssignment operator of the NSGA-II!
- Why have inifinity values to be excluded while calculating the average? Is it normal for the individual calculation to result in infinity?
- Line 58 - 67 can be rewritten as: return pointsums.Where(d => !double.IsPositiveInfinity(d)).DefaultIfEmpty(double.PositiveInfinity).Average();
- What is the difference between HyperVolume.CalculateMD and MultiDimensionalHypervolume?
- The code looks the same to me.
comment:32 Changed 8 years ago by mkommend
- Cc b.werth@gmx.at added
comment:33 Changed 8 years ago by bwerth
r14030 several fixes according to the review comments in comment 31
comment:34 Changed 8 years ago by mkommend
r14044: Refactored and improved analyzers for multi-objective test functions.
comment:35 Changed 8 years ago by mkommend
Review of Analyzers
- Why is the BestKnownFrontParameter defined in the MOTFAnalyzer, although it is only used by the hyper volume analyzers?
- HypervolumeAnalyzer
- Why is the best known hyper volume saved in a parameter of the Analyzer? Should be in the TestFunction / Problem.
- The ReferencePoint should be taken from the TestFunction (no value parameter).
- Reuse DoubleValues by setting their values instead of creating new ones (results). Similar to the changes in r14044.
- Could NonDominatedSelect.selectNonDominatedVectors (line 93) return an empty enumberable? If not the check in line 95 is obsolete.
- NormalizedHypervolumeAnalyzer
- Most of the comments for the hypervolume analyzers apply to this class as well.
- BestKnownFront Zitzler ?!?
- Always use FixedValueParameters for ValueParameters<T> where T in HeuristicLab.Data and T can be modified.
comment:36 Changed 8 years ago by mkommend
comment:37 Changed 8 years ago by mkommend
comment:38 Changed 8 years ago by mkommend
Review of TestFunctions
- IHR
- What is the purpose of the Method Y protected RealVector Y(RealVector x) { return x; }
- Why is the method G defined as abstract and implemented in derived classes, but never called?
- F2 never uses its first parameter r
- Evaluate calls F2(r,y) where y = Y(r) and Y just returns its argument. Therefore y == r. Why does F2 need two arguments at all?
- IHR5 ?
- ELLI
- Line 70 x is never used.
- SchafferN2
- Optimal ParetoFront throws NotImplementedException!
- ZDT5 ?
comment:39 Changed 8 years ago by mkommend
r14069: Moved Pareto fronts to subfolders in the according test functions folder and changed PFStore.
comment:40 Changed 8 years ago by mkommend
r14073: Worked on Multi-objective test function problem and adapted plugin dependencies.
comment:41 Changed 8 years ago by mkommend
r14085: Further refactoring of testfunction problem and analyzers.
comment:42 Changed 8 years ago by bwerth
r14090 several fixes according to the review comments 35 and 38
comment:43 Changed 8 years ago by bwerth
r14092 removed NormalizedHypervolumeAnalyzer and IMOFrontModel.cs, refactored ScatterPlotAnalyzer ,fixed bug in HypervolumeAnalyzer
comment:44 Changed 8 years ago by bwerth
r14093 altered Cloning in ScatterPlotContent
comment:45 Changed 8 years ago by bwerth
r14097 refactored Analyzers to use ResultParamters
comment:46 Changed 8 years ago by mkommend
r14108: Fixed bugs in Spacing and Hypervolume calculation, corrected plugin dependencies and minor changes to the scatter plot view.
comment:47 Changed 8 years ago by mkommend
r14111: Change plugin and folder name from HeuristicLab.Problems.MultiObjectiveTestFunctions to HeuristicLab.Problems.Testfunctions.MultiObjective and adapted namespaces and projects accordingly.
Persistence break in the branch for those already using the multi-objective test functions!
comment:48 Changed 8 years ago by mkommend
r14120: Integrated multi-objective test functions in the trunk.
comment:49 Changed 8 years ago by mkommend
r14121: Copied unit tests from branch and adapted path of multi-objective test function project file.
comment:50 Changed 8 years ago by mkommend
- Version changed from branch to 3.3.14
comment:51 Changed 8 years ago by mkommend
r14122: Renamed Testfunctions folder to TestFunctions to comply to name reference in the solution and that the compile works also with case-sensitive file systems (e.g., Linux).
comment:52 Changed 8 years ago by bburlacu
comment:53 Changed 8 years ago by mkommend
r14127: Readded accidentally deleted TestFunctions.
comment:54 Changed 8 years ago by bburlacu
r14130: Remove HintPath for DataVisualization dll and unnecessary source control xml tags from project file.
comment:55 Changed 8 years ago by pfleck
r14132 Corrected namespace of the MultiObjectiveTestFunctionParetoFrontScatterPlotView.
comment:56 Changed 8 years ago by mkommend
comment:57 Changed 8 years ago by mkommend
- Status changed from reviewing to readytorelease
comment:58 Changed 8 years ago by pfleck
r14150 Changed BestKnownFront parameter to OptionalValueParameter to allow null if no best front is known (DTLZ8 and SchafferN2 and DTLZs with objectives > 2).
comment:59 Changed 8 years ago by mkommend
comment:60 Changed 8 years ago by mkommend
comment:61 Changed 8 years ago by mkommend
- Resolution set to done
- Status changed from readytorelease to closed
r13394: Created new plugin for multi-objective test functions.