Opened 10 years ago
Last modified 5 years ago
#2361 closed feature request
Adapted symbolic classification evaluators to increase the sensitivity of a model — at Version 15
| Reported by: | ehopf | Owned by: | ehopf |
|---|---|---|---|
| Priority: | medium | Milestone: | HeuristicLab 3.3.16 |
| Component: | Problems.DataAnalysis.Symbolic.Classification | Version: | trunk |
| Keywords: | Cc: |
Description (last modified by ehopf)
Two experimental symbolic classification evaluators. Besides considering a normalized mean squared error, the first evaluator (WeightedPerformanceMeasuresEvaluator) also takes the false negative and false positive rate of the model into account. The evaluator was used in Affenzeller et al., Increasing the Sensitivity of Cancer Predictors Using Confidence Based Ensemble Modeling, EUROCAST'15. The second evaluator introduces residuals weights to the normal mean squared error.
Change History (15)
comment:1 Changed 10 years ago by ehopf
- Owner changed from mkommend to ehopf
- Status changed from new to accepted
comment:2 Changed 10 years ago by ehopf
comment:3 Changed 10 years ago by ehopf
r12209: Adapted the project settings for the SensitivityEvaluator branch.
comment:4 Changed 10 years ago by ehopf
r12210: Implemented the first prototype of the new evaluator.
comment:5 Changed 10 years ago by ehopf
r12211: Simple refactoring of SymbolicClassificationSingleObjectiveWeightedPerformanceMeasuresEvaluator.
comment:6 Changed 10 years ago by ehopf
r12212: Branched HL.Problems.DataAnalysis to fix a bug in the ClassificationPerformanceMeasuresCalculator.
comment:7 Changed 10 years ago by ehopf
r12213: Adjusted the project settings of the SensitivityEvaluator branch.
comment:8 Changed 10 years ago by ehopf
r12216: Removed redundant calculations.
comment:9 Changed 10 years ago by ehopf
r12218: Added missing estimated values bound calculation and removed duplicate code. Note that the current implementation ignores the linear scaling option.
comment:10 Changed 10 years ago by ehopf
r12302: Added a NotSupported-Exception for the case that the user activates the applyLinearScaling-option. Additionally made some execution time measurements with and without invocation of the ToArray-Extension-Method after the calculation of the bounded estimated values.
Measurements: OSGA -> Symbolic Classification -> Weighted Performance Measures Evaluator -> Seed 962761388
without ToArray(): 1.00:00:54.5801344 2.00:00:53.7666428 3.00:00:53.7478323
with ToArray(): 1.00:00:53.1704467 2.00:00:52.8202198 3.00:00:52.8622417
comment:11 Changed 10 years ago by ehopf
r12309: Adjusted Release Project settings.
comment:12 Changed 10 years ago by ehopf
r12311: Removed a unnecessary method call.
comment:13 Changed 10 years ago by mkommend
- Version changed from 3.3.11 to branch
comment:14 Changed 10 years ago by ehopf
comment:15 Changed 9 years ago by ehopf
- Description modified (diff)
- Summary changed from Adapted symbolic classification evaluator to increase the sensitivity to Adapted symbolic classification evaluators to increase the sensitivity of a model
r12204: Branched HL.Problems.DataAnalysis.Symbolic.Classification to implement the evaluator.