Improve Ensemble Classification with new Voting

[sforsten]

New voting strategies will be implemented to improve the classification ensemble modelling. Currently only majority voting is available. Weighted voting strategies should improve the results achieved with ensemble modelling. It should be possible to change the voting strategy with an additional parameter or combo box.

Decription from #1630: The voting mechanism for ensembles of classifiers based on discriminating functions should be enhanced to weight the vote of a single classifier based on the output of its discriminating function and the selected threshold. This information should also be included into the calculation of a confidence value for each sample.

[sforsten]

r7454: create branch for new voting strategies in classification ensemble modeling
r7455: delete project files, which have been accidently branched
r7456: create branch for new voting strategies in classification ensemble modeling (second try)
r7457: add additional project to branch "ClassificationEnsembleVoting"
r7458: branch creation complete

[sforsten]

r7459: first implementation of different voting strategies (currently no gui elements are available to choose between the strategies)

[sforsten]

r7464:

• selection of a voting strategy is now possible
• a more sophisticated strategy as discussed with gkronber will be implemented soon

[sforsten]

r7491:

• two new weight calculators have been implemented
• one has been deleted, because it didn't work as expected

[sforsten]

r7504:

• improvements in the usage of the WeightCalculators
• small changes in all class which inherit from the WeightCalculator class

[sforsten]

r7531:

• 2 more strategies have been implemented
• major changes in the inheritance have been made to make it possible to add strategies which don't use a voting strategy with weights
• ClassificationEnsembleSolutionEstimatedClassValuesView doesn't currently show the confidence (has been removed for test purpose)

[sforsten]

r7549:

• models can be selected with a check box
• all strategies are now finished
• major changes have been made to provide the same behaviour when getting the estimated training or test values of an ensemble

[sforsten]

r7559:

• minor bug fix
• removed unused using

[sforsten]

r7562:

• bug fix in NeighbourhoodWeightCalculator
• added GetConfidence method to IClassificationEnsembleSolutionWeightCalculator
• adjusted the confidence column in ClassificationEnsembleSolutionEstimatedClassValuesView

[sforsten]

r7596:

• added before deleted code to restore some functionality (training and test partition of a model a stored in the .hl file again)
• project files have been set to release

[sforsten]

Comments for the review:
MedianThresholdCalculator, PointCertaintyWeightCalculator and ContinuousPointCertaintyWeightCalculator only work with two class classification.

[sforsten]

r7729:

• bugfix the method GetEstimatedValues of DiscriminantClassificationWeightCalculator returns real values and not class values
• changed arguments of method DiscriminantAggregateEstimatedClassValues of DiscriminantClassificationWeightCalculator
• added two calculators

[sforsten]

r7866:

• merged r7454:7863 trunk into branch to make it compatible again
• adjusted GetConfidence of MajorityVoteWeightCalculator to be in an interval of [0, 1]

[sforsten]

r8101:

• added two calculators to test purposes
• ClassificationEnsembleSolutionEstimatedClassValuesView shows the current average confidence of the correct and wrong classified samples
• some calculators have been excluded from the project, so only relevant calculators are shown

[sforsten]

r8177: minor changes

[sforsten]

r8297:

• Corrected namespace of IClassificationEnsembleSolutionWeightCalculator interface
• Corrected calculation of confidence for test and training samples in ClassificationEnsembleSolutionEstimatedClassValuesView
• Added overload method GetConfidence to IClassificationEnsembleSolutionWeightCalculator to calculate more than one point at a time (maybe additional methods for training and test confidence could improve the performance remarkably)
• Added ClassificationEnsembleSolutionConfidenceAccuracyDependence to see how accuracy would behave, if only samples with high confidence would be classified

I reassigned the ticket to myself, because further changes have to be made, especially to improve the performance.

[sforsten]

r8329:

• added new diagram which shows correlation of accuracy and covered samples

[sforsten]

r8507: adapted ClassificationEnsembleSolutionAccuracyToCoveredSamples

[sforsten]

r8508: merged r7864:8507 trunk into branch

[sforsten]

r8510: added AUC to ClassificationEnsembleSolutionAccuracyToCoveredSamples

[sforsten]

r8534:

• merged r8508:8533 from trunk into branch
• AverageThresholdCalculator and MedianThresholdCalculator can now handle multi class classification
• changed combo boxes in ClassificationEnsembleSolutionView to drop down list

[sforsten]

r8811:

• merged r8534:8810 from trunk into branch

[sforsten]

r8814:

• improved performance of confidence calculation
• fixed bug in median confidence calculation
• fixed bug in average confidence calculation
• confidence calculation is now easier for training and test
• removed obsolete view ClassificationEnsembleSolutionConfidenceAccuracyDependence

[sforsten]

r8862: adapted the implementation of ClassificationEnsembleSolutionAccuracyToCoveredSamples to be more similar to ClassificationEnsembleSolutionAccuracyToCoveredSamples

[sforsten]

r8863:

• merged r8810:8862 from trunk into branch
• fixed exception in ClassificationEnsembleSolutionAccuracyToCoveredSamples, if no estimated values are in a partition, so nothing can be shown

[gkronber]

What is the state of this ticket. Would it be possible to merge parts of the new functionality to the trunk soon?

[jkarder]

HL architects: please rename this branch

[abeham]

r16097: renamed branch