Opened 5 years ago

Closed 4 years ago

#1919 closed defect (done)

Classification ensemble cannot be loaded if no samples are in training

Reported by: sforsten Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.8
Component: Problems.DataAnalysis Version: 3.3.8
Keywords: Cc:

Description

If an classification ensemble is created and the problem data is changed, the ensemble can be saved, but not loaded again, if the new problem data contains all samples in test and no samples in training, because the threshold cannot be calculated.

The threshold is calculated with the training partition, which can't be done if there are no samples in training.

Change History (14)

comment:1 Changed 5 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.8
  • Owner changed from sforsten to mkommend
  • Status changed from new to accepted

comment:2 Changed 5 years ago by mkommend

  • Owner changed from mkommend to sforsten
  • Status changed from accepted to reviewing

r8531: Refactored calculation of thresholds for SymbolicDiscriminantFunctionClassficationModels and removed the automatic recalculation of thresholds during solution creation.

Last edited 5 years ago by sforsten (previous) (diff)

comment:3 Changed 5 years ago by sforsten

  • Owner changed from sforsten to mkommend
  • Status changed from reviewing to readytorelease

Works correctly now

comment:4 follow-ups: Changed 5 years ago by gkronber

  • Status changed from readytorelease to reviewing

I also reviewed r8531:

  • I don't understand why the method to change the thresholds should be static when it receives the model as the first parameter anyway. Can we change this back so that the two methods for setting thresholds are instance methods?
  • the changes in the plugin infrastructure should have been committed to a different ticket. Is this a performance improvement? Please add a comment on this change here.

comment:5 in reply to: ↑ 4 Changed 5 years ago by gkronber

Replying to gkronber:

  • the changes in the plugin infrastructure should have been committed to a different ticket. Is this a performance improvement? Please add a comment on this change here.

Answering the previous question myself: The change was reverted in ticket #1923. Please see further discussion in that ticket.

Last edited 4 years ago by gkronber (previous) (diff)

comment:6 Changed 5 years ago by mkommend

  • Status changed from reviewing to assigned

comment:7 Changed 5 years ago by mkommend

  • Status changed from assigned to accepted

comment:8 in reply to: ↑ 4 Changed 5 years ago by mkommend

Replying to gkronber:

I also reviewed r8531:

  • I don't understand why the method to change the thresholds should be static when it receives the model as the first parameter anyway. Can we change this back so that the two methods for setting thresholds are instance methods?

You were right this should be instance methods. This was overseen during the relocation of the method from the solution to the model.

In addition, the same argument also applies to theScale method in all symbolic data analysis models.

comment:9 Changed 5 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from accepted to reviewing

r8550: Changed threshold calculation methods to instance instead of static methods.

comment:10 Changed 5 years ago by mkommend

r8551: Adapted forgotten evaluator to r8550.

comment:11 Changed 5 years ago by mkommend

r8552: Corrected event handling regarding changing of thresholds in DiscriminantClassificationSolutions.

comment:12 Changed 4 years ago by gkronber

  • Owner changed from gkronber to mkommend

Review comments for r8550:8852:

comment:13 Changed 4 years ago by mkommend

  • Status changed from reviewing to readytorelease

comment:14 Changed 4 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.6 to 3.3.8
Note: See TracTickets for help on using tickets.