Opened 3 weeks ago

Last modified 3 days ago

#2760 reviewing feature request

Shuffle samples in the cross-validation wrapper for data analysis algorithms

Reported by: bburlacu Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.15
Component: Algorithms.DataAnalysis Version: 3.3.14
Keywords: Cc:


The cross-validation wrapper should offer an option to shuffle the data samples.

Change History (4)

comment:1 Changed 2 weeks ago by bburlacu

  • Owner set to bburlacu
  • Status changed from new to accepted

r14864: Implement shuffling of crossvalidation samples.

comment:2 Changed 2 weeks ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from accepted to reviewing

comment:3 Changed 2 weeks ago by bburlacu

r14865: Fix issue with resources in CrossValidationView.Designer.cs

comment:4 Changed 3 days ago by gkronber

It seems that in the ensemble the information wether a point was used for training or test is not stored correctly. Reproduce:

  1. Use cross-validation with shuffling and produce an overfit model on purpose.
  2. Check line chart
  3. Expected result: errors for training predictions (yellow) are very small, errors for test predictions (red) are significantly higher.
  4. Actual result: some errors for training predictions are also high, some errors for test points are suspiciously small.
Note: See TracTickets for help on using tickets.