Opened 3 years ago

Closed 7 days ago

#2255 closed feature request (rejected)

MeanVarianceCalculator should not perform range checks of the added values

Reported by: mkommend Owned by: mkommend
Priority: medium Milestone: HeuristicLab 4.0
Component: Problems.DataAnalysis Version: 3.3.10
Keywords: Cc:

Description

The OnlineMeanAndVarianceCalculator checks if the added values are in the range -1E13 - 1E13 and if this is not the case reports an error. This behavior is unique across all online calculators and should be removed.

The reason behind this is, that datasets containing such invalid values cannot be modeled with GP and Pearson's R² evaluator (e.g., most Korns datasets), because the algorithm executions are aborted due to invalid quality values.

Change History (11)

comment:1 Changed 3 years ago by mkommend

  • Status changed from new to accepted

comment:2 Changed 3 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from accepted to reviewing

r11417: Unified behavior of online calculators by removing the range check from the OnlineMeanAndVarianceCalculator.

comment:3 Changed 3 years ago by mkommend

  • Owner changed from gkronber to mkommend
  • Status changed from reviewing to assigned

comment:4 follow-up: Changed 3 years ago by mkommend

After r11417 the symbolic regression sample unit test doesn't pass anymore, which means that this changeset breaks the reproducibility of algorithm results.

comment:5 Changed 3 years ago by mkommend

  • Status changed from assigned to accepted

comment:6 in reply to: ↑ 4 Changed 3 years ago by mkommend

Replying to mkommend:

After r11417 the symbolic regression sample unit test doesn't pass anymore, which means that this changeset breaks the reproducibility of algorithm results.

The reason is that before an individual is evaluated and checked if it produces invalid values the linear scaling and range limitation of values is performed. The linear scaling fails if an individual produces estimated values |value| > 1E13, because the linear scaling calculator reports the error invalid value added. Therefore, alpha and beta are reset to 0.0 and 1.0 (no change) and the estimated values are cut to the estimation limit boundaries. The quality calculation succeeds due to the range limitation, but results in a very low quality.

comment:7 Changed 3 years ago by mkommend

r11422: Reverted r11417 due to broken unit tests.

comment:8 Changed 2 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.11 to HeuristicLab 3.3.12

comment:9 Changed 2 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.12 to HeuristicLab 3.3.13

comment:10 Changed 19 months ago by mkommend

  • Milestone changed from HeuristicLab 3.3.13 to HeuristicLab 4.0

comment:11 Changed 7 days ago by mkommend

  • Resolution set to rejected
  • Status changed from accepted to closed

r14981: Recorded merge and reverse merge (11417,11422) in stable.

This ticket will not be implemented due to the reproducibility changes.

Note: See TracTickets for help on using tickets.