Free cookie consent management tool by TermsFeed Policy Generator

Opened 11 years ago

Closed 11 years ago

#2093 closed feature request (done)

Evaluator for symbolic data analysis that calculates the mean of logarithmic residuals

Reported by: gkronber Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.10
Component: Problems.DataAnalysis.Symbolic.Regression Version: 3.3.10
Keywords: Cc:

Description

In preliminary experiments, we found that this improves success rates for data sets with a few very large/small target values (e.g. Korns benchmark instances containing inverses of near zero values). In these cases the squared error or absolute error puts too much emphasis on modeling the outlier values. In such instances small variations are not stemming from noise but are actually the signal that should be modeled and a log-error evaluator would put more weight on such small values.

Change History (21)

comment:1 Changed 11 years ago by gkronber

  • Owner changed from mkommend to gkronber
  • Status changed from new to accepted
  • Version changed from 3.3.8 to branch

comment:2 Changed 11 years ago by gkronber

Created a feature development branch with r10202

comment:3 Changed 11 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

r10203: added log residual evaluator for symbolic regression

comment:4 Changed 11 years ago by gkronber

  • Version changed from branch to 3.3.10

comment:5 Changed 11 years ago by gkronber

  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.10
  • Version changed from 3.3.10 to branch

comment:6 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from reviewing to assigned

Despite the source code looking fine, I am unsure if this functionality should be included in trunk right now. First of all the logarithm is not defined for 0, which is not handled in the evaluator (skipping invalid values?). Second, the current implementation calculates avg(log(abs(y'-y)) which tends to -Inf for very small values (this may be an issue if one data sample was hit with a very high precision). Third, there is no warning / indication that despite linear scaling is requested, it is not performed.

comment:7 Changed 11 years ago by gkronber

r10305: changed log residual evaluator to calculate avg(log( 1 + abs(y - pred)))

comment:8 Changed 11 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from assigned to reviewing

comment:9 Changed 11 years ago by mkommend

Please put a comment / description that linear scaling is never performed for this evaluator. Otherwise everything looks fine.

comment:10 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from reviewing to assigned

comment:11 Changed 11 years ago by gkronber

  • Status changed from assigned to accepted

comment:12 Changed 11 years ago by gkronber

r10482: added a notice that the log residual evaluator does not apply linear scaling

comment:13 Changed 11 years ago by gkronber

r10483: merged trunk changes to prepare for reintegration

comment:14 Changed 11 years ago by gkronber

r10484: reintegrated branch for log residual evaluator

comment:15 Changed 11 years ago by gkronber

r10485: deleted branch for log residual evaluator

comment:16 Changed 11 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

Please review r10482 and r10484.

comment:17 Changed 11 years ago by gkronber

r10486: fixed build fail

comment:18 Changed 11 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from reviewing to readytorelease

comment:19 Changed 11 years ago by mkommend

  • Version changed from branch to 3.3.10

comment:20 Changed 11 years ago by gkronber

Depends on #2119

Last edited 11 years ago by mkommend (previous) (diff)

comment:21 Changed 11 years ago by mkommend

  • Resolution set to done
  • Status changed from readytorelease to closed

r10509: Merged r10483 and r10486 into stable.

Note: See TracTickets for help on using tickets.