Opened 11 years ago
Closed 11 years ago
#2093 closed feature request (done)
Evaluator for symbolic data analysis that calculates the mean of logarithmic residuals
Reported by: | gkronber | Owned by: | gkronber |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.10 |
Component: | Problems.DataAnalysis.Symbolic.Regression | Version: | 3.3.10 |
Keywords: | Cc: |
Description
In preliminary experiments, we found that this improves success rates for data sets with a few very large/small target values (e.g. Korns benchmark instances containing inverses of near zero values). In these cases the squared error or absolute error puts too much emphasis on modeling the outlier values. In such instances small variations are not stemming from noise but are actually the signal that should be modeled and a log-error evaluator would put more weight on such small values.
Change History (21)
comment:1 Changed 11 years ago by gkronber
- Owner changed from mkommend to gkronber
- Status changed from new to accepted
- Version changed from 3.3.8 to branch
comment:2 Changed 11 years ago by gkronber
comment:3 Changed 11 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from accepted to reviewing
r10203: added log residual evaluator for symbolic regression
comment:4 Changed 11 years ago by gkronber
- Version changed from branch to 3.3.10
comment:5 Changed 11 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.10
- Version changed from 3.3.10 to branch
comment:6 Changed 11 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from reviewing to assigned
Despite the source code looking fine, I am unsure if this functionality should be included in trunk right now. First of all the logarithm is not defined for 0, which is not handled in the evaluator (skipping invalid values?). Second, the current implementation calculates avg(log(abs(y'-y)) which tends to -Inf for very small values (this may be an issue if one data sample was hit with a very high precision). Third, there is no warning / indication that despite linear scaling is requested, it is not performed.
comment:7 Changed 11 years ago by gkronber
r10305: changed log residual evaluator to calculate avg(log( 1 + abs(y - pred)))
comment:8 Changed 11 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from assigned to reviewing
comment:9 Changed 11 years ago by mkommend
Please put a comment / description that linear scaling is never performed for this evaluator. Otherwise everything looks fine.
comment:10 Changed 11 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from reviewing to assigned
comment:11 Changed 11 years ago by gkronber
- Status changed from assigned to accepted
comment:12 Changed 11 years ago by gkronber
r10482: added a notice that the log residual evaluator does not apply linear scaling
comment:13 Changed 11 years ago by gkronber
r10483: merged trunk changes to prepare for reintegration
comment:14 Changed 11 years ago by gkronber
r10484: reintegrated branch for log residual evaluator
comment:15 Changed 11 years ago by gkronber
r10485: deleted branch for log residual evaluator
comment:16 Changed 11 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from accepted to reviewing
comment:17 Changed 11 years ago by gkronber
r10486: fixed build fail
comment:18 Changed 11 years ago by mkommend
- Owner changed from mkommend to gkronber
- Status changed from reviewing to readytorelease
comment:19 Changed 11 years ago by mkommend
- Version changed from branch to 3.3.10
comment:20 Changed 11 years ago by gkronber
Depends on #2119
comment:21 Changed 11 years ago by mkommend
- Resolution set to done
- Status changed from readytorelease to closed
Created a feature development branch with r10202