Opened 22 months ago
Closed 7 months ago
#2915 closed enhancement (done)
Add support for abs(x), AQ(x) and CubeRoot(x) for symbolic regression problems
Reported by: | gkronber | Owned by: | abeham |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.16 |
Component: | Problems.DataAnalysis.Symbolic | Version: | trunk |
Keywords: | merged | Cc: |
Description (last modified by gkronber)
It would be nice to add support for the functions abs(x), AQ(x) (analytical quotient) and CubeRoot(x).
We already support a general root function. However, this is implemented as power(x, 1/round(x)) which makes it hard to use it in combination with constants optimization.
We also provide a separate symbol for the squared root. For some problems a cube root might also be necessary.
Attachments (3)
Change History (61)
comment:1 Changed 22 months ago by gkronber
- Owner set to gkronber
- Status changed from new to accepted
comment:2 Changed 21 months ago by gkronber
comment:3 Changed 21 months ago by gkronber
r15944: added support for Abs() symbol to tree interpreter and linear interpreter as well as to the infix parser
comment:4 Changed 17 months ago by gkronber
TODO
- const opt
- all formatters
- simplifier
comment:5 Changed 16 months ago by gkronber
- Version set to branch
comment:6 Changed 16 months ago by gkronber
We should also implement the CubeRoot symbol (see # 2954) and the AnalyticalQuotient symbol (see #2937) within the branch for this ticket.
I'll change this ticket description accordingly and close the other tickets.
comment:7 Changed 16 months ago by gkronber
- Description modified (diff)
- Summary changed from Add support for the absolute value function abs(x) for symbolic regression problems to Add support for abs(x), AQ(x) and CubeRoot(x) for symbolic regression problems
Changed 16 months ago by gkronber
Changed 16 months ago by gkronber
comment:8 Changed 16 months ago by gkronber
Attached two papers discussing the usage of the analytical quotient function for GP
comment:9 Changed 16 months ago by gkronber
comment:10 Changed 16 months ago by gkronber
r16237: fixed compile error introduced in manual merge
comment:11 Changed 16 months ago by gkronber
r16238: implemented all symbols in all interpreters
comment:12 Changed 16 months ago by gkronber
r16239: added support for abs(x), cube(x), cuberoot(x), and aq(x) to:
- infix formatters, (other formatters TODO)
- infix parser
- constant opt (autodiff)
- simplifier (incomplete)
comment:13 Changed 16 months ago by gkronber
TODO for simplifier:
square(* (...)) -> (* (square ...)) and same for cubesquare(abs(x)) -> square(x)(* (aq ...) const) -> push down constant into aq, constants also include factors(* (abs ...) const) -> push down constant into abs (if positive, otherwise keep sign)- ...?
comment:14 Changed 16 months ago by gkronber
r16240: merged changes in the trunk up to current HEAD (r15951:16232) into the branch
comment:15 Changed 15 months ago by lkammere
r16304: Merge changes in the trunk to current HEAD into the branch.
comment:16 Changed 15 months ago by lkammere
r16306: Merge changes from trunk.
comment:17 Changed 14 months ago by gkronber
r16332: merged r16241:16309 (current HEAD) from HL trunk to branch
comment:18 Changed 14 months ago by gkronber
r16344: added a unit test cases for simplification of new symbols and extended the tree simplifier accordingly.
comment:19 Changed 14 months ago by gkronber
TODO:
Batched InterpreterNative Interpreter
comment:20 Changed 14 months ago by gkronber
r16345: added cube, cbrt, absolute, aq to batched interpreter (and fixed a bug in power)
comment:21 Changed 14 months ago by gkronber
r16346: fixed bugs
comment:22 Changed 14 months ago by gkronber
r16347: fixed bug in BatchInterpreter for Root symbol
comment:23 Changed 14 months ago by gkronber
r16348: fixed another bug in the BatchInterpreter for root symbols (now the linear and tree interpreter produce exactly the same results)
comment:24 Changed 14 months ago by gkronber
r16349: added abs, cube, cuberoot, and aq to native interpreter and added sqrt which was missing in the vectorized code. Added a default branch to allow easier identification of bugs because of missing operations.
comment:25 Changed 14 months ago by gkronber
r16350: merged r16333:16343 from trunk to branch (resolving a conflict in interpreter.h - the branch for sqrt)
comment:26 Changed 14 months ago by gkronber
The native interpreter produces different results than the managed interpreters when the following symbols are used
- Absolute
- sqrt
I got the exact same results for the following
- +, -, *, /
- log
- exp
- analytic quotient
- cube
- square
- power
- root
- cuberoot
comment:27 Changed 14 months ago by gkronber
r16351: fixed compile error after merge and made another change to native interpreter trying to resolve the differences when using abs and sqrt(). No success
comment:28 Changed 14 months ago by gkronber
r16352 added support for expressions like x ^ c as well as support for '{' '}' as alternatives to '(' ')' to the infix parser
comment:29 Changed 14 months ago by gkronber
r16355: small reordering of constants
comment:30 Changed 14 months ago by gkronber
r16356: merged all changes from branch to trunk
comment:32 Changed 14 months ago by gkronber
r16359: allow two different syntactical forms for power ( (a)^(b) and pow(a,b) ) in the infix formatter and merged two identical branches (for all operators)
comment:33 Changed 14 months ago by gkronber
- Version changed from branch to trunk
comment:34 Changed 14 months ago by gkronber
r16360: renamed class AnalyticalQuotient -> AnalyticQuotient
comment:35 Changed 14 months ago by gkronber
r16361: renamed file to match class name
comment:36 Changed 14 months ago by gkronber
r16362: missed a string replace AnalyticalQuotient -> AnalyticQuotient
comment:37 Changed 14 months ago by gkronber
r16365: small change in description
comment:38 Changed 14 months ago by gkronber
r16366: disabled cube and cuberoot in the OSGP sample test which failed. On my machine the algorithm produces different results and the unit test still fails. Let's check the results of the builder.
comment:39 Changed 14 months ago by gkronber
Ok, the unit test succeeded on the builder. I assume it's fine then.
comment:40 Changed 13 months ago by mkommend
- Status changed from accepted to readytorelease
Reviewed all changes in this ticket. Thanks for implementing the functionality and adapting all related classes (parser, formatter, evaluators, simplifier, ...).
This ticket however depends on #2958 (batch interpretation) that has been implemented directly in the trunk. Therefore, releasing this ticket will not be easy, because both tickets contain changes to the same classes.
comment:41 Changed 11 months ago by gkronber
r16675 : deleted branch because all changes have been merged back to trunk
comment:42 Changed 10 months ago by gkronber
- Status changed from readytorelease to assigned
Not all formatters support the new symbols yet. E.g. the Excel formatter.
comment:43 Changed 10 months ago by gkronber
r16802: added support for abs(), aq(), cube(), cuberoot() to formatters. Skipped the TSQLFormatter.
comment:44 Changed 9 months ago by gkronber
- Owner changed from gkronber to pfleck
- Status changed from assigned to reviewing
comment:45 Changed 9 months ago by gkronber
Cuberoot of negative arguments is not correctly implemented. Cuberoot(-3) is actually -Cuberoot(3). This is consistent with C++ and Math.Cbrt in .NET core.
comment:46 Changed 9 months ago by gkronber
r16905: Corrected calculation of cuberoot function in all interpreters and updated all formatters.
I tested GP/SymReg with all interpreters and got the same results with all of them.
comment:47 follow-up: ↓ 49 Changed 7 months ago by pfleck
reviewed and tested r16802 & r16905.
Formatters work well, thanks. Only a single remark: I think writing the "save" qbrt version that checks for negative x is overly defensive in some cases. For instance, Excel can handle negative bases in the POWER function, so there is no need for the save version.
I also tested the interpreters using the new symbols. All yielded the same results, except the SymbolicDataAnalysisExpressionTreeILEmittingInterpreter (see the attached script). I checked the generated IL for the short method return x < 0.0 ? -Math.Pow(-x, 1.0/3.0) : Math.Pow(x, 1.0/3.0); and compared it to the interpreter's code but I didn't spot any obvious error. Maybe there is an issue in one of the other symbols' generated IL-code.
IL_0000: nop IL_0001: ldarg.0 IL_0002: ldc.r8 00 00 00 00 00 00 00 00 IL_000B: blt.s IL_001E IL_000D: ldarg.0 IL_000E: ldc.r8 55 55 55 55 55 55 D5 3F IL_0017: call System.Math.Pow IL_001C: br.s IL_002F IL_001E: ldarg.0 IL_001F: neg IL_0020: ldc.r8 55 55 55 55 55 55 D5 3F IL_0029: call System.Math.Pow IL_002E: neg IL_002F: stloc.0 IL_0030: br.s IL_0032 IL_0032: ldloc.0 IL_0033: ret
Changed 7 months ago by pfleck
comment:48 Changed 7 months ago by pfleck
- Owner changed from pfleck to gkronber
- Status changed from reviewing to assigned
comment:49 in reply to: ↑ 47 Changed 7 months ago by gkronber
Replying to pfleck:
... think writing the "save" qbrt version that checks for negative x is overly defensive in some cases. For instance, Excel can handle negative bases in the POWER function, so there is no need for the save version.
You are right. I didn't check for that. I propose to keep the defensive code for the next release.
All yielded the same results, except the SymbolicDataAnalysisExpressionTreeILEmittingInterpreter (see the attached script).
Thanks for the test script. This was indeed a bug. I fixed this in r17028.
comment:50 Changed 7 months ago by gkronber
- Owner changed from gkronber to pfleck
- Status changed from assigned to reviewing
comment:51 Changed 7 months ago by pfleck
- Owner changed from pfleck to gkronber
- Status changed from reviewing to readytorelease
reviewed r17028: results are now the same for all interpreters
comment:52 Changed 7 months ago by abeham
- Keywords depends-2958 added
comment:53 Changed 7 months ago by abeham
- Keywords mergewith-2958 added; depends-2958 removed
comment:54 Changed 7 months ago by abeham
- Owner changed from gkronber to abeham
- Status changed from readytorelease to reviewing
comment:55 Changed 7 months ago by abeham
- Status changed from reviewing to readytorelease
comment:56 Changed 7 months ago by abeham
- Keywords depends-2520 mergewith-(2958 2866 2966) added; mergewith-2958 removed
comment:57 Changed 7 months ago by mkommend
r17072: Merged 16350, 16351, 16352, 16355, 16356, 16359, 16360, 16361, 16362, 16365, 16366 into stable.
comment:58 Changed 7 months ago by mkommend
- Keywords merged added; depends-2520 mergewith-(2958 2866 2966) removed
r17103: Merged 16675, 16802, 16905, 17028 into stable.
comment:59 Changed 7 months ago by abeham
- Resolution set to done
- Status changed from readytorelease to closed
r15943: created branch for ticket