Add support for abs(x), AQ(x) and CubeRoot(x) for symbolic regression problems

[gkronber]

It would be nice to add support for the functions abs(x), AQ(x) (analytical quotient) and CubeRoot(x).

We already support a general root function. However, this is implemented as power(x, 1/round(x)) which makes it hard to use it in combination with constants optimization.

We also provide a separate symbol for the squared root. For some problems a cube root might also be necessary.

[gkronber]

r15943: created branch for ticket

[gkronber]

r15944: added support for Abs() symbol to tree interpreter and linear interpreter as well as to the infix parser

[gkronber]

TODO

• const opt
• all formatters
• simplifier

[gkronber]

We should also implement the CubeRoot symbol (see # 2954) and the AnalyticalQuotient symbol (see #2937) within the branch for this ticket.

I'll change this ticket description accordingly and close the other tickets.

[gkronber]

Attached two papers discussing the usage of the analytical quotient function for GP

[gkronber]

r16236: merged r16083 from AnalyticalQuotient branch to AbsoluteValue branch

[gkronber]

r16237: fixed compile error introduced in manual merge

[gkronber]

r16238: implemented all symbols in all interpreters

[gkronber]

r16239: added support for abs(x), cube(x), cuberoot(x), and aq(x) to:

• infix formatters, (other formatters TODO)
• infix parser
• constant opt (autodiff)
• simplifier (incomplete)

[gkronber]

TODO for simplifier:

• square(* (...)) -> (* (square ...)) and same for cube
• square(abs(x)) -> square(x)
• (* (aq ...) const) -> push down constant into aq, constants also include factors
• (* (abs ...) const) -> push down constant into abs (if positive, otherwise keep sign)
• ...?

[gkronber]

r16240: merged changes in the trunk up to current HEAD (r15951:16232) into the branch

[lkammere]

r16304: Merge changes in the trunk to current HEAD into the branch.

[lkammere]

r16306: Merge changes from trunk.

[gkronber]

r16332: merged r16241:16309 (current HEAD) from HL trunk to branch

[gkronber]

r16344: added a unit test cases for simplification of new symbols and extended the tree simplifier accordingly.

[gkronber]

TODO:

• Batched Interpreter
• Native Interpreter

[gkronber]

r16345: added cube, cbrt, absolute, aq to batched interpreter (and fixed a bug in power)

[gkronber]

r16346: fixed bugs

[gkronber]

r16347: fixed bug in BatchInterpreter for Root symbol

[gkronber]

r16348: fixed another bug in the BatchInterpreter for root symbols (now the linear and tree interpreter produce exactly the same results)

[gkronber]

r16349: added abs, cube, cuberoot, and aq to native interpreter and added sqrt which was missing in the vectorized code. Added a default branch to allow easier identification of bugs because of missing operations.

[gkronber]

r16350: merged r16333:16343 from trunk to branch (resolving a conflict in interpreter.h - the branch for sqrt)

[gkronber]

The native interpreter produces different results than the managed interpreters when the following symbols are used

• Absolute
• sqrt

I got the exact same results for the following

• +, -, *, /
• log
• exp
• analytic quotient
• cube
• square
• power
• root
• cuberoot

[gkronber]

r16351: fixed compile error after merge and made another change to native interpreter trying to resolve the differences when using abs and sqrt(). No success

[gkronber]

r16352 added support for expressions like x ^ c as well as support for '{' '}' as alternatives to '(' ')' to the infix parser

[gkronber]

r16355: small reordering of constants

[gkronber]

r16356: merged all changes from branch to trunk

[gkronber]

r16359: allow two different syntactical forms for power ( (a)^(b) and pow(a,b) ) in the infix formatter and merged two identical branches (for all operators)

[gkronber]

r16360: renamed class AnalyticalQuotient -> AnalyticQuotient

[gkronber]

r16361: renamed file to match class name

[gkronber]

r16362: missed a string replace AnalyticalQuotient -> AnalyticQuotient

[gkronber]

r16365: small change in description

[gkronber]

r16366: disabled cube and cuberoot in the OSGP sample test which failed. On my machine the algorithm produces different results and the unit test still fails. Let's check the results of the builder.

[gkronber]

Ok, the unit test succeeded on the builder. I assume it's fine then.

[mkommend]

Reviewed all changes in this ticket. Thanks for implementing the functionality and adapting all related classes (parser, formatter, evaluators, simplifier, ...).

This ticket however depends on #2958 (batch interpretation) that has been implemented directly in the trunk. Therefore, releasing this ticket will not be easy, because both tickets contain changes to the same classes.

[gkronber]

r16675 : deleted branch because all changes have been merged back to trunk

[gkronber]

Not all formatters support the new symbols yet. E.g. the Excel formatter.

[gkronber]

r16802: added support for abs(), aq(), cube(), cuberoot() to formatters. Skipped the TSQLFormatter.

[gkronber]

Cuberoot of negative arguments is not correctly implemented. Cuberoot(-3) is actually -Cuberoot(3). This is consistent with C++ and Math.Cbrt in .NET core.

[gkronber]

r16905: Corrected calculation of cuberoot function in all interpreters and updated all formatters.

I tested GP/SymReg with all interpreters and got the same results with all of them.

[pfleck]

reviewed and tested r16802 & r16905.

Formatters work well, thanks. Only a single remark: I think writing the "save" qbrt version that checks for negative x is overly defensive in some cases. For instance, Excel can handle negative bases in the POWER function, so there is no need for the save version.

I also tested the interpreters using the new symbols. All yielded the same results, except the SymbolicDataAnalysisExpressionTreeILEmittingInterpreter (see the attached script). I checked the generated IL for the short method return x < 0.0 ? -Math.Pow(-x, 1.0/3.0) : Math.Pow(x, 1.0/3.0); and compared it to the interpreter's code but I didn't spot any obvious error. Maybe there is an issue in one of the other symbols' generated IL-code.

IL_0000:  nop         
IL_0001:  ldarg.0     
IL_0002:  ldc.r8      00 00 00 00 00 00 00 00 
IL_000B:  blt.s       IL_001E
IL_000D:  ldarg.0     
IL_000E:  ldc.r8      55 55 55 55 55 55 D5 3F 
IL_0017:  call        System.Math.Pow
IL_001C:  br.s        IL_002F
IL_001E:  ldarg.0     
IL_001F:  neg         
IL_0020:  ldc.r8      55 55 55 55 55 55 D5 3F 
IL_0029:  call        System.Math.Pow
IL_002E:  neg         
IL_002F:  stloc.0     
IL_0030:  br.s        IL_0032
IL_0032:  ldloc.0     
IL_0033:  ret         

[gkronber]

Replying to pfleck:

... think writing the "save" qbrt version that checks for negative x is overly defensive in some cases. For instance, Excel can handle negative bases in the POWER function, so there is no need for the save version.

You are right. I didn't check for that. I propose to keep the defensive code for the next release.

All yielded the same results, except the SymbolicDataAnalysisExpressionTreeILEmittingInterpreter (see the attached script).

Thanks for the test script. This was indeed a bug. I fixed this in r17028.

[pfleck]

reviewed r17028: results are now the same for all interpreters

[mkommend]

r17072: Merged 16350, 16351, 16352, 16355, 16356, 16359, 16360, 16361, 16362, 16365, 16366 into stable.

[mkommend]

r17103: Merged 16675, 16802, 16905, 17028 into stable.