Opened 4 years ago
Last modified 4 years ago
#3112 reviewing defect
Overflow in calculation of axes range in partial dependence plot
Reported by: | gkronber | Owned by: | mkommend |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.17 |
Component: | Visualization.ChartControlsExtensions | Version: | trunk |
Keywords: | Cc: |
Description (last modified by gkronber)
Models which produce a large range of outputs i.e. in the range 0 .. 1010 lead to an overflow in CalculateAxisInterval in particular in methods RoundDown() and RoundUp(). Both methods cast to an int which limits the maximum range to the 231. The cast to integer is however not required as the code should be fine with floating point arithmetic.
Change History (5)
comment:1 Changed 4 years ago by gkronber
- Description modified (diff)
comment:2 Changed 4 years ago by gkronber
- Owner set to gkronber
- Status changed from new to accepted
comment:3 Changed 4 years ago by gkronber
comment:4 Changed 4 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from accepted to reviewing
comment:5 Changed 4 years ago by gkronber
- Version set to trunk
Note: See
TracTickets for help on using
tickets.
r17861: fixed overflow because of integer cast in ChartUtil