Opened 5 weeks ago

Last modified 5 weeks ago

#3112 reviewing defect

Overflow in calculation of axes range in partial dependence plot

Reported by: gkronber Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.17
Component: Visualization.ChartControlsExtensions Version: trunk
Keywords: Cc:

Description (last modified by gkronber)

Models which produce a large range of outputs i.e. in the range 0 .. 1010 lead to an overflow in CalculateAxisInterval in particular in methods RoundDown() and RoundUp(). Both methods cast to an int which limits the maximum range to the 231. The cast to integer is however not required as the code should be fine with floating point arithmetic.

Change History (5)

comment:1 Changed 5 weeks ago by gkronber

  • Description modified (diff)

comment:2 Changed 5 weeks ago by gkronber

  • Owner set to gkronber
  • Status changed from new to accepted

comment:3 Changed 5 weeks ago by gkronber

r17861: fixed overflow because of integer cast in ChartUtil

comment:4 Changed 5 weeks ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

comment:5 Changed 5 weeks ago by gkronber

  • Version set to trunk
Note: See TracTickets for help on using tickets.