Opened 5 months ago

Last modified 5 months ago

#3112 reviewing defect

Overflow in calculation of axes range in partial dependence plot

Reported by: gkronber Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.17
Component: Visualization.ChartControlsExtensions Version: trunk
Keywords: Cc:

Description (last modified by gkronber)

Models which produce a large range of outputs i.e. in the range 0 .. 1010 lead to an overflow in CalculateAxisInterval in particular in methods RoundDown() and RoundUp(). Both methods cast to an int which limits the maximum range to the 231. The cast to integer is however not required as the code should be fine with floating point arithmetic.

Change History (5)

comment:1 Changed 5 months ago by gkronber

  • Description modified (diff)

comment:2 Changed 5 months ago by gkronber

  • Owner set to gkronber
  • Status changed from new to accepted

comment:3 Changed 5 months ago by gkronber

r17861: fixed overflow because of integer cast in ChartUtil

comment:4 Changed 5 months ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

comment:5 Changed 5 months ago by gkronber

  • Version set to trunk
Note: See TracTickets for help on using tickets.