Free cookie consent management tool by TermsFeed Policy Generator

Opened 3 years ago

Last modified 3 years ago

#3112 reviewing defect

Overflow in calculation of axes range in partial dependence plot

Reported by: gkronber Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.17
Component: Visualization.ChartControlsExtensions Version: trunk
Keywords: Cc:

Description (last modified by gkronber)

Models which produce a large range of outputs i.e. in the range 0 .. 1010 lead to an overflow in CalculateAxisInterval in particular in methods RoundDown() and RoundUp(). Both methods cast to an int which limits the maximum range to the 231. The cast to integer is however not required as the code should be fine with floating point arithmetic.

Change History (5)

comment:1 Changed 3 years ago by gkronber

  • Description modified (diff)

comment:2 Changed 3 years ago by gkronber

  • Owner set to gkronber
  • Status changed from new to accepted

comment:3 Changed 3 years ago by gkronber

r17861: fixed overflow because of integer cast in ChartUtil

comment:4 Changed 3 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from accepted to reviewing

comment:5 Changed 3 years ago by gkronber

  • Version set to trunk
Note: See TracTickets for help on using tickets.