Free cookie consent management tool by TermsFeed Policy Generator

Opened 13 years ago

Closed 13 years ago

#1609 closed defect (done)

Importing large CSV files can lead to out of memory exceptions

Reported by: gkronber Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.6
Component: Problems.DataAnalysis Version: 3.3.6
Keywords: Cc:

Description

Because the data is stored in a large monolithic two-dimensional double array (double[,]) it can become difficult to allocate a continuous memory block of the necessary size.

Change History (5)

comment:1 Changed 13 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.6
  • Owner changed from gkronber to mkommend
  • Status changed from new to accepted

comment:2 Changed 13 years ago by mkommend

r6740:

  • Corrected TableFileParser to handle empty rows correctly.
  • Refactored DataSet to store values in List<List> instead of a two-dimensional array.
  • Enable importing and storing string and datetime values.
  • Changed data access methods in dataset and adapted all concerning classes.
  • Changed interpreter to store the variable values for all rows during the compilation step.

comment:3 Changed 13 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from accepted to reviewing

comment:4 Changed 13 years ago by gkronber

  • Status changed from reviewing to readytorelease

Reviewed r6740 and made some additional changes (see #1640)

comment:5 Changed 13 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.5 to 3.3.6
Note: See TracTickets for help on using tickets.