Opened 3 years ago

Closed 2 years ago

#2238 closed enhancement (done)

Reduce memory footprint of GP solutions

Reported by: mkommend Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.11
Component: Problems.DataAnalysis.Symbolic Version: 3.3.10
Keywords: Cc:

Description

When GP experiments (classification & regression) are executed on the HIVE system they take up lots of memory and disk space. This ticket collects changes to reduce the memory consumption.

Change History (10)

comment:1 Changed 3 years ago by mkommend

  • Status changed from new to accepted

comment:2 Changed 3 years ago by mkommend

r11330: Removed unnecessary name changes event registration for result in DataAnalysisSolutions.

comment:3 Changed 3 years ago by mkommend

r11332: Removed grammar objects from GP trees used in symbolic data analysis solutions.

comment:4 Changed 3 years ago by mkommend

  • Owner changed from mkommend to gkronber
  • Status changed from accepted to reviewing

comment:5 follow-up: Changed 3 years ago by gkronber

  • Owner changed from gkronber to mkommend

Reviewed r11330 and r11332. How large is the reduction of memory consumption by setting the grammars to null? I would not have expected that this has a strong effect.

comment:6 Changed 2 years ago by gkronber

  • Summary changed from Reduce memory footpring of GP solutions to Reduce memory footprint of GP solutions

comment:7 in reply to: ↑ 5 Changed 2 years ago by mkommend

Replying to gkronber:

How large is the reduction of memory consumption by setting the grammars to null? I would not have expected that this has a strong effect.

I have tested these changes with an experiment containing 12 batchruns à 100 repetitions (1200 runs), which were executed by the Hive and where duplicate problem data objects have been removed. The zipped file size shrinks from ~27 MB to ~17.5 MB, unzipped from ~603 MB to ~297 MB and in memory according to the CLR Profiler from ~320 MB to ~130 MB. A further benefit is that due to the reduction of the unzipped file, the experiment is loaded twice as fast by the persistence.

comment:8 Changed 2 years ago by mkommend

  • Owner changed from mkommend to gkronber

comment:9 Changed 2 years ago by gkronber

  • Owner changed from gkronber to mkommend
  • Status changed from reviewing to readytorelease

comment:10 Changed 2 years ago by mkommend

  • Resolution set to done
  • Status changed from readytorelease to closed

r11416: Merged r11330 and r11332 into stable.

Note: See TracTickets for help on using tickets.