Opened 2 years ago

Closed 21 months ago

#2276 closed enhancement (done)

Create interface for Dataset class, refactor code

Reported by: bburlacu Owned by: mkommend
Priority: medium Milestone: HeuristicLab 3.3.12
Component: Algorithms.DataAnalysis Version: branch
Keywords: Cc:

Description

We should extract the essential requirements for a Dataset object into an interface, which should then be used in the code (mostly DataAnalysis.Symbolic), so that custom dataset objects can be employed in certain use cases.

Change History (41)

comment:1 Changed 2 years ago by bburlacu

r11571: Commit initial version of IDataset interface and code refactoring.

comment:2 Changed 2 years ago by bburlacu

r11572: Merged the Programmable problem projects into the branch, adjusted references and build paths.

comment:3 Changed 2 years ago by bburlacu

r11579: Removed unnecessary folders from the branch.

comment:4 Changed 2 years ago by bburlacu

  • Owner set to bburlacu
  • Status changed from new to assigned

comment:5 Changed 2 years ago by bburlacu

  • Status changed from assigned to accepted

comment:6 Changed 2 years ago by bburlacu

r11584: Removed programmable problem from this branch.

comment:7 Changed 2 years ago by bburlacu

r11589: Unsealed Dataset class, refactored GetDoubleValues, GetStringValues, GetDateTimeValues methods to internally use the same generic method, added ModifiableDataset class which adds the functionality of replacing, adding or removing rows in the dataset.

comment:8 Changed 2 years ago by bburlacu

r11621: Removed SimpleDataset class in favor of ModifiableDataset. Decided to keep the IDataset interface (even though a cast to Dataset is needed in 2 places).

comment:9 Changed 2 years ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from accepted to reviewing

comment:10 Changed 2 years ago by mkommend

  • Version changed from 3.3.10 to branch

comment:11 Changed 2 years ago by mkommend

  • Milestone changed from HeuristicLab 3.3.11 to HeuristicLab 3.3.12

comment:12 Changed 2 years ago by bburlacu

r11853: Make Arc storable constructor protected (this change was meant for the trunk version but was accidentally committed in this branch).

comment:13 Changed 2 years ago by bburlacu

  • Owner changed from mkommend to bburlacu
  • Status changed from reviewing to assigned

comment:14 Changed 2 years ago by bburlacu

r12031: Merged trunk changes.

comment:15 Changed 2 years ago by bburlacu

r12032: Extended functionality of ModifiableDataset with three new methods: AddVariable, RemoveVariable and ChangeVariableValue.

comment:16 Changed 2 years ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from assigned to reviewing

comment:17 Changed 2 years ago by bburlacu

r12105: Merged trunk changes.

comment:18 Changed 2 years ago by bburlacu

r12109: Merged trunk changes.

comment:19 Changed 2 years ago by mkommend

  • Owner changed from mkommend to bburlacu
  • Status changed from reviewing to assigned

comment:20 Changed 2 years ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from assigned to reviewing

r12141:

  • Implemented a ModifiableDatasetView which allows values in the modifiable dataset to be visually edited.
  • Made SetValue and Validate implementations explicit in the Dataset class
  • Added explicit implementations of SetValue and Validate in the ModifiableDataset
  • Renamed ChangeVariableValue method to SetVariableValue for consistency

comment:21 Changed 2 years ago by bburlacu

r12190: Changed the GetVariableType method so that it also works when there are no rows in the dataset.

comment:22 Changed 2 years ago by mkommend

r12247: Merged trunk changes into dataset refactoring branch.

comment:23 Changed 22 months ago by mkommend

r12437: Organized code in IDataset.

comment:24 Changed 22 months ago by mkommend

r12438: Merged trunk changes into dataset refactoring branch.

comment:25 follow-up: Changed 22 months ago by mkommend

Please implement the change events such as ColumnsChanged, RowsChanged, ... for the ModifiableDataset.

comment:26 Changed 22 months ago by mkommend

  • Owner changed from mkommend to bburlacu
  • Status changed from reviewing to assigned

comment:27 Changed 22 months ago by mkommend

r12452: Adapted generic GetValues<T> method in Dataset to avoid ToList calls.

comment:28 Changed 22 months ago by mkommend

r12453: Changes to reduce the memory consumption of GP.

Profiling the first 20 generations of the "GP - Symbolic Regression" sample showed a reduction of ~ 700MB (30%) of allocated bytes.

Summary for HeuristicLab 3.3 trunk.log			Summary for HeuristicLab 3.3 refactored.log
Allocated bytes:              2,280,452,065		Allocated bytes:              1,577,495,256
Relocated bytes:                 72,310,923		Relocated bytes:                 71,416,397
Final Heap bytes:               258,834,133		Final Heap bytes:               360,013,510
Objects finalized:                    6,476		Objects finalized:                    4,049
Critical objects finalized:           1,697		Critical objects finalized:           1,440
Gen 0 collections:                       18		Gen 0 collections:                       16
Gen 1 collections:                       11		Gen 1 collections:                        7
Gen 2 collections:                        8		Gen 2 collections:                        5
Induced collections:                      0		Induced collections:                      0
Gen 0 Heap bytes:               191,961,295		Gen 0 Heap bytes:               164,683,847
Gen 1 Heap bytes:                15,646,303		Gen 1 Heap bytes:                13,780,491
Gen 2 Heap bytes:                24,997,270		Gen 2 Heap bytes:                22,111,974
Large Object Heap bytes:         11,101,599		Large Object Heap bytes:          7,515,030
Handles created:                    115,616		Handles created:                     99,237
Handles destroyed:                   58,786		Handles destroyed:                   43,106
Handles surviving:  
Last edited 22 months ago by mkommend (previous) (diff)

comment:29 Changed 22 months ago by mkommend

r12454: Changed util method CalculateEvaluatedNodesPerSec to cache the rows enumerable.

comment:30 in reply to: ↑ 25 Changed 21 months ago by bburlacu

r12489: Made events in Dataset.cs virtual and overrode them in the ModifiableDataset class. Added event handlers for dataset changes.

comment:31 Changed 21 months ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from assigned to reviewing

comment:32 Changed 21 months ago by mkommend

r12505: Merged trunk changes into dataset refactoring branch.

comment:33 Changed 21 months ago by mkommend

r12508: Minor changes in modifiable dataset.

comment:34 Changed 21 months ago by mkommend

r12509: Reintegrated branch for dataset refactoring.

comment:35 Changed 21 months ago by mkommend

r12510: Removed branch for dataset refactoring.

comment:36 Changed 21 months ago by mkommend

r12511: Sealed ModifiableDataset.

comment:37 Changed 21 months ago by mkommend

r12524: Forgot to commit merge infos for dataset refactoring reintegration.

comment:38 Changed 21 months ago by mkommend

r12578: Corrected shrink menu item to only consider objects of type Dataset.

comment:39 Changed 21 months ago by mkommend

  • Status changed from reviewing to readytorelease

comment:40 Changed 21 months ago by mkommend

r12702: Merged changes to stable.

comment:41 Changed 21 months ago by mkommend

  • Resolution set to done
  • Status changed from readytorelease to closed
Note: See TracTickets for help on using tickets.