Opened 23 months ago

Last modified 3 months ago

#2520 accepted enhancement

Persistence Overhaul

Reported by: swagner Owned by: gkronber
Priority: high Milestone: HeuristicLab 4.0
Component: Persistence Version: branch
Keywords: Cc:

Description

The HeuristicLab persistence layer should be overhauled for HeuristicLab 4.0.

Change History (51)

comment:1 Changed 23 months ago by swagner

r13323: Created new branch for persistence overhaul

comment:2 Changed 23 months ago by swagner

r13324: Removed new empty branch in order to create branch of full trunk

comment:3 Changed 23 months ago by swagner

r13325: Created branch PersistenceOverhaul of trunk r13321.

comment:4 Changed 22 months ago by swagner

r13326: Created plugin for new persistence implementation.

comment:5 Changed 22 months ago by swagner

r13347: Worked on new persistence implementation.

comment:6 Changed 22 months ago by swagner

r13358: Worked on new persistence implementation.

comment:7 Changed 22 months ago by swagner

r13367: Worked on new persistence implementation.

comment:8 Changed 22 months ago by ascheibe

r13368 added guids to storable classes

comment:9 Changed 22 months ago by ascheibe

r13370 fixed unit tests

comment:10 Changed 22 months ago by swagner

r13375: Worked on new persistence implementation.

comment:11 Changed 22 months ago by ascheibe

r13386

  • fixed duplicate guids
  • adapted/added unit tests for new persistence

comment:12 Changed 22 months ago by swagner

r13387: Worked on new persistence implementation.

comment:13 Changed 22 months ago by ascheibe

r13407 worked on type and enum transformers

comment:14 Changed 22 months ago by ascheibe

r13409

  • added profiling to unit tests
  • added more unit tests

comment:15 follow-up: Changed 22 months ago by ascheibe

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

comment:16 in reply to: ↑ 15 ; follow-up: Changed 22 months ago by gkronber

Replying to ascheibe:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

comment:17 Changed 22 months ago by ascheibe

r13410 increased repetitions to get more realistic results

comment:18 in reply to: ↑ 16 Changed 22 months ago by ascheibe

Replying to gkronber:

Replying to ascheibe:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

Yeah, I know that this is not significant, but we don't have composite transformers yet, so this is all we can do at the moment ;-)

comment:19 Changed 9 months ago by gkronber

r14537: worked on persistence

comment:20 Changed 9 months ago by jkarder

r14549: worked on persistence

comment:21 Changed 8 months ago by jkarder

r14594: worked on persistence

comment:22 Changed 7 months ago by gkronber

r14711

  • renamed StorableClass -> StorableType
  • changed persistence to use GUIDs instead of type names

comment:23 Changed 7 months ago by gkronber

r14712: added GUIDs for (almost) all interface types (probably still too many) also added newlines at end of all files

comment:24 Changed 7 months ago by gkronber

r14714 added missing StorableType attributes and registered more types. All samples can be deserialized and serialized now

comment:25 Changed 7 months ago by gkronber

r14731: added tool program which adds StorableType attributes to interfaces and enums

comment:26 Changed 7 months ago by gkronber

r14739: added a sketch for a unit test for conversion (and set svn:ignore properties)

comment:27 Changed 6 months ago by gkronber

r14771: added versions to storable types and implemented conversion unit test

comment:28 Changed 5 months ago by gkronber

r14919: created a branch for reintegration

comment:29 Changed 5 months ago by gkronber

r14920: deleted branch of trunk

comment:30 Changed 5 months ago by gkronber

r14921: created a new branch from stable for reintegration

comment:31 Changed 5 months ago by gkronber

r14922: deleted branch again

comment:32 Changed 5 months ago by gkronber

r14923: created branch from trunk (again)

comment:33 Changed 5 months ago by gkronber

r14924: copied relevant files from persistence overhaul branch

comment:34 Changed 5 months ago by gkronber

r14925: moved storable attributes (construction)

comment:35 Changed 5 months ago by gkronber

r14927: changed all usages of StorableClass to use StorableType with an auto-generated GUID (did not add StorableType to other type definitions yet)

comment:36 Changed 5 months ago by gkronber

r14928: added unit tests and fixed all test cases for old persistence

comment:37 Changed 5 months ago by gkronber

r14929: fixed all unit tests for new persistence - loading & storing all samples

comment:38 Changed 5 months ago by gkronber

r14930: changed optimizer to use new persistence per default

comment:39 Changed 5 months ago by gkronber

r14931: do not store graph visualization info for the operatorgraph

comment:40 Changed 5 months ago by gkronber

r14932: check if results are the same for the GA TSP sample test

comment:41 Changed 5 months ago by gkronber

r14933: added a code fix for generating StorableTypeAttributes

comment:42 Changed 5 months ago by gkronber

r14934: svn:ignore

comment:43 Changed 5 months ago by gkronber

r14935: support persistence for OperatorGraphVisualizationInfo

comment:44 Changed 5 months ago by gkronber

  • Owner changed from swagner to gkronber
  • Status changed from new to accepted

comment:45 Changed 4 months ago by jkarder

r14985: worked on persistence code fix

  • split existing analyzer and code fix
  • added analyzer and code fix for missing storable ctors

comment:46 Changed 4 months ago by jkarder

r14989: worked on persistence code fix

  • added SequentialFixAllProvider

comment:47 Changed 4 months ago by jkarder

r14990: worked on persistence code fix

  • added missing UnitTests.cs

comment:48 Changed 4 months ago by gkronber

  • r15018: introduced StorableConstructorFlag type for StorableConstructors
  • r15020: fixed unit tests checking for exceptions

comment:49 Changed 4 months ago by jkarder

r15022: removed ColorTransformer for now as it only handled the color's argb values (StructTransformer will handle more, e.g. IsEmpty)

comment:50 Changed 3 months ago by gkronber

r15034: improved conversions

comment:51 Changed 3 months ago by gkronber

r15035: made some changes related to renaming of storable members

Note: See TracTickets for help on using tickets.