Opened 17 months ago

Last modified 9 days ago

#2520 new enhancement

Persistence Overhaul

Reported by: swagner Owned by: swagner
Priority: high Milestone: HeuristicLab 4.0
Component: Persistence Version: branch
Keywords: Cc:

Description

The HeuristicLab persistence layer should be overhauled for HeuristicLab 4.0.

Change History (27)

comment:1 Changed 17 months ago by swagner

r13323: Created new branch for persistence overhaul

comment:2 Changed 17 months ago by swagner

r13324: Removed new empty branch in order to create branch of full trunk

comment:3 Changed 17 months ago by swagner

r13325: Created branch PersistenceOverhaul of trunk r13321.

comment:4 Changed 16 months ago by swagner

r13326: Created plugin for new persistence implementation.

comment:5 Changed 16 months ago by swagner

r13347: Worked on new persistence implementation.

comment:6 Changed 16 months ago by swagner

r13358: Worked on new persistence implementation.

comment:7 Changed 16 months ago by swagner

r13367: Worked on new persistence implementation.

comment:8 Changed 16 months ago by ascheibe

r13368 added guids to storable classes

comment:9 Changed 16 months ago by ascheibe

r13370 fixed unit tests

comment:10 Changed 16 months ago by swagner

r13375: Worked on new persistence implementation.

comment:11 Changed 16 months ago by ascheibe

r13386

  • fixed duplicate guids
  • adapted/added unit tests for new persistence

comment:12 Changed 16 months ago by swagner

r13387: Worked on new persistence implementation.

comment:13 Changed 16 months ago by ascheibe

r13407 worked on type and enum transformers

comment:14 Changed 16 months ago by ascheibe

r13409

  • added profiling to unit tests
  • added more unit tests

comment:15 follow-up: Changed 16 months ago by ascheibe

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

comment:16 in reply to: ↑ 15 ; follow-up: Changed 16 months ago by gkronber

Replying to ascheibe:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

comment:17 Changed 16 months ago by ascheibe

r13410 increased repetitions to get more realistic results

comment:18 in reply to: ↑ 16 Changed 16 months ago by ascheibe

Replying to gkronber:

Replying to ascheibe:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

Yeah, I know that this is not significant, but we don't have composite transformers yet, so this is all we can do at the moment ;-)

comment:19 Changed 3 months ago by gkronber

r14537: worked on persistence

comment:20 Changed 3 months ago by jkarder

r14549: worked on persistence

comment:21 Changed 2 months ago by jkarder

r14594: worked on persistence

comment:22 Changed 4 weeks ago by gkronber

r14711

  • renamed StorableClass -> StorableType
  • changed persistence to use GUIDs instead of type names

comment:23 Changed 4 weeks ago by gkronber

r14712: added GUIDs for (almost) all interface types (probably still too many) also added newlines at end of all files

comment:24 Changed 3 weeks ago by gkronber

r14714 added missing StorableType attributes and registered more types. All samples can be deserialized and serialized now

comment:25 Changed 3 weeks ago by gkronber

r14731: added tool program which adds StorableType attributes to interfaces and enums

comment:26 Changed 3 weeks ago by gkronber

r14739: added a sketch for a unit test for conversion (and set svn:ignore properties)

comment:27 Changed 9 days ago by gkronber

r14771: added versions to storable types and implemented conversion unit test

Note: See TracTickets for help on using tickets.