Opened 6 years ago

Closed 5 years ago

#1622 closed enhancement (done)

Add calculated values to algorithm runs for further analysis.

Reported by: epitzer Owned by: epitzer
Priority: medium Milestone: HeuristicLab 3.3.6
Component: Optimization Version: 3.3.6
Keywords: Cc:

Description

Sometimes it might be necessary to add additional values to algorithm runs that can be calculated from existing results such as a normalized value by dividing through e.g. problem size.

Change History (37)

comment:1 Changed 6 years ago by epitzer

  • Status changed from new to accepted

comment:2 Changed 6 years ago by epitzer

Create RunCollectionModificationEvaluator that evaluates IRunCollectionModifiers (r6662)

comment:3 Changed 6 years ago by epitzer

Add concrete run collection modifiers to add calculated values and discretized values (r6663)

comment:4 Changed 6 years ago by epitzer

Include RunCollection modification into RunCollection View (r6664)

comment:5 Changed 6 years ago by epitzer

Add RunCollectionValueRemover modifier and disallow name change in RunCollectionFuzzifier (r6665)

comment:6 Changed 6 years ago by epitzer

  • Owner changed from epitzer to mkommend
  • Status changed from accepted to reviewing

comment:7 Changed 6 years ago by epitzer

Modify run collection to force update of cached values (r6667)

comment:8 Changed 6 years ago by epitzer

Enable grouping before fuzzification (r6668)

comment:9 Changed 6 years ago by epitzer

Add missing plugin dependency (r6669)

comment:10 Changed 6 years ago by epitzer

Also register events in cloning constructor (r6670)

comment:11 Changed 6 years ago by epitzer

Remove unnecessary suffix parameter and correctly include grouping clause in generated item name (r6671)

comment:12 Changed 6 years ago by epitzer

Enable modifications of order and number of runs in RunCollectionModificationEvaluator and implement RunCollectionSorter (r6677)

comment:13 follow-up: Changed 6 years ago by abeham

This is a welcomed feature, that I myself would have needed a couple of times already. However can you elaborate this feature a little more? I wonder about the specific needs, because I'm a bit skeptical on the way this is implemented.

The one thing I ask myself about this feature: Why don't we just make this user interface only? All I assume you would need is to add/remove and probably modify results in several runs of a RunCollection. There can be a simple dialog that allows you to do this.

I would imagine this to work like this: I have my run collection. I open the dialog where I can filter the runs (e.g. in a simple checkbox list or via a more or less fancy search string). To all filtered runs I can then add/remove or modify results.

I don't understand the necessity to build a "RunCollection Modification Evaluator" as a creatable item.

Last edited 6 years ago by abeham (previous) (diff)

comment:14 in reply to: ↑ 13 Changed 6 years ago by epitzer

Replying to abeham:

Why don't we just make this user interface only? All I assume you would need is to add/remove and probably modify results in several runs of a RunCollection. There can be a simple dialog that allows you to do this.

In my opinion the RunCollection Modification Evaluator is your simple user interface with the added benefit that you can save the modifications both for re-use with new runs and for documenting your changes to the embedded RunCollection.

I don't understand the necessity to build a "RunCollection Modification Evaluator" as a creatable item.

I think it is a good idea to remove it since it has now been embedded into the view selector of the RunCollection in r6664 and is not really needed anymore.

Last edited 6 years ago by epitzer (previous) (diff)

comment:15 Changed 6 years ago by abeham

From a UI perspective that's not a dialog at all. It's a wrapper around a collection with a special view. However, I don't see the necessity to have both a RunCollection Modification Evaluator and a RunCollection. My question is this: Why can't the RunCollectionView do the modification? Why do we need a wrapper item with another view for that?

comment:16 Changed 6 years ago by epitzer

Actually it has already been discussed with swagner and mkommend and is planned for integration into the RunCollection itself albeit not as a view-only mechanism but similar to the existing implementation.

comment:17 Changed 6 years ago by abeham

Okay, the ticket is in reviewing state, I thought this would be released that way.

comment:18 Changed 6 years ago by epitzer

  • Owner changed from mkommend to epitzer
  • Status changed from reviewing to assigned

comment:19 Changed 6 years ago by epitzer

  • Status changed from assigned to accepted

comment:20 Changed 6 years ago by epitzer

Add RunCollectionGroupCreater for re-grouping a RunCollection (r6686)

comment:21 Changed 6 years ago by epitzer

Extend calculator to support other data types, literals, stack manipulations and conditionals (r6687)

comment:22 Changed 6 years ago by epitzer

Rename RunCollectionFuzzifier to RunCollectionDiscretizer (r6690)

This commit contains both version and a modified cloning method in the old fuzzifier to enable upgrading by cloning a fuzzifier in the GUI which will then become a discretizer.

comment:23 Changed 6 years ago by epitzer

Remove obsolete RunCollectionFuzzifier (r6691)

comment:24 Changed 6 years ago by epitzer

Integrate functionality of RunCollectionModificationEvaluator directly into RunCollection and its view. (r6693)

This update includes a modified RunCollectionModificationEvaluator that automatic migrates all modifies into the RunCollection.

comment:25 Changed 6 years ago by epitzer

Remove obsolete RunCollectionModificationEvaluator (r6694)

comment:26 Changed 6 years ago by epitzer

Fix spacing of modifiers list (r6695)

comment:27 Changed 6 years ago by epitzer

Remove unnecessary reference to HeuristicLab.Parameters (r6697)

comment:28 Changed 6 years ago by epitzer

  • Owner changed from epitzer to mkommend
  • Status changed from accepted to reviewing

comment:29 Changed 5 years ago by abeham

r6902

  • fixed cloning when object has not been initialized properly

comment:30 Changed 5 years ago by epitzer

r6903 Enable discretization by empirical distribution's quantiles.

comment:31 Changed 5 years ago by epitzer

r6908 Fix quote handling and simplify tokenizer in Calculator and provide better error message.

comment:32 Changed 5 years ago by mkommend

  • Owner changed from mkommend to epitzer
  • Status changed from reviewing to assigned

The implementation looks ok, but I am quite skeptical about the usefulness of several modifiers (e.g., RunCollectionSorter or the RunCollectionRunRemover).

Additionally it would be nice if all the source files are grouped into a folder.

comment:33 Changed 5 years ago by epitzer

  • Status changed from assigned to accepted

I agree about grouping the modifiers into a folder.

Here are some examples for the application of the questionable modifiers:

  • When handling very large run collections it can become cumbersome to manually delete a lot of runs especially when handling complex criteria. For this purpose, the RunCollectionRemover can be combined with the filtering capabilities to remove only the required selection of runs.
  • The RunCollectionSorter can, for example, be used to produce a certain order of e.g. mutation operators or problem names along an axis in a bubble chart.

comment:34 Changed 5 years ago by epitzer

r7228: Move run collection modifiers into a separate folder

comment:35 Changed 5 years ago by epitzer

  • Owner changed from epitzer to mkommend
  • Status changed from accepted to reviewing

comment:36 Changed 5 years ago by mkommend

  • Owner changed from mkommend to epitzer
  • Status changed from reviewing to readytorelease

comment:37 Changed 5 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.5 to 3.3.6
Note: See TracTickets for help on using tickets.