Opened 10 years ago

Closed 10 years ago

#1516 closed enhancement (done)

Enable caching of external evaluation results

Reported by: epitzer Owned by: abeham
Priority: medium Milestone: HeuristicLab 3.3.5
Component: Problems.ExternalEvaluation Version: 3.3.5
Keywords: Cc:

Description

External evaluations are often very time consuming. It could be beneficial to enable caching of results to save computation time.

Change History (15)

comment:1 Changed 10 years ago by epitzer

  • Status changed from new to accepted

comment:2 Changed 10 years ago by epitzer

Add new CachedExternalEvaluator with replaceable permanent cache of all evaluated solutions (r6140)

comment:3 Changed 10 years ago by abeham

Hi Erik, thanks a lot for improving this! If you have some time on your hands, there are some further things that would improve the plugin. One thing would be a thread-safe version of the evaluator/client so that it works with the new parallel engine and another thing would be to enable parallelization in that a range of clients can be called so that one can run the simulations in parallel. I'd put that on our next meeting agenda if you don't mind.

comment:4 Changed 10 years ago by epitzer

That is actually exactly what I had in mind too. However, this should go into another ticket. What I still like to improve here is to impose a size limit on the cache and at the same time only kick out the values that have not been reused recently.

comment:5 Changed 10 years ago by epitzer

Add maximum capacity parameter and a least recently used replacement strategy to the evaluation cache (r6169)

comment:6 Changed 10 years ago by epitzer

prevent additional uncached execution of base class (r6172)

comment:7 Changed 10 years ago by epitzer

Prepare for parallel external evaluation (r6183)

  • move cache to problem for better visibility
  • configurable persistence of cache content
  • proper synchronization for parallel access to cache

comment:8 Changed 10 years ago by epitzer

improve synchronization of cache (r6188)

  • set locks per instance instead of static
  • remove failed evaluations from active set

comment:9 Changed 10 years ago by epitzer

Enable export of Cache to CSV file. (r6265)

comment:10 Changed 10 years ago by epitzer

fixed synchronization issue in evaluation cache and simplified event handling (r6291)

comment:11 Changed 10 years ago by epitzer

  • Owner changed from epitzer to abeham
  • Status changed from accepted to reviewing

comment:12 Changed 10 years ago by abeham

  • Status changed from reviewing to readytorelease

I reviewed the changes and have no further comments. I think the locking mechanism is reasonable and not too difficult to understand, the simplification in r6291 certainly improved readability.

comment:13 Changed 10 years ago by abeham

  • Status changed from readytorelease to reviewing

I found some trivial issues

comment:14 Changed 10 years ago by abeham

  • Status changed from reviewing to readytorelease

r6519

  • fixed code formatting
  • added some license headers
  • removed unnecessary resx file

comment:15 Changed 10 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.4 to 3.3.5
Note: See TracTickets for help on using tickets.