Opened 2 years ago

Last modified 3 days ago

#2457 accepted feature request

Create an expert system optimizer

Reported by: abeham Owned by: abeham
Priority: medium Milestone: HeuristicLab 3.3.x Backlog
Component: Optimization Version: branch
Keywords: Cc:

Description (last modified by abeham)

A prototype of an expert-system-like optimizer should be implemented that makes algorithm instance suggestions based on FLA information and performance data from an OKB.

This will build on the developments in #2431.

Change History (71)

comment:1 Changed 2 years ago by abeham

  • Description modified (diff)
  • Version changed from 3.3.12 to branch

comment:2 Changed 2 years ago by abeham

  • Status changed from new to accepted

comment:3 Changed 2 years ago by abeham

r12842:

  • Added plugins
  • Worked on properties of the optimizer

comment:4 Changed 2 years ago by abeham

r12847: Added view

comment:5 Changed 2 years ago by abeham

r12860: worked on expert system

comment:6 Changed 23 months ago by abeham

r12957: Worked on expert system

  • using newly created ERT helper class

comment:7 Changed 20 months ago by abeham

r13475: updated from trunk, worked on okb connection for downloading knowledge base

comment:8 Changed 20 months ago by abeham

r13485:

  • fixed bugs in iterated sampling view
  • worked on download runs from okb feature

comment:9 Changed 18 months ago by abeham

r13551: Added characteristic calculator for qap and adapted expert system view

comment:10 Changed 18 months ago by abeham

r13561:

  • added two types of problem instance mappings: PCA and MDS

comment:11 Changed 18 months ago by abeham

r13564:

  • Fixed bugs in IteratedAlgorithm

comment:12 Changed 18 months ago by abeham

r13568: added code to auto-sync stopping criterion with embedded algorithm in iterated algorithms

comment:13 Changed 18 months ago by abeham

r13569: fixed bugs in iterated algorithm, fixed bug in expert system view

comment:14 Changed 18 months ago by abeham

r13583:

  • Added stripped-down version of FLA branch
  • Added appropriate calculators
  • Fixed detecting maximization in RLD view

comment:15 Changed 18 months ago by abeham

r13593:

  • Changed ICharacteristicCalculator interface
  • Updated OKB Problem view

comment:16 Changed 18 months ago by abeham

r13594: Adapted branch to trunk changes

comment:17 Changed 18 months ago by abeham

r13601:

  • Added merge info

comment:18 Changed 17 months ago by abeham

r13649: fixed bugs in suggestions

comment:19 Changed 17 months ago by abeham

r13663: worked on expert system

comment:20 Changed 17 months ago by abeham

r13667: worked on expert system, created new application, merged trunk change of r13666

comment:21 Changed 17 months ago by abeham

r13668: Worked on expert system

comment:22 Changed 17 months ago by abeham

r13694: branched permutation encoding

comment:23 Changed 17 months ago by abeham

r13706: Added pre-existing solution creator for permutation and respective interface

comment:24 Changed 17 months ago by abeham

r13713: worked on seeding algorithm instances

comment:25 Changed 17 months ago by abeham

r13718: worked on expert system

comment:26 Changed 16 months ago by abeham

r13720: worked on expert-system

comment:27 Changed 16 months ago by abeham

r13722:

  • Renamed remaining files from ExpertSystem to KnowledgeCenter
  • Added ability to scatter plot to display a regression line
  • Allowed to execute multiple instances at once and displaying either only final result or tracking result
  • Split runs in seeded runs and instance runs

comment:28 Changed 16 months ago by abeham

r13743: adding relational SOM projection for solution network visualization

comment:29 Changed 16 months ago by abeham

r13744: added best-n scopes solution analyzer

comment:30 Changed 16 months ago by abeham

r13745: added line annotations to the network visu to highlight close solutions

comment:31 Changed 16 months ago by abeham

r13748: Added toolbar, fixed some bugs

comment:32 Changed 16 months ago by abeham

r13750: Added SOM projection for problem instances, fixed a bug (learningRadius was not used)

Last edited 16 months ago by abeham (previous) (diff)

comment:33 Changed 16 months ago by abeham

r13751: worked on problem instance mapping

comment:34 Changed 16 months ago by abeham

r13752:

  • improved mapping of problem instances by normalization (z-score) and handling of missing values (median)
  • enabled to select the characteristics that should be included in the mapping
  • improved speed of OKB download by calling service methods in parallel

comment:35 Changed 16 months ago by abeham

r13757:

  • worked on problem instance mapping
  • started working on improved suggestion algorithm

comment:36 Changed 16 months ago by abeham

r13759: worked on suggestion algorithm

comment:37 Changed 16 months ago by abeham

r13767: fixed cross-thread exception

comment:38 Changed 15 months ago by abeham

r13774: worked on recommendation algorithms

comment:39 Changed 15 months ago by abeham

r13787: worked on performance modeling

comment:40 Changed 15 months ago by abeham

r13791: working on recommendation algorithms

comment:41 Changed 15 months ago by abeham

r13792: merged changes from trunk

comment:42 Changed 15 months ago by abeham

r13794: worked on recommendation algorithms (x-validation)

comment:43 Changed 15 months ago by abeham

r13797: worked on testing recommendation algorithms through x-validation

comment:44 Changed 15 months ago by abeham

r13803:

  • changed expected runtime calculation
    • now outputs positive infinity instead of nan when no run was successful
    • now filters outliers in successful and unsuccessful runs by using two standard deviations of the mean of successful runs as lower bound
      • this change allows having unsuccessful runs in the database with low evaluations / runtime (e.g. due to being aborted early or from an experiment where the max budget was lower)
  • worked on recommendation algorithms
    • implemented several performance measures (absolute error, absolute log error, ndcp, kendall's tau) to evaluate the ranking

comment:45 Changed 15 months ago by abeham

r13804: removed algorithm instances that do not appear in any run

comment:46 Changed 15 months ago by abeham

r13809: added file system based cache of runs

comment:47 Changed 14 months ago by abeham

r13861: added directed walk for qap

comment:48 Changed 14 months ago by abeham

r13878: added standardization of features for recommendation and using log10 of the expected runtime for clustering

comment:49 Changed 13 months ago by abeham

r13920: added repetitions to LocalAnalysis and computing average

comment:50 Changed 6 months ago by abeham

r14599: ignore .vs directory

comment:51 Changed 6 months ago by abeham

r14600: updated branch to trunk

comment:52 Changed 6 months ago by abeham

r14661: updated branch to trunk

comment:53 Changed 6 months ago by abeham

r14664: updated branch to trunk

comment:54 Changed 6 months ago by abeham

r14665: updated branch to trunk

comment:55 Changed 6 months ago by abeham

r14666: copied MemPR algorithm from its branch to this branch

comment:56 Changed 6 months ago by abeham

r14667: added TSNE visualization of problem instance map

comment:57 Changed 5 months ago by abeham

r14678: worked on problem instance detection

comment:58 Changed 5 months ago by abeham

r14681: removed unnecessary files

comment:59 Changed 5 months ago by abeham

r14690: working on identification of problem instances

comment:60 Changed 5 months ago by abeham

r14691: working on identification of problem instances

comment:61 Changed 5 months ago by abeham

r14694: fixed memory leak in MemPR

comment:62 Changed 5 months ago by abeham

r14695: small changes to MemPR

comment:63 Changed 5 months ago by abeham

r14696: worked on instance identification

comment:64 Changed 4 months ago by abeham

r14776: working on MemPR integration

comment:65 Changed 7 weeks ago by abeham

r15031: worked on code for eurocast paper

comment:66 Changed 11 days ago by abeham

r15253: removed Algorithms.DataAnalysis from branch (t-sne is in trunk already)

comment:67 Changed 11 days ago by abeham

r15254: changed references to file reference

comment:68 Changed 11 days ago by abeham

r15255: small changes

comment:69 Changed 11 days ago by abeham

r15256: merged trunk into branch

comment:70 Changed 4 days ago by abeham

r15279: readded information analysis chart

comment:71 Changed 3 days ago by abeham

r15282: merged trunk into branch

Note: See TracTickets for help on using tickets.